I am working on Azure Service Fabric Reliable Actor implementation. Any idea/link on where can I store the Configuration value (e.g. DB connection string) and how to access that in code.
A Service Fabric application consists of the code package, a config package, and the data (https://azure.microsoft.com/en-gb/documentation/articles/service-fabric-application-model/).
You can use the config package to store and retrieve any kind of key-value pairs you need e.g. a connection string. Have a look at this article https://azure.microsoft.com/en-us/documentation/articles/service-fabric-manage-multiple-environment-app-configuration/ for more information.
You can add multiple ApplicationParameters file. Just copy and paste the same from Cloud.Xml and use for multiple environment configurations.
Steps to Make necessary changes
The values given in the Settings.xml need to be overridden in the ApplicationManifest.xml when it imports the ServiceManifest.xml .Below is the code supporting the overriding changes add them in the ApplicationManifest.xml.
a) Add the Parameter Default value first
<Parameters>
<Parameter Name="StatelessService1_InstanceCount" DefaultValue="-1" />
<!-- Default Value is set to Point to Dev Database -->
<Parameter Name="DatabaseString"DefaultValue="Server=someserver.database.windows.net\;Database=DbDev;user id=[userid];password=[Password];Trusted_Connection=false;" />
</Parameters>
b) Then override it in the ServiceManifestImport
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="StatelessServicePkg"
ServiceManifestVersion="1.0.0" />
<ConfigOverrides>
<ConfigOverride Name="Config">
<Settings>
<Section Name="DatabaseConnections">
<Parameter Name="DbString" Value="[DatabaseString]" />
</Section>
</Settings>
</ConfigOverride>
</ConfigOverrides>
</ServiceManifestImport>
The above code change will override the following code in settings.xml
<Section Name="DatabaseConnections">
<Parameter Name="DbString" Value="Server=someserver.database.windows.net\;Database=DbDev;user id=[userid];password=[Password];Trusted_Connection=false;" />
</Section>
Overall when the application is deployed the values in the ApplicationParameter DevParam.xml or QaParam.xml or ProdParam.xml will overtake all the setting values.
<Parameters>
<Parameter Name="StatelessService1_InstanceCount" Value="-1" />
<Parameter Name="DatabaseString" Value="Server=someserverqa.database.windows.net\;Database=DbQA;user id=[userid];password=[Password];Trusted_Connection=false;" />
</Parameters>
In addition to the above info, it is important to know the order in which ASF overrides application setting:
Service Fabric will always choose from the application parameter file
first (if specified), then the application manifest, and finally the
configuration package (source)
For more info:
http://www.binaryradix.com/2016/10/reading-from-configuration-within-azure.html
Related
For two environments, local and cloud, how would I set up custom settings or parameters for resources such as Sql databases, storage accounts, etc... Ideally it would be one parameter name called in code to say, point a DbContext towards a particular database, that in configurations for either a local or cloud environment be different. Thank you.
In order to have per environment variables for running Service Fabric locally and in the cloud this is what you must do:
Add your custom config section and parameters to the Settings.xml file of the Service/Actor project (located at \PackageRoot\Config\Settings.xml from the project root). Leave the parameters blank as we will be setting these elsewhere per environment. Here is an example one.
<?xml version="1.0" encoding="utf-8" ?>
<Settings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<!-- Add your custom configuration sections and parameters here -->
<Section Name="UserDatabase">
<Parameter Name="UserDatabaseConnectionString" Value="" />
</Section>
</Settings>
In the ApplicationManifest.xml file of your Service Fabric project, there will be <ServiceManifestImport> elements for each of your included projects. Underneath that will be a <ConfigOverrides> element where we will declare what values for our configs will be supplanted by values set per environment in the local and cloud xml files underneath ApplicationParameters in our Service Fabric project. In that same ApplicationManifest.xml file, you'll need to add the parameter that will be present in the local and cloud xml files, otherwise they'll be overwritten upon build.
Continuing with the example above, this is how it would be set.
<Parameters>
<Parameter Name="ServiceName_InstanceCount" DefaultValue="-1" />
<Parameter Name="UserDatabaseConnectionString" DefaultValue="" />
</Parameters>
<ConfigOverrides>
<ConfigOverride Name="Config">
<Settings>
<Section Name="UserDatabase">
<Parameter Name="UserDatabaseConnectionString" Value="[UserDatabaseConnectionString]" />
</Section>
</Settings>
</ConfigOverride>
</ConfigOverrides>
In the local.xml and cloud.xml files underneath ApplicationParameters in your Service Fabric project, you will specify your environment specific variables like so.
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/AppFabricName.ServiceFabric" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="ServiceName_InstanceCount" Value="1" />
<Parameter Name="UserDatabaseConnectionString" Value="Server=(localdb)\MsSqlLocalDb;Database=Users;User=ReadOnlyUser;Password=XXXXX;" />
</Parameters>
</Application>
Finally, in your Service/Actor you can access these per-environment configuration variables like so.
var configurationPackage = Context.CodePackageActivationContext.GetConfigurationPackageObject("Config");
var connectionStringParameter = configurationPackage.Settings.Sections["UserDatabase"].Parameters["UserDatabaseConnectionString"];
You can just use environment variables just like any other application, this also works with guest executable within service fabric unlike the settings.xml as this requires the built-in service fabric runtime.
Within your application you can access environment variables just like any other .net application though the GetEnvironmentVariable method on the Environment class:
var baseUri = Environment.GetEnvironmentVariable("SuperWebServiceBaseUri");
Then we need to setup some default environment variables values, this is done within the ServiceManifest.xml manifest file of the service.
<?xml version="1.0" encoding="utf-8" ?>
<ServiceManifest Name="MyServicePkg" Version="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<!-- snip -->
<CodePackage Name="Code" Version="1.0.0">
<!-- snip -->
<EnvironmentVariables>
<EnvironmentVariable Name="SuperWebServiceBaseUri" Value="http://localhost:12345"/>
</EnvironmentVariables>
</CodePackage>
<!-- snip -->
</ServiceManifest>
These environment variable can then be overridden within the ApplicationManifest.xml file by using the following code:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="ChileTargetType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<!-- snip -->
</Parameters>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="MyServicePkg" ServiceManifestVersion="1.0.0" />
<EnvironmentOverrides CodePackageRef="Code">
<EnvironmentVariable Name="SuperWebServiceBaseUri" Value="https://the-real-live-super-base-uri.com/"/>
</EnvironmentOverrides>
</ServiceManifestImport>
<!-- snip -->
</ApplicationManifest>
This then can be parameterised like any other application manifest setting using the local.xml and cloud.xml.
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/AppFabricName.ServiceFabric" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="MyService_SuperWebServiceBaseUri" Value="https://another-base-uri.com/" />
</Parameters>
</Application>
Then we'll have to update the ApplicationManifest.xml to support these parameters;
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="ChileTargetType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="MyService_SuperWebServiceBaseUri" DefaultValue="https://the-real-live-super-base-uri.com/" />
</Parameters>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="MyServicePkg" ServiceManifestVersion="1.0.0" />
<EnvironmentOverrides CodePackageRef="Code">
<EnvironmentVariable Name="SuperWebServiceBaseUri" Value="[MyService_SuperWebServiceBaseUri]"/>
</EnvironmentOverrides>
</ServiceManifestImport>
<!-- snip -->
</ApplicationManifest>
The above answers explain well how it is done. I want to add a sidemark, why it is that 'convoluted':
It has to be this way, as the services are intended to be self-contained. They should run by default in any application they are linked into. Independent of the application's Manifest. So the service can only rely on parameters, which are at least predefined in its own configuration.
These presettings can then be overwritten by the application. This is the only universal approach.
I've been unable to get our TFS build server to stop putting it's local build path in the {Project}.SetParameters.xml file. I'm currently trying to override it using a project parameters.xml file, but can't find the magic combination that will let me override that property. Main problem is that I don't know where that value comes from, so it's difficult to match & override it.
Current result:
<?xml version="1.0" encoding="utf-8"?>
<parameters>
<setParameter name="IIS Web Application Name" value="Default Web Site/MyService" />
<setParameter name="IIS Web Application Pool Name" value="MyService" />
<setParameter name="IisVirtualDirectoryPhysicalPath" value="F:\Builds\59\EAS\MyService.Changeline_1.1.1.0\Sources\API Layer\API_deploy" />
</parameters>
I want that last line to be:
Of course, MSFT documentation is lacking in this regard. Here's my current parameters.xml file, that does not work. No build errors, it just doesn't overwrite that IisVirtualDirectoryPhysicalPath value.
<?xml version="1.0" encoding="utf-8"?>
<parameters>
<parameter name="IisVirtualDirectoryPhysicalPath"
description="Physical path where files for this Web application will be deployed."
defaultValue="C:\Program Files\Blah\MyService\API"
tags="DestinationVirtualDirectory">
<parameterEntry kind="DestinationVirtualDirectory" scope=".*" match="^F:\\Builds[\w\s\\\.\-]+$" />
</parameter>
</parameters>
I couldn't get that parameters.xml to do anything.
If you create a MyService.wpp.targets file,
http://msdn.microsoft.com/en-us/library/ff398069(v=vs.110).aspx
Then you can add an entry into that file. It will set this value in the setparamters.xml (it wont however set the value inside the zip, which is a bit of a pain, cos what if the someone just imports the zip. But this is better than nothing.
C:\Program Files\Blah\MyService\API
How do I configure Azure Cache in web roles to use my custom IDataCacheObjectSerializer class? In case you are wondering why I want to use a custom serializer: I want to use the compact text based style JSON(.net) serialization combined with compression. In my .config files I can enable compression:
<dataCacheClient name="default" isCompressionEnabled="true"/>
But how / where do I tell Azure Cache (preview) to use my custom IDataCacheObjectSerializer class that uses JSON serialization?
Jagan Peri has a relevant blog post.
From the post:
<dataCacheClients>
<tracing sinkType="DiagnosticSink" traceLevel="Verbose" />
<!-- This is the default config which is used for all Named Caches
This can be overriden by specifying other dataCacheClient sections with name being the NamedCache name -->
<dataCacheClient name="default" useLegacyProtocol="false">
<autoDiscover isEnabled="true" identifier="WorkerRole1" />
<!--<localCache isEnabled="true" sync="TimeoutBased" objectCount="100000" ttlValue="300" />-->
<serializationProperties serializer="CustomSerializer" customSerializerType="Your.Fully.Qualified.Path.To.IDataCacheObjectSerializer,WorkerRole1" />
</dataCacheClient>
</dataCacheClients>
I need to persist all IIS logs automatically in Azure storage, how to enable it?
I see that there are different properties in DiagnosticMonitorConfiguration class, e.g. DiagnosticInfrastructureLogs, Logs, Directories, etc. Which of the above is responsible for the IIS logs? What blob/table are the logs stored to?
Thanks!
Under Directory Buffers, you have two options.. IIS Logs and Failed RequestLogs.
Here's a snippet from the diagnostic schema definition that outlines these options:
<Directories bufferQuotaInMB="1024"
scheduledTransferPeriod="PT1M">
<!-- These three elements specify the special directories
that are set up for the log types -->
<CrashDumps container="wad-crash-dumps" directoryQuotaInMB="256" />
<FailedRequestLogs container="wad-frq" directoryQuotaInMB="256" />
<IISLogs container="wad-iis" directoryQuotaInMB="256" />
<!-- For regular directories the DataSources element is used -->
<DataSources>
<DirectoryConfiguration container="wad-panther" directoryQuotaInMB="128">
<!-- Absolute specifies an absolute path with optional environment expansion -->
<Absolute expandEnvironment="true" path="%SystemRoot%\system32\sysprep\Panther" />
</DirectoryConfiguration>
<DirectoryConfiguration container="wad-custom" directoryQuotaInMB="128">
<!-- LocalResource specifies a path relative to a local
resource defined in the service definition -->
<LocalResource name="MyLoggingLocalResource" relativePath="logs" />
</DirectoryConfiguration>
</DataSources>
I've just investigated TFS Integration Tools to migrate all source code along with the history from one TFS server 2010 to another, and I have experienced a strange behaviour. Here are my actions:
Installed TFS Server locally.
Created two project collections and project in each of them (Source project and Target project).
Filled Source project with some files, made several check-outs and check-ins.
Launched TFS Integration Tools.
Created new configuration with template VersionControl.xml
Chosen Source project as a Left Source and Target project as a Right Source.
Started the migration. Everything was well, I could see all history in the progress window.
And the result of these actions is the following - nothing changed. Literally. I expected that the source control of the Target project will be filled with files from Source project's ones. But both projects remained exactly as they were.
What am I missing? What have I misunderstood about the TFS Integration Tools?
Update
Here is the config that was generated by the application:
<?xml version="1.0" encoding="utf-16"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" UniqueId="cdf29df8-c74f-4339-b96f-7eb621c1cee1" FriendlyName="TFS to TFS Version Control">
<Providers>
<Provider ReferenceName="febc091f-82a2-449e-aed8-133e5896c47a" FriendlyName="TFS 2010 Migration VC Provider" />
</Providers>
<Addins />
<SessionGroup CreationTime="2011-04-28T11:24:10.9503239Z" FriendlyName="TFS to TFS Version Control" SessionGroupGUID="394da96f-b8b6-4bc1-9b72-1c6234c4b9f1" Creator="<domain>\<user_name>" SyncIntervalInSeconds="0" SyncDurationInMinutes="0">
<MigrationSources>
<MigrationSource InternalUniqueId="181ddf3d-9cdb-461a-9dba-4338cf2a02f8" FriendlyName="<server_name> (VC)" ServerIdentifier="c0a0f4e4-4a37-4a89-ba23-fbd323680786" ServerUrl="http://<server_name>:8080/tfs/sourcecollection" SourceIdentifier="SourceProject" ProviderReferenceName="febc091f-82a2-449e-aed8-133e5896c47a" EndpointSystemName="TFS">
<Settings>
<Addins />
<UserIdentityLookup />
<DefaultUserIdProperty UserIdPropertyName="DisplayName" />
</Settings>
<CustomSettings />
<StoredCredential />
</MigrationSource>
<MigrationSource InternalUniqueId="e292262f-9479-490f-b5e3-ce7e845d1468" FriendlyName="<server_name> (VC)" ServerIdentifier="f01ebcec-ea18-4933-b3fd-751740904136" ServerUrl="http://<server_name>:8080/tfs/targetcollection" SourceIdentifier="Target" ProviderReferenceName="febc091f-82a2-449e-aed8-133e5896c47a" EndpointSystemName="TFS">
<Settings>
<Addins />
<UserIdentityLookup />
<DefaultUserIdProperty UserIdPropertyName="DisplayName" />
</Settings>
<CustomSettings />
<StoredCredential />
</MigrationSource>
</MigrationSources>
<Sessions>
<Session CreationTime="2011-04-28T11:24:10.9333256Z" SessionUniqueId="4aa097f3-9a01-4924-b562-384215b4ef2f" FriendlyName="Version Control Session" LeftMigrationSourceUniqueId="181ddf3d-9cdb-461a-9dba-4338cf2a02f8" RightMigrationSourceUniqueId="e292262f-9479-490f-b5e3-ce7e845d1468" SessionType="VersionControl">
<EventSinks />
<CustomSettings>
<SettingXml />
<SettingXmlSchema />
</CustomSettings>
<Filters>
<FilterPair Neglect="false">
<FilterItem MigrationSourceUniqueId="181ddf3d-9cdb-461a-9dba-4338cf2a02f8" FilterString="$/SourceProject" />
<FilterItem MigrationSourceUniqueId="e292262f-9479-490f-b5e3-ce7e845d1468" FilterString="$/Target" />
</FilterPair>
</Filters>
</Session>
</Sessions>
<Linking>
<CustomSettings />
<LinkTypeMappings />
</Linking>
<WorkFlowType Frequency="ContinuousManual" DirectionOfFlow="Unidirectional" SyncContext="Disabled" />
<CustomSettings />
<UserIdentityMappings EnableValidation="false">
<UserIdentityLookupAddins />
</UserIdentityMappings>
<ErrorManagement>
<ErrorRouters />
<ReportingSettings />
</ErrorManagement>
</SessionGroup>
</Configuration>
Only thing that looks out is that the value ServerIdentifier attribute on the two MigrationSource's is different, as this is a single server deployment they should be the same, can't think of a reason it should matter though - but it is the only thing that I would change.
Everything else in your config is fine.
I must admit that this is really strange, but the issue is solved. I haven't done any changes to configuration, but when I've reran the migration everything worked - all source code appeared in Target project source control along with check-ins history. Moreover, I've created another empty Target project and migration worked fine as well. It seems that the problem was caused by some temporary TFS Integration Tools fluctuation.