We have a large system built in Azure apps. It is made up of an App Service for our API and several Functions Apps for backend processing.
What's the best way to allow these apps to share configuration?
We use ARM templates currently to set up the environment variables for each app, which is fine for deploy-time, but there's nothing to keep the config in sync between the apps.
A use case might be a feature flag that controls whether a sub-system is operational. We might want this flag to be used in the API and a Functions App. At present we can manually go in and set the variable in each of the apps, but it would be easier to manage if we only had to do it in one location.
Ideally, any update to the config would be detected by Azure and trigger a restart of the service, as currently happens with the native implementation.
Is there a good, off-the-shelf, way to do this? Or will I be rolling my own with a table in a database and a lightweight function?
One way would be to use the new App Configuration service: https://learn.microsoft.com/en-us/azure/azure-app-configuration/overview.
It is meant for sharing configuration settings across components.
Note it is not meant for secrets, that's what Key Vault is for.
There is a guidance/design pattern for this from Microsoft, it can be found from here.
Best Practice in Architecture: You can use the external configuration store pattern and use a Redis Cache to share the configuration between multiple applications as described in here: https://learn.microsoft.com/en-us/azure/architecture/patterns/external-configuration-store
The approach is you can get this data from Appsettings for each environement (this can be automated in CI/CD pipeline). On first connection you store the data in RedisCache.
For senstive data: Use Keyvault to store the secrets/keys/certificates.
Related
Currently, I'm storing all key/value pairs in Application Settings, but I'm not happy with this approach. What is the recommended way to store settings for dev, test, stage, and prod? I need to make sure that prod settings are not visible to developers. Is there a way to create 4 different JSON files and define access permissions on them? Or do I need to create 4 different Function apps (or subscriptions)?
Azure App Configuration is a relatively new service that sounds like it could help in terms of managing the config values centrally with more control than individual instance App Settings.
Beyond that, you could perhaps build segregation by limiting devs to pushing code only and not accessing the hosting environment (Azure portal, etc). The layer in between would be something like Azure DevOps or Github Actions that has access to Azure, while devs are limited to pushing code that triggers deployment.
Also worth reminding ourselves that devs ultimately have a lot of access by virtue of writing the code. If they want to get at runtime data, they can, somehow. If you consider the devs untrusted, you may have bigger problems. If it's just a matter of preventing mistakes, a solid devops process is the key.
Is it possible to store data types in the Azure app configuration? Or is it left to the program calling the app configuration to declare the data type?
Absolutely. Each key-value in App Configuration has a content-type field. While you can put anything in it as long as your application understands it, we do recommend using MIME types, which is also what the out-of-box data types App Configuration uses.
More discussion can be found at
https://github.com/Azure/AppConfiguration/issues/116#issuecomment-559628997
Because it's designed to be a centralized location to store the data as key value. It works similarly to a web.config or configuration settings in App Service. So it really is designed to just be strings. If you are wanting a specific data type, it would be up to the dev to ensure it to convert it to the corresponding data type.
https://learn.microsoft.com/en-us/azure/azure-app-configuration/overview
Azure App Configuration provides a service to centrally manage application settings and feature flags. Modern programs, especially programs running in a cloud, generally have many components that are distributed in nature. Spreading configuration settings across these components can lead to hard-to-troubleshoot errors during an application deployment. Use App Configuration to store all the settings for your application and secure their accesses in one place.
I've used Go SDK for using deploying applications in Azure. I'm now planning to do the same for Azure Gov Cloud. Is there any change that has to be made in the calls from Compute, Storage, Network, Subscription client APIs to make it work for Azure Govcloud? For example for getting the list of locations, should there be any change in the API call arguments to make it work in GovCloud?
Based on the documentation you should just be able to set the AZURE_ENVIRONMENT environment variable prior to calling the NewAuthorizerFromEnvironment() method. Set the value of this environment variable to: AZUREUSGOVERNMENTCLOUD.
When interacting with sovereign clouds, which are often updated at a slower cadence than the Public cloud, you'll need to make sure that you choose API Versions which are available in your cloud. The Azure-SDK-for-Go has been specifically designed to facilitate this, putting API-Version in the path name.
In addition, as #Steve Michelotti pointed out, you'll want to inject the Government Environment so that the correct base URLs are used by the SDK.
We currently have a window service which send some notification emails to users after doing some processing on database(SQL database). Runs once in day.
We want to move this on azure cloud. One alternate is to put it on Azure VM as is. but I am finding some other best possible solution for that.
I study about recurring and on demand Web jobs but I am not sure is this is best solution.
Also is there any possibility to update configuration of service code in App.config without re-deploy the code of service on cloud. I means we can manage configuration from Azure portal.
Thanks in advance.
Update 11/4/2016
Since this was written, there are 2 additional features available in Azure that are both excellent choices depending on what functionality you need:
Azure Functions (which was based on the WebJobs described below): Serverless code that can be trigger/invoked in various ways, and has scaling support.
Azure Service Fabric: Microservice platform, with support for actor model, stateful and stateless services.
You've got 3 basic options:
Windows service running on VM
WebJob
Cloud service
There's a lot of information out there on the tradeoffs between these choices, but here's a brief summary.
VM - Advantages: you can move your service basically as it is without having to change much or any of your code. They also have the easiest connectivity with other resources in Azure (blob storage, virtual networks, etc). The disadvantage is you're giving up all the of PaaS advantages and are still stuck managing your own VM infrastructure
WebJob - Advantages: Multiple invocation options (queues, blobs, manually, queue receive loops, continuous while-loop style, etc), scheduled (would cover your case). Easy to deploy (can go with website, as a console app, automatically through Kudu), has some built in logging in Azure portal - and yes, to answer your question, you can alter the configuration in the portal itself for connection strings and app settings.
Disadvantages - you'll need to update code, you don't have access to underlying resources (if you need that), and more of something to keep in mind than a disadvantage - it uses the same resources as the webapp it's deployed with.
Web Jobs are the newest of the options, but at the same time appear to have active development going on to increase the functionality and usefulness.
Cloud Service - like a managed VM, has some deployment options, access to underlying VM if needed. Would require some code changes from your existing service.
There's nothing you've mentioned in your use case that makes me think a Web Job shouldn't be first thing you try.
(Edit: Troy Hunt has a great and relatively recent blog post illustrating most of the points I've mentioned about Web Jobs above: http://www.troyhunt.com/2015/01/azure-webjobs-are-awesome-and-you.html)
I'm trying to create a startup script or task that changes the configuration settings in the cscfg file, that executes on role start.
I'm able to access these settings, but haven't been able to successfully change them. I'm hoping for pointers on how to change settings on Role Start, or if it's even possible.
Thanks.
EDIT: What I'm trying to accomplish
I'm trying to make a service to more easily configure configuration values on Azure applications. Right now, if I want to change a setting that it the same over 7 different environments, I have to change it in 7 different .cscfg files.
My thought is I can create a webservice, that the application will query for its configuration values. The webservice will look in a storage place, like Azure Tables, and return the correct configuration values. This way, I can edit just one value in Tables, and it will be changed in the correct environments much more quickly.
I've been able to integrate this into a deployment script pretty easily (package the app, get the settings, change the cscfg file, deploy). The problem with that is every time you want to change a setting, you have to redeploy.
Black-o, given that your desire appears to be to manage the connection settings among multiple deployments (30+), I would suggestion that perhaps your need would be better met by using a separate configuration store. This could be Azure Storage (tables, or perhaps just a config file in a blob container), a relational database, or perhaps even an external configuration service.
These options require only a minimum amount of information to be placed into the cscfg file (just enough to point at and authorize against the configuration store), and allow you to maintain all the detail settings side by side.
A simple example might use a single storage account, put the configuration settings into Azure Tables, and use a "deployment" ID as the partition key. The config file for deployment then just needs the connection info for the storage location (unless you want to get by with a shared access signature), and its deployment ID. Then can then retrieve the configuration settings on role startup and cache them locally for performance improvements (either in a distributed memory cache or perhaps on the temp "local storage" drive for each instance).
The code to pull all this together shouldn't take more then a couple hours. Just make sure you also account for resiliency in case your chosen configuration provider isn't available.
The only way to change the settings during runtime is via Management API - craft the new settings and execute "Update Deployment" operation. This will be rather slow because it honors update domains. So depending on your actual problem there might be a much better way to solve it.