I have arm template to recreate resource group with resources and their settings. This works fine.
Use case:
Some developer goes to azure portal and update some settings for some resource. Is there a way how to get exact changes that can be applied to my template to take these changes in effect? (Update template in source control)
If I go to automation script in resource group I can see all resources but my template in source control is different (parameters, conditions, variables, multiple templates linked together ...). I can't see on first look what changes were done and I can't use any diff.
Maybe I missed completely something but how are you solving this issue?
Thanks.
It is not easy to see any changes to resources by comparing templates from within the portal. Best practice is to always use ARM templates (and CI/CD pipelines) to deploy ARM templates to provision resources. Keep these ARM templates under source control to track them.
Further than that, I think you have two main options to track these changes:
1) You can use the Azure Activity Log to track the changes. The Azure Activity Log is a subscription log that provides insight into subscription-level events that have occurred in Azure. This includes a range of data, from Azure Resource Manager operational data to updates on Service Health events.
2) Write a little intelligent code against the Management Plane API. A good starting point is https://resources.azure.com/subscriptions. You could write a little extract that pulls all your resources out daily and commits them to a git repo. This will only update for changes to templates. You can then analyse the delta as or when you need.
Conceptionally, the developer should never 'go[es] to azure portal and update some settings for some resource', except for his own development / unit testing work. He then should produce an updated ARM template for deployment in the TST etc environments, and document his unit-tested changes with the new template. If his update collides with your resources in TST he will probably come to you to explain his changes, and discuss the resolution.
Related
We are using Azure cloud platform and used Terraform to provision our resources using Azuredevops pipelines. So when we provisioned the resources we kept the statefiles resource wise(eg:ApIM, Appservice, AKS, Storage Accounts, etc..) and the statefiles where in sync with the actual resource.
But we have other ADO pipelines as part of our application releases, which are making some changes on the previous terraform built resources like (API creation and update, Tags update to resources, additional component creation to the base resource etc..). So those changes made our terraform states out of sync the actual resources and when we triggered the pipeline for terraform plan for those resources, number of changes are showing and some resources are showing to replace itself.
So, We need to make our existing resources statefile in sync with any kind of pipeline\manual changes from the portal and we have to follow the practice of incrementally updating the statefile.
So by searching in internet we found that we can achieve this using terraformer and planning to add a Pipeline for terraformer task that will update those changes to the existing statefiles for each resource (planning to schedule this pipeline weekly).
Is it possible to use terrafomer to make the incremental changes with both statefile and already used terraform manifests in sync.
Your using of Terraform for IaaC seems wrong. If you plan to deploy resources through terraform then those resources should not be modified by other external factors. If it does, then you lose one of terraform's key feature i.e, maintaining a state and updating the resources.
Terraformer is a completely different tool that is not suited for your direct usecase. It is used to generate tf files/state files for existing resources created via methods other than terraform (Eg: console)
My recommendation for you will be to go through the basics of terraform, Iaac and restructure your pipelines/architecture.
Below are some links that were helpful for me.
https://developer.hashicorp.com/terraform/language/state
https://www.terraform-best-practices.com/examples
https://developer.hashicorp.com/terraform/language/modules
I'm currently busy with an internship. In this internship I need to create a program which automatically creates "snapshots" of the current state of Azure Resources (And sometimes their dependencies) which need to be deployed to another environment. e.g. Acceptance -> Production. These snapshots must then be deployed to the new environment at a later date which has been coordinated with the client.
A solution can consists out of >100 Azure resources, ranging from API Managers, to LogicApps, CosmosDB's, etc. When a customer accepts or says "ok" to a few resources (= a part of the total solution) a snapshot needs to be made of that resource, in the specific state when the client said OK. That means that I also have to create a snapshot of the dependencies of that specific resource (LogicApp can depend on a CosmosDB, Keyvault etc).
And I can't just take a reference to the resource in the Acceptance environment, I need to bring that dependency over to production as well, seeing as it might be possible that another developer will continue working on said dependency which might break things.
I am bit of at a loss as to which direction to take here. I don't have a lot of experience with ARM (Templates) and I have been making several prototypes for a month now.
I have first tried to generate my own ARM (and Bicep) files through gathering information from the Azure Rest API, but I soon discovered this is not viable because I cannot extract all of the information from that API to create said ARM file.
I then looked into modifying the generated ARM files from Azure itself. Whilst this is an option, it contains a lot of information which I do not need or want to transfer over to another environment. It is also very hard to determine which parts of the generated ARM file must be deleted, updated, copied or left alone. And then I still need to recursively get the ARM templates of the dependencies and go through those in an automated way as well.
Is modifying existing ARM templates the best route to go here? Or does a similar product already exist which might help achieve my goal?
Thank you!!
In this case, I would not go with the approach to modify exported ARM templates but I would go with approach of Infrastructure as Code i.e., I would created ARM templates as granular as possible i.e., may be one template per resource at the least and store that infrastructure code in a source repository and if required version it to use it in different environments. The reason for recommending one template per resource is to take care of the dependencies in a complex environment. I know this might look like a bigger activity for the first-time implementation but once the templates are integrated into any continuous integration and continuous deployment (CI/CD) tool like Azure DevOps then all of it can be automated with the help of release pipelines for fast and reliable application and infrastructure updates. For more information in this regard, please refer this and this Azure documents.
I have created a template to deploy a service bus (ServiceBus/namespace) and three "nested"/"tiered" resources: topics (ServiceBus/namespace/topics), queues (ServiceBus/namespace/queues), and authorisation rules (ServiceBus/namespace/authorizationrules). And the resources are built correctly.
However, I want the entire service bus and it's tiered resources to be fully customisable from the parameter file. for the tiered resources this works when adding new resources, and adjusting existing resources in the parameter file however I cannot find a way to delete tiered resources that are not deployed by the template.
I have tried complete deployment mode (even though I only want resources in the service bus hierarchy to be deleted) however the tiered resources are not destroyed in the same was as resources directly beneath the resource group (i.e. the service bus itself). I have tried entirely deleting the service bus and rebuilding it, but this causes all the keys for the authorisation rules (shared access policies) to change.
If anyone knows if, and how, I can get the functionality I desire it would be greatly appreciated
If you want to "clean up" child resources that were not defined in the template that may or may not work depending on how ARM recognizes the resources. TL;DR if it's not working with complete mode as you tried it, ARM doesn't recognize those resources in a way that would clean up the extra ones.
There is an upcoming feature called deploymentStacks that will help with this scenario but today the capabilities are as you have it.
You could script a more surgical removal of those resources after deployment finishes by leveraging deploymentOperations but there's nothing more "built-in" today.
Since the pricing does not offer much choice in terms of flexibility my developer MSDN account is quickly running out of credits using Azure Time Series Insights fro a Proof of Concept. Is it somehow possible to suspend the service so no costs are incurred? I would hate to have to delete the whole thing and set it up again when we start working again on the PoC.
Currently, Azure still do not provide a way to suspend TSI environment.
Maybe you can use scripted template deployment for creating/deleting TSI environment.
With this approach, however, you are going to constantly loose your data.
On the link below there are guidelines, provided by Microsoft, on how to implement template deployment:
https://learn.microsoft.com/en-us/azure/time-series-insights/time-series-insights-manage-resources-using-azure-resource-manager-template
The general steps provided by MSFT are:
Install PowerShell
Create the template and a parameter file.
In PowerShell, log in to your Azure account.
Create a new resource group if one does not exist.
Test the deployment.
Deploy the template.
If you modify a resource through the Azure Portal, you may forget to update your ARM JSON before your next deployment and this would lead to an outdated deployment.
Also, if you have system admins taking care of your Azure Portal and your developers keeping track of ARM JSON in the source control, you need a good way to make sure to keep the environments consistent.
What would be the best practise to do that?
That's an easy question, don't modify anything through the portal. Or follow the change process as described by ITIL, or invent your own wheel.
But apart from that, there is no answer, there is no way your repository will magically sync with what you actually have.