I am working on a script to collect all resources and put them into one resource group - however, the command in Powershell used for moving resources works on a resource group by resource group basis. This means that if there are dependent/related resources in different resource groups, the command fails.
The alternative, then, is to group resources by their relation to one another and move them using some other method (probably manually through the portal/REST API.)
How can one then go through a list of resources in a subscription and group them by their dependency/relations?
IMHO one of the reason to introduce resource group is to try avoid such situation / scenario i.e., as per this Azure document, a resource group is a container that holds related resources for an Azure solution.
I believe a straight forward way or feature (to get list of dependent / related resources that are sitting in different resource groups in a subscription and group them by their dependency/relations) is currently not available or not supported. But I see this feature request raised in UserVoice / feedback forum as a related one so if interested, you may upvote it or create a new feedback in there. In general, responsible Azure product / feature team would triage / start checking feasibility and prioritizing a received feedback based on various factors like number of votes a feedback receives, feasibility, open prioritized backlog items, etc.
On the other hand, as a workaround to accomplish this requirement for now, I suggest to try coming up with an automation (to get list of dependent / related resources that are sitting in different resource groups in a subscription and group them by their dependency/relations) by leveraging Azure Platform logs (and fetching those logs at a particular time window of operation) as those logs provide detailed diagnostic and auditing information for Azure resources and the Azure platform they depend on.
Other related references:
Move resources to a new resource group or subscription
Move operation support for resources
Troubleshoot moving Azure resources to new resource group or subscription
Related
As an MSP, we manage multiple customer subscriptions through Azure Lighthouse.
Historically we've used a single Automation Account per subscription to contain solutions such as runbooks related to the Start/Stop v1 solution, Automation-based Update Management, Inventory, and Change Tracking. This Automation Account is also linked to a single Log Analytics workspace per subscription.
We've since deployed Start/Stop v2, which uses LogicApps and Azure Functions. We now have a requirement to, as part of stopping and starting some VMs, stop and start some services on the machines itself. I plan on doing this through (PowerShell) Azure Automation Runbooks, which would only stop a VM if the runbook has successfully stopped a service on it.
My question relates to whether a single monolithic Automation Account is the way to go, or whether there are any considerations to be taken if we were to implement multiple Automation Accounts.
(I've noticed Best practice to deploy Azure Automation Account Runbooks, but that's over a year ago. Things might have changed in the mean time)
The best practice related question which you have mentioned still holds good i.e., 2 major attributes to consider are pricing and logical resource allocation. One other attribute to keep in mind while deciding whether to go with single or multiple automation accounts is the limits i.e., if you go with single automation account then does the traffic in your environment or the activities that your automation account does reach the limits mentioned here? If yes, then go for multiple automation accounts approach.
This question may sound a little odd, but here it goes: A customer of ours would like to get access to certain metrics of his environment of our product which we host on Azure for the customer. It's a pretty complicated deployment, but in the end it consists of an Application Gateway, some virtual machines and a dedicated Azure SQL database.
The customer now would want to get select metrics from this deployment forward to their own DataDog subscription, e.g. VM CPU metrics, database statistics and those things. DataDog obviously supports all this information (which is good), but as a default would slurp in information from all resources within our subscription (which is not OK).
Is there a way to fine-granularly define which data is forwarded to DataDog, e.g. the resources and also which type of metrics to forward for each resource? What are my options here? Is it enough to create a service principal with a limited reading right, or can I configure this somewhere else? I am unfortunately not familiar with DataDog.
The main thing which must be prevented is that the customer due to the metrics forwarding could get access to other metrics in our subscription - we need to control the exact scope of the metrics.
The pretty straightforward solution to this issue is to create a service principal via command line, and then to assign the monitoring role to this service principal only exactly for the resources you need. This even works down to a level of specific databases for example.
Kicker: This is not possible to do in such a granularity from the UI, but the az command line accepts assigning the monitoring reader permission on a deep resource ID level, even if the UI for this is not there. By finding the resource ID from the UI, and then using the resource ID from the command line, it's possible to achieve exactly this behaviour.
We are a software company so we setup solutions for the other companies. I guess we are not unique in this regards :) so I would like to know if we should create a new subscription each time or just a resource group.
Requirements:
We should be able to bill each customer/project separably
They should be able to take control of their resources easily and move to another company
Managing them should not be a headache
What we have tried
We've tried adding a subscription for each customer. This way, we could just change the admin profile and they could completely move away from us.
The billing is also OK, since we receive a different email for each subscription, but managing them is becoming a real headache.
What I guess could work
From what I read, I guess we could work with resource groups instead of subscriptions and handle the billing part with tags (haven't tried it yet. can we?) but then I'm afraid of not being able to move it to another subscription when they've asked us.
Is it even possible? How easy is that? Does it envolve contacting support?
Has anyone tried it?
I would advise against billing using resource groups and tags. The reports are a real mess and 100% unusable. Also, its a lot of extra work for nothing (seriously, do you care if you have 1 subscription or 10?) and adds no real benefit.
Also, you can move resources across subscriptions of different tenants. Best way of handling this is doing a subscription move. That way you dont have to do anything else. They just link your subscription to another tenant and you are good.
I'm talking from a perspective of administering dozens of subscriptions, and believe me, if you move away from subscriptions to resource groups (as a billing\security boundary) you will get completely devastated by the increased complexity of what you are doing.
In my experience working with organisations that provide similar hosting services to customers, I'd say resource groups is the way to go to avoid too much segregation. It's easier for you to keep control of the resources as well as keeping the cost low if you decide to use shared compute resources such as Application Gateway, DDOS protection, etc.
Bear in mind that depending on what level of permission you're giving to your clients, they might have access to information from other clients, so it's important to come up with a good security and governance plan for the Azure environment and strictly limit what they can access.
Moving things from one subscription to another is easy as long as you're using resources within the supported move list. Check the list below:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-move-resources
You don't have to open a ticket with Microsoft to move these resources and the move can be easily done through the portal interface as long as you select all the resources and it's dependencies and you have access to both subscriptions. If your client decides to move their stuff to their own Azure subscription, they will have to give you permission on that. If the resource you're trying to move is not in the supported list, not even Microsoft can move that.
From a billing perspective, I'd say separating by RG and using tags is the way to go as that can be easily filtered in your exported Azure consumption usage report.
I just read a small post of #pierreroman with the title Azure Cloud Service VS Azure Resource Manager. Unfortunately I was not able to comment his post (There was an error saving your comment. (Cannot create comment - access denied.)). Therefore and because I think this topic is interesting for many folks who are using azure cloud services right now, I write my "comment" here and send him a tweet. Maybe he or someone else can answer my questions.
I think that more interesting than the "what is the difference between cloud services and resource groups" is the question "should we plan to move from cloud services to azure resource groups?". Is it even possible? Or are we comparing apples to oranges?
What about scaling (for example adding new instances of a worker role with a simple slider or with auto scaling)?
The comparison is kind of apples(Cloud Service) and grocery(ARM) where can manage fruits, meats, and fishes.
But, very first concept of Cloud Service was similar to the ARM. That's why sometimes confusing.
Below quote is from free ebook Azure Web Apps for Developers (download), page 12 and 13.
An Azure Resource Group is a logical container for grouping Azure resources.
Grouping resources this way helps simplify the implementation, deployment, management, and monitoring of resources in the resource group. From a billing perspective, it gives you a way to view costs for the resource group rather than for individual resources, eliminating the need to figure out which resources are related. You can think of an Azure Resource Group as a unit of management.
The last line helps to understand clearly, Azure Resource Group is a unit of management.
I'm new to Azure architecture and am trying to understand why Azure Resource Groups, which are logical deployment buckets for applications built on Azure, are associated with a region when they are defined.
At first I thought it was to provide global distribution for disaster recovery or geographic redundancy, but then I realized that a single Resource Group can contain web apps in different regions, which can provide those features via Traffic Manager. I suppose that using separate Resource Groups would help better identify which resources are in which region, but other than for organizational purposes, I can't understand what the region definition for Resource Groups implies.
(edit: removed general advice query to be more focused)
The main reason for specifying the location of a resource group is to specify a location for data/metadata for the deployment to be stored in... It also makes the API consistent (think of the paths in REST API calls) but the primary reason is storage during deployment.
The location of the resources in the group is independent/not related to the location of the group itself.
When creating a resource group, you need to provide a location for that resource group. You may be wondering, "Why does a resource group need a location? And, if the resources can have different locations than the resource group, why does the resource group location matter at all?" The resource group stores metadata about the resources. Therefore, when you specify a location for the resource group, you are specifying where that metadata is stored. For compliance reasons, you may need to ensure that your data is stored in a particular region.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview
The meta data (definition) of the Azure resource group needs to be stored somewhere. hence the location. However, resources inside a resource group are location independent and can be placed in a different region\location.
Do note that dependencies between resources can exist. A Virtual machine in West Europe obviously needs a storage account in West Europe as well, but a SQL database in the same resource group can exist in West US.
Resource groups are basically for you to decide which resources in your application you want to manage together and by manage I mean you want to deploy, manage, and monitor them as a group so at a high level you do not see them as separate components.
In general in a big ecosystem, Azure resource groups are the ones that you do not see those components(resources) in them as separate entities, instead you see them as related and interdependent parts of a single entity so you put them in one Resource Group so that using the Azure Resource Group Manager tool You can deploy, update or delete all of the resources for your application in a single, coordinated operation.
You use a template for deployment and that template can work for different environments such as testing, staging and production. You can clarify billing for your organization by viewing the rolled-up costs for the entire group.
You can find more details on Azure Resource Manager here which I believe will help you better understand the idea behind the Azure Resource Groups:
https://azure.microsoft.com/en-us/documentation/articles/resource-group-overview/
Everything in Azure relates to a physical location/Datacenter, and ARM is no different. A little while ago, not every Datacenter supported ARM, so the reason for choosing made even more sense. Now, like any other Azure resource, the decision is for the user to make, often based on their required proximity to the end user and/or legal geographic requirements.
It appears there is no obvious way to determine what region a resource group is in after it has been created. This has caused me grief per the discussion here, when I ran into problems with CDN metadata conflicts. I had to start over. Now I name my resource groups with the region. e.g.: my-resourcegroup-westus