Managing many azure subscriptions with terraform - azure

In our organiztion we have a team which manages central azure services like vpn gateway, firewall, bastion etc. It should also provision subscriptions for our software development teams, which involves managing users and groups, creating a vnet and peering it with the hub etc. The development teams manage all other relevant resources in their subscriptions.
I couldn't find an efficient way to build the IaC around the subscription management process. It seems to me, that you have to run terraform for each subscription separately, since you have to provide a subscription id in the terraform azure provider. This seems a bit complicated to me, I would rather define all subscriptions in a single file and let terraform manage them in a single run, like this:
subscriptions = {
"my-subscription-1" = {
vnet_address_space = ["10.0.4.0/27"],
snet_address_prefixes = ["10.0.4.0/27"],
users = [
"abc#example.com",
"def#example.com",
],
groups = [
"MyAD-Group",
]
},
"my-subscription-2" = {
vnet_address_space = ["10.0.4.32/27"],
snet_address_prefixes = ["10.0.4.32/27"],
users = [
"efg#example.com",
"hij#example.com",
],
groups = [
"AnotherAD-Group",
]
}
}
I know that you can define multiple providers in terraform and assign alias names, but this only works until you have 5-6 subscriptions. In my case I need to manage 50 subscriptions.
Did I miss something? How do you manage your subscriptions?

You must set Multiple Provider Configurations
provider "azurerm" {
subscription_id = "SUBSCRIPTION 1 ID"
features {}
}
provider "azurerm" {
alias = "subscription2"
subscription_id = "SUBSCRIPTION 2 ID"
features {}
}
Then, when you will run a module, you must set the alias, for example:
# Create a Resource Group in subscription 1
resource "azurerm_resource_group" "rg1" {
name = "RG-subscription-1"
location = "eastus"
}
# Create a Resource Group in subscription 2
resource "azurerm_resource_group" "rg2" {
provider = azurerm.subscription2 # this is the alias previously defined
name = "RG-subscription-2"
location = "eastus"
}
Hope this helps!

Related

Azure Terraform Build Generic Components using Infrastructure as Code

I am new to Terraform and Azure. I am trying to build a Resource Group / Resources using Terraform. Below is the design for the same.
I have written Terraform code to build Log Analytics workspace and Automation account.
Now below are my questions :
Cost Mgmt / Azure Monitor / Network Watcher / Defender for Cloud ? Can I build all these using Terraform code in this resource group or they need to manually built from Azure portal. When we create any resource on the left hand side options like Cost estimator / management are already available. Does that mean they can be easily selected from there on usage and no need to build from Terraform code ?
How does we apply Role Entitlement / Policy Assignment from Terraform code ?
Here is my code what I have written to build Automation account / Log Analytics
terraform {
required_version = ">=0.12"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~>2.0"
}
}
}
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "management" {
# Mandatory resource attributes
name = "k8s-log-analytics-test"
location = "eastus"
}
resource "random_id" "workspace" {
keepers = {
# Generate a new id each time we switch to a new resource group
group_name = azurerm_resource_group.management.name
}
byte_length = 8
}
resource "azurerm_log_analytics_workspace" "management" {
# Mandatory resource attributes
name = "k8s-workspace-${random_id.workspace.hex}"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
# Optional resource attributes
retention_in_days = 30
sku = "PerGB2018"
}
resource "azurerm_log_analytics_solution" "management" {
# Mandatory resource attributes
solution_name = "mgmyloganalytsolution"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
workspace_resource_id = azurerm_log_analytics_workspace.management.id
workspace_name = azurerm_log_analytics_workspace.management.name
plan {
publisher = "Microsoft"
product = "OMSGallery/ContainerInsights"
}
}
resource "azurerm_automation_account" "management" {
# Mandatory resource attributes
name = "mgmtautomationaccount"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
sku_name = "Basic"
}
resource "azurerm_log_analytics_linked_service" "management" {
# Mandatory resource attributes
resource_group_name = azurerm_resource_group.management.name
workspace_id = azurerm_log_analytics_workspace.management.id
read_access_id = azurerm_automation_account.management.id
}
Cost Mgmt / Azure Monitor / Network Watcher / Defender for Cloud ? Can
I build all these using Terraform code in this resource group or they
need to manually built from Azure portal. When we create any resource
on the left hand side options like Cost estimator / management are
already available. Does that mean they can be easily selected from
there on usage and no need to build from Terraform code ?
Yes , you can create Network Watcher , Azure Monitor resources & Cost Management using terraform resource blocks as azurerm_network_watcher , azurerm_network_watcher_flow_log ,azurerm_monitor_metric_alert ... , azurerm_resource_group_cost_management_export, azurerm_consumption_budget_resource_group etc. Defender for Cloud can't be built from terraform . Yes you are correct , cost management ,monitoring etc are also available on portal but there is a need for its resources to be created like budget alert etc. for simplification it has been added as a blade in portal.
How does we apply Role Entitlement / Policy Assignment from Terraform
code ?
You can use azurerm_role_assignment to assign built-in roles and use azurerm_role_definition to create a custom role and then assign it . For Policy assignment you can use this azurerm_resource_policy_assignment and remediate using azurerm_policy_insights_remediation.
For all the azure resource block you can refer the Official Registry Documentation of Terraform AzureRM Provider & Terraform AzureAD Provider.

Bootstraping an Azure service account in Terraform

I am trying to write the Terraform to create an Azure "service account" and am getting quite confused by the distinction between what Azure AD calls "Applications" and "Service Principals". Effectively, I'm trying to mimic the following Azure CLI call:
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/${subscription_id}"
The idea would be for a human administrator to run the Terraform, to set it up once, then those credentials could later be used to authenticate for the remaining IaC. (i.e., It's a bootstrapping exercise.)
I wish to do it in Terraform, rather than a Bash script, as it seems more explicit and fits with the rest of my IaC. This is what I have so far:
data "azurerm_subscription" "current" {}
data "azuread_client_config" "current" {}
resource "azuread_application" "terraform" {
display_name = "Terraform"
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_password" "terraform" {
application_object_id = azuread_application.terraform.object_id
}
# resource "azuread_service_principal" "terraform" {
# application_id = azuread_application.terraform.application_id
# owners = [data.azuread_client_config.current.object_id]
# }
#
# resource "azuread_service_principal_password" "terraform" {
# service_principal_id = azuread_service_principal.terraform.object_id
# }
resource "local_file" "azurerc" {
filename = ".azurerc"
file_permission = "0600"
content = <<EOF
export ARM_ENVIRONMENT="public"
export ARM_SUBSCRIPTION_ID="${data.azurerm_subscription.current.subscription_id}"
export ARM_TENANT_ID="${data.azuread_client_config.current.tenant_id}"
export ARM_CLIENT_ID="${azuread_application.terraform.application_id}"
export ARM_CLIENT_SECRET="${azuread_application_password.terraform.value}"
EOF
}
This runs, but later authenticating with the generated credentials gives an authentication error. Specifically, Terraforms says:
If you are accessing as application please make sure service principal is properly created in the tenant.
Clearly I haven't done that -- it's commented out in the above snippet -- but that's because this is where my understanding starts to break down. Why do I need both? Why do both the application and the service principal have password resources? If I generate passwords for both, which is the ARM_CLIENT_SECRET (I think the application password is the right one)? Then there's the role assignment: I see there's an azuread_app_role_assignment resource, but I'm having trouble unpicking it.
I am trying to write the Terraform to create an Azure "service
account" and am getting quite confused by the distinction between what
Azure AD calls "Applications" and "Service Principals".
Applications can be seen from Azure AD App registrations blade Where as Service Principals are other wise know as Enterprise Applications. The difference is well documented in this Microsoft Documentation.
This runs, but later authenticating with the generated credentials
gives an authentication error. Specifically, Terraforms says:
If you are accessing as application please make sure service principal
is properly created in the tenant.
This is because you have no associated service principal to that azure ad application which you have created from Terraform. The association is needed to access the application or authenticating to the azure environment with contributor role. When a App registration is created from portal it by default creates an association of AD app and Service principal , which by default results in creating a service principal for that app registration. It also applies the same concept when we use az ad sp create-for-rbac.
Effectively, I'm trying to mimic the following Azure CLI call:
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/${subscription_id}"
You can use the below to mimic the above command :
provider "azurerm" {
features{}
}
provider "azuread" {}
data "azurerm_subscription" "current" {}
data "azuread_client_config" "current" {}
resource "azuread_application" "terraform" {
display_name = "Ansumantest"
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_password" "terraform" {
application_object_id = azuread_application.terraform.object_id
}
resource "azuread_service_principal" "terraform" {
application_id = azuread_application.terraform.application_id
owners = [data.azuread_client_config.current.object_id]
}
resource "azurerm_role_assignment" "example" {
scope = data.azurerm_subscription.current.id
role_definition_name = "Contributor"
principal_id = azuread_service_principal.terraform.object_id
}
resource "local_file" "azurerc" {
filename = ".azurerc"
file_permission = "0600"
content = <<EOF
export ARM_ENVIRONMENT="public"
export ARM_SUBSCRIPTION_ID="${data.azurerm_subscription.current.subscription_id}"
export ARM_TENANT_ID="${data.azuread_client_config.current.tenant_id}"
export ARM_CLIENT_ID="${azuread_application.terraform.application_id}"
export ARM_CLIENT_SECRET="${azuread_application_password.terraform.value}"
EOF
}
Output :
Using the above details I created a reosurce group in the subscription :
provider "azurerm" {
features{}
subscription_id = "88073b30-cadd-459e-b90b-8442c93573ae"
tenant_id = "ab078f81-xxxx-xxxx-xxxx-620b694ded30"
client_id = "c022ec46-xxxx-xxxx-xxxx-c72a9b82f429"
client_secret = "wdV7Q~8Grxxxxxxxxxxxxxx~SCwbRrKIq9"
}
resource "azurerm_resource_group" "name" {
name = "testansterraform"
location = "west us 2"
}

AWS - Set a global budget for a project?

I am working on a Terraform project to create, update or delete AWS sandboxs for every new employee in my company. However, we would like to have a fix budget for every sandbox and if possible, not being able to exceed that budget. I read about AWS Budget Actions but I am not sure what will be the best way to integrate it with my current code.
main.tf
## this is creating a new project and I would like to set a budget of $50 for this account
resource "aws_organizations_account" "account" {
name = "sandbox"
email = "new-user+sandbox#company.com"
role_name = "myOrganizationRole"
}
You need to create aws_budgets_budget and aws_budgets_budget_action resources in the new account. To create resources in the new account, you need to create a provider for that account by assuming the administration role that was created:
provider "aws" {
alias = "sandbox"
region = "us-east-1"
...
// Specify your credentials source, same as you use for the provider you used for the aws_organizations_account resource
...
assume_role {
role_arn = "arn:aws:iam::${aws_organizations_account.account.id}:role/${aws_organizations_account.account.role_name}"
}
}
And now you can use that provider to create the budget resources in the new sandbox account (note the provider input parameters):
resource "aws_budgets_budget" "example" {
provider = aws.sandbox
name = "example"
budget_type = "USAGE"
limit_amount = "10.0"
limit_unit = "dollars"
time_period_start = "2006-01-02_15:04"
time_unit = "MONTHLY"
}
resource "aws_budgets_budget_action" "example" {
provider = aws.sandbox
budget_name = aws_budgets_budget.example.name
...
}

Adding role assignments to multiple Azure subscriptions for a managed identity using terraform

I have an Azure function app that is hosted in subscription "sub-test1" and I want to add role assignment to give the managed system identity(for app) access to the subscription "sub-test1"(current) and I have been able to do it via the following:
data "azurerm_subscription" "current" {}
data "azurerm_role_definition" "owner" {
name = "Owner"
}
resource "azurerm_role_assignment" "custom_role_assignment" {
name = "${var.random_guid}"
scope = data.azurerm_subscription.current.id
role_definition_id = "${data.azurerm_subscription.current.id}${data.azurerm_role_definition.owner.id}"
principal_id = azurerm_function_app.app.identity.0.principal_id
}
But I need to give this app access to multiple subscriptions(dynamic number) inside the tenant, say "sub-test2","sub-test3","sub-test4",etc. What is the best way I can do it using terraform only? Also, can this be done using only one "azurerm_role_assignment" resource block as shown above or do I need multiple such blocks respective to each subscription?
For this requirement, you need to have enough permission to create the role assignment for in the subscriptions. The simplest way is that you need to have the Owner role of all the subscriptions. Then you can change the code like this to achieve what you want:
data "azurerm_subscriptions" "example" {}
data "azurerm_role_definition" "example" {
name = "Owner"
}
resource "azurerm_role_assignment" "custom_role_assignment" {
count = length(data.azurerm_subscriptions.example.subscriptions.*.subscription_id)
name = "${var.random_guid}"
scope = "/subscriptions/${element(data.azurerm_subscriptions.example.subscriptions.*.subscription_id, count.index)}"
role_definition_id = "/subscriptions/${element(data.azurerm_subscriptions.example.subscriptions.*.subscription_id, count.index)}${data.azurerm_role_definition.example.id}"
principal_id = azurerm_function_app.app.identity.0.principal_id
}
Here is something different. Use the azurerm_subscriptions instead of azurerm_subscription to get all the subscriptions. But it only gets the GUID of the subscriptions. So we need to complete the resource Id of the subscriptions ourselves. Also for the role definition.

How can I use Terraform to create a service principal and use that principal in a provider?

I have read the write-ups online but they dont seem to cover this topic completely and was hoping someone who has done it may have some direction for me.
We are setting up a complicated Terraform template to satisfy our IaC requirements relating to our SaaS offering. In doing so we want the template to use the user's credentials at launch to create a new service principal in Azure AD (This part I have no problem doing). Then in the next portion of the template we are using that service principal as the provider. Problem is that it throws errors in the plan/apply because the service principal doesnt exist (aka the id is non existent due to the service provider section not running yet).
So is there a way that I can do this? Create a service principal and then us it in a provider alias that uses that service principal without splitting this into multiple templates?
In the end I want this template to create the service provider using the local user's permissions or MSI, give it RBAC to a subscription, then use that service provider to create assets in that subscription.
main.ts (root)
provider "azurerm" {
alias = "ActiveDirectory"
subscription_id = "${var.subscriptionNucleus}"
}
provider "azurerm" {
alias = "Infrastructure"
subscription_id = "${var.subscriptionInfrastructure}"
}
module "activedirectory" {
providers = { azurerm = "azurerm.ActiveDirectory"
}
source = "./modules/activedirectory"
subscription_id_infrastructure = "${var.subscriptionInfrastructure}"
}
module "infrastructure" {
providers = { azurerm = "azurerm.Infrastructure"}
source = "./modules/infrastructure"
location = "${var.location}"
application_id =
"${module.activedirectory.service_principal_application_id}"
subscription_id = "${var.subscriptionInfrastructure}"
prefix = "${var.prefix}"
}
main.ts (./modules/infrastructure)
data "azurerm_azuread_service_principal" "serviceprincipal" {
application_id = "${var.application_id}"
}
provider "azurerm" {
alias = "InfrastructureSP"
subscription_id = "${var.subscription_id}"
client_id = "${var.application_id}"
client_secret = "secret"
tenant_id =
"${data.azurerm_client_config.clientconfig.tenant_id}"
}
For Azure Service Principal, there are two ways to use the service principal.
First: If you already have a service principal and want to use it in the Terraform. You can make use of the Terraform Data and the test like this:
data "azurerm_azuread_service_principal" "sp" {
application_id = "21f3e1de-54e2-4951-9743-c280ad7bd74a"
}
output "test" {
value = "${data.azurerm_azuread_service_principal.sp.id}"
}
The screenshot of the result is here:
Second: You don't have the service principal and you can just create a service principal in the Terraform like this:
resource "azurerm_azuread_service_principal" "test" {
application_id = "${azurerm_azuread_application.test.application_id}"
}
resource "azurerm_azuread_service_principal_password" "test" {
service_principal_id = "${azurerm_azuread_service_principal.test.id}"
value = "your pasword"
end_date = "2020-01-01T01:02:03Z"
}
Then, no matter which way you choose, there is an important step you should do for most resources. The step is that you need to create the role to give the permission and then assign it to the resource which needs. You can do that like this:
resource "azurerm_role_assignment" "test" {
scope = "yourScope" # the resource id
role_definition_name = "the Role In need" # such as "Contributor"
principal_id = "your service principal id"
}
Hope this will help you.
There is currently no working "depends_on" that works with modules that is not a hack (null_reference). This means that if you are breaking your template into modules(separating concerns) your order of operation is required to be correct to complete this successfully as one module will not know that the data source of service provider has to wait on the previous module to complete. I have had to break this into 2 separate templates where the first creates the service principal and the second has the modular separation that can then use a data source of azurerm_azuread_service_principal.
Once Hashicorp can implement the module depends_on, this will become easier.

Resources