I have created terraform template (azure) with two modules. One module is for the resource group. the other is for vnet (it handles the creation of NSG and route table as well along with their association with subnets).
When I run terraform apply, it is giving an error for the route table, as the resource group is not created yet. the order of creation is showing as route table is created first and then resource group.
Is there a way to set the order of creation? in the main.tf in the root folder, module resource group is called first and then vnet.
Reconsider the idea of creating RG and resources using two modules. Ask yourself a simple question: why?
If you are 100% sure it is the right approach then use depends_on:
module "rg1" {
source = "./rg_module"
...
}
module "net1" {
source = "./network_module"
....
depends_on = [module.rg1]
}
Another alternative would be to use implicit dependence:
-Have the root module where resource group is actually defined return an output:
output "rg_name" {
value = azurerm_resource_group.root_rg.name
}
-No modifications in resource group module which calls the root module
-While creating route table(module),use the output value from resource group module:
[Assuming the variable assignment in below module is providing input to its root source using the name resource_group_name]
resource_group_name =module.rg_module["<OPTIONAL KEY IF USING FOR EACH IN RG MODULE"].rg_name
This creates an internal dependence on the resource group.
Note that it is not possible to reference arguments(actually variables) from the resource group module unless output values were defined.
You must use -out option to save the plan to a file. Like:
terraform plan -out <plan_file>
It is always recommended to use -out and save the plan file. This will ensure that the order of creation is preserved across subsequent applies.
Related
I'm not sure that there is a right answer for this and it will vary per scenario but I am curious because there isn't much documentation that I can find for a code first azure bicep infrastructure. Most examples you find show how to make a resource within a resource group, or using a module to define scope and deploy to another resource group, but what if you're trying to do more?
Let's do the following scenario: using 2 subscriptions(1 for prod, 1 for dev & qa) with 20 resource groups each containing multiple difference resources and you want to manage this within a CI/CD pipeline, plus the 3 environments: prod, qa, and dev. How would you go about this? I can think of a few scenarios but don't necessarily, but nothing sticks out as the best way to do it, maybe I'm missing something.
CI/CD portion:
Let's assume:
az account set --subscription(set our sub)
az group create --name --location (create resource group if it doesn't exist)
az deployment group create --name --resource-group --template-file --parameters(read from our files to deploy to a resource group)
You could pass an array of resource groups to loop through to create the resource group if it doesn't exist.
You could have the resource group list in a parameters file that you read from and do the same thing as above.
You could create a step for every resource group and the resources inside of it.(seems excessive?)
Bicep Portion:
Bicep restrictions: to specify scope(a resource group in our scenario) we'd have to have use modules dealing with multiple resource groups or have a step for each resource group and have a main.bicep file for the different resource groups/resources.
You could create a folder structure for each resource group and the resources inside of it with a main.bicep but that would mean you have a lot of extra deploy steps(seems excessive?).
You could have 1 main.bicep file and have a folder structure that uses a lot of modules to specify your scope while reading the resource group, resource variables etc using an environment parameters.json file.
You could create a folder for each environment, have folders with each environment then create each resource group and resources inside of it not using a parameters.json but using params in each file instead since they would be specific for each environment.
1 final issue:
Lastly let's say you want to add a step before the deployment of resources to use bicep what-if to check what resources will be updated or deleted(this is pretty important!). Last I checked there was an issue where what-if does not work for bicep modules so you wouldn't get the luxury of knowing what changes would be made prior to a deployment with the what-if. That is a pretty big safety net you'd be losing, so would you want to scratch the module strategy all together?
What would be the best way to tackle something like this while keeping it readable for average non experts to be able to hop in and work on it? I would lean towards making a folder structure using modules and reading from an environment parameters.json but I'm not convinced that's the best way, especially if what-if isn't fully working for bicep modules.
IMO this does depend a lot on the scenario, topology, permissions, etc. The way I would start thinking about this is that you want an "environment" that will vary a bit between dev/test and prod. That env has multiple resourceGroups and a dedicated subscription for each env.
In this case, I would use a single bicep "project" (e.g. main.bicep with modules) and change the deployment using parameter files (for dev/test vs. prod). The project would lay down everything needed for the environment (think greenfield). The main.bicep file is a subscription scoped deployment that will create the RGs and all the resources needed. Oversimplified example:
targetScope = 'subscription'
param sqlAdminUsername string'
param keyVaultResourceGroup
param keyVaultName string
param keyVaultSecretName string
param location string = deployment().location
resource kv 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
scope: resourceGroup(subscription().subscriptionId, keyVaultResourceGroup)
name: keyVaultName
}
resource sqlResourceGroup 'Microsoft.Resources/resourceGroups#2021-04-01' = {
name: 'shared-sql'
location: location
}
resource webResourceGroup 'Microsoft.Resources/resourceGroups#2021-04-01' = {
name: 'shared-web'
location: location
}
module sqlDeployment 'modules/shared-sql.bicep' = {
scope: resourceGroup(sqlResourceGroup.name)
name: 'sqlDeployment'
params: {
sqlAdminUsername: sqlAdminUsername
sqlAdminPassword: kv.getSecret(keyVaultSecretName)
location: location
}
}
module webDeployment 'modules/shared-web.bicep' = {
scope: resourceGroup(webResourceGroup.name)
name: 'webDeployment'
params: {
location: location
}
}
A single template + modules that creates the RGs, creates a SQL Server (via module) and an app service plan with an admin website (also via module). You can then parameterize whatever you want to for each environment.
re: what-if - what if will skip evaluation of a module if that module has a parameter that is an output of another module. If you don't pass outputs between modules then the module will be evaluated by what-if. The sample above does not pass outputs - often you don't need to do this because the information output was known by the parent (i.e. main.bicep) but sometimes you can't avoid it - ymmv.
Once you have the template designed in such a way, the pipeline is really straightforward - just deploy the template to the desired subscription.
That help?
Our terraform remote state file in Azure has been completely destroyed and we're now faced with the challenge of recreating the state file from scratch. My option is to use the Terraform import command, using the following simple syntax:
terraform import <Terraform Resource Name>.<Resource Label> <Azure Resource ID>
To import the existing resource group for example, I will create the following configuration in a main.tf file.
provider "azurerm" {
version="1.39.0"
}
# create resource group
resource "azurerm_resource_group" "rg"{
name = "rg-terraform"
location = "uksouth"
}
Now, the problem I have is as follows:
When the existing Azure resources were originally created, they were assigned names that used an extremely complex naming convention, with some characters even being randomly generated. To further compound matters, they were all unique and there are hundreds of them. All would have been rosy if they were assigned a simplistic name like "main", as is used commonly in a lot of Terraform examples, but unfortunately, that's not the case.
My question therefore, is this:
When putting together my main.tf configuration file to be used for the Import, is it an absolute requirement that my "Resource Label" (given in my Import command) has to match the original "Resource Label" name from when the resource was created?
If it is a mandatory requirement, is there any way I could retrieve the original "Resource Label" from Azure in the same way that I can for instance obtain the "Azure Resource ID" from the Azure Portal or even an Az CLI query?
How can I ensure any child resources such as Subnets are included in the Import, without having to trawl manually through the Azure Portal to identify each one of them?
No, absolutely not. Choose whatever you want.
No, Azure generally does not know about this label, it is something terraform internal.
Unfortunately you need to import each and every resource manually and separately.
Have you made absolutely sure the current state file is lost? The storage location was not versioned? Does no developer still have a local copy of the state file laying around?
I’m using terraform to spin up Aws-DMS. To spin up DMS, we need subnet groups, dms replication task, dms endpoints, dms replication instance. I’ve configured everything using terraform documentation. My question is how will terraform know which task to be completed first to spin up other dependency tasks?
Do we need to declare it somewhere in terraform or is terraform intelligent enough to run accordingly?
Terraform uses references in the configuration to infer ordering.
Consider the following example:
resource "aws_s3_bucket" "example" {
bucket = "terraform-dependencies-example"
acl = "private"
}
resource "aws_s3_bucket_object" "example" {
bucket = aws_s3_bucket.example.bucket # reference to aws_s3_bucket.example
key = "example"
content = "example"
}
In the above example, the aws_s3_bucket_object.example resource contains an expression that refers to aws_s3_bucket.example.bucket, and so Terraform can infer that aws_s3_bucket.example must be created before aws_s3_bucket_object.example.
These implicit dependencies created by references are the primary way to create ordering in Terraform. In some rare circumstances we need to represent dependencies that cannot be inferred by expressions, and so for those exceptional circumstances only we can add additional explicit dependencies using the depends_on meta argument.
One situation where that can occur is AWS IAM policies, where the graph created naturally by references will tend to have the following shape:
Due to AWS IAM's data model, we must first create a role and then assign a policy to it as a separate step, but the objects assuming that role (in this case, an AWS Lambda function just for example) only take a reference to the role itself, not to the policy. With the dependencies created implicitly by references then, the Lambda function could potentially be created before its role has the access it needs, causing errors if the function tries to take any actions before the policy is assigned.
To address this, we can use depends_on in the aws_lambda_function resource block to force that extra dependency and thus create the correct execution order:
resource "aws_iam_role" "example" {
# ...
}
resource "aws_iam_role_policy" "example" {
# ...
}
resource "aws_lambda_function" "exmaple" {
depends_on = [aws_iam_role_policy.example]
}
For more information on resource dependencies in Terraform, see Resource Dependencies in the Terraform documentation.
Terraform will automatically create the resources in an order that all dependencies can be fulfilled.
E.g.: If you set a security group id in your DMS definition as "${aws_security_group.my_sg.id}", Terraform recognizes this dependency and created the security group prior to the DMS resource.
I am new to Terraform so not even sure something like this is possible. As an example, lets say I have a template that deploys an Azure resource group and a key vault in it. And then lets say I have another template that deploys a virtual machine into the same resource group. Is it possible to do a destroy with the virtual machine template without destroying the key vault and resource group? We are trying to compartmentalize the parts of a large solution without having to put it all in a single template and we want to be able to manage each piece separately without affecting other pieces.
On a related note...we are storing state files in an Azure storage account. If we breakup our deployment into multiple compartmentalized deployments...should each deployment have its own state file or should they all use the same state file?
For larger systems it is common to split infrastructure across multiple separate configurations and apply each of them separately. This is a separate idea from (and complimentary to) using shared modules: modules allow a number of different configurations to have their own separate "copy" of a particular set of infrastructure, while the patterns described below allow an object managed by one configuration to be passed by reference to another.
If some configurations will depend on the results of other configurations, it's necessary to store these results in some data store that can be written to by its producer and read from by its consumer. In an environment where the Terraform state is stored remotely and readable broadly, the terraform_remote_state data source is a common way to get started:
data "terraform_remote_state" "resource_group" {
# The settings here should match the "backend" settings in the
# configuration that manages the network resources.
backend = "s3"
config {
bucket = "mycompany-terraform-states"
region = "us-east-1"
key = "azure-resource-group/terraform.tfstate"
}
}
resource "azurerm_virtual_machine" "example" {
resource_group_name = "${data.terraform_remote_state.resource_group.resource_group_name}"
# ... etc ...
}
The resource_group_name attribute exported by the terraform_remote_state data source in this example assumes that a value of that name was exposed by the configuration that manages the resource group using an output.
This decouples the two configurations so that they have an entirely separate lifecycle. You first terraform apply in the configuration that creates the resource group, and then terraform apply in the configuration that contains the terraform_remote_state data resource shown above. You can then apply that latter configuration as many times as you like without risk to the shared resource group or key vault.
While the terraform_remote_state data source is quick to get started with for any organization already using remote state (which is recommended), some organizations prefer to decouple configurations further by introducing an intermediate data store like Consul, which then allows data to be passed between configurations more explicitly.
To do this, the "producing" configuration (the one that manages your resource group) publishes the necessary information about what it created into Consul at a well-known location, using the consul_key_prefix resource:
resource "consul_key_prefix" "example" {
path_prefix = "shared/resource_group/"
subkeys = {
name = "${azurerm_resource_group.example.name}"
id = "${azurerm_resource_group.example.id}"
}
resource "consul_key_prefix" "example" {
path_prefix = "shared/key_vault/"
subkeys = {
name = "${azurerm_key_vault.example.name}"
id = "${azurerm_key_vault.example.id}"
uri = "${azurerm_key_vault.example.uri}"
}
}
The separate configuration(s) that use the centrally-managed resource group and key vault would then read it using the consul_keys data source:
data "consul_keys" "example" {
key {
name = "resource_group_name"
path = "shared/resource_group/name"
}
key {
name = "key_vault_name"
path = "shared/key_vault/name"
}
key {
name = "key_vault_uri"
path = "shared/key_vault/uri"
}
}
resource "azurerm_virtual_machine" "example" {
resource_group_name = "${data.consul_keys.example.var.resource_group_name}"
# ... etc ...
}
In return for the additional complexity of running another service to store these intermediate values, the two configurations now know nothing about each other apart from the agreed-upon naming scheme for keys within Consul, which gives flexibility if, for example, in future you decide to refactor these Terraform configurations so that the key vault has its own separate configuration too. Using a generic data store like Consul also potentially makes this data available to the applications themselves, e.g. via consul-template.
Consul is just one example of a data store that happens to already be well-supported in Terraform. It's also possible to achieve similar results using any other data store that Terraform can both read and write. For example, you could even store values in TXT records in a DNS zone and use the DNS provider to read, as an "outside the box" solution that avoids running an additional service.
As usual, there is a tradeoff to be made here between simplicity (with "everything in one configuration" being the simplest possible) and flexibility (with a separate configuration store), so you'll need to evaluate which of these approaches is the best fit for your situation.
As some additional context: I've documented a pattern I used successfully for a moderate-complexity system. In that case we used a mixture of Consul and DNS to create an "environment" abstraction that allowed us to deploy the same applications separately for a staging environment, production, etc. The exact technologies used are less important than the pattern, though. That approach won't apply exactly to all other situations, but hopefully there are some ideas in there to help others think about how to best make use of Terraform in their environment.
You can destroy specific resources using terraform destroy -target path.to.resource. Docs
Different parts of a large solution can be split up into modules, these modules do not even have to be part of the same codebase and can be referenced remotely. Depending on your solution you may want to break up your deployments into modules and reference them from a "master" state file that contains everything.
I am new to Terraform and trying to create some resources on Azure. To me it looks like there is some unnecessary duplication between the resource name and the attribute name in the definitions.
resource "azurerm_resource_group" "group_name" {
name = "group_name" # <-- repeated!
location = "${local.location}"
}
Is there a difference? Can I somehow set them to be the same in the spirit of this:
resource "azurerm_resource_group" "group_name" {
name = "${name}"
location = "${local.location}"
}
The two names here serve different purposes and have different scopes.
The name that appears in the block header is a local name used within a single Terraform module. It is useful when interpolating results from one resource into another, like ${azurerm_resource_group.group_name}. The remote API never sees this name; it is used only for internal references.
The name within the block is an attribute specific to the resource type itself -- azurerm_resource_group in this case. This name will be sent to the remote API and will be how the object is described within the AzureRM system itself.
In simple configurations within small organizations it is indeed possible that both of these names could match. In practice, the difference in scope between these names causes them to often vary. For example:
If there are multiple separate teams or applications sharing an AzureRM account, the name used with the API may need to be prefixed to avoid collisions with names created by other teams or applications, while the local name needs to be unique only within the module where it's defined.
In more complex usage with child modules, it's common to instantiate the same child module multiple times. In this case, the local name will be the same between all of the instances (because it's significant only within that instance) but the name used with the API will need to be adjusted for each instances so that they don't collide.
The resource name is the name you use to refer to the resource in Terraform context. The name parameter is the name given to the resource inside your provider's context. Resource don't have to have a name parameter, for example AWS Elastic IP resource doesn't have a name because AWS doesn't allow you to name them. Some of the resource like AWS Security group rule don't even translate one to one to resources you can name.