I am new to Terraform and trying to create some resources on Azure. To me it looks like there is some unnecessary duplication between the resource name and the attribute name in the definitions.
resource "azurerm_resource_group" "group_name" {
name = "group_name" # <-- repeated!
location = "${local.location}"
}
Is there a difference? Can I somehow set them to be the same in the spirit of this:
resource "azurerm_resource_group" "group_name" {
name = "${name}"
location = "${local.location}"
}
The two names here serve different purposes and have different scopes.
The name that appears in the block header is a local name used within a single Terraform module. It is useful when interpolating results from one resource into another, like ${azurerm_resource_group.group_name}. The remote API never sees this name; it is used only for internal references.
The name within the block is an attribute specific to the resource type itself -- azurerm_resource_group in this case. This name will be sent to the remote API and will be how the object is described within the AzureRM system itself.
In simple configurations within small organizations it is indeed possible that both of these names could match. In practice, the difference in scope between these names causes them to often vary. For example:
If there are multiple separate teams or applications sharing an AzureRM account, the name used with the API may need to be prefixed to avoid collisions with names created by other teams or applications, while the local name needs to be unique only within the module where it's defined.
In more complex usage with child modules, it's common to instantiate the same child module multiple times. In this case, the local name will be the same between all of the instances (because it's significant only within that instance) but the name used with the API will need to be adjusted for each instances so that they don't collide.
The resource name is the name you use to refer to the resource in Terraform context. The name parameter is the name given to the resource inside your provider's context. Resource don't have to have a name parameter, for example AWS Elastic IP resource doesn't have a name because AWS doesn't allow you to name them. Some of the resource like AWS Security group rule don't even translate one to one to resources you can name.
Related
Our terraform remote state file in Azure has been completely destroyed and we're now faced with the challenge of recreating the state file from scratch. My option is to use the Terraform import command, using the following simple syntax:
terraform import <Terraform Resource Name>.<Resource Label> <Azure Resource ID>
To import the existing resource group for example, I will create the following configuration in a main.tf file.
provider "azurerm" {
version="1.39.0"
}
# create resource group
resource "azurerm_resource_group" "rg"{
name = "rg-terraform"
location = "uksouth"
}
Now, the problem I have is as follows:
When the existing Azure resources were originally created, they were assigned names that used an extremely complex naming convention, with some characters even being randomly generated. To further compound matters, they were all unique and there are hundreds of them. All would have been rosy if they were assigned a simplistic name like "main", as is used commonly in a lot of Terraform examples, but unfortunately, that's not the case.
My question therefore, is this:
When putting together my main.tf configuration file to be used for the Import, is it an absolute requirement that my "Resource Label" (given in my Import command) has to match the original "Resource Label" name from when the resource was created?
If it is a mandatory requirement, is there any way I could retrieve the original "Resource Label" from Azure in the same way that I can for instance obtain the "Azure Resource ID" from the Azure Portal or even an Az CLI query?
How can I ensure any child resources such as Subnets are included in the Import, without having to trawl manually through the Azure Portal to identify each one of them?
No, absolutely not. Choose whatever you want.
No, Azure generally does not know about this label, it is something terraform internal.
Unfortunately you need to import each and every resource manually and separately.
Have you made absolutely sure the current state file is lost? The storage location was not versioned? Does no developer still have a local copy of the state file laying around?
Could any please answer my question : whether it is a) azurerm_resource_group b) test c) dev
resource "azurerm_resource_group" "dev"
{
name = "test"
location = "canadacentral"
}
A block resource "azurerm_resource_group" "dev" declares a resource with the address azurerm_resource_group.dev in the current module, so azurerm_resource_group.dev is how you'd refer to this object from expressions elsewhere in the same module.
The name argument inside the block is part of the data sent to the Azure provider, so it's up to the Azure provider to decide how to make use of it. In this particular case, I believe the provider will send that value as part of its requests to the Azure API to create, update, or delete this object, and so it'll become the name of the object as far as the remote system is concerned, and not the name of the object as far as Terraform is concerned.
More easy to understand:
azurerm_resource_group is the official name that the Azure developer who created the code for the provider gave it.
dev is the name you have given to the resource and is the local name. This is used to refer to this resource from elsewhere in the same Terraform module, but has no significance outside that module's scope. This is called "Resource names". (https://developer.hashicorp.com/terraform/language/resources/syntax#resource-syntax)
test is the Name which should be used for this Resource Group. (https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/resource_group#name). But I think it is easier to see it in another resource. In the resource aws_ami, the name is the Region-unique name for the AMI.(https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ami#name)
So to summarize the answer is dev. Since even if you put it on, terraform uses it to reference.
I have created terraform template (azure) with two modules. One module is for the resource group. the other is for vnet (it handles the creation of NSG and route table as well along with their association with subnets).
When I run terraform apply, it is giving an error for the route table, as the resource group is not created yet. the order of creation is showing as route table is created first and then resource group.
Is there a way to set the order of creation? in the main.tf in the root folder, module resource group is called first and then vnet.
Reconsider the idea of creating RG and resources using two modules. Ask yourself a simple question: why?
If you are 100% sure it is the right approach then use depends_on:
module "rg1" {
source = "./rg_module"
...
}
module "net1" {
source = "./network_module"
....
depends_on = [module.rg1]
}
Another alternative would be to use implicit dependence:
-Have the root module where resource group is actually defined return an output:
output "rg_name" {
value = azurerm_resource_group.root_rg.name
}
-No modifications in resource group module which calls the root module
-While creating route table(module),use the output value from resource group module:
[Assuming the variable assignment in below module is providing input to its root source using the name resource_group_name]
resource_group_name =module.rg_module["<OPTIONAL KEY IF USING FOR EACH IN RG MODULE"].rg_name
This creates an internal dependence on the resource group.
Note that it is not possible to reference arguments(actually variables) from the resource group module unless output values were defined.
You must use -out option to save the plan to a file. Like:
terraform plan -out <plan_file>
It is always recommended to use -out and save the plan file. This will ensure that the order of creation is preserved across subsequent applies.
I’m using terraform to spin up Aws-DMS. To spin up DMS, we need subnet groups, dms replication task, dms endpoints, dms replication instance. I’ve configured everything using terraform documentation. My question is how will terraform know which task to be completed first to spin up other dependency tasks?
Do we need to declare it somewhere in terraform or is terraform intelligent enough to run accordingly?
Terraform uses references in the configuration to infer ordering.
Consider the following example:
resource "aws_s3_bucket" "example" {
bucket = "terraform-dependencies-example"
acl = "private"
}
resource "aws_s3_bucket_object" "example" {
bucket = aws_s3_bucket.example.bucket # reference to aws_s3_bucket.example
key = "example"
content = "example"
}
In the above example, the aws_s3_bucket_object.example resource contains an expression that refers to aws_s3_bucket.example.bucket, and so Terraform can infer that aws_s3_bucket.example must be created before aws_s3_bucket_object.example.
These implicit dependencies created by references are the primary way to create ordering in Terraform. In some rare circumstances we need to represent dependencies that cannot be inferred by expressions, and so for those exceptional circumstances only we can add additional explicit dependencies using the depends_on meta argument.
One situation where that can occur is AWS IAM policies, where the graph created naturally by references will tend to have the following shape:
Due to AWS IAM's data model, we must first create a role and then assign a policy to it as a separate step, but the objects assuming that role (in this case, an AWS Lambda function just for example) only take a reference to the role itself, not to the policy. With the dependencies created implicitly by references then, the Lambda function could potentially be created before its role has the access it needs, causing errors if the function tries to take any actions before the policy is assigned.
To address this, we can use depends_on in the aws_lambda_function resource block to force that extra dependency and thus create the correct execution order:
resource "aws_iam_role" "example" {
# ...
}
resource "aws_iam_role_policy" "example" {
# ...
}
resource "aws_lambda_function" "exmaple" {
depends_on = [aws_iam_role_policy.example]
}
For more information on resource dependencies in Terraform, see Resource Dependencies in the Terraform documentation.
Terraform will automatically create the resources in an order that all dependencies can be fulfilled.
E.g.: If you set a security group id in your DMS definition as "${aws_security_group.my_sg.id}", Terraform recognizes this dependency and created the security group prior to the DMS resource.
I am new to Terraform so not even sure something like this is possible. As an example, lets say I have a template that deploys an Azure resource group and a key vault in it. And then lets say I have another template that deploys a virtual machine into the same resource group. Is it possible to do a destroy with the virtual machine template without destroying the key vault and resource group? We are trying to compartmentalize the parts of a large solution without having to put it all in a single template and we want to be able to manage each piece separately without affecting other pieces.
On a related note...we are storing state files in an Azure storage account. If we breakup our deployment into multiple compartmentalized deployments...should each deployment have its own state file or should they all use the same state file?
For larger systems it is common to split infrastructure across multiple separate configurations and apply each of them separately. This is a separate idea from (and complimentary to) using shared modules: modules allow a number of different configurations to have their own separate "copy" of a particular set of infrastructure, while the patterns described below allow an object managed by one configuration to be passed by reference to another.
If some configurations will depend on the results of other configurations, it's necessary to store these results in some data store that can be written to by its producer and read from by its consumer. In an environment where the Terraform state is stored remotely and readable broadly, the terraform_remote_state data source is a common way to get started:
data "terraform_remote_state" "resource_group" {
# The settings here should match the "backend" settings in the
# configuration that manages the network resources.
backend = "s3"
config {
bucket = "mycompany-terraform-states"
region = "us-east-1"
key = "azure-resource-group/terraform.tfstate"
}
}
resource "azurerm_virtual_machine" "example" {
resource_group_name = "${data.terraform_remote_state.resource_group.resource_group_name}"
# ... etc ...
}
The resource_group_name attribute exported by the terraform_remote_state data source in this example assumes that a value of that name was exposed by the configuration that manages the resource group using an output.
This decouples the two configurations so that they have an entirely separate lifecycle. You first terraform apply in the configuration that creates the resource group, and then terraform apply in the configuration that contains the terraform_remote_state data resource shown above. You can then apply that latter configuration as many times as you like without risk to the shared resource group or key vault.
While the terraform_remote_state data source is quick to get started with for any organization already using remote state (which is recommended), some organizations prefer to decouple configurations further by introducing an intermediate data store like Consul, which then allows data to be passed between configurations more explicitly.
To do this, the "producing" configuration (the one that manages your resource group) publishes the necessary information about what it created into Consul at a well-known location, using the consul_key_prefix resource:
resource "consul_key_prefix" "example" {
path_prefix = "shared/resource_group/"
subkeys = {
name = "${azurerm_resource_group.example.name}"
id = "${azurerm_resource_group.example.id}"
}
resource "consul_key_prefix" "example" {
path_prefix = "shared/key_vault/"
subkeys = {
name = "${azurerm_key_vault.example.name}"
id = "${azurerm_key_vault.example.id}"
uri = "${azurerm_key_vault.example.uri}"
}
}
The separate configuration(s) that use the centrally-managed resource group and key vault would then read it using the consul_keys data source:
data "consul_keys" "example" {
key {
name = "resource_group_name"
path = "shared/resource_group/name"
}
key {
name = "key_vault_name"
path = "shared/key_vault/name"
}
key {
name = "key_vault_uri"
path = "shared/key_vault/uri"
}
}
resource "azurerm_virtual_machine" "example" {
resource_group_name = "${data.consul_keys.example.var.resource_group_name}"
# ... etc ...
}
In return for the additional complexity of running another service to store these intermediate values, the two configurations now know nothing about each other apart from the agreed-upon naming scheme for keys within Consul, which gives flexibility if, for example, in future you decide to refactor these Terraform configurations so that the key vault has its own separate configuration too. Using a generic data store like Consul also potentially makes this data available to the applications themselves, e.g. via consul-template.
Consul is just one example of a data store that happens to already be well-supported in Terraform. It's also possible to achieve similar results using any other data store that Terraform can both read and write. For example, you could even store values in TXT records in a DNS zone and use the DNS provider to read, as an "outside the box" solution that avoids running an additional service.
As usual, there is a tradeoff to be made here between simplicity (with "everything in one configuration" being the simplest possible) and flexibility (with a separate configuration store), so you'll need to evaluate which of these approaches is the best fit for your situation.
As some additional context: I've documented a pattern I used successfully for a moderate-complexity system. In that case we used a mixture of Consul and DNS to create an "environment" abstraction that allowed us to deploy the same applications separately for a staging environment, production, etc. The exact technologies used are less important than the pattern, though. That approach won't apply exactly to all other situations, but hopefully there are some ideas in there to help others think about how to best make use of Terraform in their environment.
You can destroy specific resources using terraform destroy -target path.to.resource. Docs
Different parts of a large solution can be split up into modules, these modules do not even have to be part of the same codebase and can be referenced remotely. Depending on your solution you may want to break up your deployments into modules and reference them from a "master" state file that contains everything.