Reuse the configuration to create similar resources in Terraform - terraform

I have the below TF file, which will create a function - FirstFunction. This works perfectly.
resource "azurerm_function_app" "**firstfunction**" {
name = **var.firstfunctionname**
location = azurerm_resource_group.resourcegroupX.location
resource_group_name = azurerm_resource_group.resourcegroupX.name
app_service_plan_id = azurerm_app_service_plan.appserviceplan.id
https_only = "true"
client_affinity_enabled = "true"
app_settings = {
NS = azurerm_eventhub_namespace.eventhubns.name
Hub = azurerm_eventhub.**firsteventhub**.name
propertyX = "**firstproperty**"
LogRef = "${azurerm_storage_account.store.primary_blob_endpoint}${azurerm_storage_container.**firstlogs**.name}"
}
}
resource "azurerm_app_service_virtual_network_swift_connection" "**firstvnet**" {
app_service_id = azurerm_function_app.**firstfunction**.id
subnet_id = azurerm_subnet.snet.id
}
In the file, see the section enclosed with ****, which need to be changed to create SecondFunction, ThirdFunction and so on ...
The way i have right now is to create multiple TF files , with the same code copied and change the sections enclosed in **.
I read through the module system but understood the limitation with the module system is that I cannot refer to the other components created in the same TF root module as shown below
For e.g In the TF file, I refer to location as
location = azurerm_resource_group.resourcegroupX.location
If I do it as a module, the location should be refered to as
location = var.location_name where location_name should be defined as a variable. I cannot refer to components created with the same root module.
Can you please suggest a solution where I can create multiple components based on the similar code ? Please note that, in the above example, Im creating 2 resources in a single TF file and both of them are related.

The simplest way is that use the count property in your resources. It can help you create multiple same resources in the same code.
resource "azurerm_function_app" "myfunction" {
count = number_your_need # how many resources you want to create
name = "${var.firstfunctionname}-${count.index}"
location = azurerm_resource_group.resourcegroupX.location
resource_group_name = azurerm_resource_group.resourcegroupX.name
app_service_plan_id = azurerm_app_service_plan.appserviceplan.id
https_only = "true"
client_affinity_enabled = "true"
app_settings = {
NS = azurerm_eventhub_namespace.eventhubns.name
Hub = azurerm_eventhub.**firsteventhub**.name
propertyX = "**firstproperty**"
LogRef = "${azurerm_storage_account.store.primary_blob_endpoint}${azurerm_storage_container.**firstlogs**.name}"
}
}
resource "azurerm_app_service_virtual_network_swift_connection" "**firstvnet**" {
count = number # how many you need to create
app_service_id = element(azurerm_function_app.myfunction[*].id, count.index)
subnet_id = azurerm_subnet.snet.id
}
The same solution for the azurerm_eventhub that you need to create. For example:
resource "azurerm_eventhub" "myeventhub" {
count = number # how many you need to create
name = "${var.eventhub_name}-${count.index}"
...
}
Then you can refer it in the function app like this:
app_settings = {
NS = azurerm_eventhub_namespace.eventhubns.name
Hub = element(azurerm_eventhub.myeventhub[*].name, count.index)
propertyX = "**firstproperty**"
LogRef = "${azurerm_storage_account.store.primary_blob_endpoint}${azurerm_storage_container.**firstlogs**.name}"
}
So does all the section enclosed with ****.

Related

Terraform - azurerm_frontdoor_custom_https_configuration - 'the given key does not identify an element in this collection value'

this code has worked before, all I'm trying to do is add new frontend endpoints, routing rules, backend pools
I've tried only sharing the code snippets that I think are relevant but let me know if there's some key info you need missing
This one has stumped me for a couple days now and no matter what I've tried I cannot seem to make sense of the error. Its like its indexing out of the variable or searching for something that isn't there but there are something like 6 already there and now I'm adding another.
I'm worried that this front door code has not been ran in awhile and something has gotten screwed up in state. Especially given all the alerts on the accompanying TF docs for this resource - https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/frontdoor_custom_https_configuration
Its been quite awhile but the AzureRM version has gone through several updates - possibly from previous to 2.58 to now past 2.58. I guess I also don't know how to verify/look at the state file and ensure its correct - even looking at the 2.58 upgrade notes its just confusing.
Ideas?
The error
on ..\modules\frontdoor\main.tf line 129, in resource "azurerm_frontdoor_custom_https_configuration" "https_config":
129: frontend_endpoint_id = azurerm_frontdoor.main.frontend_endpoints[each.value]
|----------------
| azurerm_frontdoor.main.frontend_endpoints is map of string with 8 elements
| each.value is "www-sell-dev-contoso-com"
The given key does not identify an element in this collection value.
main.tf
provider "azurerm" {
features {}
}
terraform {
backend "azurerm" {
}
}
#the outputs.tf on this module output things like the frontdoor_endpoints
#the outputs.tf with main.tf also output similar values
module "coreInfraFrontDoor" {
source = "../modules/frontdoor"
resource_group_name = module.coreInfraResourceGroup.resource_group_name
frontdoor_name = "fd-infra-${terraform.workspace}-001"
enforce_backend_pools_certificate_name_check = lookup(var.enforce_backend_pools_certificate_name_check, terraform.workspace)
log_analytics_workspace_id = module.coreInfraLogAnalytics.log_analytics_workspace_id
tags = local.common_tags
health_probes = lookup(var.health_probes, terraform.workspace)
routing_rules = lookup(var.routing_rules, terraform.workspace)
backend_pools = lookup(var.backend_pools, terraform.workspace)
frontend_endpoints = lookup(var.frontend_endpoints, terraform.workspace)
prestage_frontend_endpoints = lookup(var.prestage_frontend_endpoints, terraform.workspace)
frontdoor_firewall_policy_name = "fdfwp${terraform.workspace}001"
frontdoor_firewall_prestage_policy_name = "fdfwp${terraform.workspace}prestage"
mode = lookup(var.mode, terraform.workspace)
ip_whitelist_enable = lookup(var.ip_whitelist_enable, terraform.workspace)
ip_whitelist = lookup(var.ip_whitelist, terraform.workspace)
key_vault_id = module.coreInfraKeyVault.id
}
module main.tf
resource "azurerm_frontdoor" "main" {
name = var.frontdoor_name
location = "global"
resource_group_name = var.resource_group_name
enforce_backend_pools_certificate_name_check = var.enforce_backend_pools_certificate_name_check
tags = var.tags
dynamic "routing_rule {#stuff is here obv}
dynamic "backend_pool {#also here}
#i think this is because there was an issue/needs to be some default value for the first endpoint?
frontend_endpoint {
name = var.frontdoor_name
host_name = "${var.frontdoor_name}.azurefd.net"
web_application_firewall_policy_link_id = azurerm_frontdoor_firewall_policy.main.id
}
#now the dynamic ones from vars
dynamic "frontend_endpoint" {
for_each = var.frontend_endpoints
content {
name = frontend_endpoint.value.name
host_name = frontend_endpoint.value.host_name
session_affinity_enabled = lookup(frontend_endpoint.value, "session_affinity_enabled", false)
web_application_firewall_policy_link_id = azurerm_frontdoor_firewall_policy.main.id
}
}
versions.tf
terraform {
required_version = "~> 0.14.7"
required_providers {
azurerm = "~>2.72.0"
}
}
variables.tf
variable "frontend_endpoints" {
type = map(any)
description = "List of frontend (custom) endpoints. This is in addition to the <frontend_name>.azurefd.net endpoint that this module creates by default."
default = {
dev = [
{
name = "dev-search-contoso-com"
host_name = "dev.search.contoso.com"
},
{
name = "dev-cool-contoso-com"
host_name = "dev.cool.contoso.com"
},
########################
#this is new below
########################
{
name = "dev-sell-contoso-com"
host_name = "dev.sell.contoso.com"
}
]
prod = [ #you get the idea ]
}

Getting Outputs of Resources from Reused Modules

I have the following code that will call a module and make target groups for me based off of the information I pass it in the locals variables. This works just fine, my issue being trying to get the arn of each target group it creates in an output.
locals {
targetgroups_beta = {
targetgroup1 = {
name = "example",
path = "/",
environment = "Beta"
}
targetgroup2 = {
name = "example2",
path = "/",
environment = "Beta"
}
}
}
module "target-groups"{
for_each = local.targetgroups_beta
source = ".//modules/targetgroups"
name-beta = each.value.name
path-beta = each.value.path
environment-beta = each.value.environment
vpc-id = "${module.vpc.vpc_id}"
}
The resource name in the module it is calling is target-group, so based off of what I read I should be able to refer to it by something like this:
output{
value = "${aws_lb_target_group.target-group[0].arn}"
}
When I try this I receive the following when running a terraform plan:
"Because aws_lb_target_group.target-group does not have "count" or "for_each" set, references to it must not include an index key. Remove the bracketed index to refer to the single instance of this resource."
My understanding of this is the module that the for_each is calling isn't running a for_each, so I cannot reference the resources in this way. If I do ""${aws_lb_target_group.target-group.arn}" that reference works technically, but includes the arn for every target group and I plan on adding a lot more. Is there a way to take each of these arns out of this list as its own output?
Code in the module that it is calling for reference:
resource "aws_lb_target_group" "target-group" {
name = "example-${var.name-beta}"
port = 80
protocol = "HTTP"
vpc_id = var.vpc-id
deregistration_delay = 5
tags = {
Environment = "${var.environment-beta}"
}
health_check{
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 10
interval = 15
path = var.path-beta
}
}
If I correctly understand, you are using for_each in your target-groups module. If so to get the outputs, you would have to use something like the following in your main.tf file:
module.target-groups[*].arn
The for_each will create multiple modules, not multiple resources in a single module.
Here is good info on using for_each and count with modules in terraform 0.13.
Update for one module
If you want to use only one module, you can do the following:
module "target-groups"{
target_groups_to_create = local.targetgroups_beta
source = ".//modules/targetgroups"
name-beta = each.value.name
path-beta = each.value.path
environment-beta = each.value.environment
vpc-id = "${module.vpc.vpc_id}"
}
Then in the module:
variable "target_groups_to_create" {}
resource "aws_lb_target_group" "target-group" {
for_each = var.target_groups_to_create
name = "example-${each.value.name}"
port = 80
protocol = "HTTP"
vpc_id = var.vpc-id
deregistration_delay = 5
tags = {
Environment = "${each.value.environment}"
}
health_check{
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 10
interval = 15
path = each.value.path
}
}

Terraform- How to avoid destroy and create with single state file

I have a terraform code that creates a stream analytics job, An input and output for the job too.
Below is my terraform code:
provider "azurerm" {
version = "=1.44"
}
resource "azurerm_stream_analytics_job" "test_saj" {
name = "test-stj"
resource_group_name = "myrgname"
location = "Southeast Asia"
compatibility_level = "1.1"
data_locale = "en-US"
events_late_arrival_max_delay_in_seconds = 60
events_out_of_order_max_delay_in_seconds = 50
events_out_of_order_policy = "Adjust"
output_error_policy = "Drop"
streaming_units = 3
tags = {
environment = "test"
}
transformation_query = var.query
}
resource "azurerm_stream_analytics_output_blob" "mpl_saj_op_jk_blob" {
name = var.saj_jk_blob_output_name
stream_analytics_job_name = "test-stj"
resource_group_name = "myrgname"
storage_account_name = "mystaname"
storage_account_key = "mystakey"
storage_container_name = "testupload"
path_pattern = myfolder/{day}"
date_format = "yyyy-MM-dd"
time_format = "HH"
depends_on = [azurerm_stream_analytics_job.test_saj]
serialization {
type = "Json"
encoding = "UTF8"
format = "LineSeparated"
}
}
resource "azurerm_stream_analytics_stream_input_eventhub" "mpl_saj_ip_eh" {
name = var.saj_joker_event_hub_name
stream_analytics_job_name = "test-stj"
resource_group_name = "myrgname"
eventhub_name = "myehname"
eventhub_consumer_group_name = "myehcgname"
servicebus_namespace = "myehnamespacename"
shared_access_policy_name = "RootManageSharedAccessKey"
shared_access_policy_key = "ehnamespacekey"
serialization {
type = "Json"
encoding = "UTF8"
}
depends_on = [azurerm_stream_analytics_job.test_saj]
}
Following is my tfvars input file:
query=<<EOT
myqueryhere
EOT
saj_jk_blob_output_name="outputtoblob01"
saj_joker_event_hub_name="inputventhub01"
I have no problem with the creation. Now my problem is when I want to create a new input and output for the same stream analytics job, I changed the name values alone in the tfvars file and gave terraform apply (in the same directory where first apply was given. Same state file).
Terraform is replacing the existing i/p and o/p with the new ones which is not my requirement. I want both the old one and the new one. This usecase was satisfied when imported the existing stream analytics using terraform import in a completely different folder and I used the same code. But is there way to do this without terraform import. Can this be done with a single state file itself?
State allows Terraform to know what Azure resources to add, update, or delete. What you want to do can not be done with a single state file itself unless you directly deploy resources with different names in your configuration files.
For example, if you want to create two virtual networks. You can directly create resources like this or use a count parameter on resources level for the loop.
resource "azurerm_virtual_network" "example" {
name = "examplevnet1"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
address_space = ["10.1.0.0/16"]
}
resource "azurerm_virtual_network" "example" {
name = "examplevnet2"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
address_space = ["10.2.0.0/16"]
}
When working with Terraform in a team, you can use remote state to write the state data to a remote data store, which can then be shared between all members of a team. It's recommended to store Terraform state in Azure Storage.
For more information, you could see Terraform workflow in this blog.

How do I make terraform skip that block while creating multiple resources in loop from a CSV file?

Hi I am trying to create a Terraform script which will take inputs from the user in the form of a CSV file and create multiple Azure resources.
For example if the user wants to create: ResourceGroup>Vnet>Subnet in bulk, he will provide input in CSV format as below:
resourcegroup,RG_location,RG_tag,domainname,DNS_Zone_tag,virtualnetwork,VNET_location,addressspace
csvrg1,eastus2,Terraform RG,test.sd,Terraform RG,csvvnet1,eastus2,10.0.0.0/16,Terraform VNET,subnet1,10.0.0.0/24
csvrg2,westus,Terraform RG2,test2.sd,Terraform RG2,csvvnet2,westus,172.0.0.0/8,Terraform VNET2,subnet1,171.0.0.0/24
I have written the following working main.tf file:
# Configure the Microsoft Azure Provider
provider "azurerm" {
version = "=1.43.0"
subscription_id = var.subscription
tenant_id = var.tenant
client_id = var.client
client_secret = var.secret
}
#Decoding the csv file
locals {
vmcsv = csvdecode(file("${path.module}/computelanding.csv"))
}
# Create a resource group if it doesn’t exist
resource "azurerm_resource_group" "myterraformgroup" {
count = length(local.vmcsv)
name = local.vmcsv[count.index].resourcegroup
location = local.vmcsv[count.index].RG_location
tags = {
environment = local.vmcsv[count.index].RG_tag
}
}
# Create a DNS Zone
resource "azurerm_dns_zone" "dnsp-private" {
count = 1
name = local.vmcsv[count.index].domainname
resource_group_name = local.vmcsv[count.index].resourcegroup
depends_on = [azurerm_resource_group.myterraformgroup]
tags = {
environment = local.vmcsv[count.index].DNS_Zone_tag
}
}
To be continued....
The issue I am facing here what is in the second resource group, the user don't want a resource type, suppose the user want to skip the DNS zone in the resource group csvrg2. How do I make terraform skip that block ?
Edit: What I am trying to achieve is "based on some condition in the CSV file, not to create azurerm_dns_zone resource for the resource group csvrg2"
I have provided an example of the CSV file, how it may look like below:
resourcegroup,RG_location,RG_tag,DNS_required,domainname,DNS_Zone_tag,virtualnetwork,VNET_location,addressspace
csvrg1,eastus2,Terraform RG,1,test.sd,Terraform RG,csvvnet1,eastus2,10.0.0.0/16,Terraform VNET,subnet1,10.0.0.0/24
csvrg2,westus,Terraform RG2,0,test2.sd,Terraform RG2,csvvnet2,westus,172.0.0.0/8,Terraform VNET2,subnet1,171.0.0.0/24
you had already the right thought in your mind using the depends_on function. Although, you're using a count inside, which causes from my understanding, that once the first resource[0] is created, Terraform sees the dependency as solved and goes ahead as well.
I found this post with a workaround which you might be able to try:
https://github.com/hashicorp/terraform/issues/15285#issuecomment-447971852
That basically tells us to create a null_resource like in that example:
variable "instance_count" {
default = 0
}
resource "null_resource" "a" {
count = var.instance_count
}
resource "null_resource" "b" {
depends_on = [null_resource.a]
}
In your example, it might look like this:
# Create a resource group if it doesn’t exist
resource "azurerm_resource_group" "myterraformgroup" {
count = length(local.vmcsv)
name = local.vmcsv[count.index].resourcegroup
location = local.vmcsv[count.index].RG_location
tags = {
environment = local.vmcsv[count.index].RG_tag
}
}
# Create a DNS Zone
resource "azurerm_dns_zone" "dnsp-private" {
count = 1
name = local.vmcsv[count.index].domainname
resource_group_name = local.vmcsv[count.index].resourcegroup
depends_on = null_resource.example
tags = {
environment = local.vmcsv[count.index].DNS_Zone_tag
}
}
resource "null_resource" "example" {
...
depends_on = [azurerm_resource_group.myterraformgroup[length(local.vmcsv)]]
}
or depending on your Terraform version (0.12+ which you're using guessing your syntax)
# Create a resource group if it doesn’t exist
resource "azurerm_resource_group" "myterraformgroup" {
count = length(local.vmcsv)
name = local.vmcsv[count.index].resourcegroup
location = local.vmcsv[count.index].RG_location
tags = {
environment = local.vmcsv[count.index].RG_tag
}
}
# Create a DNS Zone
resource "azurerm_dns_zone" "dnsp-private" {
count = 1
name = local.vmcsv[count.index].domainname
resource_group_name = local.vmcsv[count.index].resourcegroup
depends_on = [azurerm_resource_group.myterraformgroup[length(local.vmcsv)]]
tags = {
environment = local.vmcsv[count.index].DNS_Zone_tag
}
}
I hope that helps.
Greetings

Error: azurerm_app_service.ci_rg: resource repeated multiple times

I am trying to deploy 2 different Azure Apps in the same resource group.
These Azure Apps are defined as docker images stored in an Azure Container Registry (where I previously pushed those docker images).
I am not able to deploy both of them at the same time because I think there is something wrong in the way I am defining them as Terraform is expecting to find only one azurerm_app_service, but I am not sure how I can work around this?
When I run this command: terraform plan -var-file test.tfvars, then I see this message in the output:
Error: azurerm_app_service.ci_rg: resource repeated multiple times
How do I define "2 different resources of the same type"?
This is the content of the main.tf file (where I inject the variables defined in variables.tf with the values defined in test.tfvars):
// the resource group definition
resource "azurerm_resource_group" "ci_rg" {
name = "${var.resource_group_name}"
location = "${var.azure_location}"
}
// the app service plan definition
resource "azurerm_app_service_plan" "ci_rg" {
name = "${var.app_service_plan}"
location = "${azurerm_resource_group.ci_rg.location}"
resource_group_name = "${azurerm_resource_group.ci_rg.name}"
kind = "Linux"
sku {
tier = "Standard"
size = "S1"
capacity = 2 // for both the docker containers
}
properties {
reserved = true
}
}
// the first azure app
resource "azurerm_app_service" "ci_rg" {
name = "${var.first_app_name}"
location = "${azurerm_resource_group.ci_rg.location}"
resource_group_name = "${azurerm_resource_group.ci_rg.name}"
app_service_plan_id = "${azurerm_app_service_plan.ci_rg.id}"
site_config {
linux_fx_version = "DOCKER|${var.first_app_docker_image_name}"
}
app_settings {
"CONF_ENV" = "${var.conf_env}"
"DOCKER_REGISTRY_SERVER_URL" = "${var.docker_registry_url}",
"DOCKER_REGISTRY_SERVER_USERNAME" = "${var.docker_registry_username}",
"DOCKER_REGISTRY_SERVER_PASSWORD" = "${var.docker_registry_password}",
}
}
// the second azure app
resource "azurerm_app_service" "ci_rg" {
name = "${var.second_app_name}"
location = "${azurerm_resource_group.ci_rg.location}"
resource_group_name = "${azurerm_resource_group.ci_rg.name}"
app_service_plan_id = "${azurerm_app_service_plan.ci_rg.id}"
site_config {
linux_fx_version = "DOCKER|${var.second_app_docker_image_name}"
}
app_settings {
"CONF_ENV" = "${var.conf_env}"
"DOCKER_REGISTRY_SERVER_URL" = "${var.docker_registry_url}",
"DOCKER_REGISTRY_SERVER_USERNAME" = "${var.docker_registry_username}",
"DOCKER_REGISTRY_SERVER_PASSWORD" = "${var.docker_registry_password}",
}
}
Edit:
I am not entirely sure about how this Terraform thing works, but I think the label azurerm_app_service is taken by the "syntax of Terraform". See the docs here: https://www.terraform.io/docs/providers/azurerm/r/app_service.html
where the title is azurerm_app_service. So I don't think I can change that.
My guess would be you need to rename the second one to something else. Like this: resource "azurerm_app_service" "ci_rg_second". It obviously doesnt like the fact that it has the same name.

Resources