kubectl_manifest keeps changing the yaml_body when applying terraform apply.
For this use case, I can not use kubernetes_manifest.
Do you have any comments on the cause and suggestions for resolving this issue?
# module.mymodule.kubectl_manifest.someresource will be updated in-place
~ resource "kubectl_manifest" "someresource " {
id = ""
name = ""
~ yaml_body = (sensitive value)
# (14 unchanged attributes hidden)
}
Plan: 0 to add, 33 to change, 0 to destroy.
Terraform v1.3.1
kubectl = {
source = "gavinbunney/kubectl"
version = ">= 1.14.0"
}
Related
When using azurecaf to generate multiple names like in the following code, how do I use the results output?
resource "azurecaf_name" "names" {
name = var.appname
resource_type = "azurerm_resource_group"
resource_types = ["azurerm_mssql_database"]
prefixes = [var.environment]
suffixes = [var.resource_group_location_short]
random_length = 5
clean_input = false
}
results - The generated name for the Azure resources based in the resource_types list
How to use this? Also, can I somehow debug / print out what results looks like? (I don't know if it is an array, a key-value structure etc)
You can view the results in two common ways. It is applicable to all attributes of the resource.
[1] Exporting the attribute required as a terraform output.
When you add any attribute as an output in your code by default terraform will show you the values with terraform apply.
In your used case.
output "caf_name_result" {
value = azurecaf_name.names.result
}
output "caf_name_results" {
value = azurecaf_name.names.results
}
Apply the config with the above outputs definitions you will have the below output on your terminal.
Changes to Outputs:
+ caf_name_result = (known after apply)
+ caf_name_results = (known after apply)
azurecaf_name.names: Creating...
azurecaf_name.names: Creation complete after 0s [id=YXp1cmVybV9yZXNvdXJjZV9ncm91cAlkZXYtcmctc3RhY2tvdmVyZmxvdy15b2RncC13ZXUKYXp1cmVybV9tc3NxbF9kYXRhYmFzZQlkZXYtc3FsZGItc3RhY2tvdmVyZmxvdy15b2RncC13ZXU=]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
caf_name_result = "dev-rg-stackoverflow-yodgp-weu"
caf_name_results = tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
After the first successful terraform apply you can view them anytime when you want by using the terraform output command.
$ terraform output
caf_name_result = "dev-rg-stackoverflow-yodgp-weu"
caf_name_results = tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
You can be very specific also to check only particular output value.
$ terraform output caf_name_results
tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
[2] View your applied resources via Terraform State Commands
This is only available after the resources are applied and only in cases when terraform execution was done from the same machine where this command is running. (in simple the identity doing terraform execution satisfies all the authentication, authorization and network connectivity conditions. )
It is not recommended, just to share another option available when requiring a quick look on the resources applied.
$ terraform state list
azurecaf_name.names
$ terraform state show azurecaf_name.names
# azurecaf_name.names:
resource "azurecaf_name" "names" {
clean_input = false
id = "YXp1cmVybV9yZXNvdXJjZV9ncm91cAlkZXYtcmctc3RhY2tvdmVyZmxvdy15b2RncC13ZXUKYXp1cmVybV9tc3NxbF9kYXRhYmFzZQlkZXYtc3FsZGItc3RhY2tvdmVyZmxvdy15b2RncC13ZXU="
name = "stackoverflow"
passthrough = false
prefixes = [
"dev",
]
random_length = 5
random_seed = 1676730686950185
random_string = "yodgp"
resource_type = "azurerm_resource_group"
resource_types = [
"azurerm_mssql_database",
]
result = "dev-rg-stackoverflow-yodgp-weu"
results = {
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
}
separator = "-"
suffixes = [
"weu",
]
use_slug = true
}
[1]
[2]
I want to exempt certain policies for an Azure VM. I have the following terraform code to exempt the policies.
It uses locals to identify the scope on which policies should be exempt.
locals {
exemption_scope = try({
mg = length(regexall("(\\/managementGroups\\/)", var.scope)) > 0 ? 1 : 0,
sub = length(split("/", var.scope)) == 3 ? 1 : 0,
rg = length(regexall("(\\/managementGroups\\/)", var.scope)) < 1 ? length(split("/", var.scope)) == 5 ? 1 : 0 : 0,
resource = length(split("/", var.scope)) >= 6 ? 1 : 0,
})
expires_on = var.expires_on != null ? "${var.expires_on}T23:00:00Z" : null
metadata = var.metadata != null ? jsonencode(var.metadata) : null
# generate reference Ids when unknown, assumes the set was created with the initiative module
policy_definition_reference_ids = length(var.member_definition_names) > 0 ? [for name in var.member_definition_names :
replace(substr(title(replace(name, "/-|_|\\s/", " ")), 0, 64), "/\\s/", "")
] : var.policy_definition_reference_ids
exemption_id = try(
azurerm_management_group_policy_exemption.management_group_exemption[0].id,
azurerm_subscription_policy_exemption.subscription_exemption[0].id,
azurerm_resource_group_policy_exemption.resource_group_exemption[0].id,
azurerm_resource_policy_exemption.resource_exemption[0].id,
"")
}
and the above local is used like mentioned below
resource "azurerm_management_group_policy_exemption" "management_group_exemption" {
count = local.exemption_scope.mg
name = var.name
display_name = var.display_name
description = var.description
management_group_id = var.scope
policy_assignment_id = var.policy_assignment_id
exemption_category = var.exemption_category
expires_on = local.expires_on
policy_definition_reference_ids = local.policy_definition_reference_ids
metadata = local.metadata
}
Both the locals and azurerm_management_group_policy_exemption are part of the same module file. And Policy exemption is applied like mentioned below
module exemption_jumpbox_sql_vulnerability_assessment {
count = var.enable_jumpbox == true ? 1 : 0
source = "../policy_exemption"
name = "Exemption - SQL servers on machines should have vulnerability"
display_name = "Exemption - SQL servers on machines should have vulnerability"
description = "Not required for Jumpbox"
scope = module.create_jumbox_vm[0].virtual_machine_id
policy_assignment_id = module.security_center.azurerm_subscription_policy_assignment_id
policy_definition_reference_ids = var.exemption_policy_definition_ids
exemption_category = "Waiver"
depends_on = [module.create_jumbox_vm,module.security_center]
}
It works for an existing Azure VM. However it throws the following error while trying to provision the Azure VM and apply the policy exemption on this Azure VM.
Ideally, module.exemption_jumpbox_sql_vulnerability_assessment should get executed only after [module.create_jumbox_vm as it is defined as a dependent. But not sure why it is throwing the error
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
I tried to reproduce the scenario in my environment.
resource "azurerm_management_group_policy_exemption" "management_group_exemption" {
count = local.exemption_scope.mg
name = var.name
display_name = var.display_name
description = var.description
management_group_id = var.scope
policy_assignment_id = var.policy_assignment_id
exemption_category = var.exemption_category
expires_on = local.expires_on
policy_definition_reference_ids = local.policy_definition_reference_ids
metadata = local.metadata
}
locals {
exemption_scope = try({
...
})
Received the same error:
The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
Referring to local values , the values will be known on the apply time only, and not during the apply time .So if it is not dependent on other sources , it will expmpt policies but it is dependent on the VM which may be still in process of creation.
So target only the resource that is dependent on first ,as only when vm is created is when the exemption policy can be assigned to that vm.
Check count:using-expressions-in-count | Terraform | HashiCorp Developer
Also note that while using terraform count argument with Azure Virtual Machines ,NIC resource also to be created for each Virtual Machine resource.
resource "azurerm_network_interface" "nic" {
count = var.vm_count
name = "${var.vm_name_pfx}-${count.index}-nic"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
//tags = var.tags
ip_configuration {
name = "internal"
subnet_id = azurerm_subnet.internal.id
private_ip_address_allocation = "Dynamic"
}
}
Reference: terraform-azurerm-policy-exemptions/examples/count at main · AnsumanBal-MT/terraform-azurerm-policy-exemptions · GitHub
I have the following resource definition for additional node pools in my k8s cluster:
resource "azurerm_kubernetes_cluster_node_pool" "extra" {
for_each = var.node_pools
kubernetes_cluster_id = azurerm_kubernetes_cluster.k8s.id
name = each.key
vm_size = each.value["vm_size"]
node_count = each.value["count"]
node_labels = each.value["labels"]
vnet_subnet_id = var.subnet.id
}
Here is the output from terraform plan:
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# module.aks.azurerm_kubernetes_cluster_node_pool.extra["general"] has been changed
~ resource "azurerm_kubernetes_cluster_node_pool" "extra" {
+ availability_zones = []
id = "/subscriptions/3913c9fe-c571-4af9-bc9a-533202d41061/resourcegroups/amic-resources/providers/Microsoft.ContainerService/managedClusters/amic-k8s-01/agentPools/general"
name = "general"
+ node_taints = []
+ tags = {}
# (18 unchanged attributes hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
# module.aks.azurerm_kubernetes_cluster_node_pool.extra["general"] must be replaced
-/+ resource "azurerm_kubernetes_cluster_node_pool" "extra" {
- availability_zones = [] -> null
- enable_auto_scaling = false -> null
- enable_host_encryption = false -> null
- enable_node_public_ip = false -> null
~ id = "/subscriptions/3913c9fe-c571-4af9-bc9a-533202d41061/resourcegroups/amic-resources/providers/Microsoft.ContainerService/managedClusters/amic-k8s-01/agentPools/general" -> (known after apply)
~ kubernetes_cluster_id = "/subscriptions/3913c9fe-c571-4af9-bc9a-533202d41061/resourcegroups/amic-resources/providers/Microsoft.ContainerService/managedClusters/amic-k8s-01" -> "/subscriptions/3913c9fe-c571-4af9-bc9a-533202d41061/resourceGroups/amic-resources/providers/Microsoft.ContainerService/managedClusters/amic-k8s-01" # forces replacement
- max_count = 0 -> null
~ max_pods = 30 -> (known after apply)
- min_count = 0 -> null
name = "general"
- node_taints = [] -> null
~ orchestrator_version = "1.20.7" -> (known after apply)
~ os_disk_size_gb = 128 -> (known after apply)
- tags = {} -> null
# (9 unchanged attributes hidden)
}
Plan: 1 to add, 0 to change, 1 to destroy.
As you can see, terraform tries to force replacement of my node pool because of a change in kubernetes_cluster_id, even though there is actually no change at all on this value.
I've been able to work around this by ignoring kubernetes_cluster_id changes in the lifecycle block, but I am still puzzled as to why terraform detects a change there.
So why does Terraform find a change in this case while there are none?
I'm not proud, but I managed to work my way around this bug by using the "replace" terraform string function.
resource "azurerm_kubernetes_cluster_node_pool" "extra" {
[...]
# Use this once the bug gets fixed in the provider, and delete the workaround.
# kubernetes_cluster_id = azurerm_kubernetes_cluster.k8s.id
kubernetes_cluster_id = replace(azurerm_kubernetes_cluster.k8s.id, "resourceGroups", "resourcegroups")
[...]
}
NOTE: I'm not replacing /resourceGroups/ with /resourcegroups/ because the replace function will default to consider this is a regex replacement, which might end up duplicating your forward slashes. (I didn't test this myself)
I have fixed this weird bug by introducing lifecycle block as follows:
resource "azurerm_kubernetes_cluster_node_pool" "my-node-pool" {
name = "mynodepool"
kubernetes_cluster_id = azurerm_kubernetes_cluster.aks.id
...
lifecycle {
ignore_changes = [
kubernetes_cluster_id
]
}
}
Not the cleanest way, but it works. Cluster Id should not be changed unless you recreate whole AKS Cluster, so it should be safe.
I have a code snippet that sets the node pool name for a GKE cluster, it is very much unreadable. I would appreciate any help in making it more presentable and easy to understand what is happening.
output "test" {
value = regex("(?:[a-z](?:[-a-z0-9]{0,38}[a-z0-9])?)", lower(var.node_pool_version != "" ? var.node_pool_name != "" ? "${var.node_pool_name}-v${replace("${var.node_pool_version}",".","-")}" : "${var.name_prefix}-v${replace("${var.node_pool_version}",".","-")}" : var.node_pool_name != "" ? var.node_pool_name : "${var.name_prefix}-standard"))
}
variable "node_pool_version" {
description = "Override node_version for cluster upgrades"
type = string
default = ""
}
variable "node_pool_name" {
type = string
default = ""
}
variable "name_prefix" {
type = string
default = "develop"
}
Outputs:
❯ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
test = develop-standard
❯ terraform apply -var node_pool_version=1.16.15-gke.7800
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
test = develop-v1-16-15-gke-7800
❯ terraform apply -var node_pool_version=1.16.15-gke.7800 -var node_pool_name=develop-standard
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
test = develop-standard-v1-16-15-gke-7800
Edit:
My concern is with the extensive usage of String Functions & Conditional Expressions to generate the output. My use-case is if the user doesn't pass either node_pool_version or node_pool_name, my output should still manage to generate a node-name that can be easily identifiable. Is there a better way to re-write this code and make it more readable for anyone new to Terraform?
I suppose the best you could do is use a locals block:
variable "node_pool_version" {
description = "Override node_version for cluster upgrades"
type = string
default = ""
}
variable "node_pool_name" {
type = string
default = ""
}
variable "name_prefix" {
type = string
default = "develop"
}
locals {
version = replace("${var.node_pool_version}", ".", "-")
prefix = var.node_pool_name != "" ? var.node_pool_name : var.name_prefix
id = var.node_pool_name != "" ? local.prefix : "${local.prefix}-standard"
versioned_id = local.version != "" ? "${local.prefix}-v${local.version}" : local.id
clean_versioned_id = regex("(?:[a-z](?:[-a-z0-9]{0,38}[a-z0-9])?)", lower(local.versioned_id))
}
output "test" {
value = local.clean_versioned_id
}
I am trying to create a Security Group using terraform module terraform-aws-modules/security-group/aws. This would need VPC id which is taken from aws_vpcs data source. The VPC id requires a string value, but the aws_vpcs data source returns a list with a single value.
Please find
data "aws_vpcs" "this" {
tags = {
"Name" = "example"
}
}
module "route53_sg" {
source = "terraform-aws-modules/security-group/aws"
name = "R53_health_checkers"
description = "Security group for Route53 health checkers"
vpc_id = element([data.aws_vpcs.this.ids], 0)
ingress_cidr_blocks = [
...
...
...
]
ingress_rules = ["https-443-tcp"]
}
$ terraform apply
data.aws_lb.ext_alb: Refreshing state...
data.aws_vpcs.this: Refreshing state...
Error: Invalid value for module argument
on main.tf line 75, in module "route53_sg":
75: vpc_id = element([data.aws_vpcs.this.ids], 0)
The given value is not suitable for child module variable "vpc_id" defined at
.terraform/modules/route53_sg/terraform-aws-modules-terraform-aws-security-group-d55e4de/variables.tf:10,1-18:
string required.
vpc_id is expecting a Single string. FOLLOWING is a result from Output.tf
$ terraform apply
data.aws_lb.ext_alb: Refreshing state...
data.aws_vpcs.this: Refreshing state...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
vpc = [
"vpc-08067a598522a7b30",
]
data.aws_vpcs.this.ids is already a list, you don't need to put it into another list.
Try:
vpc_id = element(data.aws_vpcs.this.ids, 0)
EDIT: Answering questions from the comment:
It seems like the ids returned is a set instead of a list, as mentioned in a similar issue here:
https://github.com/terraform-providers/terraform-provider-aws/issues/7522
If you are using 0.12.x:
You can do
vpc_id = element(tolist(data.aws_vpcs.this.ids), 0)
If you are using 0.11.x: You can do
vpc_id = element(split(",", join(",", data.aws_vpcs.this.ids))), 0)