I am not able to create multiple Glue Jobs through Terraform. I am trying to do a count for jobname using count but when I try to do the same for job s3 script path its saying only string or single allowed." command.0.script_location must be a single value, not a list
"
I tried playing around count order but looks like for every count name its creating 2 paths
resource "aws_glue_job" "glue_ETL_jobs" {
count = "${length(var.jobnames)}"
count = "${length(var.script_location)}"
name ="${var.jobnames[count.index]}_glueETLjob"
role_arn = "${var.ETLjob_glue_role}"
command {
script_location = ["${var.script_location[count.index]}"]
}
default_arguments = {
"--job-language" = "${var.job_language}"
"--job-bookmark-option" = "${var.job_bookmark_option}"
"--TempDir" = "${var.tempdirectory}"
"--enable-continuous-cloudwatch-log" = "${var.cloud_watch_logging}"
"--enable-continuous-log-filter" = "${var.continuous_log_filter}"
"--max-capacity" = "${var.max-capacity}"
}
}
as give abv
name="${var.jobnames[count.index]}_glueETLjob"
script_location = ["${var.script_location[count.index]}"]
Related
locals {
dns_data = <<-CSV
RecordType,RecordName,RecordZone,RecordZoneId,RecordValue
A,something.example.com,something,zoneid,"x.x.x.x \n x.x.x.x"
CSV
dns_records = csvdecode(local.dns_data)
}
resource "aws_route53_record" "route53_entry" {
for_each = {for record in local.dns_records : record.RecordName => record}
name = each.value.RecordName
type = each.value.RecordType
zone_id = each.value.RecordZoneId
ttl = 60
records = [each.value.RecordValue]
}
I'm trying to create the r53 entries inside a zone using a CSV. How to handle the line breaks inside a cell? My A record cord can have multiple values, tried multiple ways with no luck. Thanks for any help.
I am having child module for Windows virtual machine.
Then I have root module (main.tf file), where I am using that child module
module "vm-win-resource" {
source = "./Modules/ServerWindows"
count = 2
vm-name = "vm-win-${random_string.rnd.result}" #OR "vm-win-${module.rnd-num.rnd-result}"
vm-rg = module.rg-resouce.rg-name
vm-location = module.rg-resouce.rg-location
nic-name = "vm-win-${random_string.rnd.result}-nic1" #OR "vm-win-${module.rnd-num.rnd-result}-nic1"
nic-rg = module.rg-resouce.rg-name
nic-location = module.rg-resouce.rg-location
nic-ip-subnet = "HERE IS SUBNET ID"
}
In same main.tf file, if I use random_string provider directly
resource "random_string" "rnd" {
length = 4
min_numeric = 4
special = false
lower = true
}
or if I create module, for random number and use it in module for virtual machine, result is same.
module "rnd-num" {
source = "./Modules/RandomNumber"
}
I get same name (generated number for both)
+ vm-win-name = [
+ [
+ "vm-win-6286",
+ "vm-win-6286",
],
]
So in both cases, value is generated only once.
Question is how can I generate random number for every loop in module for virtual machine?
Thank you for any help!
UPDATE
As workaround, I have placed provider to generate random number into virtual machine resource/module specification
resource "azurerm_windows_virtual_machine" "vm-resource" {
name = "${var.vm-name}-${random_string.rnd.result}"
resource_group_name = var.vm-rg
location = var.vm-location
size = var.vm-size
admin_username = var.vm-admin
admin_password = var.vm-adminpwd
network_interface_ids = [
azurerm_network_interface.nic-resource.id,
]
os_disk {
caching = "ReadWrite"
storage_account_type = var.vm-os-disk-type
}
source_image_reference {
publisher = var.vm-os-image.publisher
offer = var.vm-os-image.offer
sku = var.vm-os-image.sku
version = var.vm-os-image.version
}
tags = var.resource-tags
}
resource "random_string" "rnd" {
length = 4
min_numeric = 4
special = false
lower = true
}
it does the job but I would prefer to use it in main.tf file and not directly in resource/module specification, if it is possible.
A few words about how Terraform random_string works:
random_string generates a random string from specific characters. This string is generated once. Referencing its result attribute in multiple places will provide you the same output. Using it as random_string.rnd.result will not act as a function call, this means that it will provide the same value in every place.
The result value of a random_string will not change after consecutive applies. This is obvious, if we think about it. If it would change, the usage of random_string would be dangerous, since it would result in re-provisioning the resources which are referencing it.
If we want to have multiple different random strings, we have to define multiple random_string resources. For example:
resource "random_string" "rnd" {
count = 2
length = 4
min_numeric = 4
special = false
lower = true
}
module "vm-win-resource" {
source = "./Modules/ServerWindows"
count = 2
vm-name = "vm-win-${random_string.rnd[count.index].result}"
vm-rg = module.rg-resouce.rg-name
vm-location = module.rg-resouce.rg-location
nic-name = "vm-win-${random_string.rnd[count.index].result}-nic1"
nic-rg = module.rg-resouce.rg-name
nic-location = module.rg-resouce.rg-location
nic-ip-subnet = "HERE IS SUBNET ID"
}
Please note, we are using a count for the random_string resource as well.
Trying to put something together to get passed a limitation of the tfe plugin.
I have 200+ workspaces that I manage with a variable in Terraform Cloud that I need to update. All workspaces that I need to update start with "dev-workspace" in this case.
I have a data block with the following:
data "tfe_workspace_ids" "all" {
names = ["*"]
organization = "myorganization"
}
I can't do a wildcard search for these workspaces due to a limitation of the module. This data block returns a map of strings that include all of my workspaces:
aa = {
"dev-workspace-1" = "ws-anonymized"
"dev-workspace-2" = "ws-ws-anonymized"
"dev-workspace-3" = "ws-ws-anonymized"
"test-workspace-1" = "ws-ws-anonymized"
"prod-workspace-1" = "ws-ws-anonymized"
}
My problem is that I need to take this map of strings and filter it down to just return the ones that have "dev-workspace" in the key. I've tried something like the following:
resource "tfe_variable" "dev-workspace" {
for_each = contains(data.tfe_workspace_ids.all.ids, "dev-workspace")
key = "access_key"
value = "XXXX"
category = "terraform"
workspace_id = each.value
sensitive = true
description = "AWS IAM secret access key."
}
But it doesn't look like you can use contains in this manner with for_each:
Error: Error in function call
on main.tf line 16, in resource "tfe_variable" "dev-workspace":
16: for_each = contains(data.tfe_workspace_ids.all.ids, "dev-workspace")
|----------------
| data.tfe_workspace_ids.all.ids is map of string with 284 elements
Call to function "contains" failed: argument must be list, tuple, or set.
I'm not really sure what to do here, but have tried this several ways and can't figure it out. Thanks for any help.
If you want to filter, your resource could be (you have to change var.aa to the value of data.tfe_workspace_ids which produces the input map):
variable "aa" {
default = {
"dev-workspace-1" = "ws-anonymized"
"dev-workspace-2" = "ws-ws-anonymized"
"dev-workspace-3" = "ws-ws-anonymized"
"test-workspace-1" = "ws-ws-anonymized"
"prod-workspace-1" = "ws-ws-anonymized"
}
}
resource "tfe_variable" "dev-workspace" {
for_each = {for k, v in var.aa:
k => v if length(regexall("dev-workspace", k)) > 0}
key = "access_key"
value = "XXXX"
category = "terraform"
workspace_id = each.value
sensitive = true
description = "AWS IAM secret access key."
}
I have a case where I have to create an aws_vpc resource if the user does not provide vpc id. After that I am supposed to create resources with that VPC.
Now, I am applying conditionals while creating an aws_vpc resource. For example, only create VPC if existing_vpc is false:
count = "${var.existing_vpc ? 0 : 1}"
Next, for example, I have to create nodes in the VPC. If the existing_vpc is true, use the var.vpc_id, else use the computed VPC ID from aws_vpc resource.
But, the issue is, if existing_vpc is true, aws_vpc will not create a new resource and the ternary condition is anyways trying to check if the aws_vpc resource is being created or not. If it doesn't get created, terraform errors out.
An example of the error when using conditional operator on aws_subnet:
Resource 'aws_subnet.xyz-subnet' not found for variable 'aws_subnet.xyz-subnet.id'
The code resulting in the error is:
subnet_id = "${var.existing_vpc ? var.subnet_id : aws_subnet.xyz-subnet.id}"
If both things are dependent on each other, how can we create conditional resources and assign values to other configuration based on them?
You can access dynamically created modules and resources as follows
output "vpc_id" {
value = length(module.vpc) > 0 ? module.vpc[*].id : null
}
If count = 0, output is null
If count > 0, output is list of vpc ids
If count = 1 and you want to receive a single vpc id you can specify:
output "vpc_id" {
value = length(module.vpc) > 0 ? one(module.vpc).id : null
}
The following example shows how to optionally specify whether a resource is created (using the conditional operator), and shows how to handle returning output when a resource is not created. This happens to be done using a module, and uses an object variable's element as a flag to indicate whether the resource should be created or not.
But to specifically answer your question, you can use the conditional operator as follows:
output "module_id" {
value = var.module_config.skip == true ? null : format("%v",null_resource.null.*.id)
}
And access the output in the calling main.tf:
module "use_conditionals" {
source = "../../scratch/conditionals-modules/m2" # << Change to your directory
a = module.skipped_module.module_id # Doesn't exist, so might need to handle that.
b = module.notskipped_module.module_id
c = module.default_module.module_id
}
Full example follows. NOTE: this is using terraform v0.14.2
# root/main.tf
provider "null" {}
module "skipped_module" {
source = "../../scratch/conditionals-modules/m1" # << Change to your directory
module_config = {
skip = true # explicitly skip this module.
name = "skipped"
}
}
module "notskipped_module" {
source = "../../scratch/conditionals-modules/m1" # << Change to your directory
module_config = {
skip = false # explicitly don't skip this module.
name = "notskipped"
}
}
module "default_module" {
source = "../../scratch/conditionals-modules/m1" # << Change to your directory
# The default position is, don't skip. see m1/variables.tf
}
module "use_conditionals" {
source = "../../scratch/conditionals-modules/m2" # << Change to your directory
a = module.skipped_module.module_id
b = module.notskipped_module.module_id
c = module.default_module.module_id
}
# root/outputs.tf
output skipped_module_name_and_id {
value = module.skipped_module.module_name_and_id
}
output notskipped_module_name_and_id {
value = module.notskipped_module.module_name_and_id
}
output default_module_name_and_id {
value = module.default_module.module_name_and_id
}
the module
# m1/main.tf
resource "null_resource" "null" {
count = var.module_config.skip ? 0 : 1 # If skip == true, then don't create the resource.
provisioner "local-exec" {
command = <<EOT
#!/usr/bin/env bash
echo "null resource, var.module_config.name: ${var.module_config.name}"
EOT
}
}
# m1/variables.tf
variable "module_config" {
type = object ({
skip = bool,
name = string
})
default = {
skip = false
name = "<NAME>"
}
}
# m1/outputs.tf
output "module_name_and_id" {
value = var.module_config.skip == true ? "SKIPPED" : format(
"%s id:%v",
var.module_config.name,
null_resource.null.*.id
)
}
output "module_id" {
value = var.module_config.skip == true ? null : format("%v",null_resource.null.*.id)
}
The current answers here are helpful when you are working with more modern versions of terraform, but as noted by OP here they do not work when you are working with terraform < 0.12 (If you're like me and still dealing with these older versions, I am sorry, I feel your pain.)
See the relevant issue from the terraform project for more info on why the below is necessary with the older versions.
but to avoid link rot, I'll use the OPs example subnet_id argument using the answers in the github issue.
subnet_id = "${element(compact(concat(aws_subnet.xyz-subnet.*.id, list(var.subnet_id))),0)}"
From the inside out:
concat will join the splat output list to list(var.subnet_id) -- per the background link 'When count = 0, the "splat syntax" expands to an empty list'
compact will remove the empty item
element will return your var.subnet_id only when compact recieves the empty splat output.
My module takes a possibly-empty-list as input, and if that list is non-empty, creates some resources and returns a specific attribute that I need outside of the module, like so:
variable contexts {
type = "list"
}
resource "pagerduty_service" "p1" {
count = "${length(var.contexts)}"
name = "p1-${element(var.contexts, count.index)}"
description = "p1-${element(var.contexts, count.index)}"
auto_resolve_timeout = 14400
acknowledgement_timeout = 1800
escalation_policy = "${pagerduty_escalation_policy.p1.id}"
alert_creation = "create_alerts_and_incidents"
incident_urgency_rule {
type = "constant"
urgency = "high"
}
}
data "pagerduty_vendor" "cloudwatch" {
name = "Cloudwatch"
}
resource "pagerduty_service_integration" "p1_cloudwatch" {
count = "${length(var.contexts)}"
name = "Amazon Cloudwatch"
vendor = "${data.pagerduty_vendor.cloudwatch.id}"
service = "${element(pagerduty_service.p1.*.id, count.index)}"
}
output "integration_keys" {
value = "${pagerduty_service_integration.*.integration_keys}"
}
The trouble I am having is that when this module is run first with a non-empty list, thus creating the resources, it works fine. If I run it again, it fails with this exception:
* module.pagerduty.output.integration_keys: Resource 'pagerduty_service_integration.possibly_empty_resource_list' does not have attribute 'integration_key' for variable 'pagerduty_service_integration.possibly_empty_resource_list.*.integration_key'
I can't figure out a nice way to have this output return an empty list if the possibly_empty_resource_list is empty.
Any ideas?
EDIT:
I tried performing a ternary check on the output, but for some reason, using a list is not supported so this won't work however I hope it illustrates what I am trying to do:
"${length(var.contexts) > 0 ? pagerduty_service_integration.*.integration_keys : list()}"
Solution:
output "instance_id" {
value = "${element(concat(aws_instance.example.*.id, list("")), 0)}"
}
There's a section at the very bottom of the terraform upgrade to 0.11 guide here: https://www.terraform.io/upgrade-guides/0-11.html that shows what I use for counted resources
ex:
output "instance_id" { value = "${element(concat(aws_instance.example.*.id, list("")), 0)}" }
(moved over from a comment)