I've tried to get all subnet ids to add aws batch with terraform with following code:
data "aws_subnet_ids" "test_subnet_ids" {
vpc_id = "default"
}
data "aws_subnet" "test_subnet" {
count = "${length(data.aws_subnet_ids.test_subnet_ids.ids)}"
id = "${tolist(data.aws_subnet_ids.test_subnet_ids.ids)[count.index]}"
}
output "subnet_cidr_blocks" {
value = ["${data.aws_subnet.test_subnet.*.id}"]
}
Fortunately, it was working fine when I've tested like that. But when I tried to integrate with batch terraform like:
resource "aws_batch_compute_environment" "test-qr-processor" {
compute_environment_name = "test-qr-processor-test"
compute_resources {
instance_role = "${aws_iam_instance_profile.test-ec2-role.arn}"
instance_type = [
"optimal"
]
max_vcpus = 256
min_vcpus = 0
security_group_ids = [
"${aws_security_group.test-processor-batch.id}"
]
subnets = ["${data.aws_subnet.test_subnet.*.id}"]
type = "EC2"
}
service_role = "${aws_iam_role.test-batch-service-role.arn}"
type = "MANAGED"
depends_on = [ "aws_iam_role_policy_attachment.test-batch-service-role" ]
}
I've encountered following error message,
Error: Incorrect attribute value type
on terraform.tf line 142, in resource
"aws_batch_compute_environment" "test-processor": 142: subnets =
["${data.aws_subnet.test_subnet.*.id}"]
Inappropriate value for attribute "subnets": element 0: string
required.
Please let me know why, thanks.
"${data.aws_subnet.test_subnet.*.id}" is already string array type.
you should input value without [ ]
write code like :
subnets = "${data.aws_subnet.test_subnet.*.id}"
See :
Here's A document about Resource: aws_batch_compute_environment
Related
Need your kind help. I am stuck with the following resource creation. Using Terraform v1.0.6
I need to create appropriate subnets dynamically in two VPCs
variables.tf
vpc_resource_networks = {
pnw-01 = [
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.72.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "wb-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
],
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.80.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "web-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
]
]
}
main.tf
locals {
wlb_net = element(keys(var.vpc_resource_networks), 0)
}
resource "aws_subnet" "wlb" {
count = length(module.aws_vpc_app_resource)
vpc_id = element(module.aws_vpc_app_resource.*.vpc_id, count.index)
dynamic "subnet_group" {
for_each = var.vpc_resource_networks[local.wlb_net][count.index]
content {
dynamic "subnet" {
for_each = subnet_group.value
content {
cidr_block = subnet.subnet_ip
availability_zone = element(var.azs, subnet.index)
tags = {
Name = subnet.subnet_name
}
}
}
}
}
I intend to create subnets dynamically which is var.vpc_resource_networks.pnw01[0] should be on one vpc and other index on another VPC.
The above block returns
dynamic “subnet_group” {
Blocks of type “subnet_group” are not expected here.
Please assist
Looking at the resource definition of aws_subnet, I can see that, as the error message suggests, there's no property for a "subnet_group".
There are several different resource types that are subnet groups though for different services; such as DMS, DocumentDB, DAX, Elasticache, MemoryDB, Neptune, RDS, and Redshift. Search the term "subnet_group" on the left panel within the provider page.
Perhaps an AWS expert can comment here but I believe you're trying to do two things in one motion here.
First you should create the subnets and define their ranges, then you should create subnet groups that need access to different subnets for a particular service.
Here's some more information on subnets and subnet groups.
I am trying to access all groups and create groups in the below terraform code. But I am facing error This object does not have an attribute named "groups". Is there any logic I am missing here in the resource "og" "example"
for_each=toset(flatten(local.instances[*].groups)). Thanks
locals {
instances = {
test1 = {
baseUrl = "url1"
subDomain = "sd1"
groups = [
"app1",
"app2",
],
}
test2 = {
baseUrl = "url2"
subDomain = "sd2"
groups = [
"t1",
"t2",
],
}
}
}
resource "og" "example" {
for_each = toset(flatten(local.instances[*].groups))
name = each.value
description = "${each.value}-access"
}
Your local variable is a map, not a list. So it should be:
for_each = toset(flatten(values(local.instances)[*].groups))
How can I iterate over the JSON rendered data.aws_iam_policy_document documents within an aws_iam_policy?
data "aws_iam_policy_document" "role_1" {
statement {
sid = "CloudFront1"
actions = [
"cloudfront:ListDistributions",
"cloudfront:ListStreamingDistributions"
]
resources = ["*"]
}
}
data "aws_iam_policy_document" "role_2" {
statement {
sid = "CloudFront2"
actions = [
"cloudfront:CreateInvalidation",
"cloudfront:GetDistribution",
"cloudfront:GetInvalidation",
"cloudfront:ListInvalidations"
]
resources = ["*"]
}
}
variable "role_policy_docs" {
type = list(string)
description = "Policies associated with Role"
default = [
"data.aws_iam_policy_document.role_1.json",
"data.aws_iam_policy_document.role_2.json",
]
}
locals {
role_policy_docs = { for s in var.role_policy_docs: index(var.role_policy_docs, s) => s}
}
resource "aws_iam_policy" "role" {
for_each = local.role_policy_docs
name = format("RolePolicy-%02d", each.key)
description = "Custom Policies for Role"
policy = each.value
}
resource "aws_iam_role_policy_attachment" "role" {
for_each = { for p in aws_iam_policy.role : p.name => p.arn }
role = aws_iam_role.role.name
policy_arn = each.value
}
This example has been reduced down to the very basics. The policy documents are dynamically generated with the source_json and override_json conventions. I cannot simply combine the statements into a single policy document.
Terraform Error:
Error: "policy" contains an invalid JSON policy
on role.tf line 35, in resource "aws_iam_policy" "role":
35: policy = each.value
This:
variable "role_policy_docs" {
type = list(string)
description = "Policies associated with Role"
default = [
"data.aws_iam_policy_document.role_1.json",
"data.aws_iam_policy_document.role_2.json",
]
}
Is literally defining those default values as strings, so what you're getting is this:
+ role_policy_docs = {
+ 0 = "data.aws_iam_policy_document.role_1.json"
+ 1 = "data.aws_iam_policy_document.role_2.json"
}
If you tried removing the quotations around the data blocks, it will not be valid because you cannot use variables in default definitions. Instead, assign your policy documents to a new local, and use that local in your for loop instead:
locals {
role_policies = [
data.aws_iam_policy_document.role_1.json,
data.aws_iam_policy_document.role_2.json,
]
role_policy_docs = {
for s in local.role_policies :
index(local.role_policies, s) => s
}
}
In the following code block I'm trying to pass an array of server names to the attributes_json block:
resource "aws_instance" "consul-server" {
ami = var.consul-server
instance_type = "t2.nano"
key_name = var.aws_key_name
iam_instance_profile = "dna_inst_mgmt"
vpc_security_group_ids = [
"${aws_security_group.yutani_consul.id}",
"${aws_security_group.yutani_ssh.id}"
]
subnet_id = "${aws_subnet.public_1_subnet_us_east_1c.id}"
associate_public_ip_address = true
tags = {
Name = "consul-server${count.index}"
}
root_block_device {
volume_size = "30"
delete_on_termination = "true"
}
connection {
type = "ssh"
user = "chef"
private_key = "${file("${var.aws_key_path}")}"
timeout = "2m"
agent = false
host = self.public_ip
}
count = var.consul-server_count
provisioner "chef" {
attributes_json = <<-EOF
{
"consul": {
"servers": ["${split(",",aws_instance.consul-server[count.index].id)}"]
}
}
EOF
use_policyfile = true
policy_name = "consul_server"
policy_group = "aws_stage_enc"
node_name = "consul-server${count.index}"
server_url = var.chef_server_url
recreate_client = true
skip_install = true
user_name = var.chef_username
user_key = "${file("${var.chef_user_key}")}"
version = "14"
}
}
Running this gives me an error:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
(This is after declaring a count of 2 in a variable for var.consul-server_count)
Can anyone tell me what the proper way is to do this?
There are two issues here: (1) How to interpolate a comma-separated list in a JSON string ; and (2) What is causing the cyclic dependency error.
How to interpolate a list to make a valid JSON array
Use jsonencode
The cleanest method is to not use a heredoc at all and just use the jsonencode function.
You could do this:
locals {
arr = ["host1", "host2", "host3"]
}
output "test" {
value = jsonencode(
{
"consul" = {
"servers" = local.arr
}
})
}
And this yields as output:
Outputs:
test = {"consul":{"servers":["host1","host2","host3"]}}
Use the join function and a heredoc
The Chef provisioner's docs suggest to use a heredoc for the JSON string, so you can also do this:
locals {
arr = ["host1", "host2", "host3"]
sep = "\", \""
}
output "test" {
value = <<-EOF
{
"consul": {
"servers": ["${join(local.sep, local.arr)}"]
}
}
EOF
}
If I apply that:
Outputs:
test = {
"consul": {
"servers": ["host1", "host2", "host3"]
}
}
Some things to pay attention to here:
You are trying to join your hosts so that they become valid JSON in the context of a JSON array. You need to join them with ",", not just a comma. That's why I've defined a local variable sep = "\", \"".
You seem to be trying to split there when you apparently need join.
Cyclic dependency issue
The cause of the error message:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
Is that you have a cyclic dependency. Consider this simplified example:
resource "aws_instance" "example" {
count = 3
ami = "ami-08589eca6dcc9b39c"
instance_type = "t2.micro"
user_data = <<-EOF
hosts="${join(",", aws_instance.example[count.index].id)}"
EOF
}
Or you could use splat notation there too for the same result i.e. aws_instance.example.*.id.
Terraform plan then yields:
▶ terraform012 plan
...
Error: Cycle: aws_instance.example[2], aws_instance.example[1], aws_instance.example[0]
So you get a cycle error there because aws_instance.example.*.id depends on the aws_instance.example being created, so the resource depends on itself. In other words, you can't use a resources exported values inside the resource itself.
What to do
I don't know much about Consul, but all the same, I'm a bit confused tbh why you want the EC2 instance IDs in the servers field. Wouldn't the Consul config be expecting IP addresses or hostnames there?
In any case, you probably need to calculate the host names yourself outside of this resource, either as a static input parameter or something that you can calculate somehow. And I imagine you'll end up with something like:
variable "host_names" {
type = list
default = ["myhost1"]
}
resource "aws_instance" "consul_server" {
...
provisioner "chef" {
attributes_json = jsonencode(
{
"consul" = {
"servers" = var.host_names
}
})
}
}
I declared security group in following way:
resource "azurerm_network_security_group" "wan" {
count = "${var.enable_wan_subnet ? 1 : 0}"
provider = "azurerm.base"
name = "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}"
location = "${azurerm_resource_group.this.location}"
resource_group_name = "${azurerm_resource_group.this.name}"
tags = "${
merge(map("Name", format("%s-%s-%s",var.environment_name,"WAN-Subnets", "Security-Group")),
var.tags_global,
var.tags_module)
}"
}
and created output for that security group:
output "security_groups_id_wan" {
value = "${azurerm_network_security_group.wan.*.id}"
depends_on = [
"azurerm_subnet.wan",
]
}
In output i'm getting
Actual output
security_groups_id_wan = [
/subscriptions/111-222-333-4445/resourceGroups/default_resource_group/providers/Microsoft.Network/networkSecurityGroups/DF-DTAP-WAN-Subnet-Security-Group
]
How, from output, to remove all except resource name (DF-DTAP-WAN-Subnet-Security-Group)
Desired output:
security_groups_id_wan = [
DF-DTAP-WAN-Subnet-Security-Group
]
You can just use the Terraform functions and change the output value like this:
output "security_groups_id_wan" {
value = "${slice(split("/",azurerm_network_security_group.wan.*.id), length(split("/",azurerm_network_security_group.wan.*.id))-1, length(split("/",azurerm_network_security_group.wan.*.id)))}"
depends_on = [
"azurerm_subnet.wan",
]
}
With the functions, you can output every resource as you need. For more details, see Terraform Supported built-in functions.
Update
The test with an existing NSG through the Terraform data and the template here:
data "azurerm_network_security_group" "test" {
name = "azureUbuntu18-nsg"
resource_group_name = "charles"
}
output "substring" {
value = "${slice(split("/",data.azurerm_network_security_group.test.id), length(split("/",data.azurerm_network_security_group.test.id))-1, length(split("/",data.azurerm_network_security_group.test.id)))}"
}
The screenshot of the result here:
You built that name yourself with "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}" so why not just output that?
To save repeating yourself you could put that in a local and refer to it in both the resource and the output:
locals {
security_group_name = "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}"
}
resource "azurerm_network_security_group" "wan" {
count = "${var.enable_wan_subnet ? 1 : 0}"
provider = "azurerm.base"
name = "${local.security_group_name}"
# ...
}
output "security_groups_id_wan" {
value = "${local.security_group_name}"
}
Note that you also didn't need the depends_on because a) it's an output, it happens at the end of things anyway and b) you already have an implicit dependency on that resource because you used an interpolation that included the resource.
You can read more about Terraform dependencies via the Hashicorp Learn platform.
Addition to #Charles Xu's answer:Had to convert list to string first
output "subnets_id_wan" {
value = "${slice(split("/",join(",",azurerm_subnet.wan.*.id)), length(split("/",join(",",azurerm_subnet.wan.*.id)))-1, length(split("/",join(",",azurerm_subnet.wan.*.id))))}"
depends_on = [
"azurerm_subnet.wan",
]
}