I'm trying to pass a list of ingress rules to an aws_security_group resource. Rules are stored as a list of maps:
ingress = [
{
"from_port": 443,
"to_port": 443,
"protocol": "tcp",
"subnet_roles": [ "web" ]
},
{
"from_port": 3389,
"to_port": 3389,
"protocol": "tcp",
"subnet_roles": [ "private","management" ]
}
]
I'm passing these values through a module declaration and, while doing so, need to look up the CIDR blocks corresponding to the 'subnet_roles' variable. In my top level module declaration, I'm trying this:
ingress_rules = [ for rule in each.value.rules.ingress : merge(rule,
{ cidr_blocks = [
for role in rule.subnet_roles :
lookup(module.aws_vpc.subnets_by_role[role], "cidr_block", null)
]
}
)]
But this is resulting in the error: "Invalid value for "inputMap" parameter: lookup() requires a map as the first argument." But, subnets_by_role[role] is a map here's how it's constructed:
subnets_by_role = merge(
{ for subnet in aws_subnet.public :
subnet.tags.subnet_role => subnet...
},
{
for subnet in aws_subnet.private :
subnet.tags.subnet_role => subnet...
}
)
I'm not clear why lookup isn't accepting the map from subnets_by_role.
subnets_by_role is map(list(string), or written another way, key => list(string). Thus module.aws_vpc.subnets_by_role[role] returns a list. Am I missing something here?
Related
I have this local map variable
(some AWS ECS tasks definition configurations I read from .JSON files)
tasks = {
"service1" = {
task_definition = {
"cpu": 128,
"environment": [
{
"name": "DB_HOST",
"value": "X"
}
],
"essential": true,
"healthCheck": {}
}
}
"service2" = {
"task_definition" = ...
}
}
I want to change DB_HOST value based on other module output :
worth noting DB_HOST won’t appear on each service - should be changed/added
Something like that :
tasks.x.task_definition.environment[x].value = module.example.db_host
-> DB_HOST = module.example.db_host
didn’t manage to do it when looping through the map keys…
Thanks in advance !
I'm working on an tf plan what builds a json template and out of a map variable and I'm not quite sure how to use the existing looping, type, list functions to do the work. I know that I cannot pass lists or map to a data "template_file" so my thought was to build the string in a locals or null resource block and then pass that to the template
Variable
variable "boostrap_servers" {
type = map
default = {
"env01" : [
"k01.env01",
"k02.env01"
],
"env02" : [
"k01.env02"
]
}
Desired text
"connections": {
"env01": {
"properties": {
"bootstrap.servers": "k01.env01,k02.env01"
}
},
"env02": {
"properties": {
"bootstrap.servers": "k01.env02"
}
},
You may simply use the jsonencode function and list comprehension for this:
locals {
connections = jsonencode({
for cluster, servers in local.bootstrap_servers :
cluster => {
properties = {
"bootstrap.servers" = join(",", servers)
}
}
})
}
Ok, so the following works but there's a better question: why not just use the jsonencode function to build the json
locals {
clusters = [
for cluster, servers in var.boostrap_servers :
"{\"${cluster}\":{\"properties\":{\"bootstrap.servers\":\"${join(" ,", servers)}\"}}"]
connections = join(",", local.clusters)
}
Wondering if anyone has ran tackled it. So, I need to be able to generate list of egress CIDR blocks that is currently available for listing over an API. Sample output is the following:
[
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.1.191",
"cidr": "192.168.1.128/26",
"ip": "192.168.1.128",
"ip_range": {
"start": "192.168.1.128",
"end": "192.168.1.191"
},
"netmask": "255.255.255.192",
"network": "192.168.1.128",
"prefix": "26",
"size": "64"
}
},
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.160.127",
"cidr": "192.168.160.0/25",
"ip": "192.168.160.0",
"ip_range": {
"start": "192.168.160.0",
"end": "192.168.160.127"
},
"netmask": "255.255.255.128",
"network": "192.168.160.0",
"prefix": "25",
"size": "128"
}
}
]
So, I need convert it to Azure Firewall
###############################################################################
# Firewall Rules - Allow Access To TEST VMs
###############################################################################
resource "azurerm_firewall_network_rule_collection" "azure-firewall-azure-test-access" {
for_each = local.egress_ips
name = "azure-firewall-azure-test-rule"
azure_firewall_name = azurerm_firewall.public_to_test.name
resource_group_name = var.resource_group_name
priority = 105
action = "Allow"
rule {
name = "test-access"
source_addresses = local.egress_ips[each.key]
destination_ports = ["43043"]
destination_addresses = ["172.16.0.*"]
protocols = [ "TCP"]
}
}
So, bottom line is that allowed IP addresses have to be a list of strings for the "source_addresses" parameter, such as this:
["192.168.44.0/24","192.168.7.0/27","192.168.196.0/24","192.168.229.0/24","192.168.138.0/25",]
I configured data_sources.tf file:
data "http" "allowed_networks_v1" {
url = "https://testapiserver.com/api/allowed/networks/v1"
}
...and in locals.tf, I need to configure
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
egress_ips = ...
}
...and that's where I am stuck. How can parse that data in locals.tf file so I can reference it from within TF ?
Thanks a metric ton!!
I'm assuming that the list of string you are referring to are the objects under: metadata.cidr we can extract that with a for loop in a local, and also do a distinct just in case we get duplicates.
Here is a sample code
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/heldersepu/hs-scripts/master/json/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}
and here is the output of a plan on that:
terraform plan
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
Terraform will perform the following actions:
Plan: 0 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ data = [
+ "192.168.1.128/26",
+ "192.168.160.0/25",
]
Here is the code for your second sample:
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/akamalov/testfile/master/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json.egress_nat_ranges : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}
I am trying to fetch the tags of AMI using AWS CLI and want to reuse the values from the output.
I have a terraform code below which is returning outputs in string format(Maybe not sure of format) which I want to convert into a map object.
variable "ami" {
default = "ami-xxxx"
}
locals {
tags = {
"platform" = lookup(data.local_file.read_tags.content, "platform", "") #Expecting to get platform from Map of read_tags
}
}
data "template_file" "log_name" {
template = "${path.module}/output.log"
}
resource "null_resource" "ami_tags" {
provisioner "local-exec" {
command = "aws ec2 describe-tags --filters Name=resource-id,Values=${var.ami} --query Tags[*].[Key,Value] > ${data.template_file.log_name.rendered}"
}
}
data "local_file" "read_tags" {
filename = "${data.template_file.log_name.rendered}"
depends_on = ["null_resource.ami_tags"]
}
output "tags" {
value = local.tags
}
output "cli-output-tags" {
value = "${concat(data.local_file.read_tags.content)}"
}
output of cli-output-tags is below:
[
[
"ENV",
"DEV"
],
[
"Name",
"Base-AMI"
],
[
"platform",
"Linux"
]
]
How can I convert this output into Map as below using terraform/(jq command), or is there any other way to fetch required values directly from cli-output-tags output:
{
ENV = "DEV",
Name = "Base-AMI",
platform = "Linux"
}
I have also tried changing the CLI command a bit like below but still not able to fetch values as expected:
'Tags[].{Key:Key,Value:Value}'
Resulted below output:
[
{
"Key": "ENV",
"Value": "DEV"
},
{
"Key": "Name",
"Value": "Base-AMI"
},
{
"Key": "platform",
"Value": "Linux"
}
]
You could use zipmap:
output "cli-output-tags" {
value = zipmap(
jsondecode(data.local_file.read_tags.content)[*][0],
jsondecode(data.local_file.read_tags.content)[*][1]
)
}
The code first changes string data from your file to json, then
gets all first elements [*][0] (same for second elements [*][1]), and zips them into map.
How can I convert this output into Map as below
One way would be to use jq as follows (assuming cli-output-tags is the name of the file holding the JSON array of arrays):
jq -r -f '"{", (.[] | "\(.[0]) = \"\(.[1])\""), "}"' cli-output-tags
In terraform I am attempting to pass a variable (list) to a module that we built. This variable needs to be used within a aws_ecs_task_definition resource in the container_definitions.
Right now I am just starting with an empty default list defined as a variable:
variable "task_enviornment" {
type = "list"
default = []
}
My ECS task definition looks like this:
resource "aws_ecs_task_definition" "ecs_task_definition" {
family = "${var.ecs_family}"
network_mode = "awsvpc"
task_role_arn = "${aws_iam_role.iam_role.arn}"
execution_role_arn = "${data.aws_iam_role.iam_ecs_task_execution_role.arn}"
requires_compatibilities = ["FARGATE"]
cpu = "${var.fargate_cpu}"
memory = "${var.fargate_memory}"
container_definitions =<<DEFINITION
[
{
"cpu": ${var.fargate_cpu},
"image": "${var.app_image}",
"memory": ${var.fargate_memory},
"name": "OURNAME",
"networkMode": "awsvpc",
"environment": "${jsonencode(var.task_enviornment)}",
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group" : "${aws_cloudwatch_log_group.fargate-logs.name}",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "demo"
}
},
"portMappings": [
{
"containerPort": ${var.app_port},
"hostPort": ${var.app_port}
}
]
}
]
DEFINITION
}
The part I am having a problem with with the "environment" part:
"environment": "${jsonencode(var.task_enviornment)}",
I have tried a few different ways to get this to work.
If I do "environment": "${jsonencode(var.task_enviornment)}",
I get ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal string into Go struct field ContainerDefinition.Environment of type []*ecs.KeyValuePair
If I do "environment": "${var.task_enviornment}", or "environment": ["${var.task_enviornment}"],
I get At column 1, line 1: output of an HIL expression must be a string, or a single list (argument 8 is TypeList) in:
Then it just outputs the contents of container_definitions
I did also try adding default values and I was getting similar error messages. However I do need to be able to handle no values being sent in, so an empty list.
variable "task_enviornment" {
type = "list"
default = [
{
"name" = "BUCKET",
"value" = "test"
}
]
}
After a lot of investigation and a fresh set of eyes looking at this figured out the solution. I am unsure why this fixes it, and I feel like this is likely a bug.
Needed to do 2 things to fix this.
Remove type = "list" from the variable definition.
variable "task_environment" {
default = []
}
Remove the quotes when using the variable:
"environment": ${jsonencode(var.task_environment)},
The below solution should work
in variable.tf
variable "app_environments_vars" {
type = list(map(string))
default = []
description = "environment variable needed by the application"
default = [
{
"name" = "BUCKET",
"value" = "test"
},{
"name" = "BUCKET1",
"value" = "test1"
}]
and in your task definition, you can use ${jsonencode(var.app_environments_vars)} similar to
container_definitions =<<DEFINITION
[
{
"cpu": ${var.fargate_cpu},
"image": "${var.app_image}",
"memory": ${var.fargate_memory},
"name": "OURNAME",
"networkMode": "awsvpc",
"environment": ${jsonencode(var.app_environments_vars)},
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group" : "${aws_cloudwatch_log_group.fargate-logs.name}",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "demo"
}
},
"portMappings": [
{
"containerPort": ${var.app_port},
"hostPort": ${var.app_port}
}
]
}
]
My guess is that you are trying to use the type "map" instead of lists, as showed above, the removal from type specification will work.
Example:
List_sample = [1,2,3]
Map_sample = { key_name = "value" }
Reference: Terraform - Type Constraints