I am quite new to terraform. I have a provider that will accept JSON as input.
I have stored JSON config files in my project folder, eg:
{
"id": 58187729,
"name": "My dashboard",
"tags": ["mytag1", "mytag2"]
}
and load them into the resource with the below code:
resource "datadog_monitor_json" "monitor_json" {
for_each = fileset(path.module, "/monitors/*.json")
monitor = file("${path.module}/${each.key}")
}
Is there anyway I can easily append to the "tags" key or will the entire JSON need parsing somehow?
Thanks.
Here is an example on how to add extra tags:
locals {
example = jsondecode(file("${path.module}/myfile.json"))
with_extra_tags = merge(local.example,
{tags = concat(
local.example["tags"],["mytag4", "mytag3"])})
}
gives:
test = {
"id" = 58187729
"name" = "My dashboard"
"tags" = [
"mytag1",
"mytag2",
"mytag4",
"mytag3",
]
}
Related
I'm working on an tf plan what builds a json template and out of a map variable and I'm not quite sure how to use the existing looping, type, list functions to do the work. I know that I cannot pass lists or map to a data "template_file" so my thought was to build the string in a locals or null resource block and then pass that to the template
Variable
variable "boostrap_servers" {
type = map
default = {
"env01" : [
"k01.env01",
"k02.env01"
],
"env02" : [
"k01.env02"
]
}
Desired text
"connections": {
"env01": {
"properties": {
"bootstrap.servers": "k01.env01,k02.env01"
}
},
"env02": {
"properties": {
"bootstrap.servers": "k01.env02"
}
},
You may simply use the jsonencode function and list comprehension for this:
locals {
connections = jsonencode({
for cluster, servers in local.bootstrap_servers :
cluster => {
properties = {
"bootstrap.servers" = join(",", servers)
}
}
})
}
Ok, so the following works but there's a better question: why not just use the jsonencode function to build the json
locals {
clusters = [
for cluster, servers in var.boostrap_servers :
"{\"${cluster}\":{\"properties\":{\"bootstrap.servers\":\"${join(" ,", servers)}\"}}"]
connections = join(",", local.clusters)
}
Wondering if anyone has ran tackled it. So, I need to be able to generate list of egress CIDR blocks that is currently available for listing over an API. Sample output is the following:
[
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.1.191",
"cidr": "192.168.1.128/26",
"ip": "192.168.1.128",
"ip_range": {
"start": "192.168.1.128",
"end": "192.168.1.191"
},
"netmask": "255.255.255.192",
"network": "192.168.1.128",
"prefix": "26",
"size": "64"
}
},
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.160.127",
"cidr": "192.168.160.0/25",
"ip": "192.168.160.0",
"ip_range": {
"start": "192.168.160.0",
"end": "192.168.160.127"
},
"netmask": "255.255.255.128",
"network": "192.168.160.0",
"prefix": "25",
"size": "128"
}
}
]
So, I need convert it to Azure Firewall
###############################################################################
# Firewall Rules - Allow Access To TEST VMs
###############################################################################
resource "azurerm_firewall_network_rule_collection" "azure-firewall-azure-test-access" {
for_each = local.egress_ips
name = "azure-firewall-azure-test-rule"
azure_firewall_name = azurerm_firewall.public_to_test.name
resource_group_name = var.resource_group_name
priority = 105
action = "Allow"
rule {
name = "test-access"
source_addresses = local.egress_ips[each.key]
destination_ports = ["43043"]
destination_addresses = ["172.16.0.*"]
protocols = [ "TCP"]
}
}
So, bottom line is that allowed IP addresses have to be a list of strings for the "source_addresses" parameter, such as this:
["192.168.44.0/24","192.168.7.0/27","192.168.196.0/24","192.168.229.0/24","192.168.138.0/25",]
I configured data_sources.tf file:
data "http" "allowed_networks_v1" {
url = "https://testapiserver.com/api/allowed/networks/v1"
}
...and in locals.tf, I need to configure
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
egress_ips = ...
}
...and that's where I am stuck. How can parse that data in locals.tf file so I can reference it from within TF ?
Thanks a metric ton!!
I'm assuming that the list of string you are referring to are the objects under: metadata.cidr we can extract that with a for loop in a local, and also do a distinct just in case we get duplicates.
Here is a sample code
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/heldersepu/hs-scripts/master/json/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}
and here is the output of a plan on that:
terraform plan
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
Terraform will perform the following actions:
Plan: 0 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ data = [
+ "192.168.1.128/26",
+ "192.168.160.0/25",
]
Here is the code for your second sample:
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/akamalov/testfile/master/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json.egress_nat_ranges : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}
I am trying to fetch the tags of AMI using AWS CLI and want to reuse the values from the output.
I have a terraform code below which is returning outputs in string format(Maybe not sure of format) which I want to convert into a map object.
variable "ami" {
default = "ami-xxxx"
}
locals {
tags = {
"platform" = lookup(data.local_file.read_tags.content, "platform", "") #Expecting to get platform from Map of read_tags
}
}
data "template_file" "log_name" {
template = "${path.module}/output.log"
}
resource "null_resource" "ami_tags" {
provisioner "local-exec" {
command = "aws ec2 describe-tags --filters Name=resource-id,Values=${var.ami} --query Tags[*].[Key,Value] > ${data.template_file.log_name.rendered}"
}
}
data "local_file" "read_tags" {
filename = "${data.template_file.log_name.rendered}"
depends_on = ["null_resource.ami_tags"]
}
output "tags" {
value = local.tags
}
output "cli-output-tags" {
value = "${concat(data.local_file.read_tags.content)}"
}
output of cli-output-tags is below:
[
[
"ENV",
"DEV"
],
[
"Name",
"Base-AMI"
],
[
"platform",
"Linux"
]
]
How can I convert this output into Map as below using terraform/(jq command), or is there any other way to fetch required values directly from cli-output-tags output:
{
ENV = "DEV",
Name = "Base-AMI",
platform = "Linux"
}
I have also tried changing the CLI command a bit like below but still not able to fetch values as expected:
'Tags[].{Key:Key,Value:Value}'
Resulted below output:
[
{
"Key": "ENV",
"Value": "DEV"
},
{
"Key": "Name",
"Value": "Base-AMI"
},
{
"Key": "platform",
"Value": "Linux"
}
]
You could use zipmap:
output "cli-output-tags" {
value = zipmap(
jsondecode(data.local_file.read_tags.content)[*][0],
jsondecode(data.local_file.read_tags.content)[*][1]
)
}
The code first changes string data from your file to json, then
gets all first elements [*][0] (same for second elements [*][1]), and zips them into map.
How can I convert this output into Map as below
One way would be to use jq as follows (assuming cli-output-tags is the name of the file holding the JSON array of arrays):
jq -r -f '"{", (.[] | "\(.[0]) = \"\(.[1])\""), "}"' cli-output-tags
Looking for the easiest way in terraform to create a JSON string (preferably using jsonencode) or something similar i've looked at the few examples on the terraform docs but still struggling to nail the exact formatting to get this output right. This should make it pretty easy to keep the dev.env file we use in our docker-compose to transfer nicely into this large array that's used in several of our containers.
locals {
container_envs = {
ENV = "dev"
CONTAINER_TAG = "dev"
}
}
To the following type of output
[{
"name": "ENV",
"value": "dev"
},
{
"name": "CONTAINER_TAG",
"value": "dev"
}
]
I think the following should work (not sure if the order is important or not):
locals {
container_envs = {
ENV = "dev"
CONTAINER_TAG = "dev"
}
}
output "test" {
value = [for k,v in local.container_envs: { name = k, value = v }]
}
which gives:
test = [
{
"name" = "CONTAINER_TAG"
"value" = "dev"
},
{
"name" = "ENV"
"value" = "dev"
},
]
I have below terraform configuration for cognito client:
data "aws_cognito_user_pools" "re_user_pool" {
name = "${var.cognito_user_pool_name}"
}
resource "aws_cognito_user_pool_client" "app_client" {
name = "re-app-client"
user_pool_id = data.aws_cognito_user_pools.re_user_pool.id
depends_on = [data.aws_cognito_user_pools.re_user_pool]
explicit_auth_flows = ["USER_PASSWORD_AUTH"]
prevent_user_existence_errors = "ENABLED"
allowed_oauth_flows_user_pool_client = true
allowed_oauth_flows = ["code"]
allowed_oauth_scopes = ["phone", "openid", "email", "profile", "aws.cognito.signin.user.admin"]
supported_identity_providers = ["COGNITO", "Google"]
callback_urls = ["https://scnothzsf0.execute-api.ap-southeast-2.amazonaws.com/staging/signup"]
}
I references the cognito user pool which already exists on AWS. The error happens on the line user_pool_id = data.aws_cognito_user_pools.re_user_pool.id when it uses the user pool id in aws_cognito_user_pool_client.
I will get the error
Error: Error creating Cognito User Pool Client: InvalidParameterException: 1 validation error detected: Value 're-user' at 'userPoolId' failed to satisfy constraint: Member must satisfy regular expression pattern: [\w-]+_[0-9a-zA-Z]+
on infra/cognito.tf line 5, in resource "aws_cognito_user_pool_client" "app_client":
5: resource "aws_cognito_user_pool_client" "app_client" {`
It seems the format of the ID is not correct. I have read this document https://www.terraform.io/docs/providers/aws/d/cognito_user_pools.html and it has a reference attribute ids - The list of cognito user pool ids.. I wonder why it gives a list of user pool id. How can I reference this ID?
I also tried to reference it as user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0] but got an error:
Error: Invalid index
on infra/cognito.tf line 8, in resource "aws_cognito_user_pool_client" "app_client":
8: user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0]
This value does not have any indices.
The re_user_pool referenced above is defined here:
resource "aws_cognito_user_pool" "re_user_pool" {
name = "re-user"
}
I came across your question while working through this same problem. I see the question is several months old, but I'm still going to add an answer for anyone else that ends up here like I did.
First, the solution is to convert the ids value from a set to a list via the tolist function and then access it as you would any terraform list.
Caveat: In my case, I have ensured I only have one user pool for a given name, but you could get multiple user pools if you haven't followed this convention. This solution will not be a complete solution for that situation, but perhaps it will still point in the right direction.
Example code:
data "aws_cognito_user_pools" "test" {
name = "a_name"
}
output "test" {
value = "${tolist(data.aws_cognito_user_pools.test.ids)[0]"
}
Second, how I arrived at it:
I added an output block so I could see what I was working with and I commented out the problematic lines in my terraform file so I could successfully execute terraform apply. Next I ran terraform apply followed by terraform output --json (note: the apply must be successful for output to have the latest values).
Example temporary output block:
output "test" {
value = "${data.aws_cognito_user_pools.test}" // output top-level object for debugging
}
Relevant terraform apply output:
test = {
"arns" = [
"<redacted>",
]
"id" = "a_name"
"ids" = [
"us-east-1_<redacted>",
]
"name" = "a_name"
}
Relevant terraform output --json output:
"test": {
"sensitive": false,
"type": [
"object",
{
"arns": [
"set",
"string"
],
"id": "string",
"ids": [
"set",
"string"
],
"name": "string"
}
],
"value": {
"arns": [
"<redacted>"
],
"id": "a_name",
"ids": [
"us-east-1_<redacted>"
],
"name": "a_name"
}
}
As you can see, the ids portion is a set of type string. I decided to try converting ids to a list to see if I could then access the 0 index and it worked. I feel like this could be a terraform bug, but I haven't filed an issue yet.