How to iterate only the widget part of the terraform script and get all the widget in a single dashboard?
locals {
instances = csvdecode(file("${path.module}/sample.csv"))
}
// if we use count it will loop this part
resource "aws_cloudwatch_dashboard" "main" {
dashboard_name = "my-dashboard"
dashboard_body = <<EOF
{
"widgets": [
{
"type":"metric",
"x":0,
"y":0,
"width":12,
"height":6,
"properties":{
"metrics":[
for itr in local.instances.id:
[
"AWS/EC2",
"CPUUtilization",
"InstanceId",
itr // want this section to fetch the value form excel
]
],
"period":300,
"stat":"Average",
"region":"ap-south-1",
"title":"EC2 Instance CPU ",
"annotations": {
"horizontal": [
{
"label": "Untitled annotation",
"value": 2
}]}
}},]}EOF}
If your goal is to generate JSON, it's generally better to use jsonencode rather than template_file, because it can handle the JSON syntax details automatically and thus avoid the need to tweak annoying details of a text template to get the JSON right.
For example:
dashboard_body = jsonencode({
"widgets": [
"type": "metric",
"x": 0,
"y": 0,
"width": 12,
"height": 6,
"properties": {
"metrics": [
for inst in local.instances : [
"AWS/EC2",
"CPUUtilization",
"InstanceId",
inst.id,
]
],
"period": 300,
# etc, etc
},
],
})
By using jsonencode you can use any of Terraform's normal language features and functions to produce your data structure, and leave the jsonencode function to turn that into valid JSON syntax at the end.
I tried to use two template_file resources.
sample.csv for test
instance_id
i-00001
i-00002
i-00003
create a template_file resouce for loop,
data "template_file" "ec2_metric" {
count = length(local.instances)
template = jsonencode([ "AWS/EC2", "CPUUtilization", "InstanceId", element(local.instances.*.instance_id, count.index)])
}
create a template_file for whole json
data "template_file" "widgets" {
template = <<JSON
{
"widgets": [
{
"type":"metric",
"x":0,
"y":0,
"width":12,
"height":6,
"properties":{
"metrics":[
${join(", \n ", data.template_file.ec2_metric.*.rendered)}
],
"period":300,
"stat":"Average",
"region":"ap-south-1",
"title":"EC2 Instance CPU ",
"annotations": {
"horizontal": [
{
"label": "Untitled annotation",
"value": 2
}]}
}},]}
JSON
}
using template_file,
resource "aws_cloudwatch_dashboard" "main" {
dashboard_name = "my-dashboard"
dashboard_body = data.template_file.widgets.rendered
...
}
Test template_file.widgets
output "test" {
value = data.template_file.widgets.rendered
}
result
Outputs:
test = {
"widgets": [
{
"type":"metric",
"x":0,
"y":0,
"width":12,
"height":6,
"properties":{
"metrics":[
["AWS/EC2","CPUUtilization","InstanceId","i-00001"],
["AWS/EC2","CPUUtilization","InstanceId","i-00002"],
["AWS/EC2","CPUUtilization","InstanceId","i-00003"]
],
"period":300,
"stat":"Average",
"region":"ap-south-1",
"title":"EC2 Instance CPU ",
"annotations": {
"horizontal": [
{
"label": "Untitled annotation",
"value": 2
}]}
}},]}
Related
I am trying to create multiple cloudwatch alarms using input from a json file or json locally declared. Either is fine. I've removed the list of
The problem is that this is within a module, so I cannot use file.auto.tfvars.json which is how I previously did this.
File structure:
.
├── cloudwatch.tf
├── cloudwatchalarms.json
└── locals.tf
What I've tried so far:
Use jsondecode to bring in the cloudwatchalarms.json and loop over it in the following way:
cloudwatch.tf
# creates alarms based on what exists within cloudwatchalarms.auto.tfvars.json
resource "aws_cloudwatch_metric_alarm" "alarms" {
for_each = { for alarm in local.all_alarms : alarm => alarm }
alarm_name = "${var.awsResourceName}-${each.value.Identifier}"
namespace = each.value.Namespace
comparison_operator = each.value.ComparisonOperator
evaluation_periods = each.value.EvaluationPeriods
statistic = each.value.Statistic
treat_missing_data = each.value.TreatMissingData
threshold = each.value.Threshold
period = each.value.Period
metric_name = each.value.MetricName
dimensions = {
EcsServiceName = var.awsResourceName
EcsClusterName = var.ecs_cluster_name
}
}
locals.tf
# create locals to pull in cloudwatchalarms.json
locals {
cloudwatchAlarms = jsondecode(file("${path.module}/cloudwatchalarms.json"))
# loop over the cloudwatchalarms json structure
all_alarms = [for alarms in local.cloudwatchAlarms.cloudwatchAlarms : alarms.Identifier]
}
cloudwatchalarms.json
{
"cloudwatchAlarms": [
{
"Identifier": "ServiceMemoryUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "MemoryUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
},
{
"Identifier": "ServiceCPUUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "CPUUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
}
]
}
}
When using this method I get the following errors for every attribute:
Error: Unsupported attribute
on .terraform/modules/terraform-ecs-monitoring/cloudwatch.tf line 4, in resource "aws_cloudwatch_metric_alarm" "alarms":
4: alarm_name = "${var.awsResourceName}-${each.value.Identifier}"
|----------------
| each.value is "TaskStability"
This value does not have any attributes.
The second method I tried using was declaring the json structure in locals
cloudwatch.tf
# creates alarms based on what exists within cloudwatchalarms.auto.tfvars.json
resource "aws_cloudwatch_metric_alarm" "alarms" {
for_each = { for alarm in local.cloudwatchAlarms : alarm.Identifier => alarm }
alarm_name = "${var.awsResourceName}-${each.value.Identifier}"
namespace = each.value.Namespace
comparison_operator = each.value.ComparisonOperator
evaluation_periods = each.value.EvaluationPeriods
statistic = each.value.Statistic
treat_missing_data = each.value.TreatMissingData
threshold = each.value.Threshold
period = each.value.Period
metric_name = each.value.MetricName
dimensions = {
EcsServiceName = var.awsResourceName
EcsClusterName = var.ecs_cluster_name
}
}
locals.tf
locals {
cloudwatchAlarms = {
"cloudwatchAlarms": [
{
"Identifier": "ServiceMemoryUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "MemoryUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
},
{
"Identifier": "ServiceCPUUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "CPUUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
}
]
}
}
I then get this error when trying this method:
Error: Unsupported attribute
on .terraform/modules/terraform-ecs-monitoring/cloudwatch.tf line 3, in resource "aws_cloudwatch_metric_alarm" "alarms":
3: for_each = { for alarm in local.cloudwatchAlarms : alarm.Identifier => alarm }
This value does not have any attributes.
Throw to stop pipeline
Any help or pointing in the right direction would be great. Thanks.
You are closer to correctly constructing the set(object) in the first attempt, and restructuring the data in the second attempt. When combining the two, we arrive at:
locals {
cloudwatchAlarms = jsondecode(file("${path.module}/cloudwatchalarms.json"))
}
resource "aws_cloudwatch_metric_alarm" "alarms" {
for_each = { for alarm in local.cloudwatchAlarms : alarm.Identifier => alarm }
...
}
and your for_each meta-argument iterates over the set(object) (implicitly converted from the tuple type) and retrieves the value for the Identifier key in each object and the entire object. Your Map constructor in the for expression assigns the former to the key and the latter to the value, which when iterated upon will give you the desired behavior for the body of your resource block as it currently exists in the question.
Note also that:
alarm_name = "${var.awsResourceName}-${each.value.Identifier}"
can be simplified to:
alarm_name = "${var.awsResourceName}-${each.key}"
I sorted this with the following configuration:
locals.tf
# create locals to pull in cloudwatchalarms.json
locals {
cloudwatch_alarms = jsondecode(file("${path.module}/cloudwatchalarms.json"))
# loop over the cloudwatchalarms json structure
all_alarms = { for alarms in local.cloudwatch_alarms.cloudwatchAlarms : alarms.Identifier => alarms }
}
cloudwatchalarms.json
{
"cloudwatchAlarms": [
{
"Identifier": "ServiceMemoryUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "MemoryUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
},
{
"Identifier": "ServiceCPUUtilisation",
"Namespace": "AWS/ECS",
"MetricName": "CPUUtilization",
"Statistic": "Average",
"Threshold": 90,
"Period": 60,
"EvaluationPeriods": 5,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "missing"
}
]
}
}
cloudwatch.tf
# creates alarms based on what exists within cloudwatchalarms.auto.tfvars.json
resource "aws_cloudwatch_metric_alarm" "alarms" {
for_each = { for alarm in local.all_alarms : alarm.Identifier => alarm }
alarm_name = "${var.awsResourceName}-${each.value.Identifier}"
namespace = each.value.Namespace
comparison_operator = each.value.ComparisonOperator
evaluation_periods = each.value.EvaluationPeriods
statistic = each.value.Statistic
treat_missing_data = each.value.TreatMissingData
threshold = each.value.Threshold
period = each.value.Period
metric_name = each.value.MetricName
dimensions = {
EcsServiceName = var.awsResourceName
EcsClusterName = var.ecs_cluster_name
}
}
The fix was changing this line in locals.tf to the following:
all_alarms = { for alarms in local.cloudwatch_alarms.cloudwatchAlarms : alarms.Identifier => alarms }
So this is now a map.
Then in cloudwatch.tf changing the for_each to the following:
for_each = { for alarm in local.all_alarms : alarm.Identifier => alarm }
THe problem I am facing now is this. I am trying to make my policy more flexible. So I shifted them into a file instead of using EOF.
How to make the template file recognise a number value?
"${max_untagged_images}" and "${max_tagged_images}" are suppose to be numbers.
Aws lifecycle policy:
resource "aws_ecr_lifecycle_policy" "lifecycle" {
count = length(aws_ecr_repository.repo)
repository = aws_ecr_repository.repo[count.index].name
depends_on = [aws_ecr_repository.repo]
policy = var.policy_type == "app" ? data.template_file.lifecycle_policy_app.rendered : data.template_file.lifecycle_policy_infra.rendered
}
Data template:
data "template_file" "lifecycle_policy_app" {
template = file("lifecyclePolicyApp.json")
vars = {
max_untagged_images = var.max_untagged_images
max_tagged_images = var.max_tagged_images
env = var.env
}
}
Policy:
{
"rules": [
{
"rulePriority": 1,
"description": "Expire untagged images older than ${max_untagged_images} days",
"selection": {
"tagStatus": "untagged",
"countType": "sinceImagePushed",
"countUnit": "days",
"countNumber": "${max_untagged_images}"
},
"action": {
"type": "expire"
}
},
{
"rulePriority": 2,
"description": "Expire tagged images of ${env}, older than ${max_tagged_images} days",
"selection": {
"tagStatus": "tagged",
"countType": "imageCountMoreThan",
"countNumber": "${max_tagged_images}",
"tagPrefixList": [
"${env}"
]
},
"action": {
"type": "expire"
}
}
]
}
I would try the following 2 steps:
Remove the double quotes that around the "${max_tagged_images}"
Use terraform function called tonumber in order to convert it to a number:
tonumber("1")
(Follow the official documentation: https://www.terraform.io/docs/configuration/functions/tonumber.html)
I was wondering if its possible to use templatefile inside of a dashboard_body resource. I'm trying the following.
dashboard_body = <<EOF
{
"widgets": [
{
"type": "metric",
"x": 0,
"y": 0,
"width": 12,
"height": 6,
"properties": {
"metrics": [
templatefile("${path.module}/backends.tmpl", { instances = aws_instance.web })
],
"period": 300,
"stat": "Average",
"region": "us-east-1",
"title": "EC2 Instance CPU"
}
}
]
}
EOF
With templatefile
%{ for instance in instances ~}
[
"AWS/EC2",
"CPUUtilization",
"InstanceId",
"${instance.id}"
]
%{ endfor ~}
However I get the following error message when I run terraform apply.
Error: "dashboard_body" contains an invalid JSON: invalid character 'e' in literal true (expecting 'r')
on dashboards.tf line 1, in resource "aws_cloudwatch_dashboard" "main":
1: resource "aws_cloudwatch_dashboard" "main" {
Thanks in advance for your help.
Yes you can use a template file. It looks like you have a small syntax error here:
templatefile("${path.module}/backends.tmpl", { instances = aws_instance.web })
You should wrap the entire function in an interpolation block, like so:
${templatefile("${path.module}/backends.tmpl", { instances = aws_instance.web })}
In terraform I am attempting to pass a variable (list) to a module that we built. This variable needs to be used within a aws_ecs_task_definition resource in the container_definitions.
Right now I am just starting with an empty default list defined as a variable:
variable "task_enviornment" {
type = "list"
default = []
}
My ECS task definition looks like this:
resource "aws_ecs_task_definition" "ecs_task_definition" {
family = "${var.ecs_family}"
network_mode = "awsvpc"
task_role_arn = "${aws_iam_role.iam_role.arn}"
execution_role_arn = "${data.aws_iam_role.iam_ecs_task_execution_role.arn}"
requires_compatibilities = ["FARGATE"]
cpu = "${var.fargate_cpu}"
memory = "${var.fargate_memory}"
container_definitions =<<DEFINITION
[
{
"cpu": ${var.fargate_cpu},
"image": "${var.app_image}",
"memory": ${var.fargate_memory},
"name": "OURNAME",
"networkMode": "awsvpc",
"environment": "${jsonencode(var.task_enviornment)}",
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group" : "${aws_cloudwatch_log_group.fargate-logs.name}",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "demo"
}
},
"portMappings": [
{
"containerPort": ${var.app_port},
"hostPort": ${var.app_port}
}
]
}
]
DEFINITION
}
The part I am having a problem with with the "environment" part:
"environment": "${jsonencode(var.task_enviornment)}",
I have tried a few different ways to get this to work.
If I do "environment": "${jsonencode(var.task_enviornment)}",
I get ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal string into Go struct field ContainerDefinition.Environment of type []*ecs.KeyValuePair
If I do "environment": "${var.task_enviornment}", or "environment": ["${var.task_enviornment}"],
I get At column 1, line 1: output of an HIL expression must be a string, or a single list (argument 8 is TypeList) in:
Then it just outputs the contents of container_definitions
I did also try adding default values and I was getting similar error messages. However I do need to be able to handle no values being sent in, so an empty list.
variable "task_enviornment" {
type = "list"
default = [
{
"name" = "BUCKET",
"value" = "test"
}
]
}
After a lot of investigation and a fresh set of eyes looking at this figured out the solution. I am unsure why this fixes it, and I feel like this is likely a bug.
Needed to do 2 things to fix this.
Remove type = "list" from the variable definition.
variable "task_environment" {
default = []
}
Remove the quotes when using the variable:
"environment": ${jsonencode(var.task_environment)},
The below solution should work
in variable.tf
variable "app_environments_vars" {
type = list(map(string))
default = []
description = "environment variable needed by the application"
default = [
{
"name" = "BUCKET",
"value" = "test"
},{
"name" = "BUCKET1",
"value" = "test1"
}]
and in your task definition, you can use ${jsonencode(var.app_environments_vars)} similar to
container_definitions =<<DEFINITION
[
{
"cpu": ${var.fargate_cpu},
"image": "${var.app_image}",
"memory": ${var.fargate_memory},
"name": "OURNAME",
"networkMode": "awsvpc",
"environment": ${jsonencode(var.app_environments_vars)},
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group" : "${aws_cloudwatch_log_group.fargate-logs.name}",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "demo"
}
},
"portMappings": [
{
"containerPort": ${var.app_port},
"hostPort": ${var.app_port}
}
]
}
]
My guess is that you are trying to use the type "map" instead of lists, as showed above, the removal from type specification will work.
Example:
List_sample = [1,2,3]
Map_sample = { key_name = "value" }
Reference: Terraform - Type Constraints
My HTTP Response is like this:
{
"Result": [
{
"xPath": "/BB[001]",
"name": "Block001",
"folder": "\\",
"id": 13,
"information": [
{
"xPath": "/BB[001]",
"result": "BB1"
}
],
"error": []
},
{
"xPath": "/TestCases/TestCase[001]",
"name": "I_TT",
"folder": "\\Automation-Inbnd\\TT",
"id": 146,
"information": [
{
"xPath": "/TestCases/TestCase[001]",
"result": "Test1: TT1"
},
{
"xPath": "/TestCases/TestCase[001]",
"result": "Folder path: \\Automation-Inbnd\\TT"
}
],
"error": []
},
{
"xPath": "/TestCases/TestCase[002]",
"name": "TT",
"folder": "\\Automation-Inbnd\\TT",
"id": 147,
"information": [
{
"xPath": "/TestCases/TestCase[002]",
"result": "Test Case Number 2TTO"
}
],
"error": []
}
]
}
In Groovy JSR223 Post Processor I like to extract only those ids after --> "folder": "\Automation-Inbnd\TT" so in this case I like to extract only 146, and 147 and NOT 13
A solution for all Ids after
"folder": "\\Automation-Inbnd\\TT",
if we can even make TT variable it will be great since I have another test case which uses a different subfolder
so all ids only after
"folder": "\\Automation-Inbnd\\(*)",
where (*) is anything and then ,
I appreciate your help, since i have spent a lot of time on this. Thanks
As alternative solution, you could use Json Extractor post-processor, with Jsonpath like this:
$..[?(#.folder =~ /\\Automation-Inbnd\\.*?/i)].id
Full configuration would look like this:
This would generate a set of variables id_1, id_2, e.g. for your sample it would look like this:
You should just be able to use JsonSlurper:
def ids = new groovy.json.JsonSlurper()
.parseText(response)
.Result
.findAll { it.folder == '\\Automation-Inbnd\\TT' }
.id