maybe a really stupid question here and I'm guessing this is actually nothing like as complicated as I'm finding it. However, after spending hours and hours reading and trying, while I've learnt a lot about Terraform, I'm really no closer to a solution which I can understand or reuse.
I'm trying to do something that would be pretty simple in bash or python but seems to require some trickery in Terraform. I have two variables that I want to "combine" so that I have a valid block of data usable in for_each looping for resource creation. It's quite likely that I have failed to do this properly from the beginning in terms of variable declaration. So please forgive my beginner invompetence.
Just to note also, this is mostly for my own learning so not necessarily a practical example bur rather a use case that illustrates what I want to understand better. In this context, my use case is to provision workspaces in Terraform cloud and to insert the same set of private variables in to each workspace created. aka create workspaces and push in azure service principle authentication variables.
So in variables.tf I have
variable "env_names" {
type = set(string)
default = ["dev", "uat", "prod"]
}
variable "auth_vars" {
default = [
{ key = "subscription_id"
value = "XXXXXXXXXXXXXXXXXXXXXXXX"
},
{
key = "client_id"
value = "XXXXXXXXXXXXXXXXXXXX"
},
{
key = "client_secret"
value = "XXXXXXXXXXXXXXXXXXX"
},
{
key = "tenant_id"
value = "XXXXXXXXXXXXXXXXXXX"
}
]
}
Then I want to use these variables to create workspaces with the set of variables applied to each workspace. For the workspace creation, I have no problems creating them using.
resource "tfe_workspace" "cloud_workspace" {
for_each = var.env_names
name = "MyWorkspace-${each.key}"
organization = "MyOrg"
execution_mode = "remote"
auto_apply = "false"
allow_destroy_plan = "true"
global_remote_state = "false"
}
It's the creation of the variables for the workspaces which is a major headache for me. I have tried all sorts of manipulations using locals, flattening the data, doing a setproduct etc. I think it's here that I really don't know how to approach this problem. I have tried to do things like:
locals {
auth_map = flatten([
for w in var.env_names : [
for v in var.auth_vars : {
workspace = w
key = v.key
value = v.value
}
]
])
}
This does seem to create a structure that makes sense as I have a list of all the tuple values required. aka
{
key = "subscription_id"
value = "XXXXXX"
workspace = "dev"
},
{
key = "client_id"
value = "XXXXXXXXXXXX"
workspace = "dev"
},
ETC.....
I just have no idea how to get this list of tuples in to a useful form so that I can create all the variables for the workspaces. I'd want to use the data to create multiple variable blocks of the type below. I'm not really sure how to get there though.
resource "tfe_variable" "azure-credentials" {
key = ""
value = ""
category = "terraform"
workspace_id = ""
}
I'm aware that I could have simply done a few blocks of code and solved this use case long ago, but it's more for my understanding of how Terraform works and how to deal with these more complex situations. Any advice appreciated as I'm really at the start of learning Terraform properly and am well out of my depth here.
Thank you so much for taking the time
If I understand correctly, it should be:
resource "tfe_variable" "azure-credentials" {
for_each = {for idx, value in local.auth_map: idx => value}
key = each.value.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test[each.value.workspace].id
}
In the above, you convert your list of maps local.auth_map into a map, as for_each will not work with your list.
Related
I'm trying to create the following block dynamically based on a list of strings
env {
name = "SECRET_ENV_VAR"
value_from {
secret_key_ref {
name = google_secret_manager_secret.secret.secret_id
key = "1"
}
}
}
Based off documentation: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_service#example-usage---cloud-run-service-secret-environment-variables
I would like to dynamically add Secrets, and have defined the following dynamic block:
dynamic "env" {
for_each = toset(var.secrets)
content {
name = each.value
value_from {
secret_key_ref {
name = each.value
key = "1"
}
}
}
}
Where secrets is a variable of type list(string)
However, this throws an error: Blocks of type "value_from" are not expected here.
I'm not sure what I'm missing, or where I have incorrectly specified the value_from block.
Could someone point me in the right direction for fixing this up?
UPDATE;
I have also tried to implement this variable as a map, as per the suggestion in the comments on this post. (https://www.terraform.io/docs/language/expressions/dynamic-blocks.html#multi-level-nested-block-structures)
dynamic "env" {
for_each = var.secrets
content {
name = each.key
dynamic "value_from" {
for_each = env.value.name
secret_key_ref {
name = value_from.value.name
key = value_from.value.version
}
}
}
}
However, this also gives the same error. Blocks of type "value_from" are not expected here.
In this example, the secrets variable is defined as a list(any) with this value:
secrets = [
{
name = "SECRET"
version = "1"
}
]
You have to upgrade your gcp provider. Support for secrets in google_cloud_run_service was added in v3.67.0. Current version is v4.1.0, which means that you must be using very old gcp provider.
In the end, I solved this by changing the variable type to a map(any):
secrets = {
"SECRET" = "1"
}
This allowed me to create the "dynamic" env block, without needing to implement the nested dynamic block.
I am not sure if this is the right approach to do this but I want to use a variable as an attribute.
For example, I have a variable that changes based on user input: os_name = ubuntu.
I want to use this variable name like the following,
resource "aws_instance" "workerNode" {
..................
ami = data.aws_ami.${var.os_name}.image_id
..................
}
Following is an example of the data block,
data "aws_ami" "suse" {
count = "${var.os_name == "suse" ? 1 : 0}"
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
Which result the following,
"architecture" = "x86_64"
"hypervisor" = "xen"
"id" = "ami-0d3905203a039e3b0"
"image_id" = "ami-0d3905203a039e3b0"
But terraform is not allowing me to do this. Is there any way I can do this or I have to change the workflow?
In situations where it's not appropriate to gather all of your instances under a single resource using for_each (which would implicitly make that resource appear as a map of objects), you can get a similar result explicitly by writing a local value expression to construct an equivalent map:
locals {
amis = {
suse = data.aws_ami.suse
ubuntu = data.aws_ami.ubuntu
}
}
Then you can refer to local.amis["ubuntu"] or local.amis["suse"] (possibly replacing the element key with a variable, if you need to.
With that said, it does seem like there is a possible different approach for your case which would get there with only one data block:
locals {
os_ami_queries = {
suse = {
owners = ["amazon"]
filters = {
name = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
ubuntu = {
owners = ["amazon"]
filters = {
name = ["ubuntu-${var.os_version}-something-something"]
}
}
}
ami_query = local.os_ami_queries[var.os_name]
}
data "aws_ami" "selected" {
owners = local.ami_query.owners
dynamic "filter" {
for_each = local.ami_query.filters
content {
name = filter.key
values = filter.value
}
}
}
This different permutation does the OS selection before the data "aws_ami" lookup, so it can use the settings associated with whichever OS was selected by the caller. The AMI id would then be in data.aws_ami.selected.id.
With that said, this approach has the disadvantage of being quite indirect and using a dynamic block, so I'd weigh that against the readability of the alternatives to pick the one which seems subjectively easiest to follow for someone who isn't familiar with this configuration. There isn't a single answer to that because to some extent it's a matter of taste, and so if you are working in a team setting this could be something to discuss with colleagues to see which approach best suits tradeoffs like how often you expect to be adding and removing supported operating systems vs. changing the details of how you use the result.
You can make it work by specifying your AMI's with a for_each and thus getting a map which you can access by key.
My data.aws_ami.myamis looks like this:
data "aws_ami" "myamis" {
for_each = toset(["suse", "ubuntu"])
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["${each.value}*"]
}
}
For test purposes I define a variable foo like this:
variable "foo" {
type = string
default = "suse"
}
Now I can access the AMI like this:
$ tf console
> data.aws_ami.myamis[var.foo].image_id
"ami-0ea50c090ba6e85c5"
You can adapt this to suit your needs for os_name and os_version.
I have solved the issue just by using conditional expression.
I am not sure if it is a standard way of doing things but it works for me.
I have tried to emulate if/elif/else with nested conditional expression.
output "name" {
value = "${ var.os_name == "ubuntu" ? data.aws_ami.ubuntu[0].image_id : (var.os_name == "redhat" ? data.aws_ami.redhat[0].image_id : (var.os_name == "centos" ? data.aws_ami.suse[0].image_id : data.aws_ami.suse[0].image_id ))}"
}
On my concrete example:
I want to create a rancher environment resource with preconfigured members. But the number of members is supposed to be depending on a variable list. I'd imaging something like
resource "rancher_environment" "renv" {
name = "renv"
project_template_id = "atmplid"
member {
count = "${length(var.memberlist)}"
external_id = "${var.memberlist[count.index]}"
external_id_type = "exttype"
role = "owner"
}
}
This obviously doesn't work. Is there a trick to achieve this behaviour?
You can use null_resource for this. Try this
resource "null_resource" "memberlist" {
count = "${length(var.memberlist)}"
triggers {
external_id = "${var.memberlist[count.index]}"
external_id_type = "exttype"
role = "owner"
}
}
resource "rancher_environment" "renv" {
name = "renv"
project_template_id = "atmplid"
member = ["${null_resource.memberlist.*.triggers}"]
}
At long last, Terraform has just released v0.12.0-alpha1, which contains a more elegant way of solving this exact problem.
I would like to replace the 3 indepedent variables (dev_id, prod_id, stage_id), for a single list containing all the three variables, and iterate over them, applying them to the policy.
Is this something terraform can do?
data "aws_iam_policy_document" "iam_policy_document_dynamodb" {
statement {
effect = "Allow"
resources = ["arn:aws:dynamodb:${var.region}:${var.account_id}:table:${var.dynamodb_table_name}"]
actions = [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem",
]
principals {
type = "AWS"
identifiers = [
"arn:aws:iam::${var.dev_id}:root",
"arn:aws:iam::${var.prod_id}:root",
"arn:aws:iam::${var.stage_id}:root"
]
}
}
}
I looked into cycles and interpolation, but It seems that 99% of the time the interpolation is done with "count" which only works for the creation of multiple resources (I hope I am not saying a big lie).
For example, I used
principals {
count = "${length(var.list)}"
identifiers = ["arn:aws:iam::${var.list[count.index]}"]
}
but that was unsuccessful.
Is there some way of achieving the final goal of replacing those 3 variables by a single list (or map) and iterate over them?
Given you have the list of account ids, have you tried this?
var "accounts" {
default = ["123", "456", "789"]
type = "list"
}
locals {
accounts_arn = "${formatlist("arn:aws:iam::%s", var.accounts)}"
}
Then in your policy document:
principals {
type = "AWS"
identifiers = ["${local.accounts_arn}"]
}
I haven't actually tried it, but can't think of a reason it wouldn't work.
I'm creating subnets as part of a seperate terraform template and exporting the IDs as follows.
output "subnet-aza-dev" {
value = "${aws_subnet.subnet-aza-dev.id}"
}
output "subnet-azb-dev" {
value = "${aws_subnet.subnet-azb-dev.id}"
}
output "subnet-aza-test" {
value = "${aws_subnet.subnet-aza-test.id}"
}
output "subnet-azb-test" {
value = "${aws_subnet.subnet-azb-test.id}"
}
...
I'm then intending to lookup these IDs in another template which is reused to provision multiple environments. Example below shows my second template is calling a module to provision an EC2 instance and is passing through the subnet_id.
variable "environment" {
description = "Environment name"
default = "dev"
}
module "sql-1-ec2" {
source = "../modules/ec2winserver_sql"
...
subnet_id = "${data.terraform_remote_state.env-shared.subnet-aza-dev}"
}
What I'd like to do is pass the environment variable as part of the lookup for the subnet_id e.g.
subnet_id = "${data.terraform_remote_state.env-shared.subnet-aza-${var.environment}"
However I'm aware that variable interpolation isn't supported. I've tried using a map inside of the first terraform template to export them all to a 'subnet' which I could then use to lookup from the second template. This didn't work as I was unable to output variables inside of the map.
This sort of design pattern is something I've used previously with CloudFormation, however I'm much newer to terraform. Am I missing something obvious here?
Worked out a way to do this using data sources
variable "environment" {
description = "Environment name"
default = "dev"
}
module "sql-1-ec2" {
source = "../modules/ec2winserver_sql"
...
subnet_id = "${data.aws_subnet.subnet-aza.id}"
}
data "aws_subnet" "subnet-aza" {
filter {
name = "tag:Name"
values = ["${var.product}-${var.environment}-${var.environmentno}-subnet-aza"]
}
}
data "aws_subnet" "subnet-azb" {
filter {
name = "tag:Name"
values = ["${var.product}-${var.environment}-${var.environmentno}-subnet-azb"]
}
}
Whilst this works and fulfils my original need, I'd like to improve on this by moving the data blocks to within the module, so that there's less repetition. Still working on that one though...