My goal is to have something like a common.tfvars file eg:
users = {
"daniel.meier" = {
path = "/"
force_destroy = true
tag_email = "foo#example.com"
github = "dme86"
}
"linus.torvalds" = {
path = "/"
force_destroy = true
tag_email = "bar#example.com"
github = "torvalds"
}
}
Via data you'll be able to retrieve informations about the github accounts:
data "github_user" "this" {
for_each = var.users
username = each.value["github"]
}
Output of ssh keys is also possible:
output "current_github_ssh_key" {
value = values(data.github_user.this).*.ssh_keys
}
But how can i get the SSH keys from output into a resource like:
resource "aws_key_pair" "deployer" {
for_each = var.users
key_name = each.value["github"]
public_key = values(data.github_user.this).*.ssh_keys
}
If i'm trying like in this example terraform errors
Inappropriate value for attribute "public_key": string required.
which makes sense, cause the keys are a list AFAIK - but how to convert this correctly?
Output looks like this:
Changes to Outputs:
+ current_github_ssh_key = [
+ [
+ "ssh-rsa AAAAB3NzaC1yc2EAAAAD(...)ElQ==",
],
+ [
+ "ssh-rsa AAAAB3NzaC1yc2EAAGVD(...)TXxrF",
],
]
If you want to test this code you have to include a github token for your provider like:
provider "github" {
token = "123456"
}
Related
I'm trying to create a secret on GCP's Secret Manager.
The secret value is coming from Vault (HCP Cloud).
How can I pass a value of the secret if I'm using a .tfvars file for the values?
Creating the secret without .tfvars works. Other suggestions rather than data source are welcomed as well. I saw that referring locals isn't possible as well inside tfvars.
vault.tf:
provider "vault" {
address = "https://testing-vault-public-vault-numbers.numbers.z1.hashicorp.cloud:8200"
token = "someToken"
}
data "vault_generic_secret" "secrets" {
path = "secrets/terraform/cloudcomposer/kafka/"
}
main.tf:
resource "google_secret_manager_secret" "connections" {
provider = google-beta
count = length(var.connections)
secret_id = "${var.secret_manager_prefix}-${var.connections[count.index].name}"
replication {
automatic = true
}
}
resource "google_secret_manager_secret_version" "connections-version" {
count = length(var.connections)
secret = google_secret_manager_secret.connections[count.index].id
secret_data = var.connections[count.index].uri
}
dev.tfvars:
image_version = "composer-2-airflow-2.1.4"
env_size = "LARGE"
env_name = "development"
region = "us-central1"
network = "development-main"
subnetwork = "development-subnet1"
secret_manager_prefix = "test"
connections = [
{ name = "postgres", uri = "postgresql://postgres_user:XXXXXXXXXXXX#1.1.1.1:5432/"}, ## This one works
{ name = "kafka", uri = "${data.vault_generic_secret.secrets.data["kafka_dev_password"]}"
]
Getting:
Error: Invalid expression
on ./tfvars/dev.tfvars line 39:
Expected the start of an expression, but found an invalid expression token.
Thanks in advance.
Values in the tfvars files have to be static, i.e., they cannot use any kind of a dynamic assignment like when using data sources. However, in that case, using local variables [1] should be a viable solution:
locals {
connections = [
{
name = "kafka",
uri = data.vault_generic_secret.secrets.data["kafka_dev_password"]
}
]
}
Then, in the resource you need to use it in:
resource "google_secret_manager_secret" "connections" {
provider = google-beta
count = length(local.connections)
secret_id = "${var.secret_manager_prefix}-${local.connections[count.index].name}"
replication {
automatic = true
}
}
resource "google_secret_manager_secret_version" "connections-version" {
count = length(local.connections)
secret = google_secret_manager_secret.connections[count.index].id
secret_data = local.connections[count.index].uri
}
[1] https://developer.hashicorp.com/terraform/language/values/locals
If I output
cloudamqp_instance.rabbitmq.url
I'll get
amqps://test:xxxxxxxxxxx#young-white.rmq2.cloudamqp.com/test
now I have to pass substring from above output like this
provider "rabbitmq" {
endpoint = "https://young-white.rmq2.cloudamqp.com"
username = "test"
password = "xxxxxxxxxxx"
}
Is there a way to do that??
Assuming that your example is actually representative of your use-case, the following regex can parse it:
variable "s" {
default = "amqps://test:xxxxxxxxxxx#young-white.rmq2.cloudamqp.com/test"
}
locals {
parsed = regex(".+//(?P<username>.+):(?P<password>.+)#(?P<endpoint>.+)/", var.s)
}
output "test" {
value = local.parsed
}
which gives:
test = {
"endpoint" = "young-white.rmq2.cloudamqp.com"
"password" = "xxxxxxxxxxx"
"username" = "test"
}
Then you have to just add https:// to local.parsed.endpoint
I'd like to create every type of DNS record for a domain using a generic module, and so be able to call it with something like:
module "example_com_dns" {
source = "[PATH_TO_MODULES]/modules/dns"
domain = "example.com"
a_records = {
"#" = [SOME IP]
"www" = [SOME IP]
"home" = [SOME IP]
}
txt_records = {
"#" = "txt-foobar1"
"#" = "txt-foobar2"
"mail._domainkey.self" = "foobar"
}
mx_entries = {
"10" = "mail.someprovider.com"
"20" = "mail2.someprovider.com"
}
cname_records {
"cname-foo" = "cname-bar
}
}
I have something that works fine for A, CNAME , and MX records, but TXT has an edge case which I need to work around. My module has resource blocks for each type of record, which run through loops. I'll just paste the TXT one, but they're all the same:
resource "digitalocean_record" "this_txt_record" {
for_each = var.txt_records
domain = var.domain
type = "TXT"
name = each.key
value = each.value
}
This all works fine, except for the fact that since there are 2 records with "#" for their key, it results in only the last one being created (in my example above, this being "txt-foobar2"):
...
# module.example_com.digitalocean_record.this_txt_record["#"] will be created
+ resource "digitalocean_record" "this_txt_record" {
+ domain = "example.com"
+ fqdn = (known after apply)
+ id = (known after apply)
+ name = "#"
+ ttl = (known after apply)
+ type = "TXT"
+ value = "txt-foobar2"
}
I'd like for it to create both "txt-foobar1" and "txt-foobar2", even given non-unique keys in the map.
Perhaps this is the wrong way and I just need to figure out a clever loop for for parsing this structure instead?:
txt_records = [
{ "#" = "foo" },
{ "#" = "bar"},
{ "mail._domainkey.self" = "foobar"}
]
If so, I'm currently failing there too :)
Resources cannot be created by for_each'ing a list since there must be a unique key that will become part of the terraform resource name. List indexes cannot be a reliable key since your TF plan will be all messed up if you reorder items in the list.
Maps on the other hand do have unique keys by definition.
You can generate a map from a list though! I found this little trick here. Note you additionally need to manually compute the unique map key (${txt_record[0]}=${txt_record[1]} in the example below).
Your resources with updates in place:
module "example_com_dns" {
...
txt_records = [
["#", "txt-foobar1"],
["#", "txt-foobar2"],
["mail._domainkey.self", "foobar"],
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record[0]}=${txt_record[1]}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value[0]
value = each.value[1]
}
or slightly more verbose if you prefer:
module "example_com_dns" {
...
txt_records = [
{name: "#", value: "txt-foobar1"},
{name: "#", value: "txt-foobar2"},
{name: "mail._domainkey.self", value: "foobar"},
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record.name}=${txt_record.value}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value.name
value = each.value.value
}
Alternative way to already given one is to use the following:
variable "txt_records" {
default = {
"#" = ["foo", "bar"],
"mail._domainkey.self" = ["foobar"]
}
}
Then you can flatten the txt_records using:
locals {
txt_records_flat = merge([
for key, values in var.txt_records:
{for value in values:
"${key}-${value}" => {"record_name" = key, "record_value" = value}
}
]...)
}
which results in local.txt_records_flat of:
{
"#-bar" = {
"record_name" = "#"
"record_value" = "bar"
}
"#-foo" = {
"record_name" = "#"
"record_value" = "foo"
}
"mail._domainkey.self-foobar" = {
"record_name" = "mail._domainkey.self"
"record_value" = "foobar"
}
}
Then you use it:
resource "digitalocean_record" "this_txt_record" {
for_each = local.txt_records_flat
domain = var.domain
type = "TXT"
name = each.value.record_name
value = each.value.record_value
}
I'm trying to create a transfer key from a map users to SSH keys
content_users = {
"master" = [
"ssh-rsa ...",
"ssh-rsa ...",
"ssh-rsa ...",
]
"test" = [
"ssh-rsa ...",
"ssh-rsa ...",
]
}
The aws_transfer_user part is easy enough
resource "aws_transfer_user" "content" {
for_each = var.content_users
server_id = aws_transfer_server.content.id
user_name = each.key
role = aws_iam_role.transfer.arn
}
But I am trying to figure out how to do the aws_transfer_key which only accepts one ssh key
resource "aws_transfer_ssh_key" "content" {
for_each = var.content_users
server_id = aws_transfer_server.content.id
user_name = each.key
body = "... SSH key ..."
}
I am thinking it is something I just have to follow with https://www.terraform.io/docs/configuration/functions/flatten.html#flattening-nested-structures-for-for_each
resource "aws_transfer_ssh_key" "content" {
for_each = toset(flatten([
for user, keys in var.content_users : [
for key in keys : "${user}:#:${key}"
]
]))
server_id = aws_transfer_server.content.id
user_name = split(":#:", each.value)[0]
body = split(":#:", each.value)[1]
}
I did this a little different. I didn't like the idea of the resource name containing the full public ssh key value. I use the below code in a module that was created.
I used a map variable like this.
variable "customers" {
description = "A map of customer usernames and ssh public keys"
type = map(object({
keys = list(string)
}))
}
ex.
customers = {
cust1 = {
keys = [
"ssh-rsa..."
]
},
cust2 = {
keys = [
"ssh-rsa...",
"ssh-rsa...",
"ssh-rsa..."
]
},
}
First I had to flatten it and I also added an index while doing so.
locals {
customers = flatten([
for customer, sshkeys in var.customers : [
for k, v in sshkeys.keys : {
client = customer
index = k
key = v
}
]
])
}
Then I used it in the resource. The index is required otherwise the map fails and asks you to group the client keys using ..., but we do not want this.
resource "aws_transfer_ssh_key" "this" {
for_each = { for each in local.customers : "${each.client}.${each.index}" => each }
server_id = aws_transfer_server.this.id
user_name = "customer-${each.value.client}"
body = each.value.key
}
This should generate numbered resources for the transfer ssh key like the follwing...
cust1.0
cust2.0
cust2.1
cust2.2
The index value resets for each client. You could add a +1 to the index so it starts at 1 instead of 0 as well.
In the following code block I'm trying to pass an array of server names to the attributes_json block:
resource "aws_instance" "consul-server" {
ami = var.consul-server
instance_type = "t2.nano"
key_name = var.aws_key_name
iam_instance_profile = "dna_inst_mgmt"
vpc_security_group_ids = [
"${aws_security_group.yutani_consul.id}",
"${aws_security_group.yutani_ssh.id}"
]
subnet_id = "${aws_subnet.public_1_subnet_us_east_1c.id}"
associate_public_ip_address = true
tags = {
Name = "consul-server${count.index}"
}
root_block_device {
volume_size = "30"
delete_on_termination = "true"
}
connection {
type = "ssh"
user = "chef"
private_key = "${file("${var.aws_key_path}")}"
timeout = "2m"
agent = false
host = self.public_ip
}
count = var.consul-server_count
provisioner "chef" {
attributes_json = <<-EOF
{
"consul": {
"servers": ["${split(",",aws_instance.consul-server[count.index].id)}"]
}
}
EOF
use_policyfile = true
policy_name = "consul_server"
policy_group = "aws_stage_enc"
node_name = "consul-server${count.index}"
server_url = var.chef_server_url
recreate_client = true
skip_install = true
user_name = var.chef_username
user_key = "${file("${var.chef_user_key}")}"
version = "14"
}
}
Running this gives me an error:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
(This is after declaring a count of 2 in a variable for var.consul-server_count)
Can anyone tell me what the proper way is to do this?
There are two issues here: (1) How to interpolate a comma-separated list in a JSON string ; and (2) What is causing the cyclic dependency error.
How to interpolate a list to make a valid JSON array
Use jsonencode
The cleanest method is to not use a heredoc at all and just use the jsonencode function.
You could do this:
locals {
arr = ["host1", "host2", "host3"]
}
output "test" {
value = jsonencode(
{
"consul" = {
"servers" = local.arr
}
})
}
And this yields as output:
Outputs:
test = {"consul":{"servers":["host1","host2","host3"]}}
Use the join function and a heredoc
The Chef provisioner's docs suggest to use a heredoc for the JSON string, so you can also do this:
locals {
arr = ["host1", "host2", "host3"]
sep = "\", \""
}
output "test" {
value = <<-EOF
{
"consul": {
"servers": ["${join(local.sep, local.arr)}"]
}
}
EOF
}
If I apply that:
Outputs:
test = {
"consul": {
"servers": ["host1", "host2", "host3"]
}
}
Some things to pay attention to here:
You are trying to join your hosts so that they become valid JSON in the context of a JSON array. You need to join them with ",", not just a comma. That's why I've defined a local variable sep = "\", \"".
You seem to be trying to split there when you apparently need join.
Cyclic dependency issue
The cause of the error message:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
Is that you have a cyclic dependency. Consider this simplified example:
resource "aws_instance" "example" {
count = 3
ami = "ami-08589eca6dcc9b39c"
instance_type = "t2.micro"
user_data = <<-EOF
hosts="${join(",", aws_instance.example[count.index].id)}"
EOF
}
Or you could use splat notation there too for the same result i.e. aws_instance.example.*.id.
Terraform plan then yields:
▶ terraform012 plan
...
Error: Cycle: aws_instance.example[2], aws_instance.example[1], aws_instance.example[0]
So you get a cycle error there because aws_instance.example.*.id depends on the aws_instance.example being created, so the resource depends on itself. In other words, you can't use a resources exported values inside the resource itself.
What to do
I don't know much about Consul, but all the same, I'm a bit confused tbh why you want the EC2 instance IDs in the servers field. Wouldn't the Consul config be expecting IP addresses or hostnames there?
In any case, you probably need to calculate the host names yourself outside of this resource, either as a static input parameter or something that you can calculate somehow. And I imagine you'll end up with something like:
variable "host_names" {
type = list
default = ["myhost1"]
}
resource "aws_instance" "consul_server" {
...
provisioner "chef" {
attributes_json = jsonencode(
{
"consul" = {
"servers" = var.host_names
}
})
}
}