Terraform attach disks dynamic - terraform

Point: I need to dynamically attach disks created with a for_each loop to VM
resource "yandex_compute_disk" "volume" {
for_each = var.volumes
name = each.value["name"]
size = each.value["size"]
type = each.value["type"]
resource "yandex_compute_instance" "vps" {
count = var.server_count
name = var.server_count > 1 ? "${var.server_name}${count.index}" : var.server_name
platform_id = var.platform_id
zone = var.zone
....
....
....
dynamic "secondary_disk" {
for_each = var.volumes
content {
disk_id = yandex_compute_disk.volume.id -- here is problem
auto_delete = var.server_volume_auto_delete
}
}
error:
│
│ For example, to correlate with indices of a referring resource, use:
│ yandex_compute_disk.volume[each.key]```
But its not work like that yandex_compute_disk.volume[each.key].id

If the requested attribute is being exported, then this might work:
resource "yandex_compute_instance" "vps" {
count = var.server_count
name = var.server_count > 1 ? "${var.server_name}${count.index}" : var.server_name
platform_id = var.platform_id
zone = var.zone
....
....
....
dynamic "secondary_disk" {
for_each = var.volumes
content {
disk_id = yandex_compute_disk.volume[secondary_disk.name].id
auto_delete = var.server_volume_auto_delete
}
}
The important thing to note about using dynamic blocks is the iterator [1] name. If you do not set it, it will default to the label of the dynamic block, i.e., secondary_disk in this case. At a very high level, you could say that iterator name replaces the each object.
[1] https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks

Related

Terraform for_each The given key does not identify an element in this collection value

I am working on a project where I am building a "stateful" qa environment that incorporates different Win versions that correspond to different sql versions. For example, for Server 2016 I will have 3 servers with each having a different version of sql. Same thing for 2019, and 2022. I have reached the point where it will read the values correctly but then it is giving me this error:
│ Error: Invalid index
│
│ on main.tf line 60, in resource "vsphere_virtual_machine" "vm":
│ 60: guest_id = data.vsphere_virtual_machine.template[each.value.template].guest_id
│ ├────────────────
│ │ data.vsphere_virtual_machine.template is object with 2 attributes
│ │ each.value.template is "Templates/QA_2016"
│
│ The given key does not identify an element in this collection value.
Here is the code:
`
provider "vsphere" {
vim_keep_alive = 30
user = var.vsphere_user
password = var.vsphere_password
vsphere_server = var.vsphere_server
# If you have a self-signed cert
allow_unverified_ssl = true
}
#### data block see local vars
data "vsphere_datacenter" "dc" {
name = local.dc
}
data "vsphere_compute_cluster" "compute_cluster" {
name = local.cluster
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_datastore" "datastore" {
name = local.datastore
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_network" "network" {
for_each = var.vms
name = each.value.network
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_virtual_machine" "template" {
for_each = var.vms
name = each.value.template
datacenter_id = data.vsphere_datacenter.dc.id
}
##### Resource Block
resource "vsphere_virtual_machine" "vm" {
for_each = var.vms
datastore_id = data.vsphere_datastore.datastore.id
guest_id = data.vsphere_virtual_machine.template[each.value.template].guest_id
resource_pool_id = data.vsphere_compute_cluster.compute_cluster.id
# host_system_id = "${data.vsphere_datacenter.dc.id}"
firmware = data.vsphere_virtual_machine.template[each.value.template].firmware
num_cpus = local.cpu_count
memory = local.memory
scsi_type = data.vsphere_virtual_machine.template[each.value.template].scsi_type
wait_for_guest_net_timeout = -1
name = each.value.name
`
Here is the vars file:
`
locals {
dc = "DC"
cluster = "The Cluster"
datastore = "Storage_thing"
cpu_count = "4"
memory = "16384"
disk_label = "disk0"
disk_size = "250"
disk_thin = "true"
domain = "my.domain"
dns = ["xx.xx.xx.xx", "xx.xx.xx.xx"]
password = "NotMyPass"
auto_logon = true
auto_logon_count = 1
firmware = "efi"
}
#### Name your vm's here - Terraform will provision what is provided here - Add or comment out VM's as needed
variable "vms" {
type = map(any)
default = {
wqawin16sql14 = {
name = "wqawin16sql14"
network = "vm_network"
template = "Templates/QA_2016"
},
wqawin16sql17 = {
name = "wqawin16sql17"
network = "vm_network"
template = "Templates/QA_2016"
},
}
}
`
Terraform is reporting this error because your data "vsphere_virtual_machine" "template" block has for_each = var.vms and so the instance keys of that resource are the keys from your map value: "wqawin16sql14" and "wqawin16sql17".
That fails because you're trying to look up an instance using the value of the template attribute, which is "Templates/QA_2016" and therefore doesn't match any of the instance keys.
It seems like your goal here is to find one virtual machine for each distinct value of the template attributes in your input variable, and then use the guest_id of each of those VMs to populate the guest_id of the corresponding instance of resource "vsphere_virtual_machine" "vm".
If so, you'll need to make the for_each for your data resource be a collection where each element represents a template, rather than having each element represent a virtual machine to manage. One way to achieve that would be to calculate the set of all template values across all of your elements of var.vm, like this:
data "vsphere_virtual_machine" "template" {
for_each = toset(var.vms[*].template)
name = each.value
datacenter_id = data.vsphere_datacenter.dc.id
}
Notice that name is now set to just each.value because for_each is now just a set of template names, like toset(["Templates/QA_2016"]), so the values of this collection are just strings rather than objects with attributes.
With this change you should then have only one instance of this data resource whose address will be data.vmware_virtual_machine.template["Templates/QA_2016"]. This instance key now does match the template attribute in both of your VM objects, and so the dynamic lookup of the guest ID based on the template attribute of each VM object should succeed.

Produce repeating blocks inside a terraform resource

I am fairly new to terraform and trying to create a google_compute_backend_service using terraform and there are multiple backend blocks inside the resource as shown below:
resource "google_compute_backend_service" "app-backend" {
log_config {
enable = "true"
sample_rate = "1"
}
name = "app-backend"
port_name = "http-34070"
project = "my-project"
protocol = "HTTP"
session_affinity = "NONE"
timeout_sec = "30"
backend {
group = "insatnce-group1"
}
backend {
group = "instance-group2"
}
backend {
group = "instance-group3"
}
health_checks = [google_compute_http_health_check.app-http-l7.name]
}
As seen in the code block above the backend block repeats multiple times. I want to make it dynamic so I do not have to write multiple blocks manually.
I tried the following:
Created a variable in the variables.tf file that contains all the instance groups:
variable "groups" {
type = list(object({
name = string
}))
default = [{ name = "instance-group1"},
{ name = "instance-group2"},
{ name = "instance-group3"}]
}
And modified my resource block to this:
resource "google_compute_backend_service" "app-backend" {
log_config {
enable = "true"
sample_rate = "1"
}
name = "app-backend"
port_name = "http-34070"
project = "my-project"
protocol = "HTTP"
session_affinity = "NONE"
timeout_sec = "30"
dynamic "backend" {
for_each = var.groups
iterator = item
group = item.value.name
}
health_checks = [google_compute_http_health_check.app-http-l7.name]
}
However, when I execute terraform plan I get the following error:
Error: Unsupported argument
│
│ on backend_service.tf line 15, in resource "google_compute_backend_service" "app-backend":
│ 15: group = item.value.name
│
│ An argument named "group" is not expected here.
Where am I going wrong? Is there a better way to achieve this?
You can check the dynamic blocks documentation for the syntax. Otherwise, you had the right idea.
dynamic "backend" {
for_each = var.groups
content {
group = backend.value.name
}
}
You can also simplify the variable structure to make this even easier.
variable "groups" {
type = set(string)
default = ["instance-group1", "instance-group2", "instance-group3"]
}
dynamic "backend" {
for_each = var.groups
content {
group = backend.value
}
}

Terraform - How can I reference a resource containing a "for_each" argument to another resource?

I created a resource that produces a list of VM's using for_each argument. I'm having trouble trying to reference this resource to my web_private_group.
resource "google_compute_instance_group" "web_private_group" {
name = "${format("%s","${var.gcp_resource_name}-${var.gcp_env}-vm-group")}"
description = "Web servers instance group"
zone = var.gcp_zone_1
network = google_compute_network.vpc.self_link
# I've added some attempts I've tried that do not work...
instances = [
//google_compute_instance.compute_engines.self_link
//[for o in google_compute_instance.compute_engines : o.self_link]
{for k, o in google_compute_instance.compute_engines : k => o.self_link}
//google_compute_instance.web_private_2.self_link
]
named_port {
name = "http"
port = "80"
}
named_port {
name = "https"
port = "443"
}
}
# Create Google Cloud VMs
resource "google_compute_instance" "compute_engines" {
for_each = var.vm_instances
name = "${format("%s","${var.gcp_resource_name}-${var.gcp_env}-each.value")}"
machine_type = "e2-micro"
zone = var.gcp_zone_1
tags = ["ssh","http"]
boot_disk {
initialize_params {
image = "debian-10"
}
}
}
variable "vm_instances" {
description = "list of VM's"
type = set(string)
default = ["vm1", "vm2", "vm3"]
}
How can I properly link my compute_engines to my web_private_group resource within the instances= [] block?
Edit: To further clarify, how can I state the fact that there are multiple instances within my compute_engines resource?
You probably just need to use a splat expression as follows:
instances = values(google_compute_instance.compute_engines)[*].id
Moreover, a for expression can be used as well:
instances = [for res in google_compute_instance.compute_engines: res.id]

Because data.zabbix_template.template has "for_each" set, its attributes must be accessed on specific instances

I'm tring to create a zabbix template with applications defined and trigger.
I can create the template, import my hosts and associate to it.
Now when I try to add the trigger to the template, I receive the error in the object.
this is my
data.tf
data "zabbix_hostgroup" "group" {
name = "Templates"
}
data "zabbix_template" "template" {
for_each = {
common_simple = { name = "Common Simple" }
common_snmp = { name = "Common SNMP" }
class_template = { name = var.class_names[var.class_id] }
}
name = each.value.name
}
data "zabbix_proxy" "proxy" {
for_each = {
for inst in var.instances :
"${inst.instance}.${inst.site}" => inst.site
}
#host = "zabpxy01.${each.value}.mysite.local"
host = "mon-proxy1.${each.value}.mtsite.local"
}
and this is my hosts.tf:
# create host group for specific to service
resource "zabbix_hostgroup" "hostgroup" {
name = var.class_names[var.class_id]
}
# create template
resource "zabbix_template" "template" {
host = var.class_id
name = var.class_names[var.class_id]
description = var.class_names[var.class_id]
groups = [
data.zabbix_hostgroup.group.id
]
}
# create application
resource "zabbix_application" "application" {
hostid = data.zabbix_template.template.id
name = var.class_names[var.class_id]
}
# create snmp disk_total item
resource "zabbix_item_snmp" "disk_total_item" {
hostid = data.zabbix_template.template.id
key = "snmp_disk_root_total"
name = "Disk / total"
valuetype = "unsigned"
delay = "1m"
snmp_oid="HOST-RESOURCES-MIB::hrStorageSize[\"index\", \"HOST-RESOURCES-MIB::hrStorageDescr\", \"/\"]"
depends_on = [
data.zabbix_template.template
]
}
# create snmp disk_used item
resource "zabbix_item_snmp" "disk_used_item" {
hostid = data.zabbix_template.template.id
key = "snmp_disk_root_used"
name = "Disk / used"
valuetype = "unsigned"
delay = "1m"
snmp_oid="HOST-RESOURCES-MIB::hrStorageUsed[\"index\", \"HOST-RESOURCES-MIB::hrStorageDescr\", \"/\"]"
depends_on = [
data.zabbix_template.template
]
}
# create trigger > 75%
resource "zabbix_trigger" "trigger" {
name = "Disk Usage 75%"
expression = "({${data.zabbix_template.template.host}:${zabbix_item_snmp.disk_used_item.key}.last()} / {${data.zabbix_template.template.host}:${zabbix_item_snmp.disk_total_item.key}.last()}) * 100 >= 75"
priority = "warn"
enabled = true
multiple = false
recovery_none = false
manual_close = false
}
# create hosts
resource "zabbix_host" "host" {
for_each = {
for inst in var.instances : "${var.class_id}${format("%02d", inst.instance)}.${inst.site}" => inst
}
host = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["hostname"]
name = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["hostname"]
enabled = false
proxyid = data.zabbix_proxy.proxy["${each.value.instance}.${each.value.site}"].id
groups = [
zabbix_hostgroup.hostgroup.id
]
templates = concat ([
data.zabbix_template.template["common_simple"].id,
data.zabbix_template.template["common_snmp"].id,
zabbix_template.template.id
])
# add SNMP interface
interface {
type = "snmp"
ip = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
main = true
port = 161
}
# Add Zabbix Agent interface
interface {
type = "agent"
ip = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
main = true
port = 10050
}
macro {
name = "{$INTERFACE_MONITOR}"
value = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
}
macro {
name = "{$SNMP_COMMUNITY}"
value = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["snmp"]
}
depends_on = [
zabbix_hostgroup.hostgroup,
data.zabbix_template.template,
data.zabbix_proxy.proxy,
]
}
output "class_template_id" {
value = zabbix_template.template.id
description = "Template ID of created class template for items"
}
When I run "Terraform plan" I receive the error:
Error: Missing resource instance key │ │ on hosts/hosts.tf line 26,
in resource "zabbix_application" "application": │ 26: hostid =
data.zabbix_template.template.id │ │ Because
data.zabbix_template.template has "for_each" set, its attributes must
be accessed on specific instances. │ │ For example, to correlate with
indices of a referring resource, use: │
data.zabbix_template.template[each.key]
Where is my error?
Thanks for the support
UPDATE
I tried to use
output "data_zabbix_template" {
value = data.zabbix_template.template
}
but I don't see any output when I run terraform plan
I tried to modify in:
hostid = data.zabbix_template.template.class_template.id
but I continue to receive the same error:
Error: Missing resource instance key on hosts/hosts.tf line 27, in
resource "zabbix_application" "application": 27: hostid =
data.zabbix_template.template.class_template.id Because
data.zabbix_template.template has "for_each" set, its attributes must
be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
data.zabbix_template.template[each.key]
Error: Unsupported attribute on hosts/hosts.tf line 27, in resource
"zabbix_application" "application": 27: hostid =
data.zabbix_template.template.class_template.id This object has no
argument, nested block, or exported attribute named "class_template".
UPDATE:
My script for each host taht I'll add, set two existing template ("Common Simple" and "Common SNMP") and create a new template as below:
# module.mytemplate-servers_host.zabbix_template.template will be created
+ resource "zabbix_template" "template" {
+ description = "mytemplate-servers"
+ groups = [
+ "1",
]
+ host = "mytemplate-servers"
+ id = (known after apply)
+ name = "mytemplate-servers"
}
Now my scope is to add on this template an application and set two items and one trigger
When you use for_each in a data source or resource, the output of that data source or resource is a map, where the keys in the map are the same as the keys in the for_each and the values are the regular output of that data/resource for the given input value with that key.
Try using:
output "data_zabbix_template" {
value = data.zabbix_template.template
}
And you'll see what I mean. The output will look something like:
data_zabbix_template = {
common_simple = {...}
common_snmp = {...}
class_template = {...}
}
So in order to use this data source (on the line where the error is being thrown), you need to do:
hostid = data.zabbix_template.template.common_simple.id
And replace common_simple in that line with whichever key in the for_each you want to use. You'll need to do this everywhere that you use data.zabbix_template.template.

How to dynamically attach multiple volumes to multiple instances via terraform in openstack?

I have written a terraform module(v0.14) that can be used to provision multiple instances in openstack with the correct availability_zone,network-port, correct flavor, can create 1 local disk or 1 blockstorage_volume based on boolean input variables and so forth. Now I have gotten a feature request to be able to add dynamically multiple blockstorage_volumes(=network/shared storage volumes) on multiple instances.
Example of how everything dynamically works for 1 blockstorage_volume on multiple instances:
#Dynamically create this resource ONLY if boolean "volume_storage" = true
resource "openstack_blockstorage_volume_v3" "shared_storage" {
for_each = var.volume_storage ? var.nodes : {}
name = "${each.value}-${each.key}.${var.domain}"
size = var.volume_s
availability_zone = each.key
volume_type = var.volume_type
image_id = var.os_image
}
resource "openstack_compute_instance_v2" "instance" {
for_each = var.nodes
name = "${each.value}-${each.key}.${var.domain}"
flavor_name = var.flavor
availability_zone = each.key
key_pair = var.key_pair
image_id = (var.volume_storage == true ? "" : var.os_image)
config_drive = true
#Dynamically create this parameter ONLY if boolean "volume_storage" = true
dynamic "block_device" {
for_each = var.volume_storage ? var.volume : {}
content {
uuid = openstack_blockstorage_volume_v3.shared_storage[each.key].id
source_type = "volume"
destination_type = "volume"
delete_on_termination = var.volume_delete_termination
}
}
user_data = data.template_file.cloud-init[each.key].rendered
scheduler_hints {
group = openstack_compute_servergroup_v2.servergroup.id
}
network {
port = openstack_networking_port_v2.network-port[each.key].id
}
}
So let's say now that I have 2 instances, where I want to add dynamically 2 extra blockstorage_volumes to each instance, my first idea was to add 2 extra dynamic resources as a try-out:
#Dynamically create this resource if boolean "multiple_volume_storage" = true
resource "openstack_blockstorage_volume_v3" "multiple_shared_storage" {
for_each = var.multiple_volume_storage ? var.multiple_volumes : {}
name = each.value
size = var.volume_s
availability_zone = each.key
volume_type = var.volume_type
image_id = var.os_image
}
Example of 2 extra blockstorage_volumes defined in a .tf file:
variable "multiple_volumes" {
type = map(any)
default = {
dc1 = "/volume/mysql"
dc2 = "/volume/postgres"
}
}
Example of 2 instances defined in a .tf file:
nodes = {
dc1 = "app-stage"
dc2 = "app-stage"
}
Here I try to dynamically attach 2 extra blockstorage_volumes to each instance:
resource "openstack_compute_volume_attach_v2" "attach_multiple_shared_storage" {
for_each = var.multiple_volume_storage ? var.multiple_volumes : {}
instance_id = openstack_compute_instance_v2.instance[each.key].id
volume_id = openstack_blockstorage_volume_v3.multiple_shared_storage[each.key].id
}
The openstack_compute_instance_v2.instance [each.key] is obviously not correct since it now only creates 1 extra blockstorage_volume per instance. Is there a clean/elegant way to solve this? So basically to attach all given volumes in variable "multiple_volumes" to each single instance that is defined in var.nodes
Kind regards,
Jonas

Resources