Iterating over multi-level yaml values in terraform - terraform

I'm trying to shorten my terraform code for deploying azure vnets by iterating over values I provide in a yaml file. I want to write one .tf file with the code for the vnets, the subnets, the NSGs, etc. but I'm struggling to get the locals block right to correctly iterate through my yaml file (see below)
vnets:
- name: adds
location: eastus
address_space: ["10.1.0.0/24"]
subnets:
- name: adds
address_prefix: "10.1.0.0/27"
- name: dns
location: eastus
address_space: ["10.1.53.0/24"]
subnets:
- name: dns-inbound
address_prefix: "10.1.53.0/28"
- name: dns-outbound
address_prefix: "10.1.53.16/28"
Any help on how I should right my locals block would be appreciated!

This code will transform your yaml file into local map:
locals {
vnets = yamldecode(file("./test.yaml"))
vnets_map = {
for vnet in local.vnets.vnets :
vnet.name => {
address_space = vnet.address_space
location = vnet.location
subnets = {
for subnet in vnet.subnets :
subnet.name => subnet.address_prefix
}
}
}
}
output "example-output-dns-inbound-subnet" {
value = local.vnets_map.dns.subnets.dns-inbound
}
I took the liberty of changing lists to maps - it is better to navigate in terraform. Entire vnets_map object looks like this:
{
"adds" = {
"address_space" = [
"10.1.0.0/24",
]
"location" = "eastus"
"subnets" = {
"adds" = "10.1.0.0/27"
}
}
"dns" = {
"address_space" = [
"10.1.53.0/24",
]
"location" = "eastus"
"subnets" = {
"dns-inbound" = "10.1.53.0/28"
"dns-outbound" = "10.1.53.16/28"
}
}
}

Related

How to add existing route table to subnet

I am new to bicep and I am trying to attach an existing UDR named "udr-test" to 2 subnets, unfortunately with no luck.
This is my template that work fine but without UDR configuration:
var addressSpace = [
'10.0.0.0/16'
]
var subnets = [
{
name: 'Subnet1'
subnetPrefix: '10.0.2.0/24'
}
{
name: 'Subnet2'
subnetPrefix: '10.0.3.0/24'
}
]
resource VNET 'Microsoft.Network/virtualNetworks#2021-02-01' existing = {
name: 'vnet-test'
}
#batchSize(1)
resource Subnets 'Microsoft.Network/virtualNetworks/subnets#2020-11-01' = [for (sn, index) in subnets: {
name: sn.name
parent: VNET
properties: {
addressPrefix: sn.subnetPrefix
}
}]
Do you know how I can modify the template to add the UDR as well?
There is routeTable property on the subnet resource (see documentation):
// reference to existing route table
resource routeTable 'Microsoft.Network/routeTables#2022-01-01' existing = {
name: 'udr-test'
}
#batchSize(1)
resource Subnets 'Microsoft.Network/virtualNetworks/subnets#2020-11-01' = [for (sn, index) in subnets: {
name: sn.name
parent: VNET
properties: {
addressPrefix: sn.subnetPrefix
routeTable:{
id: routeTable.id // assign the route table
}
}
}]

How to use tags from yaml file - terraform

I am trying to extract certain tags from a YAML file with Terraform, but i just don't know how.
Yaml file looks like this:
---
name: subscriptionName
emailContact: someemail#domain.com
tags:
- key: "tagKey1"
value: "tagValue1"
- key: "tagKey2"
value: "tagValue2"
- key: "tagKey3"
value: "tagValue3"
- key: "tagKey4"
value: "tagValue4"
What i am interested in is getting 2 (let's say key1 and key3) key-value pairs as tags and tag resouces. I know that 'locals' plays a role here, but i am kinda new to terraform and cannot get any resource tagged.
The resources are Azure (if it matters).
The resource i am trying to tag is:
resource "azurerm_network_security_group" "nsg" {
name = "randomname"
location = "westeurope"
resource_group_name = "random_rg"
tags { }
}
If you really want two random tags, you can use random_shuffle:
locals {
loaded_yaml = yamldecode(file("./your_file.yaml"))
}
resource "random_shuffle" "indices" {
input = range(0, length(local.loaded_yaml.tags))
result_count = 2
seed = timestamp()
}
output "random_tags" {
value = [for idx in random_shuffle.indices.result:
local.loaded_yaml.tags[idx]]
}
update
For example:
tags = {
(local.loaded_yaml.tags[0].key) = local.loaded_yaml.tags[0].value
(local.loaded_yaml.tags[3].key) = local.loaded_yaml.tags[3].value
}

How to reference an object in an array of objects using Bicep

I am trying to output the referenceId of each subnet in a module that creates a virtual network with 4 subnets. I can get the first one, [0], but when I try output the others, [1], [2], [3] the deployment fails and throws the error:
The language expression property array index "1" is out of bounds
Below is the code to create the virtualNetwork and the subnets:
resource virtualNetwork 'Microsoft.Network/virtualNetworks#2018-11-01' = {
name: vNetName
location: location
tags: tags
properties: {
addressSpace: {
addressPrefixes: [
addressPrefix
]
}
subnets: subnets
}
}
subnets is a variable of type array:
var subnets = [
{
name: mgmtSubnetName
properties: {
addressPrefix: mgmtSubnetAddressPrefix
}
}
{
name: intSubnetName
properties: {
addressPrefix: intSubnetAddressPrefix
}
}
{
name: extSubnetName
properties: {
addressPrefix: extSubnetAddressPrefix
}
}
{
name: vdmsSubnetName
properties: {
addressPrefix: vdmsSubnetAddressPrefix
}
}
]
When I use the output line below, it returns and array that has 4 objects...one for each subnet created:
output subnets array = virtualNetwork.properties.subnets
Each object has the format below:
{
"name":"<value>",
"id":"<value>",
"etag":"<value>",
"properties":{
"provisioningState":"Succeeded",
"addressPrefix":"<value>",
"ipConfigurations":[
{
"id":"<value>"
}
],
"delegations":[]
},
"type":"Microsoft.Network/virtualNetworks/subnets"
}
When I use the output line below, it returns the first object in the subnets array:
output subnet1 object = virtualNetwork.properties.subnets[0]
When I use the output line below, it returns the resourceId of the first subnet:
output subnet1 string = virtualNetwork.properties.subnets[0].id
I am unable to retrieve the other objects in the array using indexes 1, 2, or 3.
I have also tried the resourceId function (example below) but the behavior is exactly the same for indexes 1, 2, or 3:
output subnet1Id string = resourceId('Microsoft.Network/VirtualNetworks/subnets', name, subnets[0].name)
You can use the below bicep template to deploy the vnet and subnets and output the subnets and subnet id's like below :
var subnets = [
{
name: 'vm-subnet'
properties: {
addressPrefix:'10.0.0.0/24'
}
}
{
name: 'webapp-subnet'
properties: {
addressPrefix:'10.0.1.0/24'
}
}
{
name: 'appgw-subnet'
properties: {
addressPrefix:'10.0.2.0/24'
}
}
{
name: 'bastion-subnet'
properties: {
addressPrefix:'10.0.3.0/24'
}
}
]
resource virtualNetwork 'Microsoft.Network/virtualNetworks#2018-11-01' = {
name: 'ansuman-vnet'
location: 'east us'
properties: {
addressSpace: {
addressPrefixes: [
'10.0.0.0/16'
]
}
subnets: subnets
}
}
output subnets array = [for (name, i) in subnets :{
subnets : virtualNetwork.properties.subnets[i]
}]
output subnetids array = [for (name, i) in subnets :{
subnets : virtualNetwork.properties.subnets[i].id
}]
output subnetappgw string = virtualNetwork.properties.subnets[2].id
output webappsubnet object = virtualNetwork.properties.subnets[1]
Outputs:
Note: I am using the latest Bicep version i.e. Bicep CLI version 0.4.1124

Looping in terraform with multi-level Maps

I have a terraform question. I want to create a vnet with multiple subnets by reading in all the variable information in from a yaml file into locals. For instance:
vnets:
vnet1:
name: "core-euw"
location: "westeurope"
address_space: ["10.0.0.0/22"]
subnets:
- name: "adds"
address_prefixes: "10.0.0.64/27"
- name: "mgmt"
address_prefixes: "10.0.0.96/27"
vnet2:
name: "core-ussc"
location: "westeurope"
address_space: ["10.1.0.0/22"]
subnets:
- name: "adds"
address_prefixes: "10.1.0.64/27"
- name: "mgmt"
address_prefixes: "10.1.0.96/27"
In my .tf file, I want to create only one resource block for azurerm_virtual_network and loop on the number of vnets (vnet1, vnet2) in the yaml file. The code I have works great for creating the two vnets, but I can't seem to figure out how to parse the subnets: section under each vnet for creating the multiple subnets within it. My .tf file is below:
provider "azurerm" {
features {}
}
locals {
settings = yamldecode(file("./settings.yaml"))
}
resource "azurerm_resource_group" "rg-network" {
name = "rg-global-core-network"
location = "South Central US"
}
resource "azurerm_virtual_network" "vnets" {
for_each = local.settings.vnets
name = "vnet-${each.value["name"]}"
resource_group_name = azurerm_resource_group.rg-network.name
location = each.value["location"]
address_space = each.value["address_space"]
dynamic "subnet" {
for_each = local.settings.vnets.subnets
content {
name = each.value["name"]
address_prefix = each.value["address_prefixes"]
}
}
}
output "name" {
value = [for subnet in local.settings.vnets]
}
In your dynamic subnet block you should not use each. Instead it should be subnet:
dynamic "subnet" {
for_each = each.subnets
content {
name = subnet.value["name"]
address_prefix = subnet.value["address_prefixes"]
}
}

Unable to pass service annotations when deploying helm chart via terraform

I have seeing some examples regarding how to pass annotations when deploying a helm chart via terraform but none of then are working as expected, in this case, im trying to create a service assining a private ip on a specific subnet, but instead, its creating a public IP.
My terraform files:
locals {
helm_general = {
# Reference values
# https://github.com/elastic/helm-charts/blob/master/elasticsearch/values.yaml
elasticsearch = {
name = "elasticsearch"
chart = "elastic/elasticsearch"
tag = "7.14.0"
namespace = "elasticsearch"
set = [
{
name = "nodeSelector.agentpool"
value = "general"
},
{
name = "replicas"
value = "1"
},
{
name = "minimumMasterNodes"
value = "1"
},
{
name = "image"
value = "docker.elastic.co/elasticsearch/elasticsearch"
},
{
name = "imageTag"
value = "7.14.0"
},
{
name = "resources.requests.cpu"
value = "10m"
},
{
name = "resources.requests.memory"
value = "128Mi"
},
{
name = "volumeClaimTemplate.reosources.requests.storage"
value = "4Gi"
},
{
name = "persistence.enabled"
value = "false"
},
{
name = "service.type"
value = "LoadBalancer"
},
{
name = "service.annotations\\.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
value = "true"
},
{
name = "service.annotations\\.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal-subnet"
value = "somesubnet"
},
]
timeout = "900"
}
}
}
Helm deployment
resource "helm_release" "helm" {
provider = helm.general
for_each = local.helm_general
name = each.value.name
chart = each.value.chart
namespace = format(each.value.namespace)
dynamic "set" {
iterator = item
for_each = each.value.set == null ? [] : each.value.set
content {
name = item.value.name
value = item.value.value
}
}
depends_on = [kubernetes_namespace.general]
}
Plan / apply output
https://i.imgur.com/uAdmblM.png
And what is currently being deployed is a public ip instead of a private ip:
Namespace: elasticsearch
Labels: app=elasticsearch-master
app.kubernetes.io/managed-by=Helm
chart=elasticsearch
heritage=Helm
release=elasticsearch
Annotations: meta.helm.sh/release-name: elasticsearch
meta.helm.sh/release-namespace: elasticsearch
Selector: app=elasticsearch-master,chart=elasticsearch,release=elasticsearch
Type: LoadBalancer
IP Families: <none>
IP: xx
IPs: xxx
LoadBalancer Ingress: redacted public ip
Port: http 9200/TCP
TargetPort: 9200/TCP
NodePort: http 32083/TCP
Endpoints:
Port: transport 9300/TCP
TargetPort: 9300/TCP
NodePort: transport 32638/TCP
Endpoints:
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 1s service-controller Ensuring load balancer
Normal EnsuredLoadBalancer <invalid> service-controller Ensured load balancer
References that i have been following:
https://github.com/hashicorp/terraform-provider-helm/issues/125
https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release
Edit:
error message:
Error: unable to decode "": resource.metadataOnlyObject.ObjectMeta: v1.ObjectMeta.Annotations: ReadString: expects " or n, but found t, error found in #10 byte of ...|nternal":true},"labe|..., bigger context
..|beta.kubernetes.io/azure-load-balancer-internal":true},"labels":{"app":"elasticsearch-master","chart|...
with helm_release.helm["elasticsearch"],
on aks-general-helm.tf line 1, in resource "helm_release" "helm":
1: resource "helm_release" "helm" {
I just faced a similar issue, and here is what worked for me:
{
name = "service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
value = "true"
},
I think the issue is how it is concatenated. The service in the chart manifest for elastic official is service.annotations:{} so you need to append .service then use \\.
if you have multiple you can try something like this, pass annotations as map and set them using
dynamic "set" {
for_each = var.ingress_annotations
content {
name = replace(set.key, ".", "\\.")
value = set.value
}
}
One way of doing this without the escape characters and keeping the original YAML format would be using values attribute of the helm_release resource. Would be curious to know if there was a specific used-case to not use it in the first place.
resource "helm_release" "helm" {
provider = helm.general
for_each = local.helm_general
name = each.value.name
chart = each.value.chart
namespace = format(each.value.namespace)
values = each.value.values ##CHANGE IS HERE ##
dynamic "set" {
iterator = item
for_each = each.value.set == null ? [] : each.value.set
content {
name = item.value.name
value = item.value.value
}
}
depends_on = [kubernetes_namespace.general]
}
The local in your case would be adjusted to below, you can still keep set for something which has a dependency on any terraform resources or with any other logical reasons.
locals {
helm_general = {
# Reference values
# https://github.com/elastic/helm-charts/blob/master/elasticsearch/values.yaml
elasticsearch = {
[...]
values = [file("${path.module}/elasticsearch-values.yaml")]
[...]
}
}
}
There has to be a new file elasticsearch-values.yaml at the same path (which can be adjusted with any relative path as per the local.helm_general.elasticsearch.values) location, where this terraform configurations exist.
# Reference values can be adapted as per the upstream chart.
# https://github.com/elastic/helm-charts/blob/master/elasticsearch/values.yaml
service:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "somesubnet"

Resources