I'm trying to add values to an Azure App Configuration Store using Bicep. I have an issue where I add a label to a keyValue.
This is my module:
#description('Configuration Store Name')
param configurationStoreName string
#description('key prefix')
param prefix string
#description('key name')
param keyName string
#description('value')
param value string
#description('content type')
param contentType string = 'string'
#description('Deployment Environment')
param deploymentEnvironment string = 'dev'
resource configurationStore 'Microsoft.AppConfiguration/configurationStores#2021-10-01- preview' existing = {
name: configurationStoreName
}
resource configurationStoreValue 'Microsoft.AppConfiguration/configurationStores/keyValues#2021-10-01-preview' = {
name: '${prefix}:${keyName}'
parent: configurationStore
properties: {
contentType: contentType
value: value
tags: {
environment: deploymentEnvironment
}
}
}
There doesn't seem to be any way to add a label, which I want to do to enable filtering.
It can be done when creating KeyValues using the Azure Portal, therefore it should be possible using Bicep.
Am I missing something, or is this missing functionality from Bicep?
EDIT April 2022
The documentation has now been updated
The keyValues resource's name is a combination of key and label. The key and label are joined by the $ delimiter. The label is optional. In the above example, the keyValues resource with name myKey creates a key-value without a label.
Percent-encoding, also known as URL encoding, allows keys or labels to include characters that are not allowed in ARM template resource names. % is not an allowed character either, so ~ is used in its place. To correctly encode a name, follow these steps:
Apply URL encoding
Replace ~ with ~7E
Replace % with ~
For example, to create a key-value pair with key name AppName:DbEndpoint and label name Test, the resource name should be AppName~3ADbEndpoint$Test.
I tried this approach and it works:
#description('Configuration Store Name')
param configurationStoreName string
#description('key prefix')
param prefix string
#description('key name')
param keyName string
#description('value')
param value string
#description('label')
param label string
#description('content type')
param contentType string = 'string'
#description('Deployment Environment')
param deploymentEnvironment string = 'dev'
resource configurationStore 'Microsoft.AppConfiguration/configurationStores#2021-10-01-preview' existing = {
name: configurationStoreName
}
var keyValueName = empty(label) ? '${prefix}:${keyName}' : '${prefix}:${keyName}$${label}'
resource configurationStoreValue 'Microsoft.AppConfiguration/configurationStores/keyValues#2021-10-01-preview' = {
name: keyValueName
parent: configurationStore
properties: {
contentType: contentType
value: value
tags: {
environment: deploymentEnvironment
}
}
}
Related
I've got the following variable in a module:
variable "container_registries" {
type = list(object({
name = string
addl_keys = list(string)
namespaces = set(string)
hostnames = list(string)
username = string
password = string
}))
default = []
}
I'm feeding the module variable as such:
container_registries = [
{
name : "server.example.com"
addl_keys : ["config.json"]
namespaces : ["flux-system", "tekton"]
hostnames : ["cr-lts.server.example.com", "cr-test.server.example.com"]
username : "foo"
password : "bar"
}
]
Now I need to create multiple Kubernetes Secrets, each in different namespaces - but with the same content. I need the Secrets in the flux-system and tekton namespace. I need the secret to look like this:
apiVersion: v1
kind: Secret
type: kubernetes.io/dockerconfigjson
metadata:
name: server.example.com
data:
.dockerconfigjson: eyJhdXRocyI6eyJjci1sdHMuc2VydmVyLmV4YW1wbGUuY29tIjp7ImF1dGgiOiJabTl2T21KaGNnbz0ifSwiY3ItdGVzdC5zZXJ2ZXIuZXhhbXBsZS5jb20iOnsiYXV0aCI6IlptOXZPbUpoY2dvPSJ9fX0K
config.json: eyJhdXRocyI6eyJjci1sdHMuc2VydmVyLmV4YW1wbGUuY29tIjp7ImF1dGgiOiJabTl2T21KaGNnbz0ifSwiY3ItdGVzdC5zZXJ2ZXIuZXhhbXBsZS5jb20iOnsiYXV0aCI6IlptOXZPbUpoY2dvPSJ9fX0K
Note that the Secret has two different keys, each with the same value. The .dockerconfigjson key is mandatory when the Secret type is set to kubernetes.io/dockerconfigjson, so it should always be included. The value is a base64 encoded JSON and the JSON looks as such:
{
"auths": {
"cr-lts.server.example.com": {
"auth": "Zm9vOmJhcgo="
},
"cr-test.server.example.com": {
"auth": "Zm9vOmJhcgo="
}
}
}
The value of auth is foo:bar (the username and password) in base64.
I've been trying and trying, but I am not getting any closer. All my attempts have felt like garbage 😰 How in the world can I achieve this with Terraform? 😅
Here's what I did to solve the problem, using #Kreetchy's answer as a base:
locals {
container_registries = toset(flatten([
for cr in var.container_registries : [
for ns in cr.namespaces : format("%s/%s", ns, cr.name)
]
]))
container_registry_data = {
for cr in var.container_registries : cr.name => {
for key in toset(concat([".dockerconfigjson"], cr.addl_keys)) : key => jsonencode({
auths = {
for hostname in cr.hostnames : hostname => {
auth = base64encode("${cr.username}:${cr.password}")
}
}
})
}
}
}
resource "kubernetes_secret" "container_registry" {
for_each = local.container_registries
metadata {
namespace = split("/", each.value)[0]
name = split("/", each.value)[1]
}
type = "kubernetes.io/dockerconfigjson"
data = local.container_registry_data[split("/", each.value)[1]]
}
Note the extra local.container_registries which is used for the loop in the resource. This local stores the namespace/name of the secret in the list. The drawback is that two secrets by the same name and different content can not be created in two namespaces. It's something I can live with :-)
I also renamed var.keys to var.addl_keys, because a key by the name of .dockerconfigjson always must exist in a kubernetes.io/dockerconfigjson kind of Secret.
#Kreetchy: Please feel free to copy the code above and put it into your answer and I'll mark it as accepted. Once done, I will edit my question to adapt the requirement to fit the answer :-)
You could achieve it this way in Terraform
locals {
secret_data = {
for registry in var.container_registries : {
for key in registry.keys : key => base64encode(jsonencode({
auths = {
for hostname in registry.hostnames : hostname => {
auth = base64encode("${registry.username}:${registry.password}")
}
}
}))
}
}
}
resource "kubernetes_secret" "example" {
for_each = flatten([for registry in var.container_registries : registry.namespaces])
metadata {
name = lookup(var.container_registries[each.key].name, each.value)
namespace = each.value
}
type = "kubernetes.io/dockerconfigjson"
data = {
for key, value in local.secret_data : key => value
}
}
This will handle a list of varying amount of entries in the container_registries variable, and create a separate kubernetes_secret resource for each namespace. The contents of the secrets will still be the encoded JSON, but this time generated dynamically based on the number of keys and hostnames specified in each entry in the container_registries variable.
This handles any number of keys, namespaces and hosts dynamically and will create separate kuberneteds_secret resource for each namespace. Content of secret is still encoded JSON, but generated dynamically based on number of keys and hostnames specified in container_registries
I don't understand the logic of the following terraform code, and not sure, but I guess it might be me, but would appreciate some help with this.
So there's the following module https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/modules/definition/main.tf
resource azurerm_policy_definition def {
name = local.policy_name
display_name = local.display_name
description = local.description
policy_type = "Custom"
mode = var.policy_mode
management_group_id = var.management_group_id
metadata = jsonencode(local.metadata)
parameters = length(local.parameters) > 0 ? jsonencode(local.parameters) : null
policy_rule = jsonencode(local.policy_rule)
lifecycle {
create_before_destroy = true
}
timeouts {
read = "10m"
}
}
and https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/modules/definition/variables.tf
variable management_group_id {
type = string
description = "The management group scope at which the policy will be defined. Defaults to current Subscription if omitted. Changing this forces a new resource to be created."
default = null
}
variable policy_name {
type = string
description = "Name to be used for this policy, when using the module library this should correspond to the correct category folder under /policies/policy_category/policy_name. Changing this forces a new resource to be created."
default = ""
validation {
condition = length(var.policy_name) <= 64
error_message = "Definition names have a maximum 64 character limit, ensure this matches the filename within the local policies library."
}
}
variable display_name {
type = string
description = "Display Name to be used for this policy"
default = ""
validation {
condition = length(var.display_name) <= 128
error_message = "Definition display names have a maximum 128 character limit."
}
}
variable policy_description {
type = string
description = "Policy definition description"
default = ""
validation {
condition = length(var.policy_description) <= 512
error_message = "Definition descriptions have a maximum 512 character limit."
}
}
variable policy_mode {
type = string
description = "The policy mode that allows you to specify which resource types will be evaluated, defaults to All. Possible values are All and Indexed"
default = "All"
validation {
condition = var.policy_mode == "All" || var.policy_mode == "Indexed" || var.policy_mode == "Microsoft.Kubernetes.Data"
error_message = "Policy mode possible values are: All, Indexed or Microsoft.Kubernetes.Data (In Preview). Other modes are only allowed in built-in policy definitions, these include Microsoft.ContainerService.Data, Microsoft.CustomerLockbox.Data, Microsoft.DataCatalog.Data, Microsoft.KeyVault.Data, Microsoft.MachineLearningServices.Data, Microsoft.Network.Data and Microsoft.Synapse.Data"
}
}
variable policy_category {
type = string
description = "The category of the policy, when using the module library this should correspond to the correct category folder under /policies/var.policy_category"
default = null
}
variable policy_version {
type = string
description = "The version for this policy, if different from the one stored in the definition metadata, defaults to 1.0.0"
default = null
}
variable policy_rule {
type = any
description = "The policy rule for the policy definition. This is a JSON object representing the rule that contains an if and a then block. Omitting this assumes the rules are located in /policies/var.policy_category/var.policy_name.json"
default = null
}
variable policy_parameters {
type = any
description = "Parameters for the policy definition. This field is a JSON object that allows you to parameterise your policy definition. Omitting this assumes the parameters are located in /policies/var.policy_category/var.policy_name.json"
default = null
}
variable policy_metadata {
type = any
description = "The metadata for the policy definition. This is a JSON object representing additional metadata that should be stored with the policy definition. Omitting this will fallback to meta in the definition or merge var.policy_category and var.policy_version"
default = null
}
variable file_path {
type = any
description = "The filepath to the custom policy. Omitting this assumes the policy is located in the module library"
default = null
}
locals {
# import the custom policy object from a library or specified file path
policy_object = jsondecode(coalesce(try(
file(var.file_path),
file("${path.cwd}/policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.root}/policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.root}/../policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.module}/../../policies/${title(var.policy_category)}/${var.policy_name}.json")
)))
# fallbacks
title = title(replace(local.policy_name, "/-|_|\\s/", " "))
category = coalesce(var.policy_category, try((local.policy_object).properties.metadata.category, "General"))
version = coalesce(var.policy_version, try((local.policy_object).properties.metadata.version, "1.0.0"))
# use local library attributes if runtime inputs are omitted
policy_name = coalesce(var.policy_name, try((local.policy_object).name, null))
display_name = coalesce(var.display_name, try((local.policy_object).properties.displayName, local.title))
description = coalesce(var.policy_description, try((local.policy_object).properties.description, local.title))
metadata = coalesce(var.policy_metadata, try((local.policy_object).properties.metadata, merge({ category = local.category },{ version = local.version })))
parameters = coalesce(var.policy_parameters, try((local.policy_object).properties.parameters, null))
policy_rule = coalesce(var.policy_rule, try((local.policy_object).properties.policyRule, null))
# manually generate the definition Id to prevent "Invalid for_each argument" on set_assignment plan/apply
definition_id = var.management_group_id != null ? "${var.management_group_id}/providers/Microsoft.Authorization/policyDefinitions/${local.policy_name}" : azurerm_policy_definition.def.id
}
and an example how to use the module https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/examples/definitions.tf
module "deny_resource_types" {
source = "..//modules/definition"
policy_name = "deny_resource_types"
display_name = "Deny Azure Resource types"
policy_category = "General"
management_group_id = data.azurerm_management_group.org.id
}
From how I see it (might be wrrong) a variable can be used as a default value to the local in a Terraform script. So how is the value for the variable policy_name used when main.tf references local.policy_name instead of var.policy_name. The variable policy_name is also not having a default value.
What am I missing ?
Thank you !
EDIT:
Thank you, very clear explanation. I think I understand it better now. So basically, when I’m calling the definition module Terraform essentially load and process that module’s configuration files (including variables.tf). I was confused to see name = local.policy_name instead of for example mode = var.policy_mode. But the way I understand it now, is that when calling this module, I can set the value for the variable policy_name, which is then further processed inside the locals section, and result is what is actual provided to the name argument required by azurerm_policy_definition https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/policy_definition. Could you please confirm that my understanding is correct?
Thank you !
policy_name does have default value, but the value is empty.
default = ""
Empty value can be default value. Terraform expects module inputs only when there is no default attribute set in the input field.
OK, so there are two scenarios at play here.
When the policy_name isn't provided to the module, it takes on its default behaviour of empty string
When a value is provided to the policy_name, the locals section transforms that value and then uses it in the code as local.policy_name for the resources. L103
policy_name = coalesce(var.policy_name, try((local.policy_object).name, null))
If you look for the coalesce function, its purpose is to return the first element that is not null/empty.
Although, I don't see the point of that logic since if both cases are null it is supposed to return null. Could've used a simple condition instead.
I hope this clarifies things more.
Ps: an empty string (""), Terraform consider as a value.
I'm setting up a terraform repo for my snowflake instance and bringing in a list of users to start managing.
I have a module called users
and have the following files:
I have a variable defined as follows.
variable "users" {
type = list(object(
{
name = string
comment = string
default_role = string
disabled = bool
must_change_password = bool
display_name = string
email = string
first_name = string
last_name = string
default_warehouse = string
}
)
)
}
now inside users.tf I want to hold a list of all my users based on the above variable, I thought I could define it as follows:
users {
user_1 = {
name = 'x'
},
user_2 = {
name = 'y'
}
}
however, when I run Terraform validate on this it gives me the error that a user block is not expected here.
Can someone tell me my error and give me some guidance if I'm doing this correctly?
My intention is to have a file to hold all my users that I then define with a dynamic block inside my main.tf file within this module.
I can then reference the dynamic block inside the outputs.tf which will give me access to the users inside said module in the global project namespace.
Looks to me like you are attempting to configuring your users as an object:
users {
user_1 = {
name = "x"
},
user_2 = {
name = "y"
}
}
but you actually set your variable constraint to a list of objects. So it should be:
users = [
{
name = "user_1"
# other fields
},
{
name = "user_2"
# other fields
}
]
Here is a full working example:
modules/users/variables.tf
variable "users" {
type = list(object({
name = string
}))
}
modules/users/outputs.tf
output "users" {
value = var.users
}
main.tf
module "users" {
source = "./modules/users"
users = [
{ name = "user_1" },
{ name = "user_2" }
]
}
output "users" {
value = module.users.users
}
plan output
Changes to Outputs:
+ users = [
+ {
+ name = "user_1"
},
+ {
+ name = "user_2"
},
]
Your config syntax and usage is completely correct here. Your config file organization is the issue here. users.tf is a Terraform variables file, and therefore should have the .tfvars extension. If you rename the file from users.tf to e.g. users.tfvars, then you can specify it as an input with the -var-file=users.tfvars argument with the CLI or otherwise as per standard usage. You can see more information in the documentation.
On a side note: it is not really best practices to manage an entire module just for managing a set of users for a specific service. If you follow this design pattern in the future, then your codebase will not scale very well, and could easily become unmanageably large.
I have a list of objects from a variable in terraform
variable "persons" {
type = list(object({
name = string,
phonenumber = string,
tshirtSize = string
}))
description = "List of person"
}
Now I want a list of the person's names so I can use it to define an AWS Resource
How can I convert this object list to a list of names
["bob", "amy", "jane"]
I'm on terraform 0.12.24, though can upgrade if needed
Updated Answer:
Use the splat expression
var.persons[*].name
https://www.terraform.io/docs/configuration/expressions.html#splat-expressions
Original Answer:
I was able to do this in locals file
locals {
names = [
for person in var.persons:
person.name
]
}
For additional reading
SEE: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each/
I have a four different environments:
dev
sit
uat
prod
The parameter group values differ for each environment. Below value will go in variable file for each environment.
Below are the parameters of an example AWS RDS parameter group. In other environments, there may be more or less parameters:
parameter {
name = "character_set_client"
value = "utf8mb4"
}
parameter {
name = "character_set_connection"
value = "utf8mb4"
}
parameter {
name = "character_set_server"
value = "utf8mb4"
}
parameter {
name = "log_bin_trust_function_creators"
value = "1"
}
I'm curious how to represent this in variables and how to write the resources to use those variables to make RDS Parameter Groups with an arbitrary number of parameters.
How do I create RDS parameter groups from an arbitrary number of input parameters?
You can use a dynamic block:
dynamic "parameter" {
for_each = var.parameters
content {
name = parameter.value.name
value = parameter.value.value
}
}
The variable can be a list of maps:
variable "parameters" {
type = list(map(string))
default = []
}
terraform.tfvars:
parameters = [
{
name = "character_set_connection"
value = "utf8mb4"
},
{
name = "character_set_server"
value = "utf8mb4"
},
{
name = "log_bin_trust_function_creators"
value = "1"
}
]