I am trying to access the Kubernetes_secret data.token attribute in terraform, but I keep on getting the error
Resource 'data.kubernetes_secret.misp_whitelist_secret' does not have attribute 'data.token' for variable 'data.kubernetes_secret.misp_whitelist_secret.data.token'
Whats the way to resolve this issue?
resource "kubernetes_service_account" "misp_whitelist_sa" {
metadata {
name = "misp-whitelist-sa"
}
}
data "kubernetes_secret" "misp_whitelist_secret" {
metadata {
name = "${kubernetes_service_account.misp_whitelist_sa.default_secret_name}"
namespace = "${kubernetes_service_account.misp_whitelist_sa.metadata.0.namespace}"
}
depends_on = [
"kubernetes_service_account.misp_whitelist_sa",
]
}
And I'm trying to access the data.token inside the terraform google_cloud_function resource
resource "google_cloudfunctions_function" "misp_whitelist_function" {
name = "${var.cluster}-misp-whitelist"
....<additional data> .....
environment_variables = {
CLUSTER = "${var.cluster}"
PROJECT = "${var.project}"
AUTH = "${data.kubernetes_secret.misp_whitelist_secret.data.token}"
}
}
The correct way to access the data secret key is:
AUTH = "${data.kubernetes_secret.misp_whitelist_secret.data["token"]}"
Ok banged my head against a wall here for a really long time. The other answer is correct, but skips a crucial step.
You need to make sure that the secret declares the correct type (and also maybe specify the annotation?)
resource "kubernetes_secret" "vault" {
metadata {
name = "vault-token"
annotations = {
"kubernetes.io/service-account.name" = "vault"
}
}
type = "kubernetes.io/service-account-token" // THIS!
}
Then, once you have the proper type specified, you can use the token
output "token" {
value = kubernetes_secret.vault.data.token
}
Related
I've got the following variable in a module:
variable "container_registries" {
type = list(object({
name = string
addl_keys = list(string)
namespaces = set(string)
hostnames = list(string)
username = string
password = string
}))
default = []
}
I'm feeding the module variable as such:
container_registries = [
{
name : "server.example.com"
addl_keys : ["config.json"]
namespaces : ["flux-system", "tekton"]
hostnames : ["cr-lts.server.example.com", "cr-test.server.example.com"]
username : "foo"
password : "bar"
}
]
Now I need to create multiple Kubernetes Secrets, each in different namespaces - but with the same content. I need the Secrets in the flux-system and tekton namespace. I need the secret to look like this:
apiVersion: v1
kind: Secret
type: kubernetes.io/dockerconfigjson
metadata:
name: server.example.com
data:
.dockerconfigjson: eyJhdXRocyI6eyJjci1sdHMuc2VydmVyLmV4YW1wbGUuY29tIjp7ImF1dGgiOiJabTl2T21KaGNnbz0ifSwiY3ItdGVzdC5zZXJ2ZXIuZXhhbXBsZS5jb20iOnsiYXV0aCI6IlptOXZPbUpoY2dvPSJ9fX0K
config.json: eyJhdXRocyI6eyJjci1sdHMuc2VydmVyLmV4YW1wbGUuY29tIjp7ImF1dGgiOiJabTl2T21KaGNnbz0ifSwiY3ItdGVzdC5zZXJ2ZXIuZXhhbXBsZS5jb20iOnsiYXV0aCI6IlptOXZPbUpoY2dvPSJ9fX0K
Note that the Secret has two different keys, each with the same value. The .dockerconfigjson key is mandatory when the Secret type is set to kubernetes.io/dockerconfigjson, so it should always be included. The value is a base64 encoded JSON and the JSON looks as such:
{
"auths": {
"cr-lts.server.example.com": {
"auth": "Zm9vOmJhcgo="
},
"cr-test.server.example.com": {
"auth": "Zm9vOmJhcgo="
}
}
}
The value of auth is foo:bar (the username and password) in base64.
I've been trying and trying, but I am not getting any closer. All my attempts have felt like garbage 😰 How in the world can I achieve this with Terraform? 😅
Here's what I did to solve the problem, using #Kreetchy's answer as a base:
locals {
container_registries = toset(flatten([
for cr in var.container_registries : [
for ns in cr.namespaces : format("%s/%s", ns, cr.name)
]
]))
container_registry_data = {
for cr in var.container_registries : cr.name => {
for key in toset(concat([".dockerconfigjson"], cr.addl_keys)) : key => jsonencode({
auths = {
for hostname in cr.hostnames : hostname => {
auth = base64encode("${cr.username}:${cr.password}")
}
}
})
}
}
}
resource "kubernetes_secret" "container_registry" {
for_each = local.container_registries
metadata {
namespace = split("/", each.value)[0]
name = split("/", each.value)[1]
}
type = "kubernetes.io/dockerconfigjson"
data = local.container_registry_data[split("/", each.value)[1]]
}
Note the extra local.container_registries which is used for the loop in the resource. This local stores the namespace/name of the secret in the list. The drawback is that two secrets by the same name and different content can not be created in two namespaces. It's something I can live with :-)
I also renamed var.keys to var.addl_keys, because a key by the name of .dockerconfigjson always must exist in a kubernetes.io/dockerconfigjson kind of Secret.
#Kreetchy: Please feel free to copy the code above and put it into your answer and I'll mark it as accepted. Once done, I will edit my question to adapt the requirement to fit the answer :-)
You could achieve it this way in Terraform
locals {
secret_data = {
for registry in var.container_registries : {
for key in registry.keys : key => base64encode(jsonencode({
auths = {
for hostname in registry.hostnames : hostname => {
auth = base64encode("${registry.username}:${registry.password}")
}
}
}))
}
}
}
resource "kubernetes_secret" "example" {
for_each = flatten([for registry in var.container_registries : registry.namespaces])
metadata {
name = lookup(var.container_registries[each.key].name, each.value)
namespace = each.value
}
type = "kubernetes.io/dockerconfigjson"
data = {
for key, value in local.secret_data : key => value
}
}
This will handle a list of varying amount of entries in the container_registries variable, and create a separate kubernetes_secret resource for each namespace. The contents of the secrets will still be the encoded JSON, but this time generated dynamically based on the number of keys and hostnames specified in each entry in the container_registries variable.
This handles any number of keys, namespaces and hosts dynamically and will create separate kuberneteds_secret resource for each namespace. Content of secret is still encoded JSON, but generated dynamically based on number of keys and hostnames specified in container_registries
I'm trying to achieve (maybe by wrong means) something like that. I'd like to be able to create few types of endpoints in Azure (KV, SA for example).
module "endpoints" {
source = "./modules/private_endpoint"
for_each = toset(var.endpoint_type)
private_connection_resource_id = "var.${each.value}.private_connection_resource_id"
Where:
Endpoint_type is a list of endpoints (its value is "storage_account"),
private_connection_resource_id is in map(any) which looks like (there are other values, but I don't think they're important at this point):
storage_account = {
private_connection_resource_id = #VALUE
...
private_connection_resource_id = "var.${each.value}.private_connection_resource_id" --- this gets translated to literal string (var.storage_account.private_connection_resource_id), where I'd like it to get translated to exact value - the id of storage account (it's hardcoded in tfvars).
Thank you in advance for any tips!
Edit: It appears that Im as dumb as they come. Should've changed the map a bit:
endpoint_type = {
storage_account = {
private_connection_resource_id = #VALUE
...
And ref in module calling to: each.value.private_connection_resource_id
You cannot construct an expression inside a string and then evaluate it. Terraform always parses the full configuration first and then executes it once already parsed.
If you want to look up values dynamically based on a key then a map is the most appropriate data structure to use for that purpose. For example, you could define a input variables endpoint_types and endpoints like this:
variable "endpoint_types" {
type = map(object({
private_connection_resource_id = string
}})
}
variable "endpoints" {
type = map(object({
type = string
}))
}
My intent with the above example is that the type attribute inside the endpoints objects is a lookup key for the other map in endpoint_types.
When you then define your module block with for_each you will first refer to var.endpoints and then look up an appropriate endpoint type in var.endpoint_types based on its selected key:
module "endpoints" {
source = "./modules/private_endpoint"
for_each = var.endpoints
private_connection_resource_id = var.endpoint_types[each.value.type].private_connection_resource_id
}
The user of the outer module would need to provide both a map of endpoints and a map describing all of the possible types those endpoints might have, like this:
endpoints = {
storage_1 = {
type = "storage"
}
storage_2 = {
type = "storage"
}
other_1 = {
type = "other"
}
}
endpoint_types = {
storage = {
private_connection_resource_id = "something"
}
other = {
private_connection_resource_id = "something_else"
}
}
I'm trying to create the following block dynamically based on a list of strings
env {
name = "SECRET_ENV_VAR"
value_from {
secret_key_ref {
name = google_secret_manager_secret.secret.secret_id
key = "1"
}
}
}
Based off documentation: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_service#example-usage---cloud-run-service-secret-environment-variables
I would like to dynamically add Secrets, and have defined the following dynamic block:
dynamic "env" {
for_each = toset(var.secrets)
content {
name = each.value
value_from {
secret_key_ref {
name = each.value
key = "1"
}
}
}
}
Where secrets is a variable of type list(string)
However, this throws an error: Blocks of type "value_from" are not expected here.
I'm not sure what I'm missing, or where I have incorrectly specified the value_from block.
Could someone point me in the right direction for fixing this up?
UPDATE;
I have also tried to implement this variable as a map, as per the suggestion in the comments on this post. (https://www.terraform.io/docs/language/expressions/dynamic-blocks.html#multi-level-nested-block-structures)
dynamic "env" {
for_each = var.secrets
content {
name = each.key
dynamic "value_from" {
for_each = env.value.name
secret_key_ref {
name = value_from.value.name
key = value_from.value.version
}
}
}
}
However, this also gives the same error. Blocks of type "value_from" are not expected here.
In this example, the secrets variable is defined as a list(any) with this value:
secrets = [
{
name = "SECRET"
version = "1"
}
]
You have to upgrade your gcp provider. Support for secrets in google_cloud_run_service was added in v3.67.0. Current version is v4.1.0, which means that you must be using very old gcp provider.
In the end, I solved this by changing the variable type to a map(any):
secrets = {
"SECRET" = "1"
}
This allowed me to create the "dynamic" env block, without needing to implement the nested dynamic block.
I am trying to figure this one out and struggling to get it right.
So i have 3 keyvaults, 1 for each region, US, Europe, Asia so i am passing this via data blocks
data "azurerm_key_vault" "existing" {
name = "mykeyvault"
resource_group_name = "myrg"
}
data "azurerm_key_vault_secret" "userlist1" {
name = "secret1"
key_vault_id = "${data.azurerm_key_vault.existing.id}"
}
data "azurerm_key_vault_secret" "userlist2"{
name = "secret2"
key_vault_id = "${data.azurerm_key_vault.existing.id}"
}
output "secret_value1" {
value = "${data.azurerm_key_vault_secret.userlist1.value}"
}
output "secret_value2" {
value = "${data.azurerm_key_vault_secret.userlist2.value}"
}
now what am struggling to put together is, if my pipeline is set to run on region = europe, how can i pass the secret value below?
module "testmod" {
source = "./test
password = "${data.azurerm_key_vault_secret.**IFREGIONISEUROPETHENPASSTHISSECRET**.value}"
}
First off, the approach you have planned to use puts your secrets in your statefile, so make sure your statefile is encrypted and well secured.
Because you want to dynamically reference the values, what you want is probably a data structure like an object, rather than a sequence of variables.
Collect your values in an object using locals:
locals {
secret_by_region = {
"asia" = data.azurerm_key_vault_secret.userlist1.value,
"europe" = data.azurerm_key_vault_secret.userlist2.value
}
}
Then reference the value from the local variable:
module "testmod" {
source = "./test
password = local.secret_by_region[var.region]
}
I'm using the HTTP data source to retrieve data from an internal service. The service returns JSON data.
I can't interpolate the returned JSON data and look up data in it.
For example:
module A
data "http" "json_data" {
url = "http://myservice/jsondata"
# Optional request headers
request_headers {
"Accept" = "application/json"
}
}
output "json_data_key" {
value = "${lookup(data.http.json_data.body, "mykey")}"
}
main.tf
provider "aws" {
region = "${var.region}"
version = "~> 0.1"
}
module "moduleA" {
source = "../../../terraform-modules/moduleA"
}
resource "aws_instance" "example" {
ami = "ami-2757f631"
instance_type = "${module.moduleA.json_data_key}"
}
The lookup function will fail to extract the key within the JSON data.
Is there any way to decode the JSON data into a terraform map ?
variable "json" {
default = "{\"foo\": \"bar\"}"
}
data "external" "json" {
program = ["echo", "${var.json}"]
}
output "map" {
value = "${data.external.json.result}"
}
Not directly related to map convertion, but here's an additional sample with jsondecode if you got a multi-value secret (=JSON) in AWS SecretsManager & you want to use separate values from it in another service as I've struggled with this.
Retrieving the secret:
data "aws_secretsmanager_secret" "oauth_client" {
name = "oauth-client"
}
data "aws_secretsmanager_secret_version" "oauth_client" {
secret_id = data.aws_secretsmanager_secret.oauth_client.id
}
Using it at Lambda, as an example:
resource "aws_lambda_function" "lambda" {
[...]
environment {
variables = {
OAUTH_CLIENT_ID = jsondecode(data.aws_secretsmanager_secret_version.oauth_client.secret_string)["client_id"]
OAUTH_CLIENT_SECRET = jsondecode(data.aws_secretsmanager_secret_version.oauth_client.secret_string)["client_secret"]
}
}
}
Ok, so it seems to be that the way to do the is by using the external data, as it return a map from a json response.
https://www.terraform.io/docs/providers/external/data_source.html
terraform version v0.10.6
Since 0.12 version of the Terraform you can use jsondecode function to decode json into a Terraform map. More details on: https://www.terraform.io/docs/configuration/functions/jsondecode.html
example from the page above:
> jsondecode("{\"hello\": \"world\"}")
{
"hello" = "world"
}
> jsondecode("true")
true