Using terraform variable in hcl extention ( vault ) - terraform

I'am tring to do automation for path and policies creation in vault.
Do you know how I can proceed please ? variables declared in terraform are not reconized in .hcl file.
I tried to rename my file client-ro-policy.hcl to client-ro-policy.tf but I have same issue
Varibales is recognized in file with .tf extention
Thanks
main.tf
# Use Vault provider
provider "vault" {
# It is strongly recommended to configure this provider through the
# environment variables:
# - VAULT_ADDR
# - VAULT_TOKEN
# - VAULT_CACERT
# - VAULT_CAPATH
# - etc.
}
acl-ro-policy.hcl
path "${var.client[0]}/k8s/preprod/*" {
capabilities = ["read"]
}
policies.tf
#---------------------
# Create policies
#---------------------
# Create 'client' policy
resource "vault_policy" "ro-client" {
name = "${var.client[0]}_k8s_preprod_ro"
policy = file("./hcl-ro-policy.tf")
}
variables.tf
variable "client" {
type = list(string)
}
variables.tfvars
client = ["titi", "toto","itutu"]
Result in vault:

Even though Terraform and Vault both use HCL as the underlying syntax of their respective configuration languages, their language interpreters are totally separate and so the Vault policy language implementation cannot make direct use of any values defined in the Terraform language.
Instead, you'll need to use the Terraform language to construct a suitable configuration for Vault. Vault supports a JSON variant of its policy language in order to make it easier to programmatically generate it, and so you can use Terraform's jsonencode function to build a JSON-based policy from the result of a Terraform expression, which may itself include references to values elsewhere in Terraform.
For example:
locals {
vault_ro_policy = {
path = {
"${var.client[0]}/k8s/preprod/*" = {
capabilities = ["read"]
}
}
}
}
resource "vault_policy" "ro-client" {
name = "${var.client[0]}_k8s_preprod_ro"
policy = jsonencode(local.var_ro_policy)
}
The value of local.vault_ro_policy should encode to JSON as follows, assuming that var.client[0] has the value "example":
{
"path": {
"example/k8s/preprod/*": {
"capabilities": ["read"]
}
}
}
Assuming that this is valid Vault JSON policy syntax (which I've not verified), this should be accepted by Vault as a valid policy. If I didn't get the JSON policy syntax exactly right then hopefully you can see how to adjust it to be valid; my expertise is with Terraform, so I focused on the Terraform language part here.

Related

Using azure azurerm_resource_provider_registration with newly created subscription

I am working on creating Azure landing zone and part of that is to enable/disable resource providers on the newly created subscriptions.
I have tried to used alias with a variable but i am getting error that i cant use variable in an alias so is there any way through which i can use this feature on multiple subscription
This is my code main.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.0.0"
}
}
}
#list of providers i want to register
locals {
# List is compiled from here
# https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-services-resource-providers
provider_list = [
"Microsoft.Storage"
]
provider_map = { for p in local.provider_list : p => p }
}
# Registering a default provider here and skipping registration
# as i will do it later
provider "azurerm" {
features {}
skip_provider_registration = true
}
# I am creating a subscription here with same alias as the name
# the subscription is being created under and EA enrollment but
# any type of subscription will do
resource "azurerm_subscription" "feature_subscription" {
billing_scope_id = "/providers/Microsoft.Billing/billingAccounts/xxx/enrollmentAccounts/xx"
alias = var.temp_alias # "test-provider-registration"
subscription_name = "test-provider-registration"
}
#this is what i have created to point out my azurerm_resource_provider_registration
#module i am using variable in alias which is failing
provider "azurerm" {
alias = var.temp_alias
subscription_id = azurerm_subscription.feature_subscription.id
features {
}
skip_provider_registration = true
}
#module through which i am registering the resource providers
module "azurerm_resource_provider_registration-provider" {
source = "../modules/azurerm_resource_provider_registration"
providers = {
azurerm = azurerm.test-provider-registration
}
feature_list = local.provider_map
}
#the module code is mentioned here
#resource "azurerm_resource_provider_registration" "provider" {
# for_each = var.feature_list
# name = each.value
#}
I am getting this error when i run it
There are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
╷
Error: Variables not allowed
│
On main.tf line 25: Variables may not be used here.
╵
╷
Error: Unsuitable value type
│
On main.tf line 25: Unsuitable value: value must be known
There is a workaround available like using this
resource "null_resource" "provider_registration" {
for_each = local.provider_map
provisioner "local-exec" {
command = "az account set --subscription ${data.azurerm_subscription.subscription.subscription_id} && az provider register --namespace ${each.value}"
}
}
but i want to use the state file for the resource registration if possible as i have more subscriptions in a loop
Error: Unsuitable value type | Unsuitable value: value must be known
Need to check:
This problem usually occurs with module sources or versions. When invoking a module, using variables instead of passing direct values for the source and some other arguments causes this error. terraform initdoesn't take variable inputs with modules in backend state configuration.
Note: And also include required version for modules.
To make it work, use direct values for module block instead of accessing from other resources.
Pass provider explicitly to make it work as detailed in terraform providers.
providers = {
azurerm = azurerm.test-provider-registration
}
feature_list = local.provider_map
}
}
module "azurerm_resource_provider_registration-provider" {
source ="../modules/azurerm_resource_provider_registration"
version = <required version> //Use latest one
After checking the above conditions, I tried same in my environment and it was initialized successfully:
Refer aliasing for the deployment in multiple subscriptions detailed by #Jeff Brown

How to generate file and re-use its content with terrafrom

I would like to generate ssh keys with a local-exec then read the content of the file.
resource "null_resource" "generate-ssh-keys-pair" {
provisioner "local-exec" {
command = <<EOT
ssh-keygen -t rsa -b 4096 -C "test" -P "" -f "testkey"
EOT
}
}
data "local_file" "public-key" {
depends_on = [null_resource.generate-ssh-keys-pair]
filename = "testkey.pub"
}
data "local_file" "private-key" {
depends_on = [null_resource.generate-ssh-keys-pair]
filename = "testkey"
}
terraform plan works but when I run the apply, I got error on testkey and testkey.pub don't exist.
Thanks
Instead of generating a file using an external command and then reading it in, I would suggest to use the Terraform tls provider to generate the key within Terraform itself, using tls_private_key:
terraform {
required_providers {
tls = {
source = "hashicorp/tls"
}
}
}
resource "tls_private_key" "example" {
algorithm = "RSA"
rsa_bits = 4096
}
The tls_private_key resource type exports two attributes that are equivalent to the two files you were intending to read in your example:
tls_private_key.example.private_key_pem: the private key in PEM format
tls_private_key.example.public_key_openssh: the public key in the format OpenSSH expects to find in .ssh/authorized_keys.
Please note the warning in the tls_private_key documentation that using this resource will cause the private key data to be saved in your Terraform state, and so you should protect that state data accordingly. That would also have been true for your approach of reading files from disk using data resources, because any value Terraform has available for use in expressions must always be stored in the state.
I run your code and there are no problems with it. It correctly generates testkey and testkey.pub.
So whatever causes it to fail for you, its not this snipped of code you've provided in the question. The fault must be outside the code snipped.
Generating an SSH key in Terraform is inherently insecure because it gets stored in the tfstate file, however I had a similar problem to solve and thought that the most secure/usable was to use a secret management service + using a cloud bucket for the backend:
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "4.3.0"
}
}
backend "gcs" {
bucket = "tfstatebucket"
prefix = "terraform/production"
}
}
//Import Ansible private key from Google Secrets Manager
resource "google_secret_manager_secret" "ansible_private_key" {
secret_id = var.ansible_private_key_secret_id
replication {
automatic = true
}
}
data "google_secret_manager_secret_version" "ansible_private_key"{
secret = google_secret_manager_secret.ansible_private_key.secret_id
version = var.ansible_private_key_secret_id_version_number
}
resource "local_file" "ansible_imported_local_private_key" {
sensitive_content = data.google_secret_manager_secret_version.ansible_private_key.secret_data
filename = var.ansible_imported_local_private_key_filename
file_permission = "0600"
}
In the case of GCP, I would add the secret in Google Secrets Manager, then use terraform import on the secret, which will in turn write it to the backend bucket. That way it doesn't get stored in Git as plain text, you can have the key file local to your project (.terraform shouldn't be under version control), and it's arguably more secure in the bucket.
So the workflow essentially is:
Human --> Secret Manager
Secret Manager --> Terraform Import --> GCS Bucket Backend
|--> Create .terraform/ssh_key
.terraform/ssh_key --> Terraform/Ansible/Whatever
Hashicorp Vault would be another way to address this

Combine terraform locals variables

Can you guide me on how to combine locals? This is not working though some online docs advise to frame this way. Appreciate your help, thanks!
# see README.md for developer guide
# prepare subscription where resources are created
locals {
location_code = "weu"
environment_code = "test"
}
locals {
kv_name = "oamp-kv-${local.environment_code}-${location_code}"
ai_name = "oamp-ai-${local.environment_code}-${location_code}"
}
# prepare azure rm configuration
provider "azurerm" {
version = "~>2.15.0"
use_msi = true
features {}
}
Validation
Error: Invalid reference
on main.tf line 20, in locals:
20: kv_name = "oamp-kv-${local.environment_code}-${location_code}"
A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.
You can reference local variables into locals as it is written in the documentation.
As shown above, local values can be referenced from elsewhere in the
module with an expression like local.common_tags, and locals can
reference each other in order to build more complex values from
simpler ones.
The error come from the fact you need to prefix with an attribute
access your resource. However you did not prefix location_code.
In you code you miss to prefix access attribute local before location_code.
What you need to do is prefix correctly your variables :
# see README.md for developer guide
# prepare subscription where resources are created
locals {
location_code = "weu"
environment_code = "test"
}
locals {
kv_name = "oamp-kv-${local.environment_code}-${local.location_code}"
ai_name = "oamp-ai-${local.environment_code}-${local.location_code}"
}
# prepare azure rm configuration
provider "azurerm" {
version = "~>2.15.0"
use_msi = true
features {}
}

Reference multiple aws_instance in terraform for template output

We want to deploy services into several regions.
Looks like because of the aws provider, we can't just use count or for_each, as the provider can't be interpolated. Thus I need to set this up manually:
resource "aws_instance" "app-us-west-1" {
provider = aws.us-west-1
#other stuff
}
resource "aws_instance" "app-us-east-1" {
provider = aws.us-east-1
#other stuff
}
I would like when running this to create a file which contains all the IPs created (for an ansible inventory).
I was looking at this answer:
https://stackoverflow.com/a/61788089/169252
and trying to adapt it for my case:
resource "local_file" "app-hosts" {
content = templatefile("${path.module}/templates/app_hosts.tpl",
{
hosts = aws_instance[*].public_ip
}
)
filename = "app-hosts.cfg"
}
And then setting up the template accordingly.
But this fails:
Error: Invalid reference
on app.tf line 144, in resource "local_file" "app-hosts":
122: hosts = aws_instance[*].public_ip
A reference to a resource type must be followed by at least one attribute
access, specifying the resource name
I am suspecting that I can't just reference all the aws_instance defined as above like this. Maybe to refer to all aws_instance in this file I need to use a different syntax.
Or maybe I need to use a module somehow. Can someone confirm this?
Using terraform v0.12.24
EDIT: The provider definitions use alias and it's all in the same app.tf, which I was naively assuming to be able to apply in one go with terraform apply (did I mention I am a beginner with terraform?):
provider "aws" {
alias = "us-east-1"
region = "us-east-1"
}
provider "aws" {
alias = "us-west-1"
region = "us-west-1"
}
My current workaround is to not do a join but simply listing them all individually:
{
host1 = aws_instance.app-us-west-1.public_ip
host2 = aws_instance.app-us-east-1.public_ip
# more hosts
}

How to use dynamic resource names in Terraform?

I would like to use the same terraform template for several dev and production environments.
My approach:
As I understand it, the resource name needs to be unique, and terraform stores the state of the resource internally. I therefore tried to use variables for the resource names - but it seems to be not supported. I get an error message:
$ terraform plan
var.env1
Enter a value: abc
Error asking for user input: Error parsing address 'aws_sqs_queue.SqsIntegrationOrderIn${var.env1}': invalid resource address "aws_sqs_queue.SqsIntegrationOrderIn${var.env1}"
My terraform template:
variable "env1" {}
provider "aws" {
region = "ap-southeast-2"
}
resource "aws_sqs_queue" "SqsIntegrationOrderIn${var.env1}" {
name = "Integration_Order_In__${var.env1}"
message_retention_seconds = 86400
receive_wait_time_seconds = 5
}
I think, either my approach is wrong, or the syntax. Any ideas?
You can't interpolate inside the resource name. Instead what you should do is as #BMW have mentioned in the comments, you should make a terraform module that contains that SqsIntegrationOrderIn inside and takes env variable. Then you can use the module twice, and they simply won't clash. You can also have a look at a similar question I answered.
I recommend using a different workspace for each environment. This allows you to specify your configuration like this:
variable "env1" {}
provider "aws" {
region = "ap-southeast-2"
}
resource "aws_sqs_queue" "SqsIntegrationOrderIn" {
name = "Integration_Order_In__${var.env1}"
message_retention_seconds = 86400
receive_wait_time_seconds = 5
}
Make sure to make the name of the "aws_sqs_queue" resource depending on the environment (e.g. by including it in the name) to avoid name conflicts in AWS.

Resources