I have a terraform config which creates an AWS IAM user with an access key, and I assign both id and secret to output variables:
...
resource "aws_iam_access_key" "brand_new_user" {
user = aws_iam_user.brand_new_user.name
}
output "brand_new_user_id" {
value = aws_iam_access_key.brand_new_user.id
}
output "brand_new_user_secret" {
value = aws_iam_access_key.brand_new_user.encrypted_secret
sensitive = true
}
Here brand_new_user_secret is declared as sensitive, so terraform output obviously does not print it.
Is there any way to get its output value without parsing the whole state file?
Trying to access it directly (terraform output brand_new_user_secret) does not work (results in an error "The output variable requested could not be found in the state file...").
Terraform version: 0.12.18
I had some hopes to avoid it, but so far I did not find a better way than parse terraform state:
terraform state pull | jq '.resources[] | select(.type == "aws_iam_access_key") | .instances[0].attributes'
which would result in a structure similar to:
{
"encrypted_secret": null,
"id": "....",
"key_fingerprint": null,
"pgp_key": null,
"secret": "....",
"ses_smtp_password": "....",
"ses_smtp_password_v4": null,
"status": "Active",
"user": "...."
}
To see the sensitive value interactively, i.e. for the purposes of analyzing/debugging the state, you can use the Terraform's console command and nonsensitive() function:
$ terraform console
> nonsensitive(aws_iam_access_key.brand_new_user.encrypted_secret)
You may need to use other functions to decode/manipulate the value before printing it.
The output command does print a sensitive output if a single output is specified. This is using Terraform v1.1.7 and 0.14.8
terraform output brand_new_user_secret
The response I get when not specifying an attribute name is:
brand_new_user_secret = (sensitive)
whereas the question said this produced an error, so the behaviour may have changed since 0.12.
See https://www.terraform.io/cli/commands/output
I'm using a hacky workaround here like this...
resource "aws_iam_access_key" "brand_new_user" {
user = aws_iam_user.brand_new_user.name
}
output "brand_new_user_id" {
value = aws_iam_access_key.brand_new_user.id
}
data "template_file" "secret" {
template = aws_iam_access_key.brand_new_user.encrypted_secret
}
output "brand_new_user_secret" {
value = data.template_file.secret.rendered
}
I haven't tried it, but the docs seem to suggest that if you want to output encrypted_secret you must supply a pgp_key to the aws_iam_access_key resource:
pgp_key - (Optional) Either a base-64 encoded PGP public key, or a keybase username in the form keybase:some_person_that_exists, for use in the encrypted_secret output attribute.
encrypted_secret - The encrypted secret, base64 encoded, if pgp_key was specified. ~> NOTE: The encrypted secret may be decrypted using the command line, for example: terraform output encrypted_secret | base64 --decode | keybase pgp decrypt.
https://www.terraform.io/docs/providers/aws/r/iam_access_key.html
I think the recommended way to do this with modern Terraform (v1.1+) is to use terraform output -json [<name>]. The -json switch bypasses Terraform's sensitive value masking.
For example, with Terraform 1.2.7, to list and unmask all outputs:
$ terraform output -json
{
"app_database_password": {
"sensitive": true,
"type": "string",
"value": "..."
}
}
Or, if you just want a single output's value without needing to parse JSON (n.b. still seems to be enclosed in quotes):
$ terraform output -json app_database_password
"..."
Related
I'am tring to do automation for path and policies creation in vault.
Do you know how I can proceed please ? variables declared in terraform are not reconized in .hcl file.
I tried to rename my file client-ro-policy.hcl to client-ro-policy.tf but I have same issue
Varibales is recognized in file with .tf extention
Thanks
main.tf
# Use Vault provider
provider "vault" {
# It is strongly recommended to configure this provider through the
# environment variables:
# - VAULT_ADDR
# - VAULT_TOKEN
# - VAULT_CACERT
# - VAULT_CAPATH
# - etc.
}
acl-ro-policy.hcl
path "${var.client[0]}/k8s/preprod/*" {
capabilities = ["read"]
}
policies.tf
#---------------------
# Create policies
#---------------------
# Create 'client' policy
resource "vault_policy" "ro-client" {
name = "${var.client[0]}_k8s_preprod_ro"
policy = file("./hcl-ro-policy.tf")
}
variables.tf
variable "client" {
type = list(string)
}
variables.tfvars
client = ["titi", "toto","itutu"]
Result in vault:
Even though Terraform and Vault both use HCL as the underlying syntax of their respective configuration languages, their language interpreters are totally separate and so the Vault policy language implementation cannot make direct use of any values defined in the Terraform language.
Instead, you'll need to use the Terraform language to construct a suitable configuration for Vault. Vault supports a JSON variant of its policy language in order to make it easier to programmatically generate it, and so you can use Terraform's jsonencode function to build a JSON-based policy from the result of a Terraform expression, which may itself include references to values elsewhere in Terraform.
For example:
locals {
vault_ro_policy = {
path = {
"${var.client[0]}/k8s/preprod/*" = {
capabilities = ["read"]
}
}
}
}
resource "vault_policy" "ro-client" {
name = "${var.client[0]}_k8s_preprod_ro"
policy = jsonencode(local.var_ro_policy)
}
The value of local.vault_ro_policy should encode to JSON as follows, assuming that var.client[0] has the value "example":
{
"path": {
"example/k8s/preprod/*": {
"capabilities": ["read"]
}
}
}
Assuming that this is valid Vault JSON policy syntax (which I've not verified), this should be accepted by Vault as a valid policy. If I didn't get the JSON policy syntax exactly right then hopefully you can see how to adjust it to be valid; my expertise is with Terraform, so I focused on the Terraform language part here.
resource "google_service_account" "myaccount" {
account_id = "dev-foo-account"
}
resource "google_service_account_key" "mykey" {
service_account_id = google_service_account.myaccount.name
}
data "google_service_account_key" "mykey" {
name = google_service_account_key.mykey.name
public_key_type = "TYPE_X509_PEM_FILE"
}
If I create a Service Account and a key like this - how do I obtain the key afterwards?
terraform output yields:
$ terraform output -json google_service_account_key
The output variable requested could not be found in the state
file. If you recently added this to your configuration, be
sure to run `terraform apply`, since the state won't be updated
with new output variables until that command is run.
You have to put that variable as an output if you want to use it after apply the plan:
output "my_private_key" {
value = data.google_service_account_key.mykey.private_key
}
To output the value of "my_private_key":
$ terraform output my_private_key
To obtain the credentials as a JSON which can later be used for authentication:
$ terraform output -raw key | base64 -d -
I would like to generate ssh keys with a local-exec then read the content of the file.
resource "null_resource" "generate-ssh-keys-pair" {
provisioner "local-exec" {
command = <<EOT
ssh-keygen -t rsa -b 4096 -C "test" -P "" -f "testkey"
EOT
}
}
data "local_file" "public-key" {
depends_on = [null_resource.generate-ssh-keys-pair]
filename = "testkey.pub"
}
data "local_file" "private-key" {
depends_on = [null_resource.generate-ssh-keys-pair]
filename = "testkey"
}
terraform plan works but when I run the apply, I got error on testkey and testkey.pub don't exist.
Thanks
Instead of generating a file using an external command and then reading it in, I would suggest to use the Terraform tls provider to generate the key within Terraform itself, using tls_private_key:
terraform {
required_providers {
tls = {
source = "hashicorp/tls"
}
}
}
resource "tls_private_key" "example" {
algorithm = "RSA"
rsa_bits = 4096
}
The tls_private_key resource type exports two attributes that are equivalent to the two files you were intending to read in your example:
tls_private_key.example.private_key_pem: the private key in PEM format
tls_private_key.example.public_key_openssh: the public key in the format OpenSSH expects to find in .ssh/authorized_keys.
Please note the warning in the tls_private_key documentation that using this resource will cause the private key data to be saved in your Terraform state, and so you should protect that state data accordingly. That would also have been true for your approach of reading files from disk using data resources, because any value Terraform has available for use in expressions must always be stored in the state.
I run your code and there are no problems with it. It correctly generates testkey and testkey.pub.
So whatever causes it to fail for you, its not this snipped of code you've provided in the question. The fault must be outside the code snipped.
Generating an SSH key in Terraform is inherently insecure because it gets stored in the tfstate file, however I had a similar problem to solve and thought that the most secure/usable was to use a secret management service + using a cloud bucket for the backend:
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "4.3.0"
}
}
backend "gcs" {
bucket = "tfstatebucket"
prefix = "terraform/production"
}
}
//Import Ansible private key from Google Secrets Manager
resource "google_secret_manager_secret" "ansible_private_key" {
secret_id = var.ansible_private_key_secret_id
replication {
automatic = true
}
}
data "google_secret_manager_secret_version" "ansible_private_key"{
secret = google_secret_manager_secret.ansible_private_key.secret_id
version = var.ansible_private_key_secret_id_version_number
}
resource "local_file" "ansible_imported_local_private_key" {
sensitive_content = data.google_secret_manager_secret_version.ansible_private_key.secret_data
filename = var.ansible_imported_local_private_key_filename
file_permission = "0600"
}
In the case of GCP, I would add the secret in Google Secrets Manager, then use terraform import on the secret, which will in turn write it to the backend bucket. That way it doesn't get stored in Git as plain text, you can have the key file local to your project (.terraform shouldn't be under version control), and it's arguably more secure in the bucket.
So the workflow essentially is:
Human --> Secret Manager
Secret Manager --> Terraform Import --> GCS Bucket Backend
|--> Create .terraform/ssh_key
.terraform/ssh_key --> Terraform/Ansible/Whatever
Hashicorp Vault would be another way to address this
Terraform Version
Terraform v0.11.11
+ provider.azurerm v1.21.0
Terraform Configuration Files
I have left many required fields for brevity (all other config worked before I added the connection strings).
# modules/function/main.tf
variable "conn-value" {}
locals {
conn = "${map("name", "mydb", "value", "${var.conn-value}", "type", "SQLAzure")}"
}
resource "azurerm_function_app" "functions" {
connection_string = "${list(local.conn)}"
# ...
}
# modules/db/main.tf
# ... other variables declared
resource "azurerm_sql_server" "server" {
# ...
}
output "connection-string" {
value = "Server=tcp:${azurerm_sql_server.server.fully_qualified_domain_name},1433;Initial Catalog=${var.catalog};Persist Security Info=False;User ID=${var.login};Password=${var.login-password};MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=200;"
}
# main.tf
module "my_db" {
source = "modules/db"
}
module "my_app" {
source = "modules/function"
conn-value = "${module.my_db.connection-string}"
# ...
}
Expected Behavior on terraform plan
The module.my_db.connection-string output resolves to a string when passed to the my_app conn-value variable and is able to be used in the map/list passed to the azurerm_function_app.functions.connection_string variable.
Actual Behavior on terraform plan
I get this error:
module.my_app.azurerm_function_app.functions: connection_string: should be a list
If I replace "${var.conn-value}" in the modules/function/main.tf locals with just a string, it works.
Update
In response to to this comment, I updated the script above with the connection string construction.
I finally found the GitHub issue that references the problem I am having (I found the issue through this gist comment). This describes the problem exactly:
Assigning values to nested blocks is not supported, but appears to work in certain cases due to a number of coincidences...
This limitation is in <= v0.11, but is apparently fixed in v0.12 with the dynamic block.
Is there an efficient way to apply validation logic to variables used in a terraform run?
Specifically I want to check the length and casing of some variables. The variables are a combination of ones declared in tfvars files, in variables.tf files, and collected during runtime by terraform.
Thanks.
Custom Validation Rules
Terraform document - Input Variables - Custom Validation Rules
Results
Failure case
provider aws {
profile="default"
}
terraform {
experiments = [variable_validation]
}
## Custom Validation Rules
variable "test" {
type = string
description = "Example to test the case and length of the variable"
default = "TEsT"
validation {
condition = length(var.test) > 4 && upper(var.test) == var.test
error_message = "Validation condition of the test variable did not meet."
}
}
Execution
$ terraform plan
Warning: Experimental feature "variable_validation" is active
on main.tf line 5, in terraform:
5: experiments = [variable_validation]
Experimental features are subject to breaking changes in future minor or patch
releases, based on feedback.
If you have feedback on the design of this feature, please open a GitHub issue
to discuss it.
Error: Invalid value for variable # <---------------------------
on main.tf line 9:
9: variable "test" {
Validation condition of the test variable did not meet.
This was checked by the validation rule at main.tf:14,3-13.
Pass case
terraform {
experiments = [variable_validation]
}
## Custom Validation Rules
variable "test" {
type = string
description = "Example to test the case and length of the variable"
default = "TESTED"
validation {
condition = length(var.test) > 4 && upper(var.test) == var.test
error_message = "Validation condition of the test variable did not meet."
}
}
Execution
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
Others
Alternatively, use null_resource local-exec to implement logic in shell script, or use external provider to send the variable to an external program to validate?
This isn't something you can currently do directly with Terraform but I find it easier to just mangle the input variables to the required format if necessary.
As an example the aws_lb_target_group resource takes a protocol parameter that currently requires it to be uppercased instead of automatically upper casing things and suppressing the diff like the aws_lb_listener resource does for the protocol (or even the protocol in the health_check block).
To solve this I just use the upper function when creating the resource:
variable "protocol" {
default = "http"
}
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
}
resource "aws_lb_target_group" "test" {
name = "tf-example-lb-tg"
port = 80
protocol = "${upper(var.protocol)}"
vpc_id = "${aws_vpc.main.id}"
}
As for checking length I just substring things to make them the right length. I currently do this for ALBs as the name has a max length of 32 and I have Gitlab CI create review environments for some services that get a name based on the slug of the Git branch name so have little control over the length that is used.
variable "environment" {}
variable "service_name" {}
variable "internal" {
default = true
}
resource "aws_lb" "load_balancer" {
name = "${substr(var.environment, 0, min(length(var.environment), 27 - length(var.service_name)))}-${var.service_name}-${var.internal ? "int" : "ext"}"
internal = "${var.internal}"
security_groups = ["${aws_security_group.load_balancer.id}"]
subnets = ["${data.aws_subnet_ids.selected.ids}"]
}
With the above then any combination of length of environment or service name will lead to the environment/service name pair being trimmed to 27 characters at most which leaves room for the extra characters that I want to specify.
Got inspired by this conversation and found the following already existing provider:
https://github.com/craigmonson/terraform-provider-validate