I've got 2 options to pass creds to terraform provider:
Setup ENV variables like FOO_PROVIDER_USERNAME & FOO_PROVIDER_PASSWORD. Update: and read them from ENV in a source code of a provider so there's no username / password vars in *.tf files.
Set it explicitly in a provider:
provider "foocloud" {
username = "admin#foocloud.org"
password = "coolpass"
}
Shall I pick #1 or #2? My concern about #2 is that those username / password might be saved to a state file which is a security concern.
EDIT: this is typically for managing secrets in resources:
A few weeks ago, I came across this great article by Yevgeniy Brikman:
https://blog.gruntwork.io/a-comprehensive-guide-to-managing-secrets-in-your-terraform-code-1d586955ace1
Out of the two options you mention, go with option 1 (like you said, option 2 will write them to the state file) but you should set the variables as sensitive.
Example:
# main.tf
resource "foocloud" {
name = "foobar"
username = var.username
password = var.password
}
# variables.tf
variable "username" {
description = "foobar"
type = string
sensitive = true
}
variable "password" {
description = "foobar"
type = string
sensitive = true
}
# command line or in text file
export TF_VAR_username=foo
export TF_VAR_password=bar
EDIT: in the case of authentication to cloud providers such as AWS you can use the credentials files among other options, as explained here:
https://blog.gruntwork.io/authenticating-to-aws-with-the-credentials-file-d16c0fbcbf9e
Related
I'm trying to provision a kubernetes cluster by creating all the certificates through vault first. It somehow makes it easy in the context of terraform, because I can insert all this information in the cloudinit config, so I don't have to rely on a node being ready and then transfer data from one to another.
In any case, the problem that I have is that vault_pki_secret_backend_cert doesn't seem to support any change to the subject field except for common_name (https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/pki_secret_backend_cert), whereas kubernetes relies on these types of certificates where the organization is specified. For example:
Subject: O = system:masters, CN = kube-etcd-healthcheck-client
I'm generating these certificates by directly using vault's intermediate certificate, so the private key is in vault. I cannot generate them separately, and I wouldn't want that anyway, because I'm trying to provision basically everything using terraform.
Any ideas how I can get around this issue?
I was able to find out the answer eventually. The only way to do this with terraform/vault seems to be configuring the backend role and add the organization parameter in that role:
https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/cert_auth_backend_role.
For example, you define the role:
resource "vault_pki_secret_backend_role" "etcd_ca_clients" {
depends_on = [ vault_pki_secret_backend_intermediate_set_signed.kube1_etcd_ca ]
backend = vault_mount.kube1_etcd_ca.path
name = "kubernetes-client"
ttl = 3600
allow_ip_sans = true
key_type = "ed25519"
allow_any_name = true
allowed_domains = ["*"]
allow_subdomains = true
organization = [ "system:masters" ]
}
And here you tell vault to generate the certificate based on that role:
resource "vault_pki_secret_backend_cert" "etcd_healthcheck_client" {
for_each = { for k, v in var.kubernetes_servers : k => v if startswith(k, "etcd-") }
depends_on = [vault_pki_secret_backend_role.etcd_ca_clients]
backend = vault_mount.kube1_etcd_ca.path
name = vault_pki_secret_backend_role.etcd_ca_clients.name
common_name = "kube-etcd-healthcheck-client"
}
The limitation makes no sense whatsoever to me, but if you don't a bulk of very different certificates, it's not all too bad and you don't have to repeat a lot of code.
Is there a way in Terraform to check if a resource in Google Cloud exists prior to trying to create it?
I want to check if the following resources below exist in my CircleCI CI/CD pipeline during a job. I have access to terminal commands, bash, and gcloud commands. If the resources do exist, I want to use them. If they do not exist, I want to create them. I am doing this logic in CircleCI's config.yml as steps where I have access to terminal commands and bash. My goal is to create my necessary infrastructure (resources) in GCP when they are needed, otherwise use them if they are created, without getting Terraform errors in my CI/CD builds.
If I try to create a resource that already exists, Terraform apply will result in an error saying something like, "you already own this resource," and now my CI/CD job fails.
Below is pseudo code describing the resources I am trying to get.
resource "google_artifact_registry_repository" "main" {
# this is the repo for hosting my Docker images
# it does not have a data source afaik because it is beta
}
For my google_artifact_registry_repository resource. One approach I have is to do a Terraform apply using a data source block and see if a value is returned. The problem with this is that the google_artifact_registry_repository does not have a data source block. Therefore, I must create this resource once using a resource block and every CI/CD build thereafter can rely on it being there. Is there a work-around to read that it exists?
resource "google_storage_bucket" "bucket" {
# bucket containing the folder below
}
resource "google_storage_bucket_object" "content_folder" {
# folder containing Terraform default.tfstate for my Cloud Run Service
}
For my google_storage_bucket and google_storage_bucket_object resources. If I do a Terraform apply using a data source block to see if these exist, one issue I run into is when the resources are not found, Terraform takes forever to return that status. It would be great if I could determine if a resource exists within like 10-15 seconds or something, and if not assume these resources do not exist.
data "google_storage_bucket" "bucket" {
# bucket containing the folder below
}
output bucket {
value = data.google_storage_bucket.bucket
}
When the resource exists, I can use Terraform output bucket to get that value. If it does not exist, Terraform takes too long to return a response. Any ideas on this?
Thanks to the advice of Marcin, I have a working example of how to solve my problem of checking if a resource exists in GCP using Terraform's external data sources. This is one way that works. I am sure there are other approaches.
I have a CircleCI config.yml where I have a job that uses run commands and bash. From bash, I will init/apply a Terraform script that checks if my resource exists, like so below.
data "external" "get_bucket" {
program = ["bash","gcp.sh"]
query = {
bucket_name = var.bucket_name
}
}
output "bucket" {
value = data.external.get_bucket.result.name
}
Then in my gcp.sh, I use gsutil to get my bucket if it exists.
#!/bin/bash
eval "$(jq -r '#sh "BUCKET_NAME=\(.bucket_name)"')"
bucket=$(gsutil ls gs://$BUCKET_NAME)
if [[ ${#bucket} -gt 0 ]]; then
jq -n --arg name "" '{name:"'$BUCKET_NAME'"}'
else
jq -n --arg name "" '{name:""}'
fi
Then in my CircleCI config.yml, I put it all together.
terraform init
terraform apply -auto-approve -var bucket_name=my-bucket
bucket=$(terraform output bucket)
At this point I check if the bucket name is returned and determine how to proceed based on that.
TF does not have any build in tools for checking if there are pre-existing resources, as this is not what TF is meant to do. However, you can create your own custom data source.
Using the custom data source you can program any logic you want, including checking for pre-existing resources and return that information to TF for future use.
There is a way to check if a resource already exists before creating the resource. But you should be aware of whether it exists. Using this approach, you need to know if the resource exists. If the resource does not exist, it'll give you an error.
I will demonstrate it by create/reading data from an Azure Resource Group. First, create a boolean variable azurerm_create_resource_group. You can set the value to true if you need to create the resource; otherwise, if you just want to read data from an existing resource, you can set it to false.
variable "azurerm_create_resource_group" {
type = bool
}
Next up, get data about the resource using the ternary operator supplying it to count, next do the same for creating the resource:
data "azurerm_resource_group" "rg" {
count = var.azurerm_create_resource_group == false ? 1 : 0
name = var.azurerm_resource_group
}
resource "azurerm_resource_group" "rg" {
count = var.azurerm_create_resource_group ? 1 : 0
name = var.azurerm_resource_group
location = var.azurerm_location
}
The code will create or read data from the resource group based on the value of the var.azurerm_resource_group. Next, combine the data from both the data and resource sections into a locals.
locals {
resource_group_name = element(coalescelist(data.azurerm_resource_group.rg.*.name, azurerm_resource_group.rg.*.name, [""]), 0)
location = element(coalescelist(data.azurerm_resource_group.rg.*.location, azurerm_resource_group.rg.*.location, [""]), 0)
}
Another way of doing it might be using terraformer to import the infra code.
I hope this helps.
This work for me:
Create data
data "gitlab_user" "user" {
for_each = local.users
username = each.value.user_name
}
Create resource
resource "gitlab_user" "user" {
for_each = local.users
name = each.key
username = data.gitlab_user.user[each.key].username != null ? data.gitlab_user.user[each.key].username : split("#", each.value.user_email)[0]
email = each.value.user_email
reset_password = data.gitlab_user.user[each.key].username != null ? false : true
}
P.S.
Variable
variable "users_info" {
type = list(
object(
{
name = string
user_name = string
user_email = string
access_level = string
expires_at = string
group_name = string
}
)
)
description = "List of users and their access to team's groups for newcommers"
}
Locals
locals {
users = { for user in var.users_info : user.name => user }
}
I'm using the below lines in a module say ABC:
additional_users = [
{
name = each.value.mysql_db_user
password = data.google_secret_manager_secret_version.each.value.mysql_db_secret.secret_data
}
]
Using a data source to fetch password from secret manager module:
data "google_secret_manager_secret_version" "mysql_db_password_001" {
project = var.project_id
provider = google-beta
secret = var.mysql_secret_id_001
depends_on = [
module.mysql_db_password_001
]
}
Using for-each to loop around multiple variables in the module ABC, but when i am using in below line I run into errors.
password = data.google_secret_manager_secret_version.each.value.mysql_db_secret.secret_data
I guess only static values can be passed while referencing here, is there some workaround to it where I can grab values for password from various datasources mysql_db_password_001 mysql_db_password_002 mysql_db_password_003 and so on.
Thank You !
Have you tried something like?
Index can also be *
password = data.google_secret_manager_secret_version. mysql_db_password_001[index]
sorry, had to answer instead of comment... to few reputation
How can I assign during creation of IAM users with below code to one group existing alredy in AWS?
resource "aws_iam_user" "developer-accounts" {
path = "/"
for_each = toset(var.names)
name = each.value
force_destroy = true
}
resource "aws_iam_user_group_membership" "developers-membership" {
user = values(aws_iam_user.developer-accounts)[*].name
groups = [data.aws_iam_group.developers.group_name]
}
With above code I’m getting
inappropriate value for attribute “user”: string required.
Users variable used:
variable "names" {
description = "account names"
type = list(string)
default = ["user-1", "user-2", "user-3",...etc]
}
2nd part of question. With below I want to create passwords for each users:
resource "aws_iam_user_login_profile" "devs_login" {
for_each = toset(var.names)
user = each.value
pgp_key = "keybase:macdrorepo"
password_reset_required = true
}
Output:
output "all_passwordas" {
value = values(aws_iam_user_login_profile.devs_login)[*].encrypted_password
}
How can I decode the passwords? Below is not working as I'm sure missing some kind of loop...
terraform output all_passwordas | base64 --decode | keybase pgp
decrypt
For your first question, the following should do the trick:
You need to iterate over all users again and attach groups to each of them:
resource "aws_iam_user_group_membership" "developers-membership" {
for_each = toset(var.names)
user = aws_iam_user.developer-accounts[each.key].name
groups = [data.aws_iam_group.developers.group_name]
}
To answer your second question: You are trying to decrypt all user passwords at once, which will not work as expected. Instead, you need to decrypt each users password one by one. You could use tools like jq to loop over terraform output -json output.
Just a small note. It's better to open two questions instead of adding multiple (unrelated) questions into one. I hope this answer helps.
I need to be able to create multiple iam accounts for my developer group with console login enabled using terraform
It's a bit tricky since we have to use keybase when enabling login profiles in terraform. Is there a way to achieve this
variable "iam_users" {
description = "List of iam users that that needs to be created"
type = "list"
}
# values assigned in .tfvars file
iam_users = ["mahela","bhanuka","duminda"]
resource "aws_iam_user_login_profile" "login_profile" {
count = "${length(var.iam_users)}"
user = "${element(var.iam_users,count.index)}"
password_reset_required = true
pgp_key = "keybase:mahela"
}
I cannot get the output for this list of users from below code since count is define in the resource
output "password" {
value = "${aws_iam_user_login_profile.login_profile.encrypted_password}"
}
Do I have to use modules when creating users? with that help to get the output of encrypted password for each user?
Also do I have to use different keybase usernames for each user? this is going to be tricky again :(
When count is set in a resource block, references to that resource in other expressions produce a list of instance objects rather than a single instance object, and so you can't access the instance attributes directly.
If you wish to return a map from user to password then you can write an output expression like the following, assuming you're using Terraform 0.12 or later:
output "passwords" {
value = { for p in aws_iam_user_login_profile.login_profile : p.user => p.encrypted_password }
}
A Terraform 0.11-compatible variant of that would be something like the following:
output "passwords" {
value = "${zipmap(aws_iam_user_login_profile.login_profile.*.user, aws_iam_user_login_profile.login_profile.*.encrypted_password)}"
}