I am trying to run a Terraform deployment via a Shell script where within the Shell script I first dynamically collect the access key for my Azure storage account and assign it to a variable. I then want to use the variable in a -var assignment on the terraform command line. This method works great when configuring the backend for remote state but it is not working for doing a deployment. The other variables used in the template are being pulled from a terraform.tfvars file. Below is my Shell script and Terraform template:
Shell script:
#!/bin/bash
set -eo pipefail
subscription_name="Visual Studio Enterprise with MSDN"
tfstate_storage_resource_group="terraform-state-rg"
tfstate_storage_account="terraformtfstatesa"
az account set --subscription "$subscription_name"
tfstate_storage_access_key=$(
az storage account keys list \
--resource-group "$tfstate_storage_resource_group" \
--account-name "$tfstate_storage_account" \
--query '[0].value' -o tsv
)
echo $tfstate_storage_access_key
terraform apply \
-var "access_key=$tfstate_storage_access_key"
Deployment template:
provider "azurerm" {
subscription_id = "${var.sub_id}"
}
data "terraform_remote_state" "rg" {
backend = "azurerm"
config {
storage_account_name = "terraformtfstatesa"
container_name = "terraform-state"
key = "rg.stage.project.terraform.tfstate"
access_key = "${var.access_key}"
}
}
resource "azurerm_storage_account" "my_table" {
name = "${var.storage_account}"
resource_group_name = "${data.terraform_remote_state.rg.rgname}"
location = "${var.region}"
account_tier = "Standard"
account_replication_type = "LRS"
}
I have tried defining the variable in my terraform.tfvars file:
storage_account = "appastagesa"
les_table_name = "appatable
region = "eastus"
sub_id = "abc12345-099c-1234-1234-998899889988"
access_key = ""
The access_key definition appears to get ignored.
I then tried not using a terraform.tfvars file, and created the variables.tf file below:
variable storage_account {
description = "Name of the storage account to create"
default = "appastagesa"
}
variable les_table_name {
description = "Name of the App table to create"
default = "appatable"
}
variable region {
description = "The region where resources will be deployed (ex. eastus, eastus2, etc.)"
default = "eastus"
}
variable sub_id {
description = "The ID of the subscription to deploy into"
default = "abc12345-099c-1234-1234-998899889988"
}
variable access_key {}
I then modified my deploy.sh script to use the line below to run my terraform deployment:
terraform apply \
-var "access_key=$tfstate_storage_access_key" \
-var-file="variables.tf"
This results in the error invalid value "variables.tf" for flag -var-file: multiple map declarations not supported for variables Usage: terraform apply [options] [DIR-OR-PLAN] being thrown.
After playing with this for hours...I am almost embarrassed as to what the problem was but I am also frustrated with Terraform because of the time I wasted on this issue.
I had all of my variables defined in my variables.tf file with all but one having default values. For the one without a default value, I was passing it in as part of the command line. My command line was where the problem was. Because of all of the documentation I read, I thought I had to tell terraform what my variables file was by using the -var-file option. Turns out you don't and when I did it threw the error. Turns out all I had to do was use the -var option for the variable that had no defined default and terraform just automagically saw the variables.tf file. Frustrating. I am in love with Terraform but the one negative I would give it is that the documentation is lacking.
Related
I am trying to provision a kubernetes cluster on Azure (AKS) with Terraform. The provisioning works quite well but I can't get the kubeconfig from kube_config_raw exported to a file.
Below is my main.tf and outputs.tf. I supressed the resource_group and user_assigned_identity resources.
This is a resource I used for creating the configuration: https://learnk8s.io/terraform-aks
main.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">=2.79.1"
}
}
}
provider "azurerm" {
features {}
subscription_id = "..."
}
resource "azurerm_kubernetes_cluster" "aks" {
name = "myCluster"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
dns_prefix = "my-cluster-dns"
default_node_pool {
name = "agentpool"
node_count = 1
os_disk_size_gb = 64
vm_size = "Standard_B2ms"
}
identity {
type = "UserAssigned"
user_assigned_identity_id = azurerm_user_assigned_identity.user_assigned_identity.id
}
depends_on = [
azurerm_user_assigned_identity.user_assigned_identity
]
}
outputs.tf - I've tried "./kubeconfig" and "kubeconfig" in the filename but nothing gets exported anywhere
resource "local_file" "kubeconfig" {
depends_on = [azurerm_kubernetes_cluster.aks]
filename = "./kubeconfig"
content = azurerm_kubernetes_cluster.aks.kube_config_raw
}
Bonus: is it possible to export it directly to the existing ~/.kube/config file? Like the az aks get-credentials command does?
outputs.tf - I've tried "./kubeconfig" and "kubeconfig" in the
filename but nothing gets exported anywhere
I tested the same code that you have and did terraform-apply , it saved the local file to the location where the apply was performed.
For example:
If I ran the main.tf file from C:\Users\user\terraform\aksconfig> as its present there then the kubeconfig file gets saved in the same path .
Output:
Bonus: is it possible to export it directly to the existing ~/.kube/config file? Like the az aks get-credentials command does?
Path where the az aks get-credentials --resource-group myresourcegroup --name myCluster stores the config file:
Code to save the script in the same path as az command:
resource "local_file" "kubeconfig" {
depends_on = [azurerm_kubernetes_cluster.aks]
filename = "C:/Users/user/.kube/config" this is where the config file gets stored
content = azurerm_kubernetes_cluster.aks.kube_config_raw
}
Output:
Exisitng config file in /.kube/config
New File overwrites the existing file :
Note: Using local_file block here will completely overwrite the file not appending the context to the previous one . If you are looking for merging the content in a single file like az command does , then its not possible from terraform.
I want my terraform script to create the resource group only when it does not exist in Azure, otherwise it should skip the creation of resource group.
Well, you can use Terraform external to execute the CLI command to check if the resource group exists or not. And then use the result to determine whether the resource group will create. Here is an example:
./main.tf
provider "azurerm" {
features {}
}
variable "group_name" {}
variable "location" {
default = "East Asia"
}
data "external" "example" {
program = ["/bin/bash","./script.sh"]
query = {
group_name = var.group_name
}
}
resource "azurerm_resource_group" "example" {
count = data.external.example.result.exists == "true" ? 0 : 1
name = var.group_name
location = var.location
}
./script.sh
#!/bin/bash
eval "$(jq -r '#sh "GROUP_NAME=\(.group_name)"')"
result=$(az group exists -n $GROUP_NAME)
jq -n --arg exists "$result" '{"exists":$exists}'
Terraform is declarative, not imperative. When using Terraform you shouldn't need to check for existing resources
to validate your tf script
terraform plan
and to apply the tf script changes
terraform apply
This will validate the resources if it already exists and create if not
I'm trying to figure out Terraform Cloud, just wondering how I would access environment variables I've set in the workspace within my files?
// main.tf
// Configure the Google Cloud provider
provider "google" {
credentials = "GOOGLE_CREDENTIALS"
project = "my-project"
region = "australia-southeast1"
}
I've set an environment variable on the cloud workspace GOOGLE_CREDENTIALS with the value being my .json key formatted to work with TFC.
Just not sure how to access it within my main.tf file like above
add credentials to the Terraform Cloud environment variable TF_VAR_credentials and set it as sensitive. then you can use the variable here :
provider "google" {
credentials = var.credentials
project = var.project
region = var.region
}
and declare your variable like this
variable "credentials" {
description = "credentials"
}
variable "project" {
description = "project"
}
variable "region" {
description = "region"
}
then on the apply command you can pass :
terraform apply \
-var "region=${REGION_FROM_ENV}" \
-var "project=${PROJECT_FROM_ENV}" \
-var "credentials=${GOOGLE_CREDENTIALS}"
here's a reference :
https://www.terraform.io/docs/commands/apply.html#var-39-foo-bar-39-
I have below two environments terraform files
devenv_variables.tfvars
testenv_variables.tfvars
Here is devenv_variables.tfvars
location = "westeurope"
resource_group_name = "devenv-cloudresources-rg"
Here is testenv_variables.tfvars
location = "westeurope"
resource_group_name = "testenv-cloudresources-rg"
Here is my main.tf
# Configure the Microsoft Azure Provider
provider "azurerm" {
version = "=2.0.0"
features {}
subscription_id = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
provider "azurerm" {
alias = "testenv"
subscription_id = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
# Create a resource group
resource "azurerm_resource_group" {
provider = "azurerm.testenv" //How do I pass provider based on variables here?
name = "${var.resource_group_name}"
location = "${var.location}"
}
My requirement is, based on passed tfvar file as parameter, it should choose the subscription.
terraform apply -var-file="devenv_variables.tfvars"
when I type below command resource shall create in test environment
terraform apply -var-file="testenv_variables.tfvars"
I think, I need to define client id, and password to login to respective subscriptions.
tfvars files should only contain the values of variables.
The declaration of variables should happen in regular tf files.
variables.tf
variable "location" {
type = "string"
description = "The azure location where the resources is created"
}
devenv_variables.tfvars
location = "West Europe"
This tutorial can also help you with some more information and examples.
All you have to do this just make same variable name in both files. when you run the command terraform plan–var-file='variable's_file_name' & terraform apply–var-file='variable's_file_name'
on that point terraform will get different value from different file.
for demo
variable.tf (contain all variables)
variable "resource_group" {
type = string
description = "Resource Group"
default = ""
}
dev.tfvars (contain development variable's values)
resource_group="name_of_my_resource_group_dev"
prod.tfvars (contain production variable's values)
resource_group="name_of_my_resource_group_prod"
now when you run command terraform plan–var-file='dev.tfvars' it take the value from dev.tfvars file.
Here is my setup,
Terraform version - Terraform v0.12.17
OS - OSX 10.15.1
Use Case - define a provider file and access the variables defined in the vars file
Files
main.tf - where the code is
provider "aws" {
}
variable "AWS_REGION" {
type = string
}
variable "AMIS" {
type = map(string)
default = {
us-west-1 = "my ami"
}
}
resource "aws_instance" "awsInstall" {
ami = var.AMIS[var.AWS_REGION]
instance_type = "t2.micro"
}
awsVars.tfvars - where the region is defined
AWS_REGION="eu-region-1"
Execution
$ terraform console
var.AWS_REGION
Error: Result depends on values that cannot be determined until after "terraform apply".
What mistake I have done, I don't see any syntax but have issues in accessing the variables, any pointers would be helpful
Thanks
Terraform does not automatically read a .tfvars file unless its filename ends with .auto.tfvars. Because of that, when you ran terraform console with no arguments Terraform did not know a value for variable AWS_REGION.
To keep your existing filename, you can pass this variables file explicitly on the command line like this:
terraform console -var-file="awsVars.tfvars"
Alternatively, you could rename the file to awsVars.auto.tfvars and then Terraform will read it by default as long as it's in the current working directory when you run Terraform.
There's more information on how you can set values for root module input variables in the Terraform documentation section Assigning Values to Root Module Variables.
Note also that the usual naming convention for input variables and other Terraform-specific objects is to keep the names in lowercase and separate words with underscores. For example, it would be more conventional to name your variables aws_region and amis.
Furthermore, if your goal is to find an AMI for the current region (the one chosen by the AWS_DEFAULT_REGION environment variable, or in the provider configuration), you could use the aws_region data source to allow Terraform to determine that automatically, so you don't have to set it as a variable at all:
variable "amis" {
type = map(string)
default = {
us-west-1 = "my ami"
}
}
data "aws_region" "current" {}
resource "aws_instance" "awsInstall" {
ami = var.amis[data.aws_region.current.name]
instance_type = "t2.micro"
}