Terraform Init throwing error while using with Azure DevOps - azure

Question

I am trying to run Terraform Open Source using Azure Devops
I have the state file stored in Azure Blobstorage
Below is my pipeline file
variables:
- group: infra-variables
trigger:
branches:
include:
- master
paths:
include:
- Terraform-Test
exclude:
- README.md
stages:
- stage: Validate
displayName: Validate
jobs:
- job: validate
pool:
vmImage: ubuntu-latest
steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller#0
displayName: Install Terraform
inputs:
terraformVersion: 'latest'
# Init
- task: TerraformCLI#0
displayName: Initialize Terraform
env:
ARM_SAS_TOKEN: $(ARM_ACCESS_KEY)
inputs:
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
commandOptions: '-backend-config=storage_account_name=$(TF_STATE_BLOB_ACCOUNT_NAME) -backend-config=container_name=$(TF_STATE_BLOB_CONTAINER_NAME) -backend-config=key=$(ARM_ACCESS_KEY)'
backendType: 'selfConfigured'
# Validate
- task: TerraformCLI#0
displayName: Validate Config
inputs:
command: 'validate'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
- stage: Plan
displayName: Plan
jobs:
- job: plan
pool:
vmImage: ubuntu-latest
steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller#0
displayName: Install Terraform
inputs:
terraformVersion: 'latest'
# Init
- task: TerraformCLI#0
displayName: Initialize Terraform
env:
ARM_SAS_TOKEN: $(ARM_ACCESS_KEY)
inputs:
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
commandOptions: '-backend-config=storage_account_name=$(TF_STATE_BLOB_ACCOUNT_NAME) -backend-config=container_name=$(TF_STATE_BLOB_CONTAINER_NAME) -backend-config=key=$(ARM_ACCESS_KEY)'
backendType: 'selfConfigured'
# Plan
- task: TerraformCLI#0
displayName: Plan Terraform Deployment
env:
ARM_SAS_TOKEN: $(ARM_ACCESS_KEY)
ARM_CLIENT_ID: $(AZURE_CLIENT_ID)
ARM_CLIENT_SECRET: $(AZURE_CLIENT_SECRET)
ARM_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(AZURE_TENANT_ID)
inputs:
command: 'plan'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
# Approve
- stage: Approve
displayName: Approve
jobs:
- job: approve
displayName: Wait for approval
pool: server
steps:
- task: ManualValidation#0
timeoutInMinutes: 60
inputs:
notifyUsers: 'pallabcd#hotmail.com'
instructions: 'Review the plan in the next hour'
- stage: Apply
displayName: Apply
jobs:
- job: apply
pool:
vmImage: ubuntu-latest
steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller#0
displayName: Install Terraform
inputs:
terraformVersion: 'latest'
# Init
- task: TerraformCLI#0
displayName: TF Init
env:
ARM_SAS_TOKEN: $(ARM_ACCESS_KEY)
inputs:
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
commandOptions: '-backend-config=storage_account_name=$(TF_STATE_BLOB_ACCOUNT_NAME) -backend-config=container_name=$(TF_STATE_BLOB_CONTAINER_NAME) -backend-config=key=$(ARM_ACCESS_KEY)'
backendType: 'selfConfigured'
# Apply
- task: TerraformCLI#0
displayName: TF Apply
env:
ARM_SAS_TOKEN: $(ARM_ACCESS_KEY)
ARM_CLIENT_ID: $(AZURE_CLIENT_ID)
ARM_CLIENT_SECRET: $(AZURE_CLIENT_SECRET)
ARM_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(AZURE_TENANT_ID)
inputs:
command: 'apply'
workingDirectory: '$(System.DefaultWorkingDirectory)/Terraform-Test'
commandOptions: '-auto-approve'
My main.tf file is given below
terraform {
required_version = "~> 1.0"
backend "azurerm" {
storage_account_name = var.storage_account_name
container_name = var.container_name
key = "terraform.tfstate"
access_key = "#{ARM_ACCESS_KEY}#"
features {}
}
required_providers {
azuread = "~> 1.0"
azurerm = "~> 2.0"
}
}
provider "azurerm" {
tenant_id = var.tenant_id
client_id = var.client_id
client_secret = var.client_secret
subscription_id = var.subscription_id
features {}
}
data "azurerm_resource_group" "az-rg-wu" {
name = "Great-Learning"
}
data "azurerm_client_config" "current" {}
When i am putting the actual storage access key in main.tf the Init is successful but if i am putting the ADO variable in the form of "#{ARM_ACCESS_KEY}#", the pipeline fails.
This variable is there in my tfvar file also and the value is set in a variable group in Azure Devops
So what i am doing wrong here

Answer 1

The service connection may cause problems. Terraform should require a proper service connection and an Azure DevOps extension as a pre-requisite. Ensure Terraform CLI is installed on the pipeline agent.
Here is the sample YAML to connect to Azure DevOps.
Step1: sample yaml file code as below
variables:
- name: TerraformBackend.ResourceGroup
value: rg-realworld-staging-001
- name: TerraformBackend.StorageAccount
value: strwstagingterraform01
- name: TerraformBackend.ContainerName
value: staging
- group: 'staging'
steps:
- task: AzureCLI#2
inputs:
azureSubscription: '[service connection]'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
az group create --location eastus --name $(TerraformBackend.ResourceGroup)
az storage account create --name $(TerraformBackend.StorageAccount) --resource-group $(TerraformBackend.ResourceGroup) --location eastus --sku Standard_LRS
az storage container create --name staging --account-name $(TerraformBackend.StorageAccount)
STORAGE_ACCOUNT_KEY=$(az storage account keys list -g $(TerraformBackend.ResourceGroup) -n $(TerraformBackend.StorageAccount) | jq ".[0].value" -r)
echo "setting storage account key variable"
echo "##vso[task.setvariable variable=ARM_ACCESS_KEY;issecret=true]$STORAGE_ACCOUNT_KEY"
- task: TerraformInstaller#0
inputs:
terraformVersion: '1.3.6'
- task: TerraformTaskV1#0
displayName: "Terraform Init"
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '[service connection]'
backendAzureRmResourceGroupName: $(TerraformBackend.ResourceGroup)
backendAzureRmStorageAccountName: $(TerraformBackend.StorageAccount)
backendAzureRmContainerName: '$(TerraformBackend.ContainerName)'
backendAzureRmKey: 'infrastructure/terraform.tfstate'
workingDirectory: '$(System.DefaultWorkingDirectory)/stag/'
- task: TerraformTaskV1#0
displayName: "Terraform Apply"
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.DefaultWorkingDirectory)/stag/'
environmentServiceNameAzureRM: '[service connection]'
commandOptions: |
-var "some_key=$(value)"
Upon init execution
Updated main tf file as below
terraform {
required_version = "~> 1.0"
backend "azurerm" {
storage_account_name = "tstate6075"
container_name = "tstate"
key = "terraform.tfstate"
access_key = "=="
******************************** // features {}
}
required_providers {
azuread = "~> 1.0"
azurerm = "~> 2.0"
}
}
provider "azurerm" {
tenant_id = "*******************"
//client_id = var.client_id
//client_secret = var.client_secret
subscription_id = "**********************"
features {}
}
data "azurerm_resource_group" "az-rg-wu" {
name = "rg-*****"
}
data "azurerm_client_config" "current" {}