Setting up terraform cloud for the first time and getting this error. Not sure why as on my local machine azure CLI is installed and the path is set, but I think has something to do with setting it in the terraform cloud platform.
Error: building AzureRM Client: please ensure you have installed Azure CLI version 2.0.79 or newer. Error parsing json result from the Azure CLI: launching Azure CLI: exec: "az": executable file not found in $PATH.
with provider["registry.terraform.io/hashicorp/azurerm"]
on versions.tf line 21, in provider "azurerm":
provider "azurerm" {
My currently tf code
versions.tf
terraform {
cloud {
organization = "myorg"
workspaces {
name = "dev"
}
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~>3.10.0"
}
}
required_version = ">= 1.2.3"
}
provider "azurerm" {
features {}
}
variables.tf
variable "tenant_id" {
description = "tenant id for azure subscription"
}
main.tf
resource "azurerm_resource_group" "testrg" {
name = "test-rg"
location = "Central US"
}
not doing anything fancy, but not sure how to get past the azure CLI error. I know where variables can be set in the terraform cloud platform, but not specifically where to set a $Path for the azure cli or even how to install azure cli in terraform cloud. On my local machine, I am logging in with az login on an account with sufficient permissions to the subscription.
I'm trying to "boil down" kavya Saraboju's answer, which is formally correct, to a bare minimum that helped me.
The Error message seems to be very confusing, if it has anything at all to do with the actual problem. I had to set the environment variables ARM_CLIENT_ID, ARM_TENANT_ID, ARM_CLIENT_SECRET and ARM_SUBSCRIPTION_ID in Terraform Cloud. Go to Terraform Cloud's web admin panel, choose your workspace, click on "Variables" and set all the required values:
Read here how to obtain the values for those variables.
I'm a beginner as well on both Terraform and Azure, but I anyway hope this answer will help anybody who stumbles across this issue.
And also, my solution is described comprehensively in this tutorial.
It looks like , you are trying to login using az login.This works for local terraform runs .
To authenticate in terraform cloud instance , you may need to use Terraform Cloud workspace variables .
Please make sure to complete below steps:
Please check , if you have created service principal. If
you're using Azure Clouds for example US Government .In the first
step you need to configure the Azure CLI to work with that Cloud.
$ az cloud set --name AzureUSGovernment
Then log in using az login and check for the subscriptions listed
and set it up for one.
$ az login
$ az account list
$ az account set --subscription="SUBSCRIPTION_ID"
Now please try to create the Service Principal which will have
permissions to actually manage resources in that particular
specified Subscription which had been set in the previous step.
$ az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"
Where you can get tenant id, clientId etc which can be
used as environment variables later. This document on creating a
service principal using the azure-cli | Terraform Registry
will guide you in detail
Now in terraform cloud workers , as we cant use az login, we can
logout of it and set the environment variables something like below
from the obtained values from previous steps.
$ export ARM_CLIENT_ID="xxxxxxxxxx"
$ export ARM_SUBSCRIPTION_ID="xxxxxxx"
$ export ARM_TENANT_ID="xxxxxx"
$ export ARM_CLIENT_SECRET="xxxxxxx"
see Configuring the Service Principal in Terraform
Then you can specify Terraform and Provider blocks
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.0.0"
}
}
}
# Configure the Microsoft Azure Provider
provider "azurerm" {
features {}
}
Then try to run terraform init > terraform plan or terraform apply which can
probably make it possible to authenticate and make terraform to run
using the Service Principal .
Reference: Using the Azure Provider with Terraform Cloud - Terraform - HashiCorp Discuss
Also do check if you have latest version of terraform, if not install and try with it.
Related
I'm implementing a Terraform template, that deploys an Azure VM, based on a custom image that resides on another tenant. I've provided permissions to an AppRegistration, and validated that using Az CLI I can deploy a VMSS referring to that same shared image.
However, if I use Terraform to deploy the VM, I get this error:
Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=403 -- Original Error: Code="LinkedAuthorizationFailed" Message="The client has permission to perform action 'Microsoft.Compute/galleries/images/versions/read' on scope '/subscriptions//resourceGroups/RG-Images/providers/Microsoft.Compute/virtualMachines/VM1', however the current tenant '' is not authorized to access linked subscription '***'."
Terraform is using the AppRegistration that was created. however, it fails with that error
I've followed this how-to, successfully, that usees Az cli.
https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/share-images-across-tenants
I understand by the error message, that the user has the permissions, but the issue is between the 2 tenants, is that it? What else can I do to fix this?
Initially please check with the RBAC permissions on the two tenants like Virtual machine contributor or Network Contributor role .
This issue with cross tenant may be even fixed in terraform azurerm
provider version 1.34.0 or later
provider "azurerm" {
version = "~> 1.34.0"
}
And you can make use of auxiliary_tenant_ids = ["<tenant2 Id>"] to mention both the tenants while using shared image gallery .See shared image gallery /terraform/github.com by #rajaie-algorithmia
provider "azurerm" {
subscription_id = "${var.subscription_id}"
client_id = "${var.client_id}"
client_secret = "${var.client_secret}"
tenant_id = "${var.tenant_id}"
auxiliary_tenant_ids = ["${var.sig_tenant_id}"] #give the other tenant Id here
}
References:
share-images-across-tenants | microsoft docs
azure portal : how-to-share-gallery-vm-images-across-azure-tenants |Ajay varma| axiom
I am currently building a terraform pipeline, where the steps are as follows:
Setting the environment
terraform init
terraform plan
terraform apply
At the current stage I want to perform terraform init without doing az login and not use az login . I am doing this on azure dev ops and was wondering how I can achieve?
You need to authenticate yourself to Azure, otherwise Terraform would never know where to place it's resources (in which subscription, which resource group and with which rights).
You can also opt using a Managed Service Identity or Service Principal with a Client Certificate.
All information regarding this you can find on the official Terraform documentation here
I have a project that is running on Azure DevOps that requires creating a KeyVault and giving a series of managed AppService identities access to secrets in that vault.
Because of Terraform not being able to give its own service connection access to the key vault(this is a bug of some kind), I am forced to create ResourceGroup and Keyvault with SP access before Terraforming.
When running terraform import on resourcegroup and Keyvault through a PowerShell task:
terraform init
$state = terraform state list
if ($state -like '*azurerm_resource_group.instancerg*' -and '*azurerm_key_vault.instancekeyvault*') {
Write-Host "Resources have already been imported!"
}
else {
terraform import azurerm_resource_group.instancerg /subscriptions/$(subscriptionid)/resourceGroups/rgname
terraform import azurerm_key_vault.instancekeyvault /subscriptions/$(subscriptionid)/resourceGroups/rgname/providers/Microsoft.KeyVault/vaults/keyvaultname
}
Failure happens on terraform import commands:
'Authenticate using a Service Principal' To authenticate to Azure
using a Service Principal, you can use the separate auth method -
instructions for which can be found here:'
My main.tf contains:
provider "azurerm" {
version = "=2.7.0"
subscription_id = var.subscriptionid
client_id = var.devopsserviceconnectionaid
client_secret = var.devopsserviceconnectionpw
tenant_id = var.tennantid
features {}
}
The variables are all linked to the proper credentials.
From what I understand Terraform should pick up on what authentication method that is being used based on the credentials that are in the block above or specific env variables (that are also present...) but somehow Terraform still thinks I'm trying to auth through Azure Cli and not Service principal.
You can use manage identities in keyvault in terraform as shown below.
object_id = azurerm_app_service.app.identity.0.principal_id
Web app is as below creating managed identity
KV as below
The order should be create web app with managed identity, then the KV then the KV access policy.
For authenticate with Azure pipelines service connection below works fine but you need to pass the arguments via the pipeline.
for further information check this blog here
Full PowerShell based implementation calling terraform with Azure DevOps pipelines is explained here. This implementation prevents any azure resources as prerequisite before terraforming. Only prerequisite is creating the SPN to enable authentication and authorization.
I have two subscriptions in Azure. Let's call them sub-dev and sub-prod. Under sub-dev I have resources for development (in a resource group rg-dev) and under sub-prod resources for production (in a resource group rg-prod).
Now, I would like to have only one state-file for both dev and prod. I can do this as I am using Terraform workspaces (dev and prod). There is a Storage Account under sub-dev (rg-dev) named tfsate. It has a container etc. The Azure backend is configured like this:
terraform {
backend "azurerm" {
resource_group_name = "rg-dev"
storage_account_name = "tfstate"
container_name = "tfcontainer"
key = "terraform.tfstate"
}
}
If I want to apply to the dev environment I have to switch Az Cli to the sub-dev. Similarly, for production, I would have to use sub-prod. I switch the default subscription with az cli:
az account set -s sub-prod
Problem is that the state's storage account is under sub-dev and not sub-prod. I will get access errors when trying to terraform init (or apply) when the default subscription is set to sub-prod.
Error: Failed to get existing workspaces: Error retrieving keys for Storage Account "tfstate": storage.AccountsClient#ListKeys: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'user#example.com' with object id '<redacted>' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope '/subscriptions/sub-prod/resourceGroups/rg-dev/providers/Microsoft.Storage/storageAccounts/tfstate' or the scope is invalid. If access was recently granted, please refresh your credentials."
I have tried couple of things:
I added subscription_id = "sub-dev"
I generated a SAS token for the tfstate storage account and added the sas_token config value (removed resource_group_name)
but in vain and getting the same error.
I tried to az logout but terraform requires me to login first. Do I have to tune the permissions in the Azure end somehow (this is hard as the Azure environment is configured by a 3rd party) or does Terraform support this kind of having your state file under different subscription setup at all?
For better or worse (I haven't experimented much with other methods of organising terraform) we use terraform in the exact way you are describing. A state file, in a remote backend, in a different subscription to my resources. Workspaces are created to handle environments for the deployment.
Our state files are specified like this:
terraform {
required_version = ">= 0.12.6"
backend "azurerm" {
subscription_id = "<subscription GUID storage account is in>"
resource_group_name = "terraform-rg"
storage_account_name = "myterraform"
container_name = "tfstate"
key = "root.terraform.tfstate"
}
}
We keep our terraform storage account in a completely different subscription to our deployments but this isn't necessary.
When configuring your state file like so, it authenticates to the remote backend via az CLI, using the context of the person interacting with the CLI. This person needs to have the "Reader & Data Access" role to the storage account in order to dynamically retrieve the storage account keys at runtime.
With the above state file configured, executing Terraform would be
az login
az account set -s "<name of subscription where you want to create resources>"
terraform init
terraform plan
terraform apply
There's another way to do that. You can use the Access Key associated with the Storage Account on the other subscription(the one you want to have the state files on) and export it as an environment variable.
Bash:
export ARM_ACCESS_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query '[0].value' -o tsv)
Powershell:
$env:ARM_ACCESS_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query '[0].value' -o tsv)
Then switch to the subscription you want to deploy to and deploy.
I am looking at this example from the terraform docs for creating an azure group:
resource "azurerm_resource_group" "test" {
name = "testResourceGroup1"
location = "West US"
tags = {
environment = "Production"
}
}
It does not specify the subscription anywhere.
How can I specify the subscription?
For your issue, you know the Terraform deploy the Azure resources through the Azure CLI. And there are four ways to authenticate.
Authenticating to Azure using the Azure CLI
Authenticating to Azure using Managed Service Identity
Authenticating to Azure using a Service Principal and a Client Certificate
Authenticating to Azure using a Service Principal and a Client Secret
If you do not set the tenant Id and subscription Id in the Terraform code, then you must use the first method in default. And you authenticate via the Azure CLI with the account that you log in the Azure CLI. So which subscription you set in the CLI then you use it for your Terraform.
But as the Terraform recommend:
We recommend using either a Service Principal or Managed Service
Identity when running Terraform non-interactively (such as when
running Terraform in a CI server) - and authenticating using the Azure
CLI when running Terraform locally.
So that you could grant the more appropriate permission for the service principal as you want.
Subscription is set when you configure Terraform to log in to Azure. The recommended way is to use an Azure AD service principal and environment variables.
To configure Terraform to use your Azure AD service principal, set the following environment variables, which are then used by the Azure Terraform modules. You can also set the environment if working with an Azure cloud other than Azure public.
ARM_SUBSCRIPTION_ID
ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_TENANT_ID
ARM_ENVIRONMENT
Reference