Having the Terraform azure state file under different subscription - azure

Question

I have two subscriptions in Azure. Let's call them sub-dev and sub-prod. Under sub-dev I have resources for development (in a resource group rg-dev) and under sub-prod resources for production (in a resource group rg-prod).
Now, I would like to have only one state-file for both dev and prod. I can do this as I am using Terraform workspaces (dev and prod). There is a Storage Account under sub-dev (rg-dev) named tfsate. It has a container etc. The Azure backend is configured like this:
terraform {
backend "azurerm" {
resource_group_name = "rg-dev"
storage_account_name = "tfstate"
container_name = "tfcontainer"
key = "terraform.tfstate"
}
}
If I want to apply to the dev environment I have to switch Az Cli to the sub-dev. Similarly, for production, I would have to use sub-prod. I switch the default subscription with az cli:
az account set -s sub-prod
Problem is that the state's storage account is under sub-dev and not sub-prod. I will get access errors when trying to terraform init (or apply) when the default subscription is set to sub-prod.
Error: Failed to get existing workspaces: Error retrieving keys for Storage Account "tfstate": storage.AccountsClient#ListKeys: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'user#example.com' with object id '<redacted>' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope '/subscriptions/sub-prod/resourceGroups/rg-dev/providers/Microsoft.Storage/storageAccounts/tfstate' or the scope is invalid. If access was recently granted, please refresh your credentials."
I have tried couple of things:
I added subscription_id = "sub-dev"
I generated a SAS token for the tfstate storage account and added the sas_token config value (removed resource_group_name)
but in vain and getting the same error.
I tried to az logout but terraform requires me to login first. Do I have to tune the permissions in the Azure end somehow (this is hard as the Azure environment is configured by a 3rd party) or does Terraform support this kind of having your state file under different subscription setup at all?

Answer 1

For better or worse (I haven't experimented much with other methods of organising terraform) we use terraform in the exact way you are describing. A state file, in a remote backend, in a different subscription to my resources. Workspaces are created to handle environments for the deployment.
Our state files are specified like this:
terraform {
required_version = ">= 0.12.6"
backend "azurerm" {
subscription_id = "<subscription GUID storage account is in>"
resource_group_name = "terraform-rg"
storage_account_name = "myterraform"
container_name = "tfstate"
key = "root.terraform.tfstate"
}
}
We keep our terraform storage account in a completely different subscription to our deployments but this isn't necessary.
When configuring your state file like so, it authenticates to the remote backend via az CLI, using the context of the person interacting with the CLI. This person needs to have the "Reader & Data Access" role to the storage account in order to dynamically retrieve the storage account keys at runtime.
With the above state file configured, executing Terraform would be
az login
az account set -s "<name of subscription where you want to create resources>"
terraform init
terraform plan
terraform apply

Answer 2

There's another way to do that. You can use the Access Key associated with the Storage Account on the other subscription(the one you want to have the state files on) and export it as an environment variable.
Bash:
export ARM_ACCESS_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query '[0].value' -o tsv)
Powershell:
$env:ARM_ACCESS_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query '[0].value' -o tsv)
Then switch to the subscription you want to deploy to and deploy.