I am deploying an azure infrastructure with Terraform. The terraform state will be stored in a subscription which will be different from the main deployment subscription. I am using alias in provider declaration. My terraform code is like below:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 2.38.0"
}
}
backend "azurerm" {
resource_group_name = "resourcegroup_name"
storage_account_name = "storageaccount_name"
container_name = "mystate"
key = "tfstatename1.tfstate"
}
}
provider "azurerm" {
features {}
}
provider "azurerm" {
features {}
alias = "second_subscription"
subscription_id = var.second_subscription_id
}
My terraform state should be stored in the subscription with alias.
How can i achieve that?
I don't think the azurerm backend configuration is taking input from the azurerm provider configuration. To some extent you could say it applies its own authentication mechanism. But there are some features they share nevertheless: e.g. both are capable of using the Azure CLI security context.
In order to explicitly target a subscription id for your backend configuration, you must add it to the backend configuration block. Like so:
backend "azurerm" {
resource_group_name = "resourcegroup_name"
storage_account_name = "storageaccount_name"
container_name = "mystate"
key = "tfstatename1.tfstate"
subscription_id = "091f1800-0de3-4fef-831a-003a74ce245f"
}
Reference: https://developer.hashicorp.com/terraform/language/settings/backends/azurerm
Related
I'm fairly new to Terraform. I'm trying to use Azure Service Principal for terraform. My code looks like below. However when I run terraform init, I get an error Error building ARM Config: obtain subscription() from Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.
(I have deliberately logged out of Azure CLI to verify that the terraform code is NOT using azure cli. And looks like it's still looking for Azure CLI.)
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.0.0"
}
}
backend "azurerm" {
resource_group_name = "rg-XXXX"
storage_account_name = "saXXXXX"
container_name = "XXXXX"
key = "XXXXX"
}
}
provider "azurerm" {
features {}
subscription_id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
client_id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
client_secret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
tenant_id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
There are missing info allowing terraform to connect to the azurerm backend.
https://developer.hashicorp.com/terraform/language/settings/backends/azurerm#resource_group_name-2
You need to add client_id, client_secret and tenant_id in the backend configuration
backend "azurerm" {
resource_group_name = "rg-XXXX"
storage_account_name = "saXXXXX"
container_name = "XXXXX"
key = "XXXXX"
client_id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
client_secret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
tenant_id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
You may use environment variables for this. You won't have to repeat them and it will allow you to commit your template to a source code repository without sensitive information.
I'm trying to create a terraform backend in my TF script. The problem is that Im getting errors that the variables are not allowed.
Here is my code:
# Configure the Azure provider
provider "azurerm" {
version = "~> 2.0"
}
# Create an Azure resource group
resource "azurerm_resource_group" "example" {
name = "RG-TERRAFORM-BACKEND"
location = "$var.location"
}
# Create an Azure storage account
resource "azurerm_storage_account" "example" {
name = "$local.backendstoragename"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
tags = "$var.tags"
}
# Create an Azure storage container
resource "azurerm_storage_container" "example" {
name = "example"
resource_group_name = azurerm_resource_group.example.name
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
# Create a Terraform backend configuration
resource "azurerm_terraform_backend_configuration" "example" {
resource_group_name = azurerm_resource_group.example.name
storage_account_name = azurerm_storage_account.example.name
container_name = azurerm_storage_container.example.name
key = "terraform.tfstate"
}
# Use the backend configuration to configure the Terraform backend
terraform {
backend "azurerm" {
resource_group_name = azurerm_terraform_backend_configuration.example.resource_group_name
storage_account_name = azurerm_terraform_backend_configuration.example.storage_account_name
container_name = azurerm_terraform_backend_configuration.example.container_name
key = azurerm_terraform_backend_configuration.example.key
}
}
What am I doing wrong? All of a sudden Terraform init is giving me the following errors:
Error: Variables not allowed
│
│ on main.tf line 65, in terraform:
│ 65: key = azurerm_terraform_backend_configuration.example.key
│
│ Variables may not be used here.
╵
I get the above error for ALL lines.
What am I doing wrong?
I tried to refactor the
azurerm_terraform_backend_configuration.example.container_name
as an interpolation - i.e. "$.." - but that didn't get accepted.
Has anything changed in Terraform? This wasn't the case a few years ago.
I have not found this resource azurerm_terraform_backend_configuration in any of the terraform-provider-azurerm documentation.
Check this URL for search results.
https://github.com/hashicorp/terraform-provider-azurerm/search?q=azurerm_terraform_backend_configuration
I am not even aware of the resource azurerm_terraform_backend_configuration but As of now, terraform-provider-azurerm does not support variables in the backend configuration.
Official documentation on Azurerm Backend
and what you are trying here is creating a Chicken-Egg problem (if I Ignore "azurerm_terraform_backend_configuration"). The initialization of terraform code needs a remote backend and the remote backend requires not just initialization but also terraform apply to the resources which are not possible.
The following are two possible solutions.
1: Create the resources required by the backend manually on the portal and then use them in your backend config. ( values in spite of any data source or variables)
2: Create the resources with the local backend and then migrate the local backend config to the remote backend.
Step 2.1: Create backend resources with local backend initially.
Provider Config
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.37.0"
}
}
required_version = ">= 1.1.0"
}
provider "azurerm" {
features {}
}
Backend resources
locals {
backendstoragename = "stastackoverflow001"
}
# variable defintions
variable "tags" {
type = map(string)
description = "(optional) Tags attached to resources"
default = {
used_case = "stastackoverflow"
}
}
# Create an Azure resource group
resource "azurerm_resource_group" "stackoverflow" {
name = "RG-TERRAFORM-BACKEND-STACKOVERFLOW"
location = "West Europe"
}
# Create an Azure storage account
resource "azurerm_storage_account" "stackoverflow" {
name = local.backendstoragename ## or "${local.backendstoragename}" but better is local.backendstoragename
location = azurerm_resource_group.stackoverflow.location
resource_group_name = azurerm_resource_group.stackoverflow.name
account_tier = "Standard"
account_replication_type = "LRS"
tags = var.tags ## or "${var.tags}" but better is var.tags
}
# Create an Azure storage container
resource "azurerm_storage_container" "stackoverflow" {
name = "stackoverflow"
storage_account_name = azurerm_storage_account.stackoverflow.name
container_access_type = "private"
}
Step 2.2: Apply the code with local backend.
terraform init
terraform plan # to view the plan
terraform apply -auto-approve # ignore `-auto-approve` if not desired auto approval on apply.
After applying you will get the message:
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Step 2.3: Update the backend configuration from local to remote.
Provider Config
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.37.0"
}
}
required_version = ">= 1.1.0"
## Add remote backend config.
backend "azurerm" {
resource_group_name = "RG-TERRAFORM-BACKEND-STACKOVERFLOW"
storage_account_name = "stastackoverflow001"
container_name = "stackoverflow"
key = "terraformstate"
}
}
Re-Initialize the terraform.
After adding your remote backend run ``terraform init -reconfigurecommand and then typeyes` to migrate your local backend to remote backend.
➜ variables_in_azurerm_backend git:(main) ✗ terraform init -reconfigure <aws:sre>
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "azurerm" backend. No existing state was found in the newly
configured "azurerm" backend. Do you want to copy this state to the new "azurerm"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Using previously-installed hashicorp/azurerm v3.37.0
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Now terraform should use the remote backend configured and also will be able to manage the resources created in the steps {2.1 && 2.2}. You can verify this by running terraform plan command and it should give No changes message.
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.
One more Side Note: Version constraints inside provider configuration blocks are deprecated and will be removed in a future version of Terraform
Special Consideraions: Use a different container key and directory for your other infrastructure terraform configurations to avoid accidental destruction of the storage account used for the backend config.
I have deployed resources on Microsoft Azure using terraform. I'm using azure storage account container to save my terraform states. I tried to configure terraform cloud to automate the deployment but I get this error.
Error: A resource with the ID "/subscriptions/{SUBSCRIPTION_ID}/resourceGroups/msk-stage-keyvault" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_resource_group" for more information.
with module.keyvault.azurerm_resource_group.msk-keyvault
on ../../modules/az-keyvault/main.tf line 2, in resource "azurerm_resource_group" "msk-keyvault":
resource "azurerm_resource_group" "msk-keyvault" {
It seems that terraform cloud is not using my backend state in my provider.tf. How do I make terraform cloud use my backend state in provider.tf.
My Backend Provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.91.0"
}
}
backend "azurerm" {
resource_group_name = "msk-configurations"
storage_account_name = "mskconfigurations"
container_name = "key-vault"
key = "stage.tfstate"
}
}
provider "azurerm" {
features {}
subscription_id = var.subscription
tenant_id = var.ternant_id
}
It looks like your main.tf has already existing keyvault state.
So Initially please check if you have already configured keyvault resource in main.tf file or if you have already imported the state.
If its already present in main.tf file , and if you are again giving it in the backend , please try to remove the one from main.tf file and then execute again.
Also please note that terraform backend needs azure storage account credentials before hand in order to store into the tfstate.
So please avoid creating storage simultaneously all account, container and then the keyvault resource to tfstate.
So if storage account is already created first , then terraform can refer it later in backend.
To preconfigure the storage account and container :
Example:
1. Create storage account and container one after the other instead in the same file:
provider "azurerm" {
features {}
}
data "azurerm_resource_group" "example" {
name = "resourcegroupname"
}
resource "azurerm_storage_account" "example" {
name = "<yourstorageaccountname>"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_container" "example" {
name = "newterraformcont"
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
Then create the msk-keyvault resource group and store the tfstate in container.
This is my already created state file in terraform (terraform.tf)
provider "azurerm" {
features {}
}
terraform {
# Configure Terraform State Storage
backend "azurerm" {
resource_group_name = "<resourcegroup>"
storage_account_name = "<storage-earliercreated>"
container_name = " newterraformcont "
key = "terraform.tfstate"
}
}
resource "azurerm_resource_group" " msk-keyvault" {
name = "<msk-keyvault>"
location = "west us"
}
Reference:
azurerm_resource_group | Resources | hashicorp/azurerm | Terraform
Registry
https://www.jorgebernhardt.com/terraform-backend
I need to store my terraform state file in a single azure storage account and deploy it to multiple cloud subscriptions(china and public cloud)..
I have my backend configured as below:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.78.0"
}
}
backend "azurerm" {
resource_group_name = "Terraform-rg"
storage_account_name = "terraformstate"
container_name = "tfstate"
subscription_id = "00000000-0000-0000-0000-000000000000"
key = "prod"
}
}
provider "azurerm" {
features {}
}
provider "azurerm" {
features {}
alias = "sub2"
subscription_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_secret = var.client_secret
tenant_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
}
However, when I am trying to terraform init with this approach, I am getting the below error:
Failed to get configured named states: Error retrieving keys for Storage Account "terraformstate": storage.AccountsClient#ListKeys: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="SubscriptionNotFound" Message="The subscription 'xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx' could not be found."
I was wondering if it is even possible to do this?
I also have a script to terraform init and terraform plan which would switch the cloud based on env..
"az cloud set --name AzureChinaCloud" for china and for non china "az cloud set --name AzureCloud"
The storage account subscription is in AzureCloud.. How do I authorise to write the state file for AzureChinaCloud in AzureCloud?
Is this approach even possible?
As mentioned in the comment section, The issue in your code is you are missing Environment parameter in the provider block . As you were manually setting the cloud environment so, the code was not able to find the Subscription in the Backend Configuration which is in public cloud .
So, the solution will be setting the parameter in the code like below and not manually using az CLI :
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.78.0"
}
}
backend "azurerm" {
resource_group_name = "Terraform-rg"
storage_account_name = "terraformstate"
container_name = "tfstate"
subscription_id = "00000000-0000-0000-0000-000000000000"
key = "prod"
}
}
provider "azurerm" {
features {}
}
provider "azurerm" {
features {}
alias = "USGovernment"
subscription_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_secret = var.client_secret
tenant_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
environment = "usgovernment"
}
provider "azurerm" {
features {}
alias = "China"
subscription_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
client_secret = var.client_secret
tenant_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
environment = "china"
}
Reference:
Docs overview | hashicorp/azurerm | Terraform Registry
SO thread
I am following this guide https://github.com/hashicorp/terraform/blob/v0.13/website/docs/configuration/modules.html.md#passing-providers-explicitly
This is a brand new Terraform project ie no resources and an empty state file! The version of terraform is Terraform v0.14.5.
In the calling module I have my providers set up as follows
terraform {
backend "azurerm" {
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
}
provider "azurerm" {
alias = "lz"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
provider "azurerm" {
alias = "prd"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
And I am calling modules passing multiple providers like this
module "prod" {
source = "../../../Terraform/modules/LandingZone"
providers = {
azurerm.lz = azurerm.lz
azurerm.prd = azurerm.prd
}
}
In the called module I have this in the providers.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
backend "azurerm" {
}
}
provider "azurerm" {
alias = "lz"
features{}
}
provider "azurerm" {
alias = "prd"
features{}
}
Up to this point init and plan works fine. However when I try to create resources like this
resource "azurerm_resource_group" "this" {
provider= azurerm.lz
for_each = var.rg_names
name = each.value
location = "Australia Southeast"
}
I get this error message
Error: Provider configuration not present
To work with module.trn.module.rg.azurerm_resource_group.this its original
provider configuration at
module.trn.module.rg.provider["registry.terraform.io/hashicorp/azurerm"].lz is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.trn.module.rg.azurerm_resource_group.this, after which you can remove
the provider configuration again.
From the guide you have linked,
A proxy configuration block is one that contains only the alias argument.
Your provider blocks in your module have more than just the alias argument, so they are probably not being set up as proxy provider configurations. Try removing the features{} part from the provider blocks inside your module.
Referring to this guide you should declare configuration_aliases inside terraform block for required providers. You can try this also.