I have deployed resources on Microsoft Azure using terraform. I'm using azure storage account container to save my terraform states. I tried to configure terraform cloud to automate the deployment but I get this error.
Error: A resource with the ID "/subscriptions/{SUBSCRIPTION_ID}/resourceGroups/msk-stage-keyvault" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_resource_group" for more information.
with module.keyvault.azurerm_resource_group.msk-keyvault
on ../../modules/az-keyvault/main.tf line 2, in resource "azurerm_resource_group" "msk-keyvault":
resource "azurerm_resource_group" "msk-keyvault" {
It seems that terraform cloud is not using my backend state in my provider.tf. How do I make terraform cloud use my backend state in provider.tf.
My Backend Provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.91.0"
}
}
backend "azurerm" {
resource_group_name = "msk-configurations"
storage_account_name = "mskconfigurations"
container_name = "key-vault"
key = "stage.tfstate"
}
}
provider "azurerm" {
features {}
subscription_id = var.subscription
tenant_id = var.ternant_id
}
It looks like your main.tf has already existing keyvault state.
So Initially please check if you have already configured keyvault resource in main.tf file or if you have already imported the state.
If its already present in main.tf file , and if you are again giving it in the backend , please try to remove the one from main.tf file and then execute again.
Also please note that terraform backend needs azure storage account credentials before hand in order to store into the tfstate.
So please avoid creating storage simultaneously all account, container and then the keyvault resource to tfstate.
So if storage account is already created first , then terraform can refer it later in backend.
To preconfigure the storage account and container :
Example:
1. Create storage account and container one after the other instead in the same file:
provider "azurerm" {
features {}
}
data "azurerm_resource_group" "example" {
name = "resourcegroupname"
}
resource "azurerm_storage_account" "example" {
name = "<yourstorageaccountname>"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_container" "example" {
name = "newterraformcont"
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
Then create the msk-keyvault resource group and store the tfstate in container.
This is my already created state file in terraform (terraform.tf)
provider "azurerm" {
features {}
}
terraform {
# Configure Terraform State Storage
backend "azurerm" {
resource_group_name = "<resourcegroup>"
storage_account_name = "<storage-earliercreated>"
container_name = " newterraformcont "
key = "terraform.tfstate"
}
}
resource "azurerm_resource_group" " msk-keyvault" {
name = "<msk-keyvault>"
location = "west us"
}
Reference:
azurerm_resource_group | Resources | hashicorp/azurerm | Terraform
Registry
https://www.jorgebernhardt.com/terraform-backend
Related
I have a no. of API management services setup on Azure already but they are not provisioned using Terraform. Now I want to using terraform to update the backend HTTP endpoint and policies. Is it possible to do so? Is there any data sources for apim I can use? Any example I can refer to?
I tried in my environment and got below results:
Initially, I created Azure APIM management service using below terraform code.
main.tf
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "example" {
name = "rg3"
location = "West Europe"
}
resource "azurerm_api_management" "example" {
name = "venkat-demo"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
publisher_name = "My Company"
publisher_email = "company#terraform.io"
sku_name = "Developer_1"
}
resource "azurerm_api_management_api" "example" {
name = "venkatdemo326"
resource_group_name = azurerm_resource_group.example.name
api_management_name = azurerm_api_management.example.name
revision = "1"
display_name = "vjtestAPI"
path = "example"
protocols = ["https"]
import {
content_format = "swagger-link-json"
content_value = "http://conferenceapi.azurewebsites.net/?format=json"
}
}
Portal:
Now I want to using terraform to update the backend HTTP endpoint and policies
You can use the below terraform code to create resources that manage the backend HTTP endpoint and policies of that service.
main.tf
provider "azurerm" {
features {}
}
data "azurerm_api_management" "example" {
name = "venkat-demo"
resource_group_name = data.azurerm_resource_group.example.name
}
resource "azurerm_api_management_backend" "example" {
name = "backend1"
api_management_name = data.azurerm_api_management.example.name
resource_group_name = data.azurerm_api_management.example.resource_group_name
protocol = "http"
url = "https://example.com/api"
}
resource "azurerm_api_management_policy" "example" {
api_management_id = data.azurerm_api_management.example.id
xml_content = <<XML
<policies>
<inbound>
<find-and-replace from="xyz" to="abc" />
</inbound>
</policies>
XML
}
Console:
Portal:
Reference:
azurerm_api_management_backend | Resources | hashicorp/azurerm | Terraform Registry
azurerm_api_management_policy | Resources | hashicorp/azurerm | Terraform Registry
If you want to change an Azure resource (in this case Azure APIM), you'll have to first declare the resource with the corresponding resource type (in this case azurerm_api_management) and then to import your existing resource into your Terraform's state.
After importing, on the subsequent execution of terraform plan, you'll see the changes terraform would like to execute based on the difference between your declaration and the current state of your APIM instance.
You cannot use a terraform data source (like this) if you want to change (=manage) the resource. Data sources can only be used to read attributes from resources.
I'm trying to create a terraform backend in my TF script. The problem is that Im getting errors that the variables are not allowed.
Here is my code:
# Configure the Azure provider
provider "azurerm" {
version = "~> 2.0"
}
# Create an Azure resource group
resource "azurerm_resource_group" "example" {
name = "RG-TERRAFORM-BACKEND"
location = "$var.location"
}
# Create an Azure storage account
resource "azurerm_storage_account" "example" {
name = "$local.backendstoragename"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
tags = "$var.tags"
}
# Create an Azure storage container
resource "azurerm_storage_container" "example" {
name = "example"
resource_group_name = azurerm_resource_group.example.name
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
# Create a Terraform backend configuration
resource "azurerm_terraform_backend_configuration" "example" {
resource_group_name = azurerm_resource_group.example.name
storage_account_name = azurerm_storage_account.example.name
container_name = azurerm_storage_container.example.name
key = "terraform.tfstate"
}
# Use the backend configuration to configure the Terraform backend
terraform {
backend "azurerm" {
resource_group_name = azurerm_terraform_backend_configuration.example.resource_group_name
storage_account_name = azurerm_terraform_backend_configuration.example.storage_account_name
container_name = azurerm_terraform_backend_configuration.example.container_name
key = azurerm_terraform_backend_configuration.example.key
}
}
What am I doing wrong? All of a sudden Terraform init is giving me the following errors:
Error: Variables not allowed
│
│ on main.tf line 65, in terraform:
│ 65: key = azurerm_terraform_backend_configuration.example.key
│
│ Variables may not be used here.
╵
I get the above error for ALL lines.
What am I doing wrong?
I tried to refactor the
azurerm_terraform_backend_configuration.example.container_name
as an interpolation - i.e. "$.." - but that didn't get accepted.
Has anything changed in Terraform? This wasn't the case a few years ago.
I have not found this resource azurerm_terraform_backend_configuration in any of the terraform-provider-azurerm documentation.
Check this URL for search results.
https://github.com/hashicorp/terraform-provider-azurerm/search?q=azurerm_terraform_backend_configuration
I am not even aware of the resource azurerm_terraform_backend_configuration but As of now, terraform-provider-azurerm does not support variables in the backend configuration.
Official documentation on Azurerm Backend
and what you are trying here is creating a Chicken-Egg problem (if I Ignore "azurerm_terraform_backend_configuration"). The initialization of terraform code needs a remote backend and the remote backend requires not just initialization but also terraform apply to the resources which are not possible.
The following are two possible solutions.
1: Create the resources required by the backend manually on the portal and then use them in your backend config. ( values in spite of any data source or variables)
2: Create the resources with the local backend and then migrate the local backend config to the remote backend.
Step 2.1: Create backend resources with local backend initially.
Provider Config
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.37.0"
}
}
required_version = ">= 1.1.0"
}
provider "azurerm" {
features {}
}
Backend resources
locals {
backendstoragename = "stastackoverflow001"
}
# variable defintions
variable "tags" {
type = map(string)
description = "(optional) Tags attached to resources"
default = {
used_case = "stastackoverflow"
}
}
# Create an Azure resource group
resource "azurerm_resource_group" "stackoverflow" {
name = "RG-TERRAFORM-BACKEND-STACKOVERFLOW"
location = "West Europe"
}
# Create an Azure storage account
resource "azurerm_storage_account" "stackoverflow" {
name = local.backendstoragename ## or "${local.backendstoragename}" but better is local.backendstoragename
location = azurerm_resource_group.stackoverflow.location
resource_group_name = azurerm_resource_group.stackoverflow.name
account_tier = "Standard"
account_replication_type = "LRS"
tags = var.tags ## or "${var.tags}" but better is var.tags
}
# Create an Azure storage container
resource "azurerm_storage_container" "stackoverflow" {
name = "stackoverflow"
storage_account_name = azurerm_storage_account.stackoverflow.name
container_access_type = "private"
}
Step 2.2: Apply the code with local backend.
terraform init
terraform plan # to view the plan
terraform apply -auto-approve # ignore `-auto-approve` if not desired auto approval on apply.
After applying you will get the message:
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Step 2.3: Update the backend configuration from local to remote.
Provider Config
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.37.0"
}
}
required_version = ">= 1.1.0"
## Add remote backend config.
backend "azurerm" {
resource_group_name = "RG-TERRAFORM-BACKEND-STACKOVERFLOW"
storage_account_name = "stastackoverflow001"
container_name = "stackoverflow"
key = "terraformstate"
}
}
Re-Initialize the terraform.
After adding your remote backend run ``terraform init -reconfigurecommand and then typeyes` to migrate your local backend to remote backend.
➜ variables_in_azurerm_backend git:(main) ✗ terraform init -reconfigure <aws:sre>
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "azurerm" backend. No existing state was found in the newly
configured "azurerm" backend. Do you want to copy this state to the new "azurerm"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Using previously-installed hashicorp/azurerm v3.37.0
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Now terraform should use the remote backend configured and also will be able to manage the resources created in the steps {2.1 && 2.2}. You can verify this by running terraform plan command and it should give No changes message.
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.
One more Side Note: Version constraints inside provider configuration blocks are deprecated and will be removed in a future version of Terraform
Special Consideraions: Use a different container key and directory for your other infrastructure terraform configurations to avoid accidental destruction of the storage account used for the backend config.
on the way to create aks via terraform, here i want to create azure storage account & want to use the same same account to store the terraform state file.
however getting below error
│ Error: Error loading state: Error retrieving keys for Storage Account "azurerm_resource_group.aks_rg.name": storage.AccountsClient#ListKeys: Invalid input: autorest/validation: validation failed: parameter=accountName constraint=MaxLength value="azurerm_resource_group.aks_rg.name" details: value length must be less than or equal to 24
│
#Create Resource Group
resource "azurerm_resource_group" "aks_rg" {
location = "${var.location}"
name = "${var.global-prefix}-${var.cluster-id}-${var.environment}-azwe-aks-rg"
}
#Create Storage Account & Container
resource "azurerm_storage_account" "storage_acc" {
name = "${var.cluster-id}-storage-account"
resource_group_name = azurerm_resource_group.aks_rg.name
location = azurerm_resource_group.aks_rg.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_container" "storage_container" {
name = "${var.cluster-id}-storage-account-container"
storage_account_name = azurerm_storage_account.storage_acc.name
container_access_type = "private"
}
#store terraform state in remote container
terraform {
# Configure Terraform State Storage
backend "azurerm" {
resource_group_name = "azurerm_resource_group.aks_rg.name"
storage_account_name = "azurerm_storage_container.storage_acc.name"
container_name = "azurerm_storage_container.storage_container.name"
key = "terraform.tfstate"
}
}
You need to first create the storage account and container then while creating the aks cluster you need to give the below:
terraform {
# Configure Terraform State Storage
backend "azurerm" {
resource_group_name = "azurerm_resource_group.aks_rg.name"
storage_account_name = "azurerm_resource_group.aks_rg.name"
container_name = "powermeprodtfstate"
key = "terraform.tfstate"
}
}
Instead of creating the storage account and container ins the same file while storing the terraform tfstate.
Example:
Create storage account and container:
provider "azurerm" {
features {}
}
data "azurerm_resource_group" "example" {
name = "resourcegroupname"
}
resource "azurerm_storage_account" "example" {
name = "yourstorageaccountname"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_container" "example" {
name = "terraform"
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
Then create the aks resource group and store the tfstate in container.
provider "azurerm" {
features {}
}
terraform {
# Configure Terraform State Storage
backend "azurerm" {
resource_group_name = "resourcegroup"
storage_account_name = "storageaccountnameearliercreated"
container_name = "terraform"
key = "terraform.tfstate"
}
}
resource "azurerm_resource_group" "aks_rg" {
name = "aks-rg"
location = "west us"
}
Reference:
How to store the Terraform state file in Azure Storage. » Jorge Bernhardt
I want to update the details of an expired SP through terraform. I can regenerate the SP by changing the expiration date for the SP. but the SP details are been stored in the keyvault. So while updating the keyvault with the same id/secret it errors out. Is there a way to update/delete the key_vault secret through terraform ?
resource "azurerm_key_vault_secret" "sp_arm_client_id"
{
name = "ARM-CLIENT-ID"
value = az_sp.app_id key_vault_id = data.azurerm_key_vault.storable_kvs[each.key].id
}
I tested your scenario in my environment and I was successfully able to do the changes and it got stored in current version and the previous one was in older version using the below code:
provider "azuread" {}
provider "azurerm" {
features{}
}
data "azuread_client_config" "current" {}
data "azuread_application" "appreg" {
display_name="ansumanterraformtest"
}
resource "azuread_application_password" "apppass" {
application_object_id = data.azuread_application.appreg.object_id
end_date_relative = "3h"
}
data "azurerm_key_vault" "kv" {
name = "kvname"
resource_group_name = "ansumantest"
}
resource "azurerm_key_vault_secret" "demo_sp_client_id" {
name = "demo-sp-client-id"
value = data.azuread_application.appreg.application_id
key_vault_id = data.azurerm_key_vault.kv.id
}
resource "azurerm_key_vault_secret" "demo_sp_client_secret" {
name = "demo-sp-client-secret"
value =azuread_application_password.apppass.value
key_vault_id = data.azurerm_key_vault.kv.id
}
Output:
Note: You might be getting the error if that secret in keyvault was not created from terraform . If it was created from portal or any other source then you have to first import that secret to terraform state and then change it so that the terraform can manage it .
Import command:
terraform import azurerm_key_vault_secret.example "https://example-keyvault.vault.azure.net/secrets/example/fdf067c93bbb4b22bff4d8b7a9a56217"
Reference:
azurerm_key_vault_secret | Resources | hashicorp/azurerm | Terraform Registry
My terraform design depends on a pre-provisioned keyvault containing secrets to be used by app services. I imported this key vault into my remote state. I can see it has been imported. Now when I run terraform plan, it acts as if it does not know about the imported resource.
This is how my terraform looks like
provider "azurerm" {
version="=2.20.0"
skip_provider_registration="true"
features{}
}
terraform {
backend "azurerm" {}
}
resource "azurerm_key_vault" "kv" {
name = "${var.env}ActicoDQM-kv"
}
module "app_service_plan"{
source = "./modules/app-service-plan"
...redacted for brevity
tags = var.tags
}
module "app-service"{
source = "./modules/app-service"
...redacted for brevity
tags = var.tags
key_vault_id = azurerm_key_vault.kv.key_vault_id
}
Adding an access policy for the app service inside the module
resource "azurerm_app_service" "app" {
... redacted for brevity
}
identity {
type = "SystemAssigned"
}
}
resource "azurerm_key_vault_access_policy" "app" {
key_vault_id = var.key_vault_id
tenant_id = azurerm_app_service.app.identity[0].tenant_id
object_id = azurerm_app_service.app.identity[0].principal_id
secret_permissions = ["get", "list"]
}
There seems to be some missing link in my understanding, because now when I do
terraform plan
It acts as if it doesn't know about imported keyvault
Error: Missing required argument
on main.tf line 19, in resource "azurerm_key_vault" "kv":
19: resource "azurerm_key_vault" "kv" {
The argument "tenant_id" is required, but no definition was found.
Even though you're importing an existing keyvault into your terraform state you need to fully define all required arguments according to keyvault resource docs.
At minimum your keyvault resource should specify these arguments:
resource "azurerm_key_vault" "kv" {
name = "${var.env}ActicoDQM-kv"
location = ..
resource_group_name = ..
sku_name = "standard" or "premium"
tenant_id = data.azurerm_client_config.current.tenant_id
}
You can expose the tenant_id using a data resource:
data "azurerm_client_config" "current" {
}