I have worked with terraform before, where terraform can place the tfstate files in S3. Does terraform also support azure blob storage as a backend? What would be the commands to set the backend to be azure blob storage?
As of Terraform 0.7 (not currently released but you can compile from source) support for Azure blob storage has been added.
The question asks for some commands, so I'm adding a little more detail in case anyone needs it. I'm using Terraform v0.12.24 and azurerm provider v2.6.0. You need two things:
Create a storage account (general purpose v2) and a container for storing your states.
Configure your environment and your main.tf
As for the second point, your terraform block in main.tf should contain a "azurerm" backend:
terraform {
required_version = "=0.12.24"
backend "azurerm" {
storage_account_name = "abcd1234"
container_name = "tfstatecontainer"
key = "example.prod.terraform.tfstate"
}
provider "azurerm" {
version = "=2.6.0"
features {}
subscription_id = var.subscription_id
}
Before calling to plan or apply, init the ARM_ACCESS_KEY variable with a bash export:
export ARM_ACCESS_KEY=<storage access key>
Finally, run the init command:
terraform init
Now, if you run terraform plan you will see the tfstate created in the container. Azure has a file locking feature built in, in case anyone tries to update the state file at the same time.
Related
I want to create two different workspaces on Terraform Cloud: One for DEV environment, the other for PROD environment.
I am trying to create them hust using a single configuration file. The infrastructure will be the same just in two different Azure subscriptions with different credentials.
Here the code I am trying:
terraform {
required_version = ">= 1.1.0"
required_providers {
#https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.40.0"
}
}
cloud {
organization = "mycompany"
workspaces {
tags = ["dev", "prod"]
}
}
}
I am watching the documentantion. It seems like inside the cloud -> workspace command I just can use either name or tags attributes. It is required I have at least one of them in my configuration.
Now in my Terraform Cloud account, I have two workspaces: 1 with the tag prod and one with the tag dev.
I set the envinroment variable:
$Env:TF_WORKSPACE="mycompany-infrastructure-dev"
And I try to initialize Terraform Cloud:
terraform init
But I get this error:
Error: Invalid workspace selection
Terraform failed to find workspace "mycompany-infrastructure-dev" with the tags specified in your configuration: │ [dev, prod]
How can I create one configuration that I can use with different environment/workspaces?
Thank you
First, I ran the similar code as yours in my environment and received an error shown below: It prompted me to use terraform login to generate a token for accessing the organization on Terraform Cloud.
The login was successful, and the browser generated an API token.
Token received and entered.
Logged into Terraform cloud as shown:
In Terraform Cloud -> Organizations, I created a new organization:
Script for creating different workspaces from a single configuration file:
cloud {
organization = "mycompanyone"
workspaces {
tags = ["dev", "prod"]
}
}
Taken your script and made a few changes as seen below:
Terraform will prompt for basic concerns while initializing, as shown here.
Now run terraform init or terraform init -upgrade.
terraform initialized successfully:
How to disable public access to Azure storage account but still accessible from cloudshell.
What I have and works:
Az-storage account that contains "terraform.tfstate" with public access
main.tf file in my "Azure Cloudshell" with "backend" config for remote statefile
terraform {
required_version = ">= 1.2.4"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 2.98.0"
}
}
# To store the state in a storage account
# Benefit=working with team and if local shell destroyed -> state=lost)
backend "azurerm" {
resource_group_name = "RG-Telco-tf-statefiles"
storage_account_name = "telcostatefiles"
container_name = "tf-statefile-app-1"
key = "terraform.tfstate"
}
}
This works perfectly.
But if I restrict public access in the storage account, my "Azure Cloudshell" has no permission to the statefile anymore.
How can I make it work and what are the best security best practices in this case?
I think this is what you need.
After you set this, you can make a network restriction rule and you can allow the cloud shell virtual network.
Some other best practices:
The storage account that stores the state file should be in a separate resource group and have a delete lock on it.
1 SAS token per user renewed every 6 months with a scope at the folder level, one container per project, and per environment
Storage with redundancy in a paired region for reading access in case of issues
I am noticing this wierd error since I moved whole of my code from 1.42.0 provider version to 2.19.0. I am creating several keyvault secrets. Earlier it when I try to execute a plan after appplying once, it used to refresh the state and proceed, but now after updating the provider version, I am noticing the below error.
Error: A resource with the ID "https://mytestingvault.vault.azure.net/secrets/hub-access/060e71ecd1084cb5a6a496f77a2aea5c" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_key_vault_secret" for more information.
Additionally I have added lifecycle ignore changes to see if it could skip reading the vault secret changes but unfortunately the same error is shown. Only way to get rid of the error is to delete the secret. What am i wrong here?
lifecycle {
ignore_changes = [
value,name
]
}
The Terraform key vault documentation says:
Terraform will automatically recover a soft-deleted Key Vault during
Creation if one is found - you can opt out of this using the features
block within the Provider block.
You should configure your Terraform to stop recovering the softly deleted Key Vault like this:
provider "azurerm" {
features {
key_vault {
recover_soft_deleted_key_vaults = false
}
}
}
If you want Terraform to purge any softly deleted Key Vaults when using terraform destroy use this additional line:
provider "azurerm" {
features {
key_vault {
purge_soft_delete_on_destroy = true
recover_soft_deleted_key_vaults = false
}
}
}
You probably need to read up on the general topic of Terraform state management. This is not specific to your Key Vault secret. Some resource (your secret) exists that was not created by the terraform workspace that you are just executing. TF does not like that. So you either need to import this pre-existing resource into the state of this workspace, or delete it so that it can be re-created (and thereby managed)
The issue was that keyvault even though was deleted, we had to purge it via cli using aws cli purge.
I'm building terraform scripts to orcastrate Azure deployment. I used Azure blob storage to store a tfstate file. This file is shared with several pipelines IAC pipelines.
If for instance I create an Azure Resource Group with terraform, when that is done, I try to create a new custom role, terraform plan will mark the Resource Group for destruction.
This is the script for the role creation:
terraform {
backend "azurerm" {
storage_account_name = "saiac"
container_name = "tfstate"
key = "dev.terraform.tfstate"
resource_group_name = "rg-devops"
}
}
data "azurerm_subscription" "primary" {
}
resource "azurerm_role_definition" "roles" {
count = length(var.roles)
name = "${var.role_prefix}${var.roles[count.index]["suffix_name"]}${var.role_suffix}"
scope = "${data.azurerm_subscription.primary.id}"
permissions {
actions = split(",", var.roles[count.index]["actions"])
not_actions = split(",", var.roles[count.index]["not_actions"])
}
assignable_scopes = ["${data.azurerm_subscription.primary.id}"]
}
and this is script for resource group creation:
terraform {
backend "azurerm" {
storage_account_name = "saiac"
container_name = "tfstate"
key = "dev.terraform.tfstate"
resource_group_name = "rg-devops"
}
}
resource "azurerm_resource_group" "rg" {
count = "${length(var.rg_purposes)}"
name = "${var.rg_prefix}-${var.rg_postfix}-${var.rg_purposes[count.index]}"
location = "${var.rg_location}"
tags = "${var.rg_tags}"
}
If I remove the backend block, everything works as expected, does that mean I need the backend block?
Terraform use the .tfstate file to check and compare your code and existing cloud infra structure, it is like backbone of terraform.
If your code and existing infra is differe, terraform will destroy it and apply code changes.
To overcome this, terraform provides the import facility, you can import the existing resource and terraform will update it's .tfstate file.
This .tfstate file must be specify into your backend.tf file,best practices is to store your .tfstate file on cloude storage not in local directory.
When you run the terraform init command, it will check for the .tfstate file.
below is the sample file for backend.tf file (aws s3 is used):
backend "s3" {
bucket = "backends.terraform.file"
key = "my-terraform.tfstate_key"
region = "my-region-1"
encrypt = "false"
acl = "bucket-owner-full-control"
}
}
A terraform backend is not required for terraform. If you do not use it however no one else will be able to pull your code and run your terraform. The state will ONLY be stored in your .terraform directory. This means if you lose your local files your in trouble. It is recommended to use a backend that also supports state locking which azurerm does. With a backend in place the state will get pulled on terraform init after pulling the repo.
Does Terraform Cloud support the -target flag when running terraform plan...?
It doesn't seem like there's an option to turn on or off this feature in Terraform Cloud. I'm wondering if this means that Terraform Cloud as a whole does not support Module targeting, or if there is an option in my instance of Terraform Cloud that turns off this feature.
Expected Result: Terraform successfully creates the plan.
Actual Result: Terraform reports the following error:
Error: Resource targeting is currently not supported
The "remote" backend does not support resource targeting at this time.
Edit 9/30/19:
I'm using Terraform Cloud's "Remote Executor" and Terraform version 0.12.9.
If you refer to https://github.com/hashicorp/terraform/blob/a8d01e3940b406a3c974bfaffd0ca5f534363cc7/backend/remote/backend_plan.go#L73 you see that there is an explicit check in Terraform Cloud backend that disables Plan targeting
In the same piece of code you can find all checks and limitations that are currently in effect in Terraform Cloud.
If you need those features, you can migrate your backend to something like aws s3: https://www.terraform.io/docs/backends/types/s3.html
To learn more about backends refer to: https://www.terraform.io/docs/backends/index.html
# main.tf
resource "null_resource" "test" {
}
resource "null_resource" "test2" {
}
terraform {
backend "remote" {
hostname = "app.terraform.io"
organization = "<my-org>"
workspaces {
name = "<my-workspace>"
}
}
}
I didn't face the error above when running terraform plan:
❯ terraform plan -target=null_resource.test -out=plan.tfplan
Acquiring state lock. This may take a few moments...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# null_resource.test will be created
+ resource "null_resource" "test" {
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
This plan was saved to: plan.tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "plan.tfplan"
Releasing state lock. This may take a few moments...
Here is my version:
❯ terraform version
Terraform v0.12.6
+ provider.null v2.1.2
Your version of Terraform is out of date! The latest version
is 0.12.9. You can update by downloading from www.terraform.io/downloads.html