I am looking to pass in the provider_version into terragrunt.hcl as a variable to make upgrading / setting the version easier. However this is my current code:
terraform {
backend "s3" {}
required_version = "~> 0.12"
required_providers {
aws = {
source = "hashicorp/aws"
version = "${var.aws_provider_version}"
}
}
}
I am getting an error
61: version = "${var.aws_provider_version}" Variables may not be used here.
Is there a known workaround or is this not possible?
Terraform doesn't support variables in blocks that are inputs to terraform itself, like provider blocks or lifecycle attributes.
You may be able to use code generation to set up a small providers.tf file before running terraform if you need to update your provider version at build time.
Related
We are using hashicorp/google provider #3.90.0 version but for one specific resource we want to use hashicorp/google provider #4.31.0 and continue using #3.90.0 everywhere else. Is there a way to use different provider version for just one block:
as of now provider.tf:
terraform {
required_version = ">= 0.13.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 3.45, <= 3.90.0"
}
}
}
main.tf here in this block we want to use version 4.31.0 for google provider:
resource "google_storage_bucket" "cdn-bucket" {
project = var.project_id
name = "cdn-${var.project_id}"
location = "US"
storage_class = "MULTI_REGIONAL"
There is no way to use multiple versions of the same provider in the same configuration. You will need to either make all of your modules have some provider version they are all mutually compatible with, or to split your configuration into multiple parts so that each part can depend on a different version of the provider and be applied separately.
I had the following Terraform configuration for my backend that worked for a while,
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
}
}
backend "azurerm" {
resource_group_name = "my-rg"
storage_account_name = "my-sa"
####!!!!! BELOW USED TO WORK !!!!###
provider = azurerm.mysub
key = "terraform.tfstate"
}
}
provider "azurerm" {
skip_provider_registration = true
subscription_id = "xxxxxx-xxxxx-xxxxx-xxxxx"
alias = "mysub"
features {
}
}
However after an upgrade it said that provider was not allowed in this block (I dont recall teh exact error message). So instead I changed it as such,
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
}
}
backend "azurerm" {
resource_group_name = "my-rg"
storage_account_name = "my-sa"
###!!!! Direct Reference to Subscription ID !!!!###
subscription_id = "xxxxxx-xxxxx-xxxxx-xxxxx"
key = "terraform.tfstate"
}
}
However now it says,
│ Error: Backend initialization required, please run "terraform init"
│
│ Reason: Backend configuration changed for "azurerm"
│
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
│ use the current configuration.
│
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
If I want it to remember the state so that it knows how to destroy the same later, what command should I use? I have already deployed several iteration of resources in this RG using terraform and want to keep that intact.
terraform init, OR,
terraform init -reconfigure, OR,
terraform init -migrate-state??
Since the backend location has not changed, I want to continue as is but just have it ignore the backend block update from using "provider" to using the "subscription_id". Which command do I use?
Thanks in advance!
The terraform init documentation says the following about this situation:
Re-running init with an already-initialized backend will update the working directory to use the new backend settings. Either -reconfigure or -migrate-state must be supplied to update the backend configuration.
The -migrate-state option will attempt to copy existing state to the new backend, and depending on what changed, may result in interactive prompts to confirm migration of workspace states. The -force-copy option suppresses these prompts and answers "yes" to the migration questions. This implies -migrate-state.
The -reconfigure option disregards any existing configuration, preventing migration of any existing state.
The decision point here is whether you want Terraform to take explicit action to try to copy the state to the new location (-migrate-state) or whether you want Terraform to just forget the old settings entirely and just use the new settings directly.
You said that the physical location is unchanged and instead you've just written the same information a different way, and so -reconfigure is the option that matches that situation: there's no need for any explicit migration here because the state is already available at the "new" location (which is functionally the same as the old location, but Terraform can't know that).
Note that it has never been valid to associate a backend configuration with a provider, so whatever you had working before wasn't working in the way you thought it was.
The azurerm backend has the behavior of looking for the ARM_SUBSCRIPTION_ID environment variable if you don't explicitly set subscription_id in its configuration, so I'd guess that you were previously running Terraform in a context where that environment variable was set and thus the backend was able to find the appropriate subscription ID to use even though you hadn't explicitly set it.
Why the backend wasn't rejecting the invalid argument provider is unclear to me. That suggests a bug in either Terraform or in the backend itself, which has since been fixed and thus Terraform is now correctly reporting that there is no argument provider declared in that backend's configuration schema.
I am not sure if this is the proper way to do this, but I was able to get this resolved without the initialization message coming up anymore by updated the local terraform.tfstate file.
Changed this line
"subscription_id": null,
to,
"subscription_id": "xxxx-xxxx-xxxx-xxxx",
Description:
I am trying to use an Elasticsearch provider for Terraform. Since there is no official one from Elastic or from Hashicorp I am trying to use a community one "https://registry.terraform.io/providers/phillbaker/elasticsearch/latest".
Terraform version: Terraform v0.14.4
Code:
I tried to put everything in 1 .tf file. I also tried to create a separate module for the resources like Hashicorp recommends. Both methods generate the same error message.
terraform {
required_providers {
elk = {
source = "phillbaker/elasticsearch"
version = "1.5.1"
}
}
}
provider "elk" {
url = "https://<my_elk_server>"
}
resource "elasticsearch_index" "index" {
name = var.elasticsearch_index_name
}
Problem:
terraform init isn't able to find the appropriate provider in the Terraform Registry for some reason.
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/elasticsearch...
- Finding phillbaker/elasticsearch versions matching "1.5.1"...
- Installing phillbaker/elasticsearch v1.5.1...
- Installed phillbaker/elasticsearch v1.5.1 (self-signed, key ID 02AD42CD82B6A957)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
Error: Failed to query available provider packages
https://www.terraform.io/docs/plugins/signing.html
Could not retrieve the list of available versions for provider
hashicorp/elasticsearch: provider registry registry.terraform.io does not have
a provider named registry.terraform.io/hashicorp/elasticsearch
If you have just upgraded directly from Terraform v0.12 to Terraform v0.14
then please upgrade to Terraform v0.13 first and follow the upgrade guide for
that release, which might help you address this problem.
No tfstate files are being generated.
How do I use third party providers from the Terraform Registry ?
In your required_providers block you've told Terraform that you intend to refer to this provider as "elk" within this module:
elk = {
source = "phillbaker/elasticsearch"
version = "1.5.1"
}
Typically you'd set the local name of the provider to be the same as the "type" portion of the provider source address, like this:
elasticsearch = {
source = "phillbaker/elasticsearch"
version = "1.5.1"
}
If you change the local name in this way then use references to elasticsearch elsewhere in the module should then refer to the community provider as you intended.
Note that means you'll also need to change the provider block so it has a matching local name:
provider "elasticsearch" {
url = "https://<my_elk_server>"
}
A different approach here would be to continue to use elk as the name and then change the rest of the configuration to properly refer to that non-default name. I don't recommend doing this, because typically I'd expect the local name to only mismatch the type in the unusual case where your module depends on two providers with the same type name, but I'm mentioning this in the hope that it helps to understand how the Terraform language infers provider dependencies when not given explicitly:
terraform {
required_providers {
elk = {
source = "phillbaker/elasticsearch"
version = "1.5.1"
}
}
}
# "elk" here is matched with the local names in the
# required_providers block, so this will work.
provider "elk" {
url = "https://<my_elk_server>"
}
# This "elasticsearch_" prefix causes Terraform to look
# for a provider with the local name "elasticsearch"
# by default...
resource "elasticsearch_index" "index" {
# ...so if you've given the provider a different local
# name then you need to associate the resource with
# the provider configuration explicitly:
provider = elk
name = var.elasticsearch_index_name
}
I expect most Terraform users would find the above approach surprising, so in the interests of using familiar Terraform idiom I'd suggest instead following my first suggestion of renaming the local name to elasticsearch, which will then allow the automatic resource-to-provider association to work.
So, after testing it seems putting the whole code in the same .tf file does the job.
terraform {
required_providers {
elasticsearch = {
source = "phillbaker/elasticsearch"
version = "1.5.1"
}
}
}
provider "elasticsearch" {
url = "http://127.0.0.1:9200"
}
resource "elasticsearch_index" "index" {
name = var.index_name
}
If you want to create a separate module for it you can just source it from another module:
module "elastic" {
index_name = var.index_name
source = "./modules/elastic"
}
Check Martin's answer for more information.
In a CI/CD context, I would like to define provider versions outside my terraform configuration using TF_VAR_ environment variables.
I'm trying to use input variable to set the version of helm provider in versions.tf (terraform 0.12) but it seems not allowed :
Error: Invalid provider_requirements syntax
on versions.tf line 3, in terraform:
3: helm = "${var.helm_version}"
provider_requirements entries must be strings or objects.
Error: Variables not allowed
on versions.tf line 3, in terraform:
3: helm = "${var.helm_version}"
Variables may not be used here.
How can I configure this ?
If it's not possible, how I can manage the terraform provider version outside my configuration ?
Cannot be done. I wish it could be done. terraform init resolves and downloads the providers, you won't have access to variables at that point.
Each terraform block can contain a number of settings related to
Terraform's behavior. Within a terraform block, only constant values
can be used; arguments may not refer to named objects such as
resources, input variables, etc, and may not use any of the Terraform
language built-in functions.
https://www.terraform.io/docs/configuration/terraform.html
As #thekbb says, it's not possible to get access to version variable during terraform init at least in 0.12.20. However, I've below workaround to manage providers outside your configuration.
You could use alias with provider configuration to achieve this. Let's assume you want 1.3.0 version of helm. Rather than passing it as a var, you could define it statically with an alias like below.
provider "helm" {
alias = "helm-stable"
version = "1.3.0" (the version you pass via TF_VAR_helm_version)
kubernetes {
host = "https://104.196.242.174"
username = "ClusterMaster"
password = "MindTheGap"
client_certificate = file("~/.kube/client-cert.pem")
client_key = file("~/.kube/client-key.pem")
cluster_ca_certificate = file("~/.kube/cluster-ca-cert.pem")
}
}
Then, in your resource or data providers, you could point to a particular provider like below::
data "some_ds" "example" {
name = "dummy"
provider = helm.helm-stable
}
For more details, refer to the below links::
providers
allow variable in provider field
I see a file like this in my Terraform code:
terraform {
required_version = ">= 0.12"
}
but I'm not clear what the outer terraform {..} block is for and can't find any documentation.
I've seen output, resource, data, etc but not terraform. Any suggestions (or documentation)?
It is basically for the top level configuration of your terraform environment. For example, the version lock you are using. And you can set the your "backend"/source of tfstate file and more.
With my experience, you would need this require_version and backend specified to guarantee your environment is clean and consistent.
Here is the doc for terraoform block: https://www.terraform.io/docs/configuration/terraform.html