I would like to use the same terraform template for several dev and production environments.
My approach:
As I understand it, the resource name needs to be unique, and terraform stores the state of the resource internally. I therefore tried to use variables for the resource names - but it seems to be not supported. I get an error message:
$ terraform plan
var.env1
Enter a value: abc
Error asking for user input: Error parsing address 'aws_sqs_queue.SqsIntegrationOrderIn${var.env1}': invalid resource address "aws_sqs_queue.SqsIntegrationOrderIn${var.env1}"
My terraform template:
variable "env1" {}
provider "aws" {
region = "ap-southeast-2"
}
resource "aws_sqs_queue" "SqsIntegrationOrderIn${var.env1}" {
name = "Integration_Order_In__${var.env1}"
message_retention_seconds = 86400
receive_wait_time_seconds = 5
}
I think, either my approach is wrong, or the syntax. Any ideas?
You can't interpolate inside the resource name. Instead what you should do is as #BMW have mentioned in the comments, you should make a terraform module that contains that SqsIntegrationOrderIn inside and takes env variable. Then you can use the module twice, and they simply won't clash. You can also have a look at a similar question I answered.
I recommend using a different workspace for each environment. This allows you to specify your configuration like this:
variable "env1" {}
provider "aws" {
region = "ap-southeast-2"
}
resource "aws_sqs_queue" "SqsIntegrationOrderIn" {
name = "Integration_Order_In__${var.env1}"
message_retention_seconds = 86400
receive_wait_time_seconds = 5
}
Make sure to make the name of the "aws_sqs_queue" resource depending on the environment (e.g. by including it in the name) to avoid name conflicts in AWS.
Related
We want to deploy services into several regions.
Looks like because of the aws provider, we can't just use count or for_each, as the provider can't be interpolated. Thus I need to set this up manually:
resource "aws_instance" "app-us-west-1" {
provider = aws.us-west-1
#other stuff
}
resource "aws_instance" "app-us-east-1" {
provider = aws.us-east-1
#other stuff
}
I would like when running this to create a file which contains all the IPs created (for an ansible inventory).
I was looking at this answer:
https://stackoverflow.com/a/61788089/169252
and trying to adapt it for my case:
resource "local_file" "app-hosts" {
content = templatefile("${path.module}/templates/app_hosts.tpl",
{
hosts = aws_instance[*].public_ip
}
)
filename = "app-hosts.cfg"
}
And then setting up the template accordingly.
But this fails:
Error: Invalid reference
on app.tf line 144, in resource "local_file" "app-hosts":
122: hosts = aws_instance[*].public_ip
A reference to a resource type must be followed by at least one attribute
access, specifying the resource name
I am suspecting that I can't just reference all the aws_instance defined as above like this. Maybe to refer to all aws_instance in this file I need to use a different syntax.
Or maybe I need to use a module somehow. Can someone confirm this?
Using terraform v0.12.24
EDIT: The provider definitions use alias and it's all in the same app.tf, which I was naively assuming to be able to apply in one go with terraform apply (did I mention I am a beginner with terraform?):
provider "aws" {
alias = "us-east-1"
region = "us-east-1"
}
provider "aws" {
alias = "us-west-1"
region = "us-west-1"
}
My current workaround is to not do a join but simply listing them all individually:
{
host1 = aws_instance.app-us-west-1.public_ip
host2 = aws_instance.app-us-east-1.public_ip
# more hosts
}
New to Terraform here. I'm trying to create multiple projects (in Google Cloud) using Terraform. The problem is I've to execute multiple resources to completely set up a project. I tried count, but how can I tie multiple resources sequentially using count? Here are the following resources I need to execute per project:
Create project using resource "google_project"
Enable API service using resource "google_project_service"
Attach the service project to a host project using resource "google_compute_shared_vpc_service_project" (I'm using shared VPC)
This works if I want to create a single project. But, if I pass a list of projects as input, how can I execute all the above resources for each project in that list sequentially?
Eg.
Input
project_list=["proj-1","proj-2"]
Execute the following sequentially:
resource "google-project" for "proj-1"
resource "google_project_service" for "proj-1"
resource "google_compute_shared_vpc_service_project" for "proj-1"
resource "google-project" for "proj-2"
resource "google_project_service" for "proj-2"
resource "google_compute_shared_vpc_service_project" for "proj-2"
I'm using Terraform version 0.11 which does not support for loops
In Terraform, you can accomplish this using count and the two interpolation functions, element() and length().
First, you'll give your module an input variable:
variable "project_list" {
type = "list"
}
Then, you'll have something like:
resource "google_project" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
resource "google_project_service" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
resource "google_compute_shared_vpc_service_project" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
And of course you'll have your other configuration in those resource declarations as well.
Note that this pattern is described in Terraform Up and Running, Chapter 5, and there are other examples of using count.index in the docs here.
A small update to this question/answer (terraform 0.13 and above). The count or length is not advisable to use anymore due to the way that terraforms works, let's imagine the next scenario:
Suppose you have an array with 3 elements: project_list=["proj-1","proj-2","proj-3"], once you apply that if you want to delete the "proj-2" item from your array once you run the plan, terraform will modify your second element to "proj-3" instead of removing It from the list (more info in this good post). The solution to get the proper behavior is to use the for_each function as follow:
variable "project_list" {
type = list(string)
}
resource "google_project" {
for_each = toset(var.project_list)
name = each.value
}
resource "google_project_service" {
for_each = toset(var.project_list)
name = each.value
}
resource "google_compute_shared_vpc_service_project" {
for_each = toset(var.project_list)
name = each.value
}
Hope this helps! 👍
In file1.tf (generated by kops) I have a resource like this:
resource "aws_vpc" "my-vpc-tf-id" {
...
}
The resource ID was dynamically generated by kops and also added to terraform.tfvars (so it can be used in other places in the .tf files):
my_var = "my-vpc-tf-id"
Now I would like to reference the VPC resource from file2.tf without hardcoding its name:
resource "aws_security_group" "db" {
...
vpc_id = "${aws_vpc.${var.my_var}.id}"
...
}
but Terraform complains that the ${var.my_var} is not allowed. So instead I define this in file2.tf:
resource "aws_security_group" "db" {
...
vpc_id = "${aws_vpc.{{MY_VAR_VAL}}.id}"
...
}
and I use sed to replace the placeholder with the value. This works well but complicates certain other tasks so I was wondering if there were other ways of achieving this without using sed or hardcoding the my_var value (just Terraform's HCL).
The normal way to do this is to use data sources to look up the thing you want to refer to.
The VPC data source allows you to filter based on a number of different things but a typical one is to use the Name tag:
data "aws_vpc" "selected" {
tags {
Name = "${var.vpc}"
}
}
And then you can refer to this VPC with:
resource "aws_security_group" "db" {
...
vpc_id = "${data.aws_vpc.selected.id}"
...
}
The two cases are
(i) if they are both vpc and security group are in same run
we can directly refer vpc id in security group without any data sources
resource "aws_security_group" "db" {
...
vpc_id = "${aws_vpc.my-vpc-tf-id.id}"
...
}
(ii) if they are run differently (using same remote state files or importing config)
use data source as mentioned above by ydaetskcoR
Is there an efficient way to apply validation logic to variables used in a terraform run?
Specifically I want to check the length and casing of some variables. The variables are a combination of ones declared in tfvars files, in variables.tf files, and collected during runtime by terraform.
Thanks.
Custom Validation Rules
Terraform document - Input Variables - Custom Validation Rules
Results
Failure case
provider aws {
profile="default"
}
terraform {
experiments = [variable_validation]
}
## Custom Validation Rules
variable "test" {
type = string
description = "Example to test the case and length of the variable"
default = "TEsT"
validation {
condition = length(var.test) > 4 && upper(var.test) == var.test
error_message = "Validation condition of the test variable did not meet."
}
}
Execution
$ terraform plan
Warning: Experimental feature "variable_validation" is active
on main.tf line 5, in terraform:
5: experiments = [variable_validation]
Experimental features are subject to breaking changes in future minor or patch
releases, based on feedback.
If you have feedback on the design of this feature, please open a GitHub issue
to discuss it.
Error: Invalid value for variable # <---------------------------
on main.tf line 9:
9: variable "test" {
Validation condition of the test variable did not meet.
This was checked by the validation rule at main.tf:14,3-13.
Pass case
terraform {
experiments = [variable_validation]
}
## Custom Validation Rules
variable "test" {
type = string
description = "Example to test the case and length of the variable"
default = "TESTED"
validation {
condition = length(var.test) > 4 && upper(var.test) == var.test
error_message = "Validation condition of the test variable did not meet."
}
}
Execution
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
Others
Alternatively, use null_resource local-exec to implement logic in shell script, or use external provider to send the variable to an external program to validate?
This isn't something you can currently do directly with Terraform but I find it easier to just mangle the input variables to the required format if necessary.
As an example the aws_lb_target_group resource takes a protocol parameter that currently requires it to be uppercased instead of automatically upper casing things and suppressing the diff like the aws_lb_listener resource does for the protocol (or even the protocol in the health_check block).
To solve this I just use the upper function when creating the resource:
variable "protocol" {
default = "http"
}
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
}
resource "aws_lb_target_group" "test" {
name = "tf-example-lb-tg"
port = 80
protocol = "${upper(var.protocol)}"
vpc_id = "${aws_vpc.main.id}"
}
As for checking length I just substring things to make them the right length. I currently do this for ALBs as the name has a max length of 32 and I have Gitlab CI create review environments for some services that get a name based on the slug of the Git branch name so have little control over the length that is used.
variable "environment" {}
variable "service_name" {}
variable "internal" {
default = true
}
resource "aws_lb" "load_balancer" {
name = "${substr(var.environment, 0, min(length(var.environment), 27 - length(var.service_name)))}-${var.service_name}-${var.internal ? "int" : "ext"}"
internal = "${var.internal}"
security_groups = ["${aws_security_group.load_balancer.id}"]
subnets = ["${data.aws_subnet_ids.selected.ids}"]
}
With the above then any combination of length of environment or service name will lead to the environment/service name pair being trimmed to 27 characters at most which leaves room for the extra characters that I want to specify.
Got inspired by this conversation and found the following already existing provider:
https://github.com/craigmonson/terraform-provider-validate
We are trying to create Terraform modules for below activities in AWS, so that we can use them where ever that is required.
VPC creation
Subnets creation
Instance creation etc.
But while creating these modules we have to define the provider in all above listed modules. So we decided to create one more module for provider so that we can call that provider module in other modules (VPC, Subnet, etc.).
Issue in above approach is that it is not taking provider value, and asking for the user input for region.
Terraform configuration is as follow:
$HOME/modules/providers/main.tf
provider "aws" {
region = "${var.region}"
}
$HOME/modules/providers/variables.tf
variable "region" {}
$HOME/modules/vpc/main.tf
module "provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
}
$HOME/modules/vpc/variables.tf
variable "vpc_cidr" {}
variable "environment" {}
variable "region" {}
$HOME/main.tf
module "dev_vpc" {
source = "modules/vpc"
vpc_cidr = "${var.vpc_cidr}"
environment = "${var.environment}"
region = "${var.region}"
}
$HOME/variables.tf
variable "vpc_cidr" {
default = "192.168.0.0/16"
}
variable "environment" {
default = "dev"
}
variable "region" {
default = "ap-south-1"
}
Then when running terraform plan command at $HOME/ location it is not taking provider value and instead asking for the user input for region.
I need help from the Terraform experts, what approach we should follow to address below concerns:
Wrap provider in a Terraform module
Handle multiple region use case using provider module or any other way.
I knew a long time back that it wasn't possible to do this because Terraform built a graph that required a provider for any resource before it included any dependencies and it didn't used to be possible to force a dependency on a module.
However since Terraform 0.8 it is now possible to set a dependency on modules with the following syntax:
module "network" {
# ...
}
resource "aws_instance" "foo" {
# ...
depends_on = ["module.network"]
}
However, if I try that with your setup by changing modules/vpc/main.tf to look something like this:
module "aws_provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
depends_on = ["module.aws_provider"]
}
And run terraform graph | dot -Tpng > graph.png against it it looks like the graph doesn't change at all from when the explicit dependency isn't there.
This seems like it might be a potential bug in the graph building stage in Terraform that should probably be raised as an issue but I don't know the core code base well enough to spot where the change needs to be.
For our usage we use symlinks heavily in our Terraform code base, some of which is historic from before Terraform supported other ways of doing things but could work for you here.
We simply define the provider in a single .tf file (such as environment.tf) along with any other generic config needed for every place you would ever run Terraform (ie not at a module level) and then symlink this into each location. That allows us to define the provider in a single place with overridable variables if necessary.
Step 1
Add region alias in the main.tf file where you gonna execute the terraform plan.
provider "aws" {
region = "eu-west-1"
alias = "main"
}
provider "aws" {
region = "us-east-1"
alias = "useast1"
}
Step 2
Add providers block inside your module definition block
module "lambda_edge_rule" {
providers = {
aws = aws.useast1
}
source = "../../../terraform_modules/lambda"
tags = var.tags
}
Step 3
Define "aws" as providers inside your module. ( source = ../../../terraform_modules/lambda")
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 2.7.0"
}
}
}
resource "aws_lambda_function" "lambda" {
function_name = "blablabla"
.
.
.
.
.
.
.
}
Note: Terraform version v1.0.5 as of now.