I am trying to call data from a remote state to reference a vpc_id for a network acl. When I run plan/apply, I receive the error "This object has no argument, nested block, or exported attribute named "vpc_id"."
I've tried using "data.terraform_remote_state.*.vpc_id", as well as "${}" syntax. I tried defining the data.remote info in the variables.tf for the child module, and the parent module.
I ultimately need to be able to call this module for different VPCs/subnets dynamically.
The relevant VPC already exists and all modules are initialized.
s3 bucket stage/network/vpc/terraform.tfstate:
"outputs": {
"vpc_id": {
"value": "vpc-1234567890",
"type": "string"
}
},
enter code here
modules/network/acl/main.tf:
data "terraform_remote_state" "stage-network" {
backend = "s3"
config = {
bucket = "bucket"
key = "stage/network/vpc/terraform.tfstate"
}
}
resource "aws_network_acl" "main" {
vpc_id = data.terraform_remote_state.stage-network.vpc_id
# acl variables here
stage/network/acl/main.tf:
data "terraform_remote_state" "stage-network" {
backend = "s3"
config = {
bucket = "bucket"
key = "stage/network/vpc/terraform.tfstate"
}
}
module "create_acl" {
source = "../../../modules/network/acl/"
vpc_id = var.vpc_id
# vpc_id = data.terraform_remote_state.stage-network.vpc_id
# vpc_id = "${data.terraform_remote_state.stage-network.vpc_id}"
# vpc_id = var.data.terraform_remote_state.stage-network.vpc_id
I am expecting the acl parent module to be able to associate to the VPC, and from there the child module to be able to configure the variables.
This is one of the breaking changes that the 0.12.X version of Terraform introduce.
The terraform_remote_state data source has changed slightly for the v0.12 release to make all of the remote state outputs available as a single map value, rather than as top-level attributes as in previous releases.
In previous releases, a reference to a vpc_id output exported by the remote state data source might have looked like this:
data.terraform_remote_state.vpc.vpc_id
This value must now be accessed via the new outputs attribute:
data.terraform_remote_state.vpc.outputs.vpc_id
Source: https://www.terraform.io/upgrade-guides/0-12.html#remote-state-references
In first state:
.....
output "expose_vpc_id" {
value = "${module.network.vpc_id}"
}
In another state, to share between terraform configs:
data "terraform_remote_state" "remote" {
backend = "s3"
config = {
bucket = "terraform-ex1"
key = "tera-ex1.tfstate"
region = "us-east-1"
}
}
output "vpc_id" {
value = "${data.terraform_remote_state.remote.outputs.expose_vpc_id}"
}
Related
I am working with terraform and trying to output the security group ID in the form of an output and pull it from the local terraform state file and use that information in a different resource in my case it would be a aws_eks_cluster in the vpc_config session.
In the module that has the security group:
output "security_group_id" {
value = aws_security_group.a_group.id
}
In the module that reads the output (the backend config dependends on which backend type you are using and how it is configured):
data "terraform_remote_state" "security_group" {
backend = "s3"
config {
bucket = "your-terraform-state-files"
key = "your-state-file-key.tfstate"
region = "us-east-1"
}
}
locals {
the_security_group_id = data.terraform_remote_state.security_group.outputs.security_group_id
}
I'm trying to setup some IaC for a new project using Hashicorp Terraform on AWS. I'm using modules because I want to be able to reuse stuff across multiple environments (staging, prod, dev, etc.)
I'm struggling to understand where I have to set an output variable within a module, and how I then use that in another module. Any pointers to this would be greatly appreciated!
I need to use some things created in my VPC module (subnet IDs) when creating EC2 machines. My understanding is that you can't reference something from one module in another, so I am trying to use an output variable from the VPC module.
I have the following in my site main.tf
module "myapp-vpc" {
source = "dev/vpc"
aws_region = "${var.aws_region}"
}
module "myapp-ec2" {
source = "dev/ec2"
aws_region = "${var.aws_region}"
subnet_id = "${module.vpc.subnetid"}
}
dev/vpc simply sets some values and uses my vpc module:
module "vpc" {
source = "../../modules/vpc"
aws_region = "${var.aws_region}"
vpc-cidr = "10.1.0.0/16"
public-subnet-cidr = "10.1.1.0/24"
private-subnet-cidr = "10.1.2.0/24"
}
In my vpc main.tf, I have the following at the very end, after the aws_vpc and aws_subnet resources (showing subnet resource):
resource "aws_subnet" "public" {
vpc_id = "${aws_vpc.main.id}"
map_public_ip_on_launch = true
availability_zone = "${var.aws_region}a"
cidr_block = "${var.public-subnet-cidr}"
}
output "subnetid" {
value = "${aws_subnet.public.id}"
}
When I run terraform plan I get the following error message:
Error: module 'vpc': "subnetid" is not a valid output for module "vpc"
Outputs need to be passed up through each module explicitly each time.
For example if you wanted to output a variable to the screen from a module nested below another module you would need something like this:
child-module.tf
output "child_foo" {
value = "foobar"
}
parent-module.tf
module "child" {
source = "path/to/child"
}
output "parent_foo" {
value = "${module.child.child_foo}"
}
main.tf
module "parent" {
source = "path/to/parent"
}
output "main_foo" {
value = "${module.parent.parent_foo}"
}
I am configuring remote state in terraform like:
provider "aws" {
region = "ap-southeast-1"
}
terraform {
backend "s3" {
bucket = "xxx-artifacts"
key = "terraform_state.tfstate"
region = "ap-southeast-1"
}
}
data "terraform_remote_state" "s3_state" {
backend = "s3"
config {
bucket = "xxx-artifacts"
key = "terraform_state.tfstate"
region = "ap-southeast-1"
}
}
It seems very duplicated tho, why is it like that? I have the same variables in terraform block and the terraform_remote_state data source block. Is this actually required?
The terraform.backend configuration is for configuring where to store remote state for the Terraform context/directory where Terraform is being ran from.
This allows you to share state between different machines, backup your state and also co-ordinate between usages of a Terraform context via state locking.
The terraform_remote_state data source is, like other data sources, for retrieving data from an external source, in this case a Terraform state file.
This allows you to retrieve information stored in a state file from another Terraform context and use that elsewhere.
For example in one location you might create an aws_elasticsearch_domain but then need to lookup the endpoint of the domain in another context (such as for configuring where to ship logs to). Currently there isn't a data source for ES domains so you would need to either hardcode the endpoint elsewhere or you could look it up with the terraform_remote_state data source like this:
elasticsearch/main.tf
resource "aws_elasticsearch_domain" "example" {
domain_name = "example"
elasticsearch_version = "1.5"
cluster_config {
instance_type = "r4.large.elasticsearch"
}
snapshot_options {
automated_snapshot_start_hour = 23
}
tags = {
Domain = "TestDomain"
}
}
output "es_endpoint" {
value = "$aws_elasticsearch_domain.example.endpoint}"
}
logstash/userdata.sh.tpl
#!/bin/bash
sed -i 's/|ES_DOMAIN|/${es_domain}/' >> /etc/logstash.conf
logstash/main.tf
data "terraform_remote_state" "elasticsearch" {
backend = "s3"
config {
bucket = "xxx-artifacts"
key = "elasticsearch.tfstate"
region = "ap-southeast-1"
}
}
data "template_file" "logstash_config" {
template = "${file("${path.module}/userdata.sh.tpl")}"
vars {
es_domain = "${data.terraform_remote_state.elasticsearch.es_endpoint}"
}
}
resource "aws_instance" "foo" {
# ...
user_data = "${data.template_file.logstash_config.rendered}"
}
According to the documentation, to use s3 and not a local terraform.tfstate file for state storage, one should configure a backend more or less as follows:
terraform {
backend "s3" {
bucket = "my-bucket-name"
key = "my-key-name"
region = "my-region"
}
}
I was
using a local (terraform.tfstate) file
added the above snippet in my provided.tf file
run (again) terraform init
was asked by terraform to migrate my state to the above bucket
...so far so good...
But then comes this confusing part about terraform_remote_state ...
Why do I need this?
Isn't my state now saved remotely (on the aforemenetioned s3 bucket) already?
terraform_remote_state isn't for storage of your state its for retrieval in another terraform plan if you have outputs. It is a data source. For example if you output your Elastic IP Address in one state:
resource "aws_eip" "default" {
vpc = true
}
output "eip_id" {
value = "${aws_eip.default.id}"
}
Then wanted to retrieve that in another state:
data "terraform_remote_state" "remote" {
backend = "s3"
config {
bucket = "my-bucket-name"
key = "my-key-name"
region = "my-region"
}
}
resource "aws_instance" "foo" {
...
}
resource "aws_eip_association" "eip_assoc" {
instance_id = "${aws_instance.foo.id}"
allocation_id = "${data.terraform_remote_state.remote.eip_id}"
}
edit: If you are retrieving outputs in Terraform > 0.12 you need to include outputs
data "terraform_remote_state" "remote" {
backend = "s3"
config {
bucket = "my-bucket-name"
key = "my-key-name"
region = "my-region"
}
}
resource "aws_instance" "foo" {
...
}
resource "aws_eip_association" "eip_assoc" {
instance_id = "${aws_instance.foo.id}"
allocation_id = "${data.terraform_remote_state.remote.outputs.eip_id}"
}
Remote State allows you to collaborate with other team members, and central location to store your infrastructure state.
Apart from that by enabling s3 versioning, you can have versioning for state file, to track changes.
If a terraform script uses a module that has outputs, it's possible to access those module outputs in using the -module option for the terraform output command:
$ terraform output --help
Usage: terraform output [options] [NAME]
Reads an output variable from a Terraform state file and prints
the value. If NAME is not specified, all outputs are printed.
Options:
-state=path Path to the state file to read. Defaults to
"terraform.tfstate".
-no-color If specified, output won't contain any color.
-module=name If specified, returns the outputs for a
specific module
-json If specified, machine readable output will be
printed in JSON format
If I store that state file in S3 or some such, I can then reference the outputs of the main script by using the terraform_remote_state data provider.
data "terraform_remote_state" "base_networking" {
backend = "s3"
config {
bucket = "${var.remote_state_bucket}"
region = "${var.remote_state_region}"
key = "${var.networking_remote_state_key}"
}
}
resource "aws_instance" "my_instance" {
subnets = "${data.terraform_remote_state.base_networking.vpc_id}"
}
Is it possible to access the module outputs that are present in the state file as well? I'm looking for something like "${data.terraform_remote_state.base_networking.module.<module_name>.<output>}" or similar.
Yes, you can access remote state outputs from your own modules. You just need to "propagate" the outputs.
E.g., let's say you have something like this, your base_networking infrastructure, which contains a module for creating your VPC, and you want that VPC ID to be accessible via remote state:
base_networking/
main.tf
outputs.tf
vpc/
main.tf
outputs.tf
In base_networking/main.tf you create your VPC using your base_networking/vpc module:
module "vpc" {
source = "./vpc"
region = "${var.region}"
name = "${var.vpc_name}"
cidr = "${var.vpc_cidr}"
}
In base_networking/vpc/outputs.tf in your module you have an id output:
output "id" {
value = "${aws_vpc.vpc.id}"
}
In base_networking/outputs.tf you also have a vpc_id output that propagates module.vpc.id:
output "vpc_id" {
value = "${module.vpc.id}"
With that you can now access vpc_id using something like:
data "terraform_remote_state" "base_networking" {
backend = "s3"
config = {
bucket = "${var.remote_state_bucket}"
region = "${var.remote_state_region}"
key = "${var.networking_remote_state_key}"
}
}
[...]
vpc_id = "${data.terraform_remote_state.base_networking.vpc_id}"