Terraform try/catch in a module? - terraform

Terraform v0.12.x, AWS provider
I'm trying to write a generic module that will either return an existing EBS snapshot of a passed-in EBS volume name, or take a snapshot of a passed-in EBS volume name. Either way it should return a snapshot id.
Here's the code to get an existing snapshot.
data "aws_ebs_snapshot" "snapshot" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
filter {
name = "status"
values = ["completed"]
}
}
output "snapshot_id" {
value = data.aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
and here's the code to take a snapshot.
data "aws_ebs_volume" "ebs" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
volume_id = data.aws_ebs_volume.ebs.id
}
output "snapshot_id" {
value = aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
Is it possible to do this? If so how? I know I can separate them into 2 separate modules, but humor me.
# try/catch block is of course pseudo-code
try {
# Get an existing snapshot id
data "aws_ebs_snapshot" "snapshot" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_name]
}
filter {
name = "status"
values = ["completed"]
}
}
output "snapshot_id" {
value = data.aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
}
catch() {
# Get the volume id and take a snapshot
data "aws_ebs_volume" "ebs" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
volume_id = data.aws_ebs_volume.ebs.id
}
output "snapshot_id" {
value = aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
}
I know try/catch blocks are not used this way in Terraform, so how can I achieve what I want?

The situation you've described doesn't seem like one where it's necessary to make a dynamic decision based on the remote system, because you can tell entirely from the input variables whether the caller is specifying a snapshot id or a volume id:
variable "ebs_name" {
type = string
default = null
}
variable "ebs_id" {
type = string
default = null
}
data "aws_ebs_snapshot" "snapshot" {
count = var.ebs_name != null ? 1 : 0
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_name]
}
filter {
name = "status"
values = ["completed"]
}
}
data "aws_ebs_volume" "ebs" {
count = var.ebs_id != null ? 1 : 0
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
count = var.ebs_id != null ? 1 : 0
volume_id = data.aws_ebs_volume.ebs[count.index].id
}
output "snapshot_id" {
# Return either the generated snapshot or the given
# snapshot. If the caller specified both for some
# reason then the generated snapshot takes priority.
# This will produce an error if neither var.ebs_name
# nor var.ebs_id is set, because the result will have
# no elements.
value = concat(
aws_ebs_snapshot.snapshot[*].id,
data.aws_ebs_snapshot.snapshot[*].id,
)[0]
description = "Jenkins master snapshot id"
}
For completeness in case someone else finds this answer in future I want to note that the Module Composition guide suggests just directly writing out the straightforward read or create code for each case rather than making dynamic decisions in cases like these, but I showed the dynamic example above because you suggested (by reference to the possibility of using two modules to address this) that you'd already considered and decided against using a composition style.

At first glance you might think that you could check if the snapshot exists via the data source, and then use something like count on the resource to create one if the data source didn't return anything. Unfortunately that's not how Terraform works because the data source will throw an error if it can't find a match, causing Terraform to exit.
See the official response from HashiCorp here, when asked for the sort of capability you are looking for.
The sort of dynamic decision-making that is being requested here runs
counter to Terraform's design goals, since it makes the configuration
a description of what might possibly be rather than what is.
In general this sort of thing is handled better via AWS CLI scripts, or something like a Python/boto3 script, instead of Terraform.

Related

Terraform - Build Fails Non-existent AMI

I have the following code in Terraform. The issue is the image no longer exits and it's failing to build.
data "aws_ami" "ami_ai_demos" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200323"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"]
}
I could update values with the current image but do not want to rebuild the image. Is there anyway to work around this issue?
It was most probably deprecated thus not searchable anymore. Just use direct ami ID for your region instead of filters.
https://cloud-images.ubuntu.com/locator/
cloud-images.ubuntu.com
Thank you for the suggestions. As my requirements are that the EC2 instance is not rebuilt, I will have to remove it from Terraform state. This makes sense as so long as Terraform see's it it'll want to act on the resource.
https://www.terraform.io/cli/commands/state/rm
The better option would be to use like this, as this will always pick up the latest image avalaible and your build will not fail.
data "aws_ami" "ubuntu" {
most_recent = true
owners = ["099720109477"]
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
}
filter {
name = "root-device-type"
values = ["ebs"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}

Terraform - How to create optional data block

I have created a few data blocks for finding the right AMIs.
I have 4 data blocks to find 4 AMIs based 4 os. Example,
data "aws_ami" "centos" {
owners = ["123456789"]
most_recent = true
filter {
name = "name"
values = ["CentOS*${var.os-version}*x86_64"]
}
}
data "aws_ami" "suse" {
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os-version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
I call them like the following ami_id=data.${os_name}.image-id
So what I want is to run only the data block that has been called. If the user chooses "suse" then only the suse data block will run. Not all of them.
It's an issue right now because users choose versions based on the os. For example, 16.04 only works for ubuntu not any other so the other data block throws exception like the following,
Error: Your query returned no results. Please change your search criteria and try again.
on main.tf line 79, in data "aws_ami" "suse":
79: data "aws_ami" "suse" {
So how can I achieve it?
An approach can be a combination of Terraform modules and count. For instance, you could structure your project like this:
Your data lookups for AMIs can be a module in a subfolder. The input to this module would be your os-version.
Inside this module you can have variable validation to validate the input OS is valid. Then, you can construct each of your data blocks. Each data block would have a count value where the lookup would either happen (a value of 1) or not (a value of 0) depending on which os-version was passed in.
I built the following as a simple example.
# Main Terraform project
module "ami" {
source = "./ami"
arch = "arm64"
}
output "ami_arn" { value = module.ami.arn }
# AMI module
variable "arch" {
type = string
validation {
condition = (
var.arch == "x86" || var.arch == "arm64"
)
error_message = "Valid architectures are 'x86' or 'arm64'."
}
}
data "aws_ami" "aws_linux_2_x86" {
count = var.arch == "x86" ? 1 : 0
most_recent = true
owners = ["amazon"]
filter {
name = "image-id"
values = ["ami-0742b4e673072066f"]
}
}
data "aws_ami" "aws_linux_2_arm64" {
count = var.arch == "arm64" ? 1 : 0
most_recent = true
owners = ["amazon"]
filter {
name = "image-id"
values = ["ami-015f1226b535bd02d"]
}
}
output "arn" { value = var.arch == "x86" ? data.aws_ami.aws_linux_2_x86[0].arn : data.aws_ami.aws_linux_2_arm64[0].arn }
This example does not filter on os-version because I can't easily mock your environment. However, the implementation is the same.
The output of the AMI module is an ARN, or any other valid output you choose which you can then pass on to building your EC2 instance.
You don't necessarily have to use modules in this case and can have the same level of validation and count actions taken in your main code. I chose this approach assuming you had an EC2 module you were coding for.

terraform - how to use variables inside attributes

I am not sure if this is the right approach to do this but I want to use a variable as an attribute.
For example, I have a variable that changes based on user input: os_name = ubuntu.
I want to use this variable name like the following,
resource "aws_instance" "workerNode" {
..................
ami = data.aws_ami.${var.os_name}.image_id
..................
}
Following is an example of the data block,
data "aws_ami" "suse" {
count = "${var.os_name == "suse" ? 1 : 0}"
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
Which result the following,
"architecture" = "x86_64"
"hypervisor" = "xen"
"id" = "ami-0d3905203a039e3b0"
"image_id" = "ami-0d3905203a039e3b0"
But terraform is not allowing me to do this. Is there any way I can do this or I have to change the workflow?
In situations where it's not appropriate to gather all of your instances under a single resource using for_each (which would implicitly make that resource appear as a map of objects), you can get a similar result explicitly by writing a local value expression to construct an equivalent map:
locals {
amis = {
suse = data.aws_ami.suse
ubuntu = data.aws_ami.ubuntu
}
}
Then you can refer to local.amis["ubuntu"] or local.amis["suse"] (possibly replacing the element key with a variable, if you need to.
With that said, it does seem like there is a possible different approach for your case which would get there with only one data block:
locals {
os_ami_queries = {
suse = {
owners = ["amazon"]
filters = {
name = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
ubuntu = {
owners = ["amazon"]
filters = {
name = ["ubuntu-${var.os_version}-something-something"]
}
}
}
ami_query = local.os_ami_queries[var.os_name]
}
data "aws_ami" "selected" {
owners = local.ami_query.owners
dynamic "filter" {
for_each = local.ami_query.filters
content {
name = filter.key
values = filter.value
}
}
}
This different permutation does the OS selection before the data "aws_ami" lookup, so it can use the settings associated with whichever OS was selected by the caller. The AMI id would then be in data.aws_ami.selected.id.
With that said, this approach has the disadvantage of being quite indirect and using a dynamic block, so I'd weigh that against the readability of the alternatives to pick the one which seems subjectively easiest to follow for someone who isn't familiar with this configuration. There isn't a single answer to that because to some extent it's a matter of taste, and so if you are working in a team setting this could be something to discuss with colleagues to see which approach best suits tradeoffs like how often you expect to be adding and removing supported operating systems vs. changing the details of how you use the result.
You can make it work by specifying your AMI's with a for_each and thus getting a map which you can access by key.
My data.aws_ami.myamis looks like this:
data "aws_ami" "myamis" {
for_each = toset(["suse", "ubuntu"])
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["${each.value}*"]
}
}
For test purposes I define a variable foo like this:
variable "foo" {
type = string
default = "suse"
}
Now I can access the AMI like this:
$ tf console
> data.aws_ami.myamis[var.foo].image_id
"ami-0ea50c090ba6e85c5"
You can adapt this to suit your needs for os_name and os_version.
I have solved the issue just by using conditional expression.
I am not sure if it is a standard way of doing things but it works for me.
I have tried to emulate if/elif/else with nested conditional expression.
output "name" {
value = "${ var.os_name == "ubuntu" ? data.aws_ami.ubuntu[0].image_id : (var.os_name == "redhat" ? data.aws_ami.redhat[0].image_id : (var.os_name == "centos" ? data.aws_ami.suse[0].image_id : data.aws_ami.suse[0].image_id ))}"
}

Create nested resource parameter blocks based on conditional in terraform

I am trying to create a terraform module that creates a compute instance. I want the resource to have an attached disk if and only if I have a variable attached_disk_enabled set to true during module invocation. I have this:
resource "google_compute_disk" "my-disk" {
name = "data"
type = "pd-ssd"
size = 20
count = var.attached_disks_enabled ? 1 : 0
}
resource "google_compute_instance" "computer" {
name = "computer"
boot_disk {
...
}
// How do I make this disappear if attached_disk_enabled == false?
attached_disk {
source = "${google_compute_disk.my-disk.self_link}"
device_name = "computer-disk"
mode = "READ_WRITE"
}
}
Variables have been declared for the module in vars.tf. Module invocation is like this:
module "main" {
source = "../modules/computer"
attached_disk_enabled = false
...
}
I know about dynamic blocks and how to use for loop to iterate over a list and set multiple blocks, but I'm not sure how to exclude a block from a resource using this method:
dynamic "attached-disk" {
for_each in var.disk_list
content {
source = "${google_compute_disk.my-disk.*.self_link}"
device_name = "computer-disk-${count.index}"
mode = "READ_WRITE"
}
}
I want if in place of for_each. Is there a way to do this?
$ terraform version
Terraform v0.12.0
Because your disk resource already has the conditional attached to it, you can use the result of that resource as your iterator and thus avoid specifying the conditional again:
dynamic "attached_disk" {
for_each = google_compute_disk.my-disk
content {
source = attached_disk.value.self_link
device_name = "computer-disk-${attached_disk.key}"
mode = "READ_WRITE"
}
}
To answer the general question: if you do need a conditional block, the answer is to write a conditional expression that returns either a single-item list or an empty list:
dynamic "attached_disk" {
for_each = var.attached_disk_enabled ? [google_compute_disk.my-disk[0].self_link] : []
content {
source = attached_disk.value
device_name = "computer-disk-${attached_disk.key}"
mode = "READ_WRITE"
}
}
However, in your specific situation I'd prefer the former because it describes the intent ("attach each of the disks") more directly.

How can I get the Instance ID of the EMR master instance in Terraform?

The following code gives me a list of all EC2 instances that are part of my cluster:
data "aws_instances" "emrMaster" {
instance_tags {
Name = "emr-cluster-name"
}
}
But when I try to narrow the list using the AWS generated tag for the master instance, I get the same list.
data "aws_instances" "emrMaster" {
instance_tags {
Name = "emr-cluster-name"
"aws:elasticmapreduce:instance-group-role" = "MASTER"
}
}
If I remove the quotes on the key name, I get a illegal character error due to the colons in the key name.
data "aws_instances" "emrMaster" {
instance_tags {
Name = "emr-cluster-name"
aws:elasticmapreduce:instance-group-role = "MASTER"
}
}
Is there a better way to do this, am I doing something wrong, or have I uncovered a bug in Terraform?
I am using Terraform v0.11.7
I am looking to capture this value so I can build specific cloudwatch alerts for the master instance that are different from the Core instances.
For this purpose better use aws_instance (not aws_instances)
resource "aws_emr_cluster" "emr-cluster" {
....
}
data "aws_instance" "master" {
filter {
name = "tag:Name"
values = ["${aws_emr_cluster.emr-cluster.name}"]
}
filter {
name = "tag:aws:elasticmapreduce:instance-group-role"
values = ["MASTER"]
}
}
And then just use output:
output "master_id" {
value = "${data.aws_instance.master.id}"
}

Resources