I have the following code in Terraform. The issue is the image no longer exits and it's failing to build.
data "aws_ami" "ami_ai_demos" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200323"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"]
}
I could update values with the current image but do not want to rebuild the image. Is there anyway to work around this issue?
It was most probably deprecated thus not searchable anymore. Just use direct ami ID for your region instead of filters.
https://cloud-images.ubuntu.com/locator/
cloud-images.ubuntu.com
Thank you for the suggestions. As my requirements are that the EC2 instance is not rebuilt, I will have to remove it from Terraform state. This makes sense as so long as Terraform see's it it'll want to act on the resource.
https://www.terraform.io/cli/commands/state/rm
The better option would be to use like this, as this will always pick up the latest image avalaible and your build will not fail.
data "aws_ami" "ubuntu" {
most_recent = true
owners = ["099720109477"]
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
}
filter {
name = "root-device-type"
values = ["ebs"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
Related
I have created a few data blocks for finding the right AMIs.
I have 4 data blocks to find 4 AMIs based 4 os. Example,
data "aws_ami" "centos" {
owners = ["123456789"]
most_recent = true
filter {
name = "name"
values = ["CentOS*${var.os-version}*x86_64"]
}
}
data "aws_ami" "suse" {
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os-version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
I call them like the following ami_id=data.${os_name}.image-id
So what I want is to run only the data block that has been called. If the user chooses "suse" then only the suse data block will run. Not all of them.
It's an issue right now because users choose versions based on the os. For example, 16.04 only works for ubuntu not any other so the other data block throws exception like the following,
Error: Your query returned no results. Please change your search criteria and try again.
on main.tf line 79, in data "aws_ami" "suse":
79: data "aws_ami" "suse" {
So how can I achieve it?
An approach can be a combination of Terraform modules and count. For instance, you could structure your project like this:
Your data lookups for AMIs can be a module in a subfolder. The input to this module would be your os-version.
Inside this module you can have variable validation to validate the input OS is valid. Then, you can construct each of your data blocks. Each data block would have a count value where the lookup would either happen (a value of 1) or not (a value of 0) depending on which os-version was passed in.
I built the following as a simple example.
# Main Terraform project
module "ami" {
source = "./ami"
arch = "arm64"
}
output "ami_arn" { value = module.ami.arn }
# AMI module
variable "arch" {
type = string
validation {
condition = (
var.arch == "x86" || var.arch == "arm64"
)
error_message = "Valid architectures are 'x86' or 'arm64'."
}
}
data "aws_ami" "aws_linux_2_x86" {
count = var.arch == "x86" ? 1 : 0
most_recent = true
owners = ["amazon"]
filter {
name = "image-id"
values = ["ami-0742b4e673072066f"]
}
}
data "aws_ami" "aws_linux_2_arm64" {
count = var.arch == "arm64" ? 1 : 0
most_recent = true
owners = ["amazon"]
filter {
name = "image-id"
values = ["ami-015f1226b535bd02d"]
}
}
output "arn" { value = var.arch == "x86" ? data.aws_ami.aws_linux_2_x86[0].arn : data.aws_ami.aws_linux_2_arm64[0].arn }
This example does not filter on os-version because I can't easily mock your environment. However, the implementation is the same.
The output of the AMI module is an ARN, or any other valid output you choose which you can then pass on to building your EC2 instance.
You don't necessarily have to use modules in this case and can have the same level of validation and count actions taken in your main code. I chose this approach assuming you had an EC2 module you were coding for.
I am not sure if this is the right approach to do this but I want to use a variable as an attribute.
For example, I have a variable that changes based on user input: os_name = ubuntu.
I want to use this variable name like the following,
resource "aws_instance" "workerNode" {
..................
ami = data.aws_ami.${var.os_name}.image_id
..................
}
Following is an example of the data block,
data "aws_ami" "suse" {
count = "${var.os_name == "suse" ? 1 : 0}"
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
Which result the following,
"architecture" = "x86_64"
"hypervisor" = "xen"
"id" = "ami-0d3905203a039e3b0"
"image_id" = "ami-0d3905203a039e3b0"
But terraform is not allowing me to do this. Is there any way I can do this or I have to change the workflow?
In situations where it's not appropriate to gather all of your instances under a single resource using for_each (which would implicitly make that resource appear as a map of objects), you can get a similar result explicitly by writing a local value expression to construct an equivalent map:
locals {
amis = {
suse = data.aws_ami.suse
ubuntu = data.aws_ami.ubuntu
}
}
Then you can refer to local.amis["ubuntu"] or local.amis["suse"] (possibly replacing the element key with a variable, if you need to.
With that said, it does seem like there is a possible different approach for your case which would get there with only one data block:
locals {
os_ami_queries = {
suse = {
owners = ["amazon"]
filters = {
name = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
ubuntu = {
owners = ["amazon"]
filters = {
name = ["ubuntu-${var.os_version}-something-something"]
}
}
}
ami_query = local.os_ami_queries[var.os_name]
}
data "aws_ami" "selected" {
owners = local.ami_query.owners
dynamic "filter" {
for_each = local.ami_query.filters
content {
name = filter.key
values = filter.value
}
}
}
This different permutation does the OS selection before the data "aws_ami" lookup, so it can use the settings associated with whichever OS was selected by the caller. The AMI id would then be in data.aws_ami.selected.id.
With that said, this approach has the disadvantage of being quite indirect and using a dynamic block, so I'd weigh that against the readability of the alternatives to pick the one which seems subjectively easiest to follow for someone who isn't familiar with this configuration. There isn't a single answer to that because to some extent it's a matter of taste, and so if you are working in a team setting this could be something to discuss with colleagues to see which approach best suits tradeoffs like how often you expect to be adding and removing supported operating systems vs. changing the details of how you use the result.
You can make it work by specifying your AMI's with a for_each and thus getting a map which you can access by key.
My data.aws_ami.myamis looks like this:
data "aws_ami" "myamis" {
for_each = toset(["suse", "ubuntu"])
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["${each.value}*"]
}
}
For test purposes I define a variable foo like this:
variable "foo" {
type = string
default = "suse"
}
Now I can access the AMI like this:
$ tf console
> data.aws_ami.myamis[var.foo].image_id
"ami-0ea50c090ba6e85c5"
You can adapt this to suit your needs for os_name and os_version.
I have solved the issue just by using conditional expression.
I am not sure if it is a standard way of doing things but it works for me.
I have tried to emulate if/elif/else with nested conditional expression.
output "name" {
value = "${ var.os_name == "ubuntu" ? data.aws_ami.ubuntu[0].image_id : (var.os_name == "redhat" ? data.aws_ami.redhat[0].image_id : (var.os_name == "centos" ? data.aws_ami.suse[0].image_id : data.aws_ami.suse[0].image_id ))}"
}
Right now I'm doing this:
data "aws_ami" "windows-image" {
most_recent = true
filter {
name = "name"
values = ["Windows_Server-2019-English-Full-Base-*"]
}
owners = ["amazon"]
}
But I want to experiment with some outdated amis. There is no oldest param. If I could get aws_ami to return a list, I was thinking I could use reverse() on it, but it seems I need to do filtering otherwise I get this error:
Error: Your query returned more than one result. Please try a more specific search criteria, or set `most_recent` attribute to true.
I just saw there is a separate resource for getting a list of ids.
data "aws_ami_ids" "windows-image" {
sort_ascending = true // sort by oldest to newest
filter {
name = "name"
values = ["Windows_Server-2019-English-Full-Base-*"]
}
owners = ["amazon"]
}
output "test" {
value = data.aws_ami_ids.windows-image.ids[0]
}
Terraform v0.12.x, AWS provider
I'm trying to write a generic module that will either return an existing EBS snapshot of a passed-in EBS volume name, or take a snapshot of a passed-in EBS volume name. Either way it should return a snapshot id.
Here's the code to get an existing snapshot.
data "aws_ebs_snapshot" "snapshot" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
filter {
name = "status"
values = ["completed"]
}
}
output "snapshot_id" {
value = data.aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
and here's the code to take a snapshot.
data "aws_ebs_volume" "ebs" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
volume_id = data.aws_ebs_volume.ebs.id
}
output "snapshot_id" {
value = aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
Is it possible to do this? If so how? I know I can separate them into 2 separate modules, but humor me.
# try/catch block is of course pseudo-code
try {
# Get an existing snapshot id
data "aws_ebs_snapshot" "snapshot" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_name]
}
filter {
name = "status"
values = ["completed"]
}
}
output "snapshot_id" {
value = data.aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
}
catch() {
# Get the volume id and take a snapshot
data "aws_ebs_volume" "ebs" {
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
volume_id = data.aws_ebs_volume.ebs.id
}
output "snapshot_id" {
value = aws_ebs_snapshot.snapshot.id
description = "Jenkins master snapshot id"
}
}
I know try/catch blocks are not used this way in Terraform, so how can I achieve what I want?
The situation you've described doesn't seem like one where it's necessary to make a dynamic decision based on the remote system, because you can tell entirely from the input variables whether the caller is specifying a snapshot id or a volume id:
variable "ebs_name" {
type = string
default = null
}
variable "ebs_id" {
type = string
default = null
}
data "aws_ebs_snapshot" "snapshot" {
count = var.ebs_name != null ? 1 : 0
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_name]
}
filter {
name = "status"
values = ["completed"]
}
}
data "aws_ebs_volume" "ebs" {
count = var.ebs_id != null ? 1 : 0
most_recent = true
filter {
name = "tag:Name"
values = [var.ebs_id]
}
}
// Take a snapshot of the green EBS resource
resource "aws_ebs_snapshot" "snapshot" {
count = var.ebs_id != null ? 1 : 0
volume_id = data.aws_ebs_volume.ebs[count.index].id
}
output "snapshot_id" {
# Return either the generated snapshot or the given
# snapshot. If the caller specified both for some
# reason then the generated snapshot takes priority.
# This will produce an error if neither var.ebs_name
# nor var.ebs_id is set, because the result will have
# no elements.
value = concat(
aws_ebs_snapshot.snapshot[*].id,
data.aws_ebs_snapshot.snapshot[*].id,
)[0]
description = "Jenkins master snapshot id"
}
For completeness in case someone else finds this answer in future I want to note that the Module Composition guide suggests just directly writing out the straightforward read or create code for each case rather than making dynamic decisions in cases like these, but I showed the dynamic example above because you suggested (by reference to the possibility of using two modules to address this) that you'd already considered and decided against using a composition style.
At first glance you might think that you could check if the snapshot exists via the data source, and then use something like count on the resource to create one if the data source didn't return anything. Unfortunately that's not how Terraform works because the data source will throw an error if it can't find a match, causing Terraform to exit.
See the official response from HashiCorp here, when asked for the sort of capability you are looking for.
The sort of dynamic decision-making that is being requested here runs
counter to Terraform's design goals, since it makes the configuration
a description of what might possibly be rather than what is.
In general this sort of thing is handled better via AWS CLI scripts, or something like a Python/boto3 script, instead of Terraform.
I'm creating subnets as part of a seperate terraform template and exporting the IDs as follows.
output "subnet-aza-dev" {
value = "${aws_subnet.subnet-aza-dev.id}"
}
output "subnet-azb-dev" {
value = "${aws_subnet.subnet-azb-dev.id}"
}
output "subnet-aza-test" {
value = "${aws_subnet.subnet-aza-test.id}"
}
output "subnet-azb-test" {
value = "${aws_subnet.subnet-azb-test.id}"
}
...
I'm then intending to lookup these IDs in another template which is reused to provision multiple environments. Example below shows my second template is calling a module to provision an EC2 instance and is passing through the subnet_id.
variable "environment" {
description = "Environment name"
default = "dev"
}
module "sql-1-ec2" {
source = "../modules/ec2winserver_sql"
...
subnet_id = "${data.terraform_remote_state.env-shared.subnet-aza-dev}"
}
What I'd like to do is pass the environment variable as part of the lookup for the subnet_id e.g.
subnet_id = "${data.terraform_remote_state.env-shared.subnet-aza-${var.environment}"
However I'm aware that variable interpolation isn't supported. I've tried using a map inside of the first terraform template to export them all to a 'subnet' which I could then use to lookup from the second template. This didn't work as I was unable to output variables inside of the map.
This sort of design pattern is something I've used previously with CloudFormation, however I'm much newer to terraform. Am I missing something obvious here?
Worked out a way to do this using data sources
variable "environment" {
description = "Environment name"
default = "dev"
}
module "sql-1-ec2" {
source = "../modules/ec2winserver_sql"
...
subnet_id = "${data.aws_subnet.subnet-aza.id}"
}
data "aws_subnet" "subnet-aza" {
filter {
name = "tag:Name"
values = ["${var.product}-${var.environment}-${var.environmentno}-subnet-aza"]
}
}
data "aws_subnet" "subnet-azb" {
filter {
name = "tag:Name"
values = ["${var.product}-${var.environment}-${var.environmentno}-subnet-azb"]
}
}
Whilst this works and fulfils my original need, I'd like to improve on this by moving the data blocks to within the module, so that there's less repetition. Still working on that one though...