I'm trying to create configurations in terraform that I can later pass to modules (I'm doing this to work around the lack of "count" in modules).
The closest thing I got was using a null_data_source but the problem with that is that it only supports a single level of properties in inputs:
data "null_data_source" "my_data" {
count = var.my_data_count
inputs = {
settings = { ... } //this doesn't work
}
}
Then I looked at the docs of how to create a custom provider but couldn't work around the types that terraform supports - TypeMap will automatically turned into map[string]string unless I pass in the Elem property but that also only accepts terraform defined types (it doesn't accept standard golang types e.g.: map[string]interface{} or interface{}).
Does anyone know a way to get unstructured data as config like this?
There is no such thing as "unstructured data" in Terraform: every value has an associated type. However, in Terraform 0.12 introduced two structural types that allow for different element/attribute types to be mixed together inside a single value, which is not possible for the collection types.
You can use Local Values if you need to factor out the expressions for these structural values for use in multiple locations:
locals {
your_data = {
settings = {
foo = "bar"
baz = []
}
}
}
Although the details of this often don't matter, Terraform will see the above as being of the following type:
object({
settings = object({
foo = string
baz = tuple([])
})
})
As the author of a module, you can associated with each variable a type constraint that can both check that the given value has the appropriate type and give Terraform some hints to interpret such a value differently. For example, if baz in the above example were a list of strings whose length isn't fixed by the module (often the case) then you can specify it as such in your type constraint:
variable "example" {
type = object({
settings = object({
foo = string
baz = list(string)
})
})
}
Then the caller can pass in the local value we constructed earlier:
module "example" {
source = "./modules/example"
example = local.your_data
}
Terraform will then take the tuple([]) value from the local value and convert it automatically to list(string), in this case creating an empty list of strings.
For Terraform 0.11 your options are more limited, because it does not have structural types. In that case, the usual approach is to flatten the structure into many separate variables and set them separately, but then it's not possible to conveniently construct them all in one place and pass them as a single value.
Related
I try to pass a list of security groups to create ec2 instance.
variables.tf
variable "parameters" {
type = map(any)
}
terraform.tfvars.json
{
"parameters": {
"ami": "ami1234",
"vpc_security_group_ids": "sg-1234,sg-wxyz"
}
}
Note, teffaform does not allow to use list for security groups as it requires all element type of map be the same. So I have to use comma-separated string.
resource "aws_instance" "worker" {
...
vpc_security_group_ids = ["${split(",",var.parameters.vpc_security_group_ids)}"]
}
I copy some online code to split the string, but terraform complains that the variable is only known after apply.
As I think you've already understood, the any in map(any) represents asking Terraform to automatically infer the element type of the map, and so Terraform will study the given value and automatically choose a single concrete type to replace any. In the case of your example here, the result would be map(string) because all of the values in your map are strings.
However, the example you've described here appears to me to more suited to be an object type rather than a map. Maps are intended for arbitrary key/value pairs where all of the elements represent the same kind of thing and can therefore be of the same type. Object types are for describing a single thing that has multiple different properties of different types.
Therefore I would suggest rewriting your type constraint to properly describe the data structure you're expecting, which seems to be the following based on context:
variable "parameters" {
type = object({
ami = string
vpc_security_group_ids = set(string)
})
}
set(string) matches the provider's type constraint for vpc_security_group_ids in aws_instance, since security groups don't have any meaningful ordering when associated with an EC2 instance and so it wouldn't make sense to use a list.
With this data type in place then you should be able to just assign the variable values directly, because they will already be of the expected types:
resource "aws_instance" "worker" {
ami = var.parameters.ami
vpc_security_group_ids = var.parameters.vpc_security_group_ids
}
When developing a terraform module, I sometimes find myself in the need to define different input variables for the same resources. For example, right now I need the NAME and ARN of the same AWS/ECS cluster for my module, so, I defined two variables in my module: ecs_cluster_arn and ecs_cluster_name.
For the sake of DRY, it would be very nice if I could just define the input variable ecs_cluster of type aws_ecs_cluster and the just use whatever I need inside my module.
I can't seem to find a way to do this. Does anyone know if it's possible?
You can define an input variable whose type constraint is compatible with the schema of the aws_ecs_cluster resource type. Typically you'd write a subset type constraint that contains only the attributes the module actually needs. For example:
variable "ecs_cluster" {
type = object({
name = string
arn = string
})
}
Elsewhere in the module, you can use var.ecs_cluster.name and var.ecs_cluster.arn to refer to those attributes. The caller of the module can pass in anything that's compatible with that type constraint, which includes a whole instance of the aws_ecs_cluster resource type, but would also include a literal object containing just those two attributes:
module "example" {
# ...
ecs_cluster = aws_ecs_cluster.example
}
module "example" {
# ...
ecs_cluster = {
name = "blah"
arn = "arn:aws:yada-yada:blah"
}
}
In many cases this would also allow passing the result of the corresponding data source instead of the managed resource type. Unfortunately for this pairing in particular the data source for some reason uses the different attribute name cluster_name and therefore isn't compatible. That's unfortunate, and not the typical design convention for pairs of managed resource type and data source with the same name; I assume it was a design oversight.
module "example" {
# ...
# This doesn't actually work for the aws_ecs_cluster
# data source because of a design quirk, but this would
# be possible for most other pairings such as
# the aws_subnet managed resource type and data source.
ecs_cluster = data.aws_ecs_cluster.example
}
Is there a way to detect the type of a variable in Terraform? Say, I have a module input variable of type any, can I do some kind of switch, depending on the type?
variable "details" {
type = any
}
local {
name = var.details.type == map ? var.details["name"] : var.details
}
What I want to archive is, to be able to pass either a string as shorthand or a complex object with additional keys.
module "foo" {
details = "my-name"
}
or
module "foo" {
details = {
name = "my-name"
age = "40"
}
}
I know this example doesn't make much sense and you would like to suggest to instead use two input vars with defaults. This example is just reduced to the minimal (non)working example. The end goal is to have a list of IAM policy statements, so it is going to be a list of lists of objects.
Terraform v0.12.20 introduced a new function try which can be used to concisely select between different ways of retrieving a value, taking the first one that wouldn't produce an error.
variable "person" {
type = any
# Optional: add a validation rule to catch invalid types,
# though this feature remains experimental in Terraform v0.12.20.
# (Since this is experimental at the time of writing, it might
# see breaking changes before final release.)
validation {
# If var.person.name succeeds then var.person is an object
# which has at least the "name" attribute.
condition = can(var.person.name) || can(tostring(var.person))
error_message = "The \"person\" argument must either be a person object or a string giving a person's name."
}
}
locals {
person = try(
# The value of the first successful expression will be taken.
{name = tostring(var.person)}, # If the value is just a string
var.person, # If the value is not a string (directly an object)
)
}
Elsewhere in the configuration you can then write local.person.name to obtain the name, regardless of whether the caller passed an object or a string.
The remainder of this answer is an earlier response that now applies only to Terraform versions between v0.12.0 and v0.12.20.
There is no mechanism for switching behavior based on types in Terraform. Generally Terraform favors selecting specific types so that module callers are always consistent and Terraform can fully validate the given values, even if that means a little extra verbosity in simpler cases.
I would recommend just defining details as an object and having the caller explicitly write out the object with the name attribute, in order to be more explicit and consistent:
variable "details" {
type = object({
name = string
})
}
module "example" {
source = "./modules/example"
details = { name = "example" }
}
If you need to support two different types, the closest thing in the Terraform language would be to define two variables and detect which one is null:
variable "details" {
type = object({
name = string
})
default = null
}
variable "name" {
type = string
default = null
}
local {
name = var.name != null ? var.name : var.details.name
}
However since there is not currently a way to express that exactly one of those two must be specified, the module configuration you write must be ready to deal with the possibility that both will be set (in the above example, var.name takes priority) or that neither will be set (in the above example, the expression would produce an error, but not a very caller-friendly one).
terraform v1.0+ introduces a new function type() for this purpose. See https://www.terraform.io/language/functions/type
I have a variable that the user will input during run time. Lets say the variable name is region. However, I want the execution to be only successful if the user picks a value from one of the values defined in a list/ choices.
how can I restrict it so the user's selection has to match values that are considered acceptable in the variable definition?
Stumbled across this question.
Since v0.13.0 input validation has been possible directly via the input variables. Thus you can actually achieve this with a snippet such as below.
variable "test_variable" {
type = string
description = "some test value"
validation {
condition = contains(["item1", "item2", "item3"], var.test_variable)
error_message = "Valid values for var: test_variable are (item1, item2, item3)."
}
}
Read more here - https://www.hashicorp.com/blog/custom-variable-validation-in-terraform-0-13
One solution:
variable "values_list" {
description = "acceptable values"
type = "list"
default = ["true", "false"]
}
variable "somevar" {
description = "must be true or false"
}
resource "null_resource" "is_variable_value_valid" {
count = "${contains(var.values_list, var.somevar) == true ? 0 : 1}"
"ERROR: The somevar value can only be: true or false" = true
}
If you pass a value different than "true" or "false" for the "somevar" variable, Terraform will throw an error and stop. The disadvantage is that you have to list all values in the default block of values_list.
Source: https://stackoverflow.com/a/54256780/1364793
Terraform currently has no first-class feature for this, but you can achieve the desired effect (albeit with a less-helpful error message) by asking Terraform to look the value up in a map:
variable "example" {
description = "must be a, b, or c"
}
locals {
allowed_example_values = { for v in ["a", "b", "c"] : v => v }
checked_example = local.allowed_example_values[var.example] # will fail if var.example is invalid
}
Because the values in the allowed_example_values map are the same as the keys, you can then use local.checked_example in place of var.example everywhere else in the module to set up the dependency graph such that it's impossible for an invalid value to be used.
Some caveats/limitations:
You can't customize the error message that Terraform will return when the value is invalid. Instead, Terraform will return the generic error about the key not matching any element in the map. However, it will include the source code snippet from the affected line in the error message (in Terraform 0.12 or later) so the comment at the end of that line should be visible in the error message and thus provide an additional clue to the user as to what might be wrong.
This works fully only for string values, because map keys are always strings in Terraform. Using other primitive types can work as a result of Terraform's automatic conversions from bool and number to string, but you should be sure to explicitly declare the type of the variable (using e.g. type = number) to ensure that Terraform will normalize incoming values into the expected type before looking them up in the map. This technique will not work at all for collection and structural types.
I wouldn't suggest using strings "true" and "false" since Terraform has a bool type which can represent that more intuitively. If you are using Terraform 0.11 and earlier then it's not possible to constrain to bool, but in Terraform 0.12 and later you can declare a variable as type = bool and then Terraform will handle the validation automatically.
I am creating a series of resources in terraform (in this case, dynamo DB table). I want to apply IAM policies to subgroups of them. E.g.
resource "aws_dynamodb_table" "foo" {
count = "${length(var.tables)}"
name = "foo-${element(var.tables,count.index)}"
tags {
Name = "foo-${element(var.tables,count.index)}"
Environment = "<unsure how to get this>"
Source = "<unsure how to get this>"
}
}
All of these share some common element, e.g. var.sources is a list composed of the Cartesian product of var.environments and var.sources:
environments = ["dev","qa","prod"]
sources = ["a","b","c"]
So:
tables = ["a:dev","a:qa","a:prod","b:dev","b:qa","b:prod","c:dev","c:qa","c:prod"]
I want to get the arns of the created dynamo tables that have, e.g. c (i.e. those with the name ["c:dev","c:qa","c:prod"]) or prod(i.e. those with the name ["a:prod","b:prod","c:prod"]).
Is there any sane way to do this with terraform 0.11 (or even 0.12 for that matter)?
I am looking to:
group the dynamo db table resources by some of the inputs (environment or source) so I can apply some policy to each group
Extract the input for each created one so I can apply the correct tags
I was thinking of, potentially, instead of creating the cross-product list, to create maps for each input:
{
"a": ["dev","qa","prod"],
"b": ["dev","qa","prod"],
"c": ["dev","qa","prod"]
}
or
{
"dev": ["a","b","c"],
"qa": ["a","b","c"],
"prod": ["a","b","c"]
}
It would make it easy to find the target names for each one, since I can look up by the input, but that only gives me the names, but not make it easy to get the actual resources (and hence the arns).
Thanks!
A Terraform 0.12 solution would be to derive the cartesian product automatically (using setproduct) and use a for expression to shape it into a form that's convenient for what you need. For example:
locals {
environments = ["dev", "qa", "prod"]
sources = ["a", "b", "c"]
tables = [for pair in setproduct(local.environments, local.sources) : {
environment = pair[0]
source = pair[1]
name = "${pair[1]}:${pair[0]}"
})
}
resource "aws_dynamodb_table" "foo" {
count = length(local.tables)
name = "foo-${local.tables[count.index].name}"
tags {
Name = "foo-${local.tables[count.index].name}"
Environment = local.tables[count.index].environment
Source = local.tables[count.index].source
}
}
At the time I write this the resource for_each feature is still in development, but in a near-future Terraform v0.12 minor release it should be possible to improve this further by making these table instances each be identified by their names, rather than by their positions in the local.tables list:
# (with the same "locals" block as in the above example)
resource "aws_dynamodb_table" "foo" {
for_each = { for t in local.tables : t.name => t }
name = "foo-${each.key}"
tags {
Name = "foo-${each.key}"
Environment = each.value.environment
Source = each.value.source
}
}
As well as cleaning up some redundancy in the syntax, this new for_each form will cause Terraform to identify this instances with addresses like aws_dynamodb_table.foo["a:dev"] instead of aws_dynamodb_table.foo[0], which means that you'll be able to freely add and remove members of the two initial lists without causing churn and replacement of other instances because the list indices changed.
This sort of thing would be much harder to achieve in Terraform 0.11. There are some general patterns that can help translate certain 0.12-only constructs to 0.11-compatible features, which might work here:
A for expression returning a sequence (one with square brackets around it, rather than braces) can be simulated with a data "null_data_source" block with count set, if the result would've been a map of string values only.
A Terraform 0.12 object in a named local value can in principle be replaced with a separate simple map of local value for each object attribute, using a common set of keys in each map.
Terraform 0.11 does not have the setproduct function, but for sequences this small it's not a huge problem to just write out the cartesian product yourself as you did in the question here.
The result will certainly be very inelegant, but I expect it's possible to get something working on Terraform 0.11 if you apply the above ideas and make some compromises.