Pseudo-code:
module "foo-1" {
source="./foo"
input=1
}
module "foo-2" {
source="./foo"
input=2
}
module "foo-3"
source="./foo"
input=3
}
...etc...
(The module ./foo outputs a unique id based on the input value)
Problem:
I would like to be able to arbitrarily instantiate/call the ./foo module and have access to the unique id from each module instances. I can't see a way to do this with Terraform as the output syntax either requires a unique val=expression per module instantiation. Splat expressions on the module object (module.*.id) are unfortunately (and not surprisingly) not supported.
I'm guessing that this can't be done in terraform but would love to be wrong.
Because each of those modules is entirely separate from Terraform's perspective, to gather their results together into a single value will require writing an expression to describe that. For example:
locals {
foos = [
module.foo_1,
module.foo_2,
module.foo_3,
]
}
With a definition like that, elsewhere in your module you can then write an expression like local.foos[*].id to collect up all of the ids across all of the modules.
Related
When writing terraform modules, one is commonly writing pass through variables/inputs for dependent objects.
How can I write the variable so that the description/type just references the dependent description?
I imagine something like
variable "foo" {
type = dependant.resource.foo.var.type
description = dependant.resource.foo.var.description
default = "module default"
}
Variable descriptions are metadata used by Terraform itself (specifically: by documentation mechanisms like Terraform Registry) and are not data visible to your module code.
Each module's input variables and output values must be entirely self-contained. Mechanisms like the Terraform Registry rely on this so that they are able to generate the documentation for a module only by reference to that module, without any need to fetch and analyze any other modules or other dependencies.
If you do intend to have a variable just "pass through" to a child module or to a resource configuration then you will need to redeclare its type and description in your module.
I would also suggest considering the advice in the documentation section When to write a module; directly passing through a variable to a child object isn't necessarily a wrong thing to do, but it can result from a module not raising the level of abstraction and therefore not passing the test described there.
In such cases, it can be better to use module composition instead of nested modules.
In this case would mean that the caller of your module would themselves call the other module you are currently wrapping. Instead of your module taking that other module's input variables as its own, it would be declared to accept the other module's output values as its input, so that the caller can pass the object representing the first module result into the second module:
module "first" {
# ...
}
module "second" {
# ...
first = module.first
}
Inside this "second" module, the input variable first would be declared as requiring an object type with attributes matching whatever subset of the output values of "first" that the second module depends on.
Suppose I had a module (or resource, or locals block, or whatever)
module "example" {
foo = "foo"
bar = "${foo}" # why wont' this work??
}
It's frustrating to me that the terraform "language" doesn't support this. How am I supposed to write simple DRY code without being able to reference variables?
Edit: I think that example is too contrived. Here's a better one:
module "example" {
domain = "example.com"
uri = "https://${domain}" # why wont' this work??
}
It is supported, but it depends on your module. Your module must output foo first. Then you can do:
module "exmaple" {
source = "./mymodule1"
foo = "dfsaf"
bar = module.exmaple.foo
}
The simplest example of mymodule1 would be:
variable "foo" {
default = 1
}
variable "bar" {
default = 2
}
output "foo" {
value = var.foo
}
But in your example it really does not make sense doing that, as bar is always same as foo, thus it shouldn't even be exposed.
The better and more natrual way would be:
locals {
foo = "foo"
}
module "exmaple" {
source = "./mymodule1"
foo = local.foo
bar = local.foo
}
Your question seems to be two different questions: how can you achieve the goal you described, and also why didn't the thing you tried first work.
Others already answered how you can factor out values to use in multiple locations, and so I wanted to just try to address the second question about why this didn't work, in case it's useful for your onward journey with Terraform.
The Terraform language is designed to appear as a heirarchical data structure rather than as linear code to be executed, which of course has some significant tradeoffs. In return for hopefully making a well-written Terraform configuration read like a manifest of what should exist rather than a program for creating it, the Terraform language does somewhat obscure the real control flow and symbol scoping that you might be accustomed to in other languages.
For your question in particular, I think it's important to recognize that the arguments inside a module block are not like declarations of variables, but are instead more like arguments passed to a function.
For your second example then, it might help to see it reworked into a function-call-like syntax:
module "example" {
source = "./mymodule1"
domain = "example.com"
uri = "https://${domain}"
}
# NOTE: pseudocode
example(domain: "example.com", uri: "https://#{domain}")
In most imperative-style languages, function arguments bind to symbols inside the module rather than outside of it, and so defining domain as "example.com" here doesn't make that symbol visible to other subsequent arguments in the same call.
Now taking Marcin's final example, adapted to resemble your second example:
locals {
domain = "example.com"
}
module "example" {
source = "./mymodule1"
domain = local.domain
url = "https://${local.domain}"
}
# NOTE: pseudocode
domain = "example.com"
example(domain: domain, uri: "https://#{domain}")
Of course this is also exposing another difference with Terraform compared to general-purpose languages, which is that it doesn't have a shared variable scope with everything assigned into it and instead places local variables inside a namespace local. But despite the difference in syntax, they are local values scoped to the module they are defined with, and so you can refer to local.domain anywhere else in that same module. (Note: this is a whole-module scope, and not a lexical scope as you might be accustomed to in other languages.)
A similar principle applies to resource, data, and provider blocks; those two are more like arguments to a function than they are variable declarations, but just shaped in a different syntax with the goal of it appearing as a series of declarations rather than as sequential code.
Let's take the dns_a_record_set data source, for example, if one does:
data "dns_a_record_set" "test" {
## Purposely passing a DNS that would not resolve
host = "thisDnsDoesNotResolve.org"
}
output "test" {
value = data.dns_a_record_set.test.addrs
}
The error returned is, as expected:
Error: error looking up A records for "thisDnsDoesNotResolve.org": lookup thisDnsDoesNotResolve.org on 1.1.1.1:53: no such host
│
│ with data.dns_a_record_set.test,
│ on main.tf line 1, in data "dns_a_record_set" "test":
│ 1: data "dns_a_record_set" "test" {
But, is it possible to recover from this error and have the test output assigned with a default value?
A naive trial of mine was to use the try function, but that obviously does not work, as the error does not happens when data is accessed, but rather when fetched, as shown in the error above.
output "test" {
## Seems like it doesn't even reach here
value = try(data.dns_a_record_set.test.addrs, ["127.0.0.1"])
}
So, is this even possible to recover from this error?
And, then, how?
The purpose of data resources in Terraform is to declare dependencies on objects managed elsewhere and, as a side-effect, obtain information about them to use in other parts of your module in a similar way as you might with objects that are managed by your current module.
If your module only depends on this DNS record in certain situations then a typical approach would be to use either the count or for_each meta-argument to explicitly declare in which situations your module depends on this object and which situations it doesn't.
It's hard to show a specific example without more information about your underlying motivation for this, but here's a contrived example where the hostname can be directly provided by the module caller as an input variable, and the dependency on that external object is only relevant when that variable is set (non-null):
variable "hostname" {
type = string
default = null # this variable is optional
}
data "dns_a_record_set" "test" {
count = length(var.hostname[*])
host = var.hostname
}
output "ip_addresses" {
value = one(data.dns_a_record_set.test[*].addrs)
}
The above uses two additional language features I didn't mention yet in this comment, which I'll describe briefly for completeness:
Applying the splat operator [*] to a non-list value converts the value to be a list which either has one element (if the value is non-null) or zero elements (if the value is null). Therefore there will be one instance of this data resource if and only if var.hostname is non-null.
The one function does essentially the opposite of that: if given a one-element list it will return the first element, and if given a zero-element list it will return null.
These two features taken together allow you to concisely pivot between values that might be null and lists of zero or one element, which is useful in situations like this where the "null-ness" of a valid decides how many of something should be declared.
Note that the try function only "catches" expression evaluation failures. It isn't relevant to this situation because what failed was the declared dependency on the external object; the error was returned when Terraform asked the provider to retrieve the dns_a_record_set object, not in the downstream expression which referred to that.
(Indeed, Terraform would never actually evaluate that expression in your case, because when there's a dependency from object B to object A, a failure of object A halts processing before reaching object B.)
I have a module a in terraform which creates a text file , i need to use that text file in another module b, i am using locals to pull the content of that text file like below in module b
locals {
ports = split("\n", file("ports.txt") )
}
But the terraform expects this file to be present at the start itself, throws error as below
Invalid value for "path" parameter: no file exists at
path/ports.txt; this function works only with files
that are distributed as part of the configuration source code, so if this file
will be created by a resource in this configuration you must instead obtain
this result from an attribute of that resource.
What am i missing here? Any help on this would be appreciated. Is there any depends_on for locals, how can i make this work
Modules are called from within other modules using module blocks. Most arguments correspond to input variables defined by the module. To reference the value from one module, you need to declare the output in that module, then you can call the output value from other modules.
For example, I suppose you have a text file in module a.
.tf file in module a
output "textfile" {
value = file("D:\\Terraform\\modules\\a\\ports.txt")
}
.tf file in module b
variable "externalFile" {
}
locals {
ports = split("\n", var.externalFile)
}
# output "b_test" {
# value = local.ports
# }
.tf file in the root module
module "a" {
source = "./modules/a"
}
module "b" {
source = "./modules/b"
externalFile = module.a.textfile
depends_on = [module.a]
}
# output "module_b_output" {
# value = module.b.b_test
# }
For more reference, you could read https://www.terraform.io/docs/language/modules/syntax.html#accessing-module-output-values
As the error message reports, the file function is only for files that are included on disk as part of your configuration, not for files generated dynamically during the apply phase.
I would typically suggest avoiding writing files to local disk as part of a Terraform configuration, because one of Terraform's main assumptions is that any objects you manage with Terraform will persist from one run to the next, but that could only be true for a local file if you always run Terraform in the same directory on the same computer, or if you use some other more complex approach such as a network filesystem. However, since you didn't mention why you are writing a file to disk I'll assume that this is a hard requirement and make a suggestion about how to do it, even though I would consider it a last resort.
The hashicorp/local provider includes a data source called local_file which will read a file from disk in a similar way to how a more typical data source might read from a remote API endpoint. In particular, it will respect any dependencies reflected in its configuration and defer reading the file until the apply step if needed.
You could coordinate this between modules then by making the output value which returns the filename also depend on whichever resource is responsible for creating the file. For example, if the file were created using a provisioner attached to an aws_instance resource then you could write something like this inside the module:
output "filename" {
value = "D:\\Terraform\\modules\\a\\ports.txt"
depends_on = [aws_instance.example]
}
Then you can pass that value from one module to the other, which will carry with it the implicit dependency on aws_instance.example to make sure the file is actually created first:
module "a" {
source = "./modules/a"
}
module "b" {
source = "./modules/b"
filename = module.a.filename
}
Then finally, inside the module, declare that input variable and use it as part of the configuration for a local_file data resource:
variable "filename" {
type = string
}
data "local_file" "example" {
filename = var.filename
}
Elsewhere in your second module you can then use data.local_file.example.content to get the contents of that file.
Notice that dependencies propagate automatically aside from the explicit depends_on in the output "filename" block. It's a good practice for a module to encapsulate its own behaviors so that everything needed for an output value to be useful has already happened by the time a caller uses it, because then the rest of your configuration will just get the correct behavior by default without needing any additional depends_on annotations.
But if there is any way you can return the data inside that ports.txt file directly from the first module instead, without writing it to disk at all, I would recommend doing that as a more robust and less complex approach.
In URL https://docs.puppetlabs.com/references/glossary.html#plusignment-operator, here is its explanation:
The +> operator, which allows you to add values to resource attributes using the (‘plusignment’) syntax. Useful when you want to override resource attributes without having to respecify already declared values.
For example, I have code like this:
Package {
require => File['/etc/apt/apt.conf.d/no-cache'],
}
package { 'php5-cgi':
ensure => present,
}
Package[ 'php5-cli' ] {
require +> Package['php5-cgi'],
}
What did the operator +> mean here?
Other sample:
subscribe +> Sshkey['www.example.com']
The attribute in question takes a value that is composed of the right hand side and whatever it would have taken otherwise.
In your example, the package { 'php5-cgi' } would normally use a require value of File['/etc/apt/apt.conf.d/no-cache'], since it is the default for all package resources. Through the plusignment, you end up with a value of
require => [ File['/etc/apt/apt.conf.d/no-cache'], Package['php5-cgi'] ]
The php5-cli package builds a relation to both the referenced file and the php5-cgi package.
The same logic will apply to the subscribe metaparameter from your second example. I cannot comment on the whole semantics without more context.
The plusignment works for all resource attributes, although the resulting array values will not make sense for many of them.