I have two lists which need to be joined before being used in for_each. Can the setunion be used directly with lists or each list needs to be casted using toset before setunion can be used?
The setunion function knows that its arguments are supposed to be sets and so it will automatically convert from list to set if you pass in a list value.
In practice the following two expressions are equivalent:
toset(concat(list_a, list_b))
setunion(list_a, list_b)
The difference is only in the method Terraform takes to calculate the result. In the first case it will first construct a new list which is the concatenation of list_a and list_b, and then convert that result to a set. In the second case it'll convert list_a and list_b to sets first and then calculate the union of those two sets. Because of the definition of setunion, these two methods will always produce the same result.
I think you may be asking this question because for for_each Terraform does not automatically convert from a list to a set.
The for_each argument has a special rule to disable Terraform's normal automatic conversion rule in this case because during the early alpha/beta testing period for for_each the team observed that people were often confused about what assigning a list to for_each would mean:
Some people expecting it to behave like count = length(list), while other people expected it to behave like for_each = toset(list).
Because the difference between these is subtle and may cause problems with later updates if the wrong interpretation were used, the Terraform team chose to force module authors to be explicit about which of these two interpretations they mean, either by explicitly converting the list to a set or by using the count argument instead to indicate that they want to use numeric indices for instance keys.
However, Terraform doesn't actually care how you produce the set of strings assigned to for_each, so you can use any expression whose result is of that type. setunion when given sets of strings (or anything that can convert to sets of strings) will always produce a set of string as a result, and so a call to that function is also a valid thing to assign to for_each:
for_each = setunion(a, b)
No explicit conversion is needed in this case because setunion inherently returns a list as part of its definition.
Related
The title is really all there is to the question: how would you compute the values of a Fibonacci sequence (first N values, where N is an input variable) and store them in a Terraform local variable?
This could, of course, be done with an external data source, but I'm looking for a way to do it in pure Terraform.
There's no real need to actually do this, but the Fibonacci sequence is a representation of a problem I need to solve in Terraform (where values in a list depend on previous values of that same list).
I think the easiest way would be to create your own external data source for that, as in TF you can't access existing list elements during iteration when you create the lists itself.
And specific to Fibonacci sequence. I would just per-compute its values, and then just read any number of values I need from a list in TF or a file. Usually you would know a possible maximum number of those elements your app requires. Thus there is no reason to recalculate it every single time.
I have managed to do it in Python, since the interpreter provide the bytecodes, as shown below.
From the bytecodes it was easy to apply a hashing function.
import dis
from inspect import signature
# Add signature.
fargs = list(signature(f).parameters.keys())
if not fargs:
raise Exception(f"Missing function input parameters.")
clean = [fargs]
# Clean line numbers.
groups = [l for l in dis.Bytecode(f).dis().split("\n\n") if l]
for group in groups:
lines = [segment for segment in group.split(" ") if segment][1:]
clean.append(lines)
How could I extract a similar kind of deterministic universally unique ยน function identifier in Haskell?
Are there built-in libraries or GHC extensions for that?
1 - unique, probabilistically speaking
* - A function not in the math sense, since the possibilities to check programatically the equivalence of programs are very limited. So I am only interested in what is possible and costless attainable, i.e., generate a unique identity for functions with the exact same code (or bytecode, to avoid formatting noise). In my specific use case, even the signatures and parameter names should match, but that is not relevant for the point here, which is about how to inspect Haskell code at runtime.
Within a single run of the program, you might consider StableName. You can make a StableName for any value (including a function/closure), get a hash value for a StableName, and compare two StableNames for equality. If two things have the same StableName, then they're definitely the same. If they have the same StableName hash, then they're probably the same. Note that if you just need to be able to check equality and don't need the hashes, one option is reallyUnsafePtrEquality#, which just compares any two values by address.
In 0.11 we had a module that create a google host. The module will create
it in one of two ways depending on the arguments passed in, using one of
two different resource statements, i.e.
resource "google_compute_instance" "this_public" {
count = var.instance_count * var.flag
}
resource "google_compute_instance" "this_private" {
count = var.instance_count * (1 - var.flag)
}
(Grossly simplified, but equivalent)
For output we'd collect both possible resources, slam them together and take which ever one wasn't blank. (given the code, one OR the other would generate, never both, so it's guaranteed that one of these strucutures is empty.)
this_instance_id = compact(
concat(
coalescelist(
google_compute_instance.this_public.*.id,
google_compute_instance.this_private.*.id,
),
[""],
),
)
This fails miserably now because tf 0.12 cares that one of those two is empty,
and it can't collesce it.
I can't see how to make this work again without completely redesigning it
with lots and lots of for loops on the various elements.
Given that these two resources are essentially mutually exclusive, I think an expression like the following would produce an equivalent result:
concat(
google_compute_instance.this_public.*.id,
google_compute_instance.this_private.*.id,
)
Since one of those two lists is guaranteed to be empty, this will always be a list of var.instance_count items concatenated with an empty list, which I think gets the result you were looking for here.
A different way to do it, which may be more explicit to future readers of this configuration, would be to rely on the conditional operator which is now fully general for all types of value in Terraform 0.12:
var.flag != 0 ?
google_compute_instance.this_public.*.id :
google_compute_instance.this_private.*.id
For example one of my entities has two sets of IDs.
One that is continuous (which apparently is necessary to create the EntitySet), and one to use as a foreign key when merging with my other table.
This results in featuretools including the ID in the set of features to aggregate. SUM(ID) isn't a feature I am interested in though.
Is there a way to include certain feature when running deep feature synthesis?
There are three ways to exclude features when calling ft.dfs.
Use the ignore_variables to specify variables in an entity that should not be used to create features. It is a dictionary mapping an entity id to a list of variable names to ignore.
Use drop_contains to drop features that contain any of the strings
listed in this parameter.
Use drop_exact to drop features that exactly match any of the strings listed in this parameter.
Here is a example usage of all three in a ft.dfs call
ft.dfs(target_entity="customers"],
ignore_variables={
"transactions": ["amount"],
"customers": ["age", "gender", "date_of_birth"]
}, # ignore these variables
drop_contains=["customers.SUM("], # drop features that contain these strings
drop_exact=["STD(transactions.quanity)"], # drop features named exactly this
...
)
These 3 parameters are all documented here.
The final thing to consider if you are getting features you don't want is the variable types of the variables in your entity set. If you are seeing the sum of an ID variable that must mean that featuretools thinks the ID variable is a numeric value. If you tell featuretools it is an ID it will not apply a numeric aggregation to it.
How do I get numeric values from the RepresentationModel?
Say, after traversing a document, I have a YamlScalarNode. It has a string Value, which I can, of course, try to convert to a number, but I'd expect YAML to detect the type and present it as int or double etc. (perhaps via descendants from YamlScalarNode, whose type I could detect).
Is there an official way to do it that I'm missing?
Note that I can't use Serialization: the document structure does not directly map to a class; it can be a recursive definition of arbitrary depth, and the end values are either scalar numbers or sequences of numbers (vectors).
Also, can YamlDotNet handle numerical keys in mappings? This means that keys 1 and 01 should be considered duplicates. I believe YAML specification requires that, but I'm not certain...
The YAML schemas specify how scalars are to be interpreted. Ideally, you would look at the tag of a scalar to establish its type according to the selected schema. However, YamlDotNet does not yet implement them. For now you will have to do that yourself.