Currently developing a TF template inclusive of some glue services. When attempting to init the template, I am prompted with the below error:
Error: Invalid template control keyword
on main.tf, in resource "aws_glue_classifier" "SrcPlanClassifier":
grok_pattern = "%{DATA:col},%{DATA:col2},%{DATA:col3},"%{DATA:col4}",% .
{DATA:col5},%{DATA:col6},%{DATA:col7},%{DATA:col8},%{DATA:col9},%{DATA:col10},% .
{DATA:col11},%{DATA:col12},%{DATA:col}13$"
"DATA" is not a valid template control keyword.
This template is the result of translating an existing CloudFormation template to Terraform, so the pattern has worked in the past. From what I can tell in the AWS documentation for Glue, the DATA term is an acceptable built-in classifier. That implies to me that this is an issue on the Terraform end. What am I missing here?
Thanks in advance for the help!
Terraform is understanding the %{ sequences as introducing a template directive, which then fails because if and for are the only keywords that are allowed to follow that %{ marker in the Terraform language.
To use %{ literally in your string, you can write %%{ instead to escape the introducer. Terraform will see %%{ and produce %{ in the resulting string:
grok_pattern = "%%{DATA:col},%%{DATA:col2},%%{DATA:col3},"%%{DATA:col4}",% .
{DATA:col5},%%{DATA:col6},%%{DATA:col7},%%{DATA:col8},%%{DATA:col9},%%{DATA:col10},% .
{DATA:col11},%%{DATA:col12},%%{DATA:col}13$"
(I think there were some line-wrapping problems in the message you shared so I've updated it as best I could for the escaping but you may need to do some additional escaping yourself. The general idea is to replace every %{ with %%{ in your quoted string.)
Another option, for complicated expressions whose readability is hurt significantly by this much escaping, is to move the relevant string into a separate file and then have Terraform read that file:
grok_pattern = file("${path.module}/grok_pattern.txt")
The file function just takes the text in the given file verbatim, and does not parse it for template sequences or any other special markers.
Related
I installed terraform and trying to do eval basic expression.
I am getting weird errors.
e.g what's wrong with this expression?
variable "BN" { default = "X" }
or
x = 3
for loops works but it needs to be wrapped into square brackets.
[ for bn in [1, 2, 3] : "%{if bn == 2} ok %{else} bad ${bn} %{endif}" ]
terraform console makes impression that you made a syntax error, but I think at most it could be semantic one.
As for now this utility looks very limited.
Is there any other place I can play with terraform where?
Of the three examples you shared, only the third one is actually a Terraform language expression. An expression is something that generates a value which you can then assign to a resource type argument.
Your first example is a variable declaration. The string you assigned to its default argument is an example of an expression - a literal string expression - but that overall block declares a new variable in a configuration, so it's not something you can evaluate to produce a value.
Your second example seems to assigning the value 3 to an argument called x, but that doesn't mean anything without some additional context: x would need to be an argument inside a block in order to be meaningful, and even then it wouldn't be an expression, but rather the definition of an argument called x using the expression 3.
If you want to experiment with the Terraform language then the best way to do it is to make a file whose name has the suffix .tf in a new directory, and then run the main Terraform commands terraform init, terraform plan, etc in that directory. The terraform console command is for evaluating hypothetical expressions against your configuration, but until you've actually written a configuration you'll have nothing other than literal values to refer to.
I am trying to deploy a load balancer template in which I port in various names to form a concatenation. This is so that the parameters template affects the naming of back end address pools and other outputs. My current concatenation code is this:
"[concat(resourceId('Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/frontendIpConfigurations/', variables('subnet1name'), '-FrontEnd'))]"
However, when I attempt to run this in Jenkins, I get the following error message:
"Unable to process template language expressions for resource '/subscriptions/****/resourceGroups/test-networks-hub-rg/providers/Microsoft.Network/loadBalancers/test-firewall-lb' at line '1' and column '2581'. 'Unable to evaluate template language function 'resourceId': function requires exactly one multi-segmented argument which must be resource type including resource provider namespace. Current function arguments 'Microsoft.Network/loadBalancers/,test-firewall-lb,/frontendIpConfigurations/,test-firewall-subnet,-FrontEnd'
It seems like the resourceId is not recognised because it is not being concatenated properly. I checked the Microsoft guidance, but I can't seem to work out where I am going wrong.
Can anyone see my error?
Okay, turns out I just had a bracket in the wrong place. The correct syntax is this:
"[concat(resourceId('Microsoft.Network/loadBalancers/', variables('loadBalancerName')), '/frontendIpConfigurations/', variables('subnet1name'), '-FrontEnd')]"
Remove concat() and just use the resourceID function, e.g.
[resourceId('Microsoft.Network/loadBalancers/frontendIpConfigurations', variables('loadBalancerName'), concat(variables('subnet1name'), '-FrontEnd'))]
Learning that function/pattern will help quite a bit when you start using resources in other groups or subscriptions.
I'm using the terraform template_file data resource to create a file that should be written to the dynamically-created EC2 instance when I apply the stack. In other words, I want this file to be created in the home folder of the newly-created EC2 instance. However, this file contains curly bracket syntax ${}, which terraform is trying to interpolate. How can I escape these curly brackets?
As background, I'm using the cloud-config syntax to write these files.
Ex:
${username} should be written to the file, not interpolated in terraform.
Even when I use the double dollar sign $$, terraform still fails because it can't find the variable:
... failed to render : <template_file>:105,18-26: Unknown variable; There is no variable named "username".
Terraform uses a fairly unique escape for curly braces:
Sequence
Replacement
$${
Literal ${, without beginning an interpolation sequence.
%%{
Literal %{, without beginning a template directive sequence.
Documentation for reference: https://www.terraform.io/docs/language/expressions/strings.html
FYI I ended up working around this by writing the template in another file, then reading it into the terraform stack using the file method:
data "template_file" "config" {
template = "${file("./user_data.tpl")}"
}
For example if you would like to add into Tags {{_payload.ref}} as it was in my case, I have done it this way:
tags = ["CRITICAL", "\u007B\u007B_payload.ref\u007D\u007D"]
so
\uNNNN Unicode character from the basic multilingual plane (NNNN is four hex digits)
ref.page: https://www.terraform.io/language/expressions/strings
I'm trying to use the keys expression in Terraform to grab a list of keys (from a map variable) and assign it to a local variable. Here is the code snippet:
locals {
project_name_list = keys(${var.project_map})
}
However, I'm getting the following error:
Unknown token: 29:22 IDENT keys
Am I missing something here. Nowhere can I find an example of this expression. As bad as it is, even the official documentation does not help -https://www.terraform.io/docs/configuration/functions/keys.html
HashiCorp has really done a bad job of elaborating the nuances of Terraform for beginners on their website.
Terraform functions need to be wrapped in expression syntax to show that it's not a literal value: "${}"
So try this: project_name_list = "${keys(var.project_map)}"
The example in the documentation is written as though being run from the terraform command line, which already assumes the command is a HCL expression and doesn't require that syntax.
UPDATE
I said above that the expression syntax is to show that it's not a literal value. It's probably more accurate to speak of it as expression syntax vs. configuration syntax. Configuration syntax is the first level of interpolation, which forms the basic structure of your terraform file with resource blocks, data blocks, etc. The second interpolation level is expression syntax which is used to generate values used by your configuration.
Thinking of it in these terms makes better sense of the error message, Unknown token, because terraform is attempting to read it as a configuration key word.
I had compared it to a literal value because it's in the same position as where a literal value would be.
Using terraform, I'd like to see if there's a way — with the template rendering system — to define a template_file in a terraform module (base template) and then "extend" or "append" to the rendering in the instantiation rather than replaced.
I can currently define the template_file in either location, but would like to know if I can build upon the module's template on a per-instance basis. The specific use case is for userdata on AWS EC2s which vary slightly from instance type to instance type.
Thank you,
Neurax
This is what I ended up finding out.
In the module. Define an a template_file for the base template. Then define an output (for example named "module_template") whose value is equal to the rendered version of the template_file.
Then in the instantiation, define another template_file with a var equal to something like mod_temp = "${module.module_name.module_template}", and then in the template_file, reference that variable where necessary with ${mod_temp}.
I was thinking about trying to escape values in the "super template" so that maybe they would get interpolated during the "sub template's" rendering. Initial tests have not been successful.