Terraform: Is there a way to ignore whitepace changes when generating a plan? - terraform

I wanted to ask if there is a way to ignore whitespace changes when creating a terraform plan.
This question is related to this one, I created a new one because I wanted to give a new example of the issue.
Terraform shows unnecessary changes due to whitespace
For example, when running
terraform plan
I get the following change for a helm provider resource
# helm_release.cert-manager will be updated in-place
~ resource "helm_release" "cert-manager" {
id = "cert-manager"
name = "cert-manager"
~ values = [
- <<-EOT
installCRDs: true
EOT,
+ <<-EOT
installCRDs: true
EOT,
]
# (27 unchanged attributes hidden)
}
I found out that the change was due to line endings. Deployed was CRLF and my local source file had LF as line ending.
Is there an option to ignore whitespaces and/or line ending characters?

It's typically the responsibility of the provider itself to determine whether the prior value and the new value are equivalent despite not being exactly equal, and so making this work automatically would require a change to the provider itself to notice that this argument is defined as being YAML and YAML doesn't ascribe any meaning to the decision between CRLF and just LF. The provider would ideally perform this check itself and thus avoid you needing to worry about it, and I would suggest opening a feature request with the provider developer to see if they would be interested in handling that.
However, if a provider isn't performing that job correctly itself then you can potentially work around it by doing your own normalization of the value using Terraform language features, so that the value passed to the provider is always the same when the meaning is the same.
One straightforward way to achieve that in this case would be to round-trip the value through both yamldecode and yamlencode, thereby normalizing the input to be in the style that yamlencode produces:
values = [yamlencode(yamldecode(var.something))]
If you want to be more surgical about it and only normalize the line endings, you could use replace to remove the CR character from any CRLF pair:
values = [replace(var.something, "\r\n", "\n")]
The above solution assumes that the difference in whitespace is being caused by something in your module, such as if you're storing your Terraform configuration in a misconfigured Git repository that's rewriting LF to CRLF when you clone it on a Windows system. This config-based normalization can undo that sort of transformation so that the provider will always see the value in the same way.
This solution cannot address problems that are caused by the provider itself misbehaving. Unfortunately some providers have bugs where they will silently rewrite the stored values for some arguments during the "refresh" step, regardless of how you wrote it in the configuration. In that case the only recourse is to fix the provider, because that incorrect value is originating inside the provider itself and isn't under the control of the module author.

Related

Can you use a templatefile as an output value in terraform without the EOF and EOT artifacts?

I'm building out my infrastructure in terraform and one of the outputs that I would like is the ssh config formatted for the newly created server.
I created a template file:
Host ${serverLabel}
HostName ${hostname}
User ${user}
IdentityFile ${identityFile}
And used it in an output:
output "ssh-config" {
description = "The new server's ssh config"
value = templatefile("${path.module}/ssh-config.tpl", {
serverLabel=aws_instance.minecraft_ec2.tags.Name
hostname= aws_instance.minecraft_ec2.public_ip
user= "ec2-user",
identityFile = "~/.ssh/minecraft_key"
})
}
Which works fine except for one little hiccup. I end up with some heredoc tags in the output:
It's not the end of the world because I can just copy formatted text between the heredocs, but is there a way of using a template file as an output value without getting the tags??
Update
In reply to Mark B's question:
What happens if you run terraform output --raw?
Apparently when I output it with the --raw flag the heredoc tags go away
Which feels weird, but reading the description for the output command I guess makes sense:
-raw - If specified, Terraform will convert the specified output value to a string and print that string directly to the output, without any special formatting. This can be convenient when working with shell scripts, but it only supports string, number, and boolean values. Use -json instead for processing complex data types.
So I guess the heredoc tags are considered "special formatting". Makes sense, thought it was not what I expected.
It appears this is how Terraform now outputs multiline values, similar to how regular values had quotes added to them a few versions ago. To get the values without quotes or heredoc tags wrapping them, use terraform output --raw.

How to keep Gitlab variable masked even the value is changed?

We have a masked variable for example called SECRET_JAMES_BOND with value /vault/abc. During the pipeline execution, the log shows as [MASKED]. When we changed the value to ABCDE, the value was shown. Even we don't change it, we can easily get the value from the log if we perform SECRET_JAMES_BOND = $(echo $SECRET_JAMES_BOND) | base64). By echoing the base64 value, it will display on the UI because the value has changed. When we copy the encoded base64 string and decode it, we get the actual secret. How can we prevent a masked variable from echoing if its value has been changed ? Shouldn't masked variables show [MASKED] even the value is different from its original value ?
How can we prevent a masked variable from echoing if its value has been changed
In short: you can't if someone with access is determined to do so.
Masking is not meant to prevent developers with access to control CI steps from revealing secrets through jobs. It is meant to prevent disclosure by accident. There are endless ways that a value could be exfiltrated through the job output it would be impossible to cover them all.
If you wanted to prevent the base64 representation of the secret from being shown, you can register that value as another masked variable, but it must be done in advance.
Shouldn't masked variables show [MASKED] even the value is different from its original value ?
Good arguments could be made that GitLab should mask some common variations of masked variables, such as base64, url-encoded, backwards, etc. Other CI services (Travis CI for example) do this.
However, this would still be ONLY for the purposes of accidental disclosure.
For example, another way I've seen developers accidentally reveal secrets is by using curl with the -v flag.
script:
- curl -v https://myusername:${SECRET_JAMES_BOND}#myhost.example.com/secret
In the above example, the output of curl will show (in part) an output like:
> Authorization: Basic bXl1c2VybmFtZTpzZWNyZXQ=
So, if GitLab also masked base64 variations of a password, it would have prevented this accidental disclosure.
But like I mentioned, there's endless ways to output a secret: url-encoding, Caesar cipher, rot-N (rot16, rot24, rot32, etc) or even custom ways like just echoing each character one line at a time, potentially in reverse order.
So, really, you can only realistically prevent accidents, not malicious exfiltration by a user with access to execute code in a job.

Why are my build pipelines replacing specific values with asterisks? - Azure DevOps

My team is working to integrate an infrastructure-as-code scanning solution into our build pipelines and we've discovered that the string "GCP" is being replaced with three asterisks when tasks are being executed in our build pipelines. This isn't unique to one task either whereas I created a bash script to execute and list our our repository and all directories that start with "GCP" are replaced by the three asterisks. The only variable set using the "GCP" value is the "system.teamProject" variable and we are not using any secret values that I know of and there are no variable groups used.
Any help would be greatly appreciated. Thanks!
Bash Asterisk Output "ls -a"
IaC Scanning Asterisk Task Failure
If you have set any secret variables in your pipeline, or have linked any variable groups that contain secret variables (include the secrets from the connected external and remote services services), generally the values of these secrets will be masked as asterisks.
When you try to print the values of the secrets to the output logs, the values will display as asterisks in the logs. If you try to output the values into a text file, the values will still display as asterisks in the file.
In addition, if a string that is not set as secret but its substrings are the values of some existing secrets in the pipeline, these substring parts may be masked as asterisks when trying to output this string.
If you do not set any secrets, for us to investigate this issue further, would you like to share us with the actual value that was masked as asterisks in the the logs? We well investigate and evaluate whether this string contains some special or sensitive characters that may be automatically identified as secrets by Azure DevOps.

Configure optional DHCP usage in Terraform for vSphere

Using Terraform to configure vSphere vms, I'd like to be able to provide an IP address (and gateway and netmask) in the tfvars file, but have the vm default to using DHCP if the values are not provided. I know it will use DHCP if the 'vsphere_virtual_machine' resources' 'customize' block contains an empty 'network_interface' block. I was hoping that be giving a default value of "" to the settings in the variables.tf file I could set values if present and use DHCP if not, but I get an error stating:
Error: module.vm.vsphere_virtual_machine.node:
clone.0.customize.0.network_interface.0.ipv4_netmask: cannot parse ''
as int: strconv.ParseInt: parsing "": invalid syntax
So putting in a blank string won't parse, and it won't just leave the whole network_interface blank if the values are blank.
I can't use COUNT on a subresource, so the only thing I've come up with so far is to put two entire, nearly identical, 'vsphere_virtual_machine' resources into my module and then put COUNT statements on both so only one gets created, depending on whether the network settings are provided or not, but man, does that seem ugly...?
I think you are in luck. I've been waiting for this exact same problem to be solved since almost a year now.
Lo and behold, Terraform v0.12.0-alpha1:
They now support dynamic block definitions instead of just static ones
Enjoy, while I'm gonna throw away a couple of hundreds of lines worth of hacks just like the one you mentioned...

How to get non standard tags in rpm query

I wanted to add things such as Size, BuildHost, BuildDate etc in rpm query but adding this thing in spec file results in unknown tag?? How can I do this so that these things are reflected when i give the rpm query command?
These tags are determined when the package is built; they cannot be forced to specific values.
For example BuildHost is hardcoded in rpmbuild and cannot be changed. There is RFE https://bugzilla.redhat.com/show_bug.cgi?id=1309367 to allow it modify from command line. But right now you cannot change it by any tag in spec file nor by passing some option on command line to rpmbuild.
I assume it will be very similar to other values you specified.
RPM5 permits arbitrary unique tag names to be added to header metadata.
The tag names are configured in a colon separated list in a macro. Then the new tags can be used in spec files and can be extracted using --queryformat.
All arbitrary tags are string (or string array) valued.

Resources