I've deployed my infra using Terraform and I noticed that I have some interesting information in the state (terraform.tfstate) file of terraform which I would like to extract. For example
$ terraform state show 'packet_device.worker'
id = 6015bg2b-b8c4-4925-aad2-f0671d5d3b13
billing_cycle = hourly
created = 2015-12-17T00:06:56Z
facility = ewr1
...
which I would like to transform somehow to
$ terraform state show 'packet_device.worker.id'
6015bg2b-b8c4-4925-aad2-f0671d5d3b13
But adding the id at the end doesn't seem to work. Any suggestions how I can achieve this behaviour?
Terraform state show command is used to retrieve all the attributes of a given resource and you won't be able to fetch a single attribute from it as the argument is resource ADDRESS and is used to refer a resource specifically. Documented in https://www.terraform.io/docs/internals/resource-addressing.html
What you can do is store the resource attribute in output value and use the command
terraform output {output-valaue-extractor}
Refer: https://www.terraform.io/docs/configuration/outputs.html
You can utilize terraform show -json and jq to get a specific value out of a Terraform state file.
terraform show -json <state_file> | jq '.values.root_module.resources[] | select(.address=="<terraform_resource_name>") | .values.<property_name>'
You have a state file named terraform.tfstate and a Terraform resource as packet_device.worker and you want to get id. Then it would be as follows:
terraform show -json terraform.tfstate | jq '.values.root_module.resources[] | select(.address=="packet_device.worker") | .values.id'
terraform.tfstate also can be omitted since it is a default name for a state file.
The primary way to export information from a Terraform configuration is to declare Output Values in your root module. You can then access them using terraform output once the apply has completed. If you need that information in a machine-readable way, you can alternatively run terraform output -json from the consuming program and parse the output as JSON.
If you are in an unusual situation where you need programmatic access to all values in the state (for example, if you were implementing some sort of generic Terraform state visualization tool) then you can instead use terraform show -json, which will print out all of the data from the state in a JSON format.
If you are accessing only specific values, perhaps to integrate with some other system in an automation solution, I'd recommend using explicit Output Values because then it's explicit to future maintainers what the interface with the caller is, and so they are less likely to accidentally break the caller by e.g. refactoring the packet_device.worker resource into a child module, which would cause it to appear in a different place in the state. The usual assumption is that the resources inside a module are an implementation detail of that module and thus that you can safely refactor them as needed as long as the output values remain unchanged.
If you want to get the exact value and are willing to install jq, the other answers here are great!
If you're looking for a quick answer to manually copy/paste, etc., piping to grep does the trick.
ex:
terraform state show 'packet_device.worker' | grep "id"
which would show the relevant line(s), like:
id = 6015bg2b-b8c4-4925-aad2-f0671d5d3b13
Related
I'm building out my infrastructure in terraform and one of the outputs that I would like is the ssh config formatted for the newly created server.
I created a template file:
Host ${serverLabel}
HostName ${hostname}
User ${user}
IdentityFile ${identityFile}
And used it in an output:
output "ssh-config" {
description = "The new server's ssh config"
value = templatefile("${path.module}/ssh-config.tpl", {
serverLabel=aws_instance.minecraft_ec2.tags.Name
hostname= aws_instance.minecraft_ec2.public_ip
user= "ec2-user",
identityFile = "~/.ssh/minecraft_key"
})
}
Which works fine except for one little hiccup. I end up with some heredoc tags in the output:
It's not the end of the world because I can just copy formatted text between the heredocs, but is there a way of using a template file as an output value without getting the tags??
Update
In reply to Mark B's question:
What happens if you run terraform output --raw?
Apparently when I output it with the --raw flag the heredoc tags go away
Which feels weird, but reading the description for the output command I guess makes sense:
-raw - If specified, Terraform will convert the specified output value to a string and print that string directly to the output, without any special formatting. This can be convenient when working with shell scripts, but it only supports string, number, and boolean values. Use -json instead for processing complex data types.
So I guess the heredoc tags are considered "special formatting". Makes sense, thought it was not what I expected.
It appears this is how Terraform now outputs multiline values, similar to how regular values had quotes added to them a few versions ago. To get the values without quotes or heredoc tags wrapping them, use terraform output --raw.
I wanted to ask if there is a way to ignore whitespace changes when creating a terraform plan.
This question is related to this one, I created a new one because I wanted to give a new example of the issue.
Terraform shows unnecessary changes due to whitespace
For example, when running
terraform plan
I get the following change for a helm provider resource
# helm_release.cert-manager will be updated in-place
~ resource "helm_release" "cert-manager" {
id = "cert-manager"
name = "cert-manager"
~ values = [
- <<-EOT
installCRDs: true
EOT,
+ <<-EOT
installCRDs: true
EOT,
]
# (27 unchanged attributes hidden)
}
I found out that the change was due to line endings. Deployed was CRLF and my local source file had LF as line ending.
Is there an option to ignore whitespaces and/or line ending characters?
It's typically the responsibility of the provider itself to determine whether the prior value and the new value are equivalent despite not being exactly equal, and so making this work automatically would require a change to the provider itself to notice that this argument is defined as being YAML and YAML doesn't ascribe any meaning to the decision between CRLF and just LF. The provider would ideally perform this check itself and thus avoid you needing to worry about it, and I would suggest opening a feature request with the provider developer to see if they would be interested in handling that.
However, if a provider isn't performing that job correctly itself then you can potentially work around it by doing your own normalization of the value using Terraform language features, so that the value passed to the provider is always the same when the meaning is the same.
One straightforward way to achieve that in this case would be to round-trip the value through both yamldecode and yamlencode, thereby normalizing the input to be in the style that yamlencode produces:
values = [yamlencode(yamldecode(var.something))]
If you want to be more surgical about it and only normalize the line endings, you could use replace to remove the CR character from any CRLF pair:
values = [replace(var.something, "\r\n", "\n")]
The above solution assumes that the difference in whitespace is being caused by something in your module, such as if you're storing your Terraform configuration in a misconfigured Git repository that's rewriting LF to CRLF when you clone it on a Windows system. This config-based normalization can undo that sort of transformation so that the provider will always see the value in the same way.
This solution cannot address problems that are caused by the provider itself misbehaving. Unfortunately some providers have bugs where they will silently rewrite the stored values for some arguments during the "refresh" step, regardless of how you wrote it in the configuration. In that case the only recourse is to fix the provider, because that incorrect value is originating inside the provider itself and isn't under the control of the module author.
I currently have a dev.auto.tfavrs file with a few dozen variables and their values for my application such as:
DB_NUM_RECORDS_PER_EXECUTION = "Something here"
QUEUE_API_KEY = "Something here"
application = "My Appplication"
Application_Secret_Key = "Some Secret Key here"
All variables are defined in a variables.tf file, with the sensitive ones assigned a blank value.
I run terraform plan in this manner:
terraform apply -var-file=env/dev.auto.tfvars
(I have also qa.auto.tfvars and prod.auto.tfvars for the other environments)
I want to inject certain sensitive values to certain keys such as :
Application_Secret_Key via The Terraform Cloud Variable sets so developers don't have to.
I have added Application_Secret_Key in the TF Cloud Variable set.
but when i run the above terraform plan, Terraform Cloud is not injecting the value stored in the variable set...instead it assigns the blank value as defined in my configuration.
It is my understanding that the auto.tfvars files take precedence and over write the terraform.tfavars file in Terraform Cloud. Hence the sensitive values are Blank
I do not want to add dozens of variables in the TF Cloud Variable set...only certain sensitive ones.
Is this possible?
Thanks in advance
I was wondering if it's possible to grab different data dynamically based on variables like so
data.terraform_remote_state.vm.outputs.vm_***var.vmname***
Or something similar? i dont have the option to redesign the outputs currently, and this would greatly lower the chance of making failure upon creating new terraform deployments
thanks!
There are Input Variables available in Terraform. These variables allow you to define inputs expected at the time of terraform apply. The values may be entered via an interactive terminal or provided in a .tfvars file.
variable "vmname" {
type = string
description = "The name of the virtual machine."
}
Then you can use them by expansion:
"data.terraform_remote_state.vm.outputs.vm_${var.vmname}"
For additional reference, see https://www.terraform.io/docs/language/values/variables.html
I would like to extract the data given from terraform state show. According to documentation we should use terraform show -json . https://www.terraform.io/docs/cli/commands/state/show.html
The output of terraform state show is intended for human consumption,
not programmatic consumption. To extract state data for use in other
software, use terraform show -json and decode the result using the
documented structure.
Not sure how to use terrafrom state show in conjuction with terrafrom show
$ terraform state show 'packet_device.worker'
# packet_device.worker:
resource "packet_device" "worker" {
billing_cycle = "hourly"
created = "2015-12-17T00:06:56Z"
facility = "ewr1"
hostname = "prod-xyz01"
id = "6015bg2b-b8c4-4925-aad2-f0671d5d3b13"
locked = false
}
The terraform state show command displays information on a single Terraform resource and does not support the -json flag. The command terraform show dumps the entire state, and does support the -json flag. Unlike the output from terraform state show, the output of terraform show -json is documented and intended for programmatic consumption.
If you want to obtain the info on a particular resource as displayed by terraform state show, you can extract it from the full-state JSON, for example by using jq:
terraform show -json | \
jq '.values.root_module.resources[] | select(.address == "packet_device.worker") | .values'
Whether this makes sense depends on what it is you're trying to do.