Is there any way for external data sources to find if this is a plan, apply or destro operation - terraform

Question

I have a Python program that is being called by an "External Data Source" explained here:
https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/data_source
The code looks like something like this:
data "external" "example" {
program = ["python", "${path.module}/example-data-source.py"]
query = {
# arbitrary map from strings to strings, passed
# to the external program as the data query.
id = "abc123"
}
}
Is there technique or workaround for my external Python program to know which terraform operation (plan, apply or destroy) is calling the external data?
Is there any way for a hcl code to find what operation is it being called under?
Update #1
I understand that we can pass parameters as query argument.
The question is that is there any way a Terraform script know which Terraform operation type is it being called under? so that I have the operation (Apply/Destroy) in a terraform variable then pass it as query argument to the Python program?

Answer 1

The external data source isn't able to infer what the Terraform operation type is as it is only sent the data in the query argument, as mentioned and shown in the data source's documentation.
As an example, if we look at the example in the documentation:
#!/bin/bash
# Exit if any of the intermediate steps fail
set -e
# Extract "foo" and "baz" arguments from the input into
# FOO and BAZ shell variables.
# jq will ensure that the values are properly quoted
# and escaped for consumption by the shell.
eval "$(jq -r '#sh "FOO=\(.foo) BAZ=\(.baz)"')"
# Placeholder for whatever data-fetching logic your script implements
FOOBAZ="$FOO $BAZ"
# Safely produce a JSON object containing the result value.
# jq will ensure that the value is properly quoted
# and escaped to produce a valid JSON string.
jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'
We can see that it's reading the foo and baz keys from whatever is sent to the program via stdin.
If we ran this in the most simple way we can see that it does as we expect:
data "external" "example" {
program = ["sh", "${path.module}/example-data-source.sh"]
query = {
foo = "abc1234"
baz = "123abc"
}
}
output "example" {
value = data.external.example
}
Gives the following output:
Outputs:
example = {
"id" = "-"
"program" = tolist([
"sh",
"./example-data-source.sh",
])
"query" = tomap({
"baz" = "123abc"
"foo" = "abc1234"
})
"result" = tomap({
"foobaz" = "abc1234 123abc"
})
"working_dir" = tostring(null)
}
Just to check that there's nothing sneaky there that Terraform communicates over stdin we can modify the script to log out to a file what is passed on stdin (note that this will break the example script so that jq no longer sees the input from Terraform):
#!/bin/bash
# Exit if any of the intermediate steps fail
set -e
# Log stdin inputs to an output file so we only output the formatted
# JSON on stdout for the external data source to work correctly.
cat - >> log.out
# Extract "foo" and "baz" arguments from the input into
# FOO and BAZ shell variables.
# jq will ensure that the values are properly quoted
# and escaped for consumption by the shell.
eval "$(jq -r '#sh "FOO=\(.foo) BAZ=\(.baz)"')"
# Placeholder for whatever data-fetching logic your script implements
FOOBAZ="$FOO $BAZ"
# Safely produce a JSON object containing the result value.
# jq will ensure that the value is properly quoted
# and escaped to produce a valid JSON string.
jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'
And then if we run a plan we can see just the expected JSON in the log.out file:
{"baz":"123abc","foo":"abc1234"}
If you could access the operation type in your Terraform code instead then you could pass that in as a parameter to your external data source but that's also not exposed anywhere that you can access unfortunately.
It's also worth mentioning that as a data source, it's expected to be side effect free to run the script. This means it shouldn't really matter what operation Terraform is running when it invokes your data source. If you need to do something else that isn't side effect free then you might need to consider another option such as a provisioner, potentially attached to a null_resource but all of these options are meant to be last resorts where you need an escape hatch. Most of the time it's better to either raise a feature request in the appropriate provider or to call these things separately and orchestrate them together via a wrapper script or some orchestration software that calls them in succession.

Answer 2

Haven't tested this, but you should be able to get terraform's pid with os.getppid(). Then you can get more information with psutil (available in pip), /proc (linux only) or subprocessing a ps command. Finding the verb on the command line may be brittle, but depending on context, it can probably work.
That said, if you need to do this, you should reconsider how your code is organized. A pure data source shouldn't care.