I'm using Terraform v0.14.4 with GCP. I have a Cloud Run service that won't be managed with Terraform (it might exist or not), and I want to read its url.
If the service exists this works ok:
data "google_cloud_run_service" "myservice" {
name = "myservice"
location = "us-central1"
}
output "myservice" {
value = data.google_cloud_run_service.myservice.status[0].url
}
But if it doesn't exist, I can't get it to work!. What I've tried:
data.google_cloud_run_service.myservice.*.status[*].url
status is null
length(data.google_cloud_run_service.myservice) > 0 ? data.google_cloud_run_service.myservice.*.status[0].url : ""
Tried with join("", data.google_cloud_run_service.myservice.*.status)
I get this error: data.google_cloud_run_service.myservice is object with 9 attributes
coalescelist(data.google_cloud_run_service.myservice.*.status, <...>)
It just returns [null], and using compact over the result gets me a Invalid value for "list" parameter: element 0: string required.
Any ideas?
It seems like you are working against the design of this data source a little here, but based on the error messages you've shown it seems like the behavior is that status is null when the requested object doesn't exist and is a list when it does, and so you'll need to write an expression that can deal with both situations.
Here's my attempt, based only on the documentation of the resource along with some assumptions I'm making based on the error message you included:
output "myservice" {
value = (
data.google_cloud_run_service.myservice.status != null ?
data.google_cloud_run_service.myservice.status[0].url :
null
)
}
There is another potentially-shorter way to write that, relying on the try function's ability to catch dynamic errors and return a fallback value, although this does go against the recommendations in the documentation in that it forces an unfamiliar future reader to do a bit more guesswork to understand in which situations the expression might succeed and which it might return the fallback:
output "myservice" {
value = try(data.google_cloud_run_service.myservice.status[0].url, null)
}
Related
I have the following structure
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
policy = each.value
}
module "groups" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.groups
name = each.key
type = each.value.type
policies = each.value.policies
depends_on = [
module.policies
]
}
Policies and groups are declared in a yaml file from which through yamldecode the corresponding variables to for_each are created.
Is there any way to make sure that the policies passed to policies = each.value.policies of the groups module DO exist?
I mean, OK I have the depends_on clause, but I want to also provision for typos in the yaml file and other similar situations.
The usual way to declare a dependency on an external object (managed elsewhere) in Terraform is to use a data block using a data source defined by the provider responsible for that object. If the goal is only to verify that the object exists then it's enough to declare the data source and then have your downstream object's configuration refer to anything about its result, just so Terraform can see that the data source is a dependency and so should be resolved first.
Unfortunately it seems like the hashicorp/vault provider doesn't currently have a data source for declaring a dependency on a policy, although there is a feature request for it.
Assuming that it did exist then the pattern might look something like this:
data "vault_policy" "needed" {
for_each = var.config.policies
name = each.value
}
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
# Accessing this indirectly via the data resource tells
# Terraform that it must complete the data lookup before
# planning anything which depends on this "policy" argument.
policy = data.vault_policy.needed[each.key].name
}
Without a data source for this particular object type I don't think there will be an elegant way to solve this, but you may be able to work around it by using a more general data source like hashicorp/external's external data source for collecting data by running an external program that prints JSON.
Again because you don't actually seem to need any specific data from the policy and only want to check whether it exists, it would be sufficient to write an external program which queries vault and then exists with an unsuccessful status if the request fails, or prints an empty JSON object {} if the request succeeds.
data "external" "vault_policy" {
for_each = var.config.policies
program = ["${path.module}/query-vault"]
query = {
policy_name = each.value
}
}
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
policy = data.external.vault_policy.query.policy_name
}
I'm not familiar enough with Vault to suggest a specific implementation of this query-vault program, but you may be able to use a shell script wrapping the vault CLI program if you follow the advice in Processing JSON in shell scripts. You only need to do the input parsing part of that, because your result would be communicated either by exit 1 to signal failure or echo '{}' followed by exiting successfully to signal success.
I am deploying an App Gateway on Azure using a Bicep template (relevant pieces shown below).
var applicationGatewayId = resourceId('Microsoft.Network/applicationGateways', applicationGatewayName)
resource applicationGateway 'Microsoft.Network/applicationGateways#2021-08-01' = {
properties: {
urlPathMaps: [
{
properties: {
defaultBackendAddressPool: {
id: '${applicationGatewayId}/backendAddressPools/backendpool-test'
}
}
]
}
}
My question is about the id of the backendAddressPool in the example. I get a warning when compiling: Warning use-resource-id-functions: If property "id" represents a resource ID, it must use a symbolic resource reference, be a parameter or start with one of these functions: extensionResourceId, guid, if, reference, resourceId, subscription, subscriptionResourceId, tenantResourceId. [https://aka.ms/bicep/linter/use-resource-id-functions].
I tried using ${applicationGateway.id}/backendAddressPools/backendpool-test, but that results in a cyclic reference error. For other resource types I used resourceId(), but for this example I wouldn't know how to.
I tried, e.g., resourceId('Microsoft.Network/ApplicationGatewayBackendAddressPool', '${prefix}-backendpool-infocat'), but that seems to result in a different resource type altogether (doesn't compile into the same id, at least).
This question is applicable to other subresources too, such as:
applicationGateway.urlPathMaps.defaultBackendAddressPool
applicationGateway.urlPathMaps.pathRules.backendAddressPool
applicationGateway.urlPathMaps.pathRules.backendHttpSettings
...
So how does one refer to these subresources properly, when there's no readily defined resourceType to be used in resourceId()? Can the warnings be avoided?
Thanks in advance!
Try this:
id:resourceId('Microsoft.Network/applicationGateways/backendAddressPools', applicationGatewayName, applicationGatewayBackendAddressPoolName)
I want to install a CRD with terraform, I was hoping it would be easy as doing this:
data "http" "crd" {
url = "https://raw.githubusercontent.com/kubernetes-sigs/application/master/deploy/kube-app-manager-aio.yaml"
request_headers = {
Accept = "text/plain"
}
}
resource "kubernetes_manifest" "install-crd" {
manifest = data.http.crd.body
}
But I get this error:
can't unmarshal tftypes.String into *map[string]tftypes.Value, expected
map[string]tftypes.Value
Trying to convert it to yaml with yamldecode also doesn't work because yamldecode doesn't support multi-doc yaml files.
I could use exec, but I was already doing that while waiting for the kubernetes_manifest resource to be released. Does kubernetes_manifest only support a single resource or can it be used to create several from a raw text manifest file?
kubernetes_manifest (emphasis mine)
Represents one Kubernetes resource by supplying a manifest attribute
That sounds to me like it does not support multiple resources / a multi doc yaml file.
However you can manually split the incoming document and yamldecode the parts of it:
locals {
yamls = [for data in split("---", data.http.crd.body): yamldecode(data)]
}
resource "kubernetes_manifest" "install-crd" {
count = length(local.yamls)
manifest = local.yamls[count.index]
}
Unfortunately on my machine this then complains about
'status' attribute key is not allowed in manifest configuration
for exactly one of the 11 manifests.
And since I have no clue of kubernetes I have no idea what that means or wether or not it needs fixing.
Alternatively you can always use a null_resource with a script that fetches the yaml document and uses bash tools or python or whatever is installed to convert and split and filter the incoming yaml.
I got this to work using kubectl provider. Eventually kubernetes_manifest should work as well, but it is currently (v2.5.0) still beta and has some bugs. This example only uses kind+name, but for full uniqueness, it should also include the API and the namespace params.
resource "kubectl_manifest" "cdr" {
# Create a map { "kind--name" => yaml_doc } from the multi-document yaml text.
# Each element is a separate kubernetes resource.
# Must use \n---\n to avoid splitting on strings and comments containing "---".
# YAML allows "---" to be the first and last line of a file, so make sure
# raw yaml begins and ends with a newline.
# The "---" can be followed by spaces, so need to remove those too.
# Skip blocks that are empty or comments-only in case yaml began with a comment before "---".
for_each = {
for pair in [
for yaml in split(
"\n---\n",
"\n${replace(data.http.crd.body, "/(?m)^---[[:blank:]]*(#.*)?$/", "---")}\n"
) :
[yamldecode(yaml), yaml]
if trimspace(replace(yaml, "/(?m)(^[[:blank:]]*(#.*)?$)+/", "")) != ""
] : "${pair.0["kind"]}--${pair.0["metadata"]["name"]}" => pair.1
}
yaml_body = each.value
}
Once Hashicorp fixes kubernetes_manifest, I would recommend using the same approach. Do not use count+element() because if the ordering of the elements change, Terraform will delete/recreate many resources without needed it.
resource "kubernetes_manifest" "crd" {
for_each = {
for value in [
for yaml in split(
"\n---\n",
"\n${replace(data.http.crd.body, "/(?m)^---[[:blank:]]*(#.*)?$/", "---")}\n"
) :
yamldecode(yaml)
if trimspace(replace(yaml, "/(?m)(^[[:blank:]]*(#.*)?$)+/", "")) != ""
] : "${value["kind"]}--${value["metadata"]["name"]}" => value
}
manifest = each.value
}
P.S. Please support Terraform feature request for multi-document yamldecode. Will make things far easier than the above regex.
Terraform can split a multi-resource yaml (---) for you (docs):
# fetch a raw multi-resource yaml
data "http" "knative_serving_crds" {
url = "https://github.com/knative/serving/releases/download/knative-v1.7.1/serving-crds.yaml"
}
# split raw yaml into individual resources
data "kubectl_file_documents" "knative_serving_crds" {
content = data.http.knative_serving_crds.body
}
# apply each resource from the yaml one by one
resource "kubectl_manifest" "knative_serving_crds" {
depends_on = [kops_cluster_updater.updater]
for_each = data.kubectl_file_documents.knative_serving_crds.manifests
yaml_body = each.value
}
Is it possible to somehow create an arbitrary attribute from a variable? Here is what I am trying to achieve.
How I currently do it (now deprecated in 3.0.0):
resource "aws_lb_listener_rule" "example" {
condition {
field = var.condition_field
values = var.condition_values
}
}
The new syntax requires a nested block with the condition field. But my condition is stored in a variable:
resource "aws_lb_listener_rule" "example" {
condition {
var.condition_field {
values = var.condition_values
}
}
}
Is it possible to somehow create an arbitrary attribute from a variable?
or: Can I store a nested attribute block in a variable?
Background on my question: I am currently trying to upgrade from 2.70.0 to 3.0.0 and there are quite a few breaking changes in my system. One of them includes the aws_lb_listener_rule. If it is not possible to create the attribute from the variable I would have to either pin the version or change the module API used by a ton of projects.
It actually seems like it is not possible to do that. The closes thing I have found that allows me to use 3.0.0 without changing my module variables and with that all the Terraform scripts that use it are dynamic conditional blocks.
dynamic "condition" {
for_each = var.field == "path-pattern" ? [var.field] : []
content {
path_pattern {
values = var.patterns
}
}
}
This is repeated for all possible var.field values.
variable "cidr" {
type = map(string)
default = {
development = "x.1.0.0/16"
qa = "x.1.0.0/16"
default = "x.1.0.0/16"
}
}
variable "network_address_space" {
default = lookup(var.cidr, var.environment_name,"default")
}
Am getting error that "Error: Function calls not allowed"
variable "subnet_address_space": cidr_subnet2_address_space = cidrsubnet(var.network_address_space,8,1)
A Terraform Input Variable is analogous to a function argument in a general-purpose programming language: its value comes from an expression in the calling module, not from the current module.
The default mechanism allows us to substitute a value for when the caller doesn't specify one, but because variables are intended for getting data into a module from the outside, it doesn't make sense to set the default to something from inside that module: that would cause the result to potentially be something the caller of the module could never actually specify, because they don't have access to the necessary data.
Terraform has another concept Local Values which are roughly analogous to a local variable within a function in a general-purpose programming language. These can draw from function results and other objects in the current module to produce their value, and so we can use input variables and local values together to provide fallback behaviors like you've illustrated in your question:
var "environment_name" {
type = string
}
var "environment_default_cidr_blocks" {
type = map(string)
default = {
development = "10.1.0.0/16"
qa = "10.2.0.0/16"
}
}
var "override_network_range" {
type = string
default = null # If not set by caller, will be null
}
locals {
subnet_cidr_block = (
var.override_network_range != null ?
var.override_network_range :
var.environment_default_cidr_blocks[var.environment_name]
)
}
Elsewhere in the module you can use local.subnet_cidr_block to refer to the final CIDR block selection, regardless of whether it was set explicitly by the caller or by lookup into the table of defaults.
When a module uses computation to make a decision like this, it is sometimes useful for the module to export its result as an Output Value so that the calling module can make use of it too, similar to how Terraform resources also export additional attributes recording decisions made by the provider or by the remote API:
output "subnet_cidr_block" {
value = local.subnet_cidr_block
}
As stated in Interpolate variables inside .tfvars to define another variable by the Hashicorp person, it is intended to be constant by design.
Input variables are constant values passed into the root module, and so they cannot contain interpolations or other expressions that do not yield a constant value.
We cannot use variables in backend either as in Using variables in terraform backend config block.
These are the things we Terraform users tripped on at some point, I suppose.