Edit Fixed syntax for curl command
I am trying to use the available list of Googlebot IPs in my Terraform configuration.
https://developers.google.com/static/search/apis/ipranges/googlebot.json
Here is the error that I am seeing.
! terraform plan
data.external.get_googlebot_ips_using_shell: Reading...
╷
│ Error: Unexpected External Program Results
│
│ with data.external.get_googlebot_ips_using_shell,
│ on main.tf line 18, in data "external" "get_googlebot_ips_using_shell":
│ 18: program = ["bash","./get_googlebot_ips.sh"]
│
│ The data source received unexpected results after executing the program.
│
│ Program output must be a JSON encoded map of string keys and string values.
│
│ If the error is unclear, the output can be viewed by enabling Terraform's logging at TRACE level. Terraform documentation on logging: https://www.terraform.io/internals/debugging
│
│ Program: /usr/local/bin/bash
│ Result Error: invalid character '{' after top-level value
Here is my main.tf
data "external" "get_googlebot_ips_using_shell" {
program = ["bash","./get_googlebot_ips.sh"]
}
output "get_googlebot_ips_using_shell" {
value = data.external.get_googlebot_ips_using_shell.result.ipv4Prefix
}
Here is what get-googlebot-bot-ips.sh contains
curl https://developers.google.com/static/search/apis/ipranges/googlebot.json \
| jq '.prefixes[] | {ipv6Prefix,ipv4Prefix} | with_entries( select( .value != null ) )'
How should I format the returned JSON so that I can use this information in Terraform?
I have tried to format the data that I am returning in jq in different ways, but I cannot quite get the format correct for what Terraform is looking for.
This is an example that you can reproduce what you did on the shell script by transforming on HCL:
data "http" "example" {
url = "https://developers.google.com/static/search/apis/ipranges/googlebot.json"
# Optional request headers
request_headers = {
Accept = "application/json"
}
}
output "test" {
# value = jsondecode(data.http.example.response_body).prefixes
value = {
"ipv4Prefix" = compact([for i in jsondecode(data.http.example.response_body).prefixes : try(i.ipv4Prefix, "")])
"ipv6Prefix" = compact([for i in jsondecode(data.http.example.response_body).prefixes : try(i.ipv6Prefix, "")])
}
}
References:
http terraform provider
jsonencode terraform function
You might consider using the Terraform http data source. This will do the request for you and nicely place the JSON response into Terraform friendly data source. See this example:
data "http" "googlebot" {
url = "https://developers.google.com/static/search/apis/ipranges/googlebot.json"
request_headers = {
Accept = "application/json"
}
}
output googlebot_ip_prefixes { value = data.http.googlebot.body }
Related
Why i'm not using the ID directly:
I have multiple datalake's where the filesystem is deployed. It throws error "resource not found" during the deployment.
What i'm trying to achieve now:
i am trying to use concat function and create the ID's. which is throwing an error.
module.adlsfs["adlsfilesystem1"].time_sleep.wait_few_mins_fs: Refreshing state... [id=2022-07-23T21:45:55Z]
╷
│ Error: Invalid function argument
│
│ on ../../../tf-core-module/adls/fs/filesystem.tf line 20, in resource "azurerm_storage_data_lake_gen2_filesystem" "storagedlsgen2fs":
│ 20: storage_account_id = concat("/subscriptions/",data.azurerm_subscription.current.id,"/resourceGroups/rsg-test/providers/Microsoft.Storage/storageAccounts/",each.value.staname)
│
│ Invalid value for "seqs" parameter: all arguments must be lists or tuples; got string.
╵
╷
│ Error: Invalid function argument
│
│ on ../../../tf-core-module/adls/fs/filesystem.tf line 20, in resource "azurerm_storage_data_lake_gen2_filesystem" "storagedlsgen2fs":
│ 20: storage_account_id = concat("/subscriptions/",data.azurerm_subscription.current.id,"/resourceGroups/rsg-test/providers/Microsoft.Storage/storageAccounts/",each.value.staname)
│
│ Invalid value for "seqs" parameter: all arguments must be lists or tuples; got string.
data "azurerm_subscription" "current" {
}
locals {
staname = toset([
for pair in sort(var.sta_name) : {
staname = pair
}
])
}
//**********************************************************
// Create File System in Datalake
//**********************************************************
resource "azurerm_storage_data_lake_gen2_filesystem" "storagedlsgen2fs" {
for_each = { for p in local.staname : jsonencode(p) => p }
name = var.adlsfilesystems
storage_account_id = concat("/subscriptions/",data.azurerm_subscription.current.id,"/resourceGroups/resourcegroup/providers/Microsoft.Storage/storageAccounts/",each.value.staname)
}
Is it even possible to use the function here? and how can i solve this.
thank you
I think that instead of concat, you want join:
storage_account_id = join("",["/subscriptions/",data.azurerm_subscription.current.id,"/resourceGroups/resourcegroup/providers/Microsoft.Storage/storageAccounts/",each.value.staname])
I am using below experimental block module in terraform settings block to use optional function .
experiments = [module_variable_optional_attrs]
I want to make variable urlMaps which is list(object) as optional . When I dont give urlMaps as a input to terraform.tfvars , I am expecting it send a null vaule. But it is giving exception mentioned below. If its not possible, is there any alternative to achieve this.
terraform.tfvars
urlMaps = [
{
hosts = ["example.com"]
paths = ["/image"]
service = "backend-service"
}
]
Variables.tf
variable "urlMaps" {
description = "Netowork endpoint group region"
type = optional(list(object({
hosts = list(string)
paths = list(string)
service = string
})))
}
Exception
│ Error: Invalid type specification
│
│ on lb/variables.tf line 30, in variable "urlMaps":
│ 30: type = optional(list(object({
│
│ Keyword "optional" is valid only as a modifier for object type attributes.
There are two posibilities:
Add a default value to your urlMaps.
Make it a normal list(map), not list(object).
I'm trying to use for_each with a terraform module creating datadog synthetic tests. The object names in an s3 bucket are listed and passed as the set for the for_each. The module reads the content of each file using the each.value passed in by the calling module as the key. I hardcoded the s3 object key value in the module during testing and it was working. When I attempt to call the module from main.tf, passing in the key name dynamically from the set it fails with the below error.
│ Error: Error in function call
│
│ on modules\Synthetics\trial2.tf line 7, in locals:
│ 7: servicedef = jsondecode(data.aws_s3_object.tdjson.body)
│ ├────────────────
│ │ data.aws_s3_object.tdjson.body is ""
│
│ Call to function "jsondecode" failed: EOF.
main.tf
data "aws_s3_objects" "serviceList" {
bucket = "bucketname"
}
module "API_test" {
for_each = toset(data.aws_s3_objects.serviceList.keys)
source = "./modules/Synthetics"
S3key = each.value
}
module
data "aws_s3_object" "tdjson" {
bucket = "bucketname"
key = var.S3key
}
locals {
servicedef = jsondecode(data.aws_s3_object.tdjson.body)
Keys = [for k,v in local.servicedef.Endpoints: k]
}
Any clues as to what's wrong here?
Thanks
Check out the note on the aws_s3_object data source:
The content of an object (body field) is available only for objects which have a human-readable Content-Type (text/* and application/json). This is to prevent printing unsafe characters and potentially downloading large amount of data which would be thrown away in favour of metadata.
Since it's successfully getting the data source (not throwing an error), but the body is empty, this is very likely to be your issue. Make sure that your S3 object has the Content-Type metadata set to application/json. Here's a Stack Overflow question/answer on how to do that via the CLI; you can also do it via the AWS console, API, or Terraform (if you created the object via Terraform).
EDIT: I found the other issue. Check out the syntax for using for_each with toset:
resource "aws_iam_user" "the-accounts" {
for_each = toset( ["Todd", "James", "Alice", "Dottie"] )
name = each.key
}
The important bit is that you should be using each.key instead of each.value.
How to control flow when resource for_each need to fetch data from data source which is having count
Its my requirement to read data from data source only so I have build the code like it
I am getting problem in below line:
command = "echo ${data.null_data_source.values.*.outputs}"
I am getting error as :
╷ │ Error: Invalid template interpolation value │ │ on
foreachloop.tf line 79, in resource "null_resource"
"null_resource_simple": │ 79: command = "echo
${data.null_data_source.values.*.outputs}" │ ├──────────────── │
│ data.null_data_source.values is tuple with 2 elements │ │ Cannot
include the given value in a string template: string required.
│ Error: Invalid template interpolation value │ │ on foreachloop.tf
line 79, in resource "null_resource" "null_resource_simple": │ 79:
command = "echo ${data.null_data_source.values.*.outputs}" │
├──────────────── │ │ data.null_data_source.values is tuple with 2
elements │ │ Cannot include the given value in a string template:
string required.
Sample code is as below:
variable "create_access_only" {
default = true
//default = false
}
locals {
test1 = "new value db"
test2 = "new value web"
string_values = "${local.test1},${local.test2}"
list_test_data = "${split(",", local.string_values)}"
}
output "testlist" {
value = local.list_test_data
}
variable "data_source_key" {
default = "test"
}
data "null_data_source" "values" {
count = "${length(local.list_test_data)}"
inputs = {
data_source_key = "${local.list_test_data[count.index]}"
}
}
resource "null_resource" "null_resource_simple" {
for_each = var.create_access_only ? toset(local.list_test_data) : []
provisioner "local-exec" {
command = "echo ${data.null_data_source.values.*.outputs}"
// command = "echo ${data.null_data_source.values}"
// command = "echo ${each.key} ${each.value.name1} ${each.value.name2}"
// command = "echo ${each.key}"
// command = "echo ${data.null_data_source.values["${each.key}"]}"
// command = "echo ${data.null_data_source.values["${count.index}"]}"
// command = "echo ${local.list_test_data}"
}
}
I need to get the data from the datasource response actually.
My requirement is that datasource's 1st value will reflected in resource first loop and so on
Also, I need to get the id from the response of the datasource , I know I am using null_data_source here but my actual script datasource will give me id and other stuff in response
example : data.datasourcename.varaiablename.namespace[0].id
One way to fix this issue is by using the for_each with the null_data_source as well. However, when running the code, here is one warning:
╷
│ Warning: Deprecated Resource
│
│ with data.null_data_source.values,
│ on data.tf line 105, in data "null_data_source" "values":
│ 105: data "null_data_source" "values" {
│
│ The null_data_source was historically used to construct intermediate values to re-use elsewhere in configuration, the same can now be achieved using locals
│
│ (and 2 more similar warnings elsewhere)
╵
As you can see in the output, the same thing can be achieved with local variables. Here is the code that works:
resource "null_resource" "null_resource_simple" {
for_each = var.create_access_only ? toset(local.list_test_data) : []
provisioner "local-exec" {
command = "echo ${data.null_data_source.values[each.key].outputs["data_source_key"]}"
}
}
data "null_data_source" "values" {
for_each = toset(local.list_test_data)
inputs = {
data_source_key = each.key
}
}
variable "create_access_only" {
default = true
//default = false
}
variable "data_source_key" {
default = "test"
}
output "testlist" {
value = local.list_test_data
}
Note that when using toset the each.key and each.value will be the same [1].
Here is the output of the terraform plan command:
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# null_resource.null_resource_simple["new value db"] will be created
+ resource "null_resource" "null_resource_simple" {
+ id = (known after apply)
}
# null_resource.null_resource_simple["new value web"] will be created
+ resource "null_resource" "null_resource_simple" {
+ id = (known after apply)
}
Plan: 2 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ testlist = [
+ "new value db",
+ "new value web",
]
[1] https://www.terraform.io/language/meta-arguments/for_each#the-each-object
Hello I am trying to create a azuread_application_password for azuread_application to use it for authentication during backend configuration.
resource "azuread_application_password" "application_password" {
application_object_id = azuread_application.app-tf.object_id
end_date = timeadd(timestamp(), "720h")
}
output "client_secret" {
description = "Client Secret"
value = azuread_application_password.application_password.value
}
Since I am doing the provisioning through terraform, I need to see the application_password or client_secret after creation so I can use that value.
│ Error: Output refers to sensitive values
│
│ on main.tf line 47:
│ 47: output "client_secret" {
│
│ To reduce the risk of accidentally exporting sensitive data that was intended to be only internal, Terraform requires
│ that any root module output containing sensitive data be explicitly marked as sensitive, to confirm your intent.
│
│ If you do intend to export this data, annotate the output value as sensitive by adding the following argument:
│ sensitive = true
I understand this might now be safest, but I believe that is only way to create and retrieve client_secret while using terraform, so how can I work around this error and get the value?
Use nonsensitive function to disable masking:
output "client_secret" {
description = "Client Secret"
value = nonsensitive(azuread_application_password.application_password.value)
}