I was trying to create a dynamic cloud-config files using templatefile(file, vars). In my template I invoke the jsonencode(var) function in order to render valid yaml. As a mre :
#cloud-init
${jsonencode({
"groups": [ for g in groups :
{
g.name
}
],
"users": [ for u in users :
{
"name": u.name,
"gecos": u.gecos,
"homedir": u.homedir,
}
})}
locals {
json-user-data = templatefile(
"${path.module}/templates/cloud-init.tpl",
{
groups = var.groups,
users = var.users
}
}
What I'd like to obtain is something like :
groups:
- ubuntu: [root,sys]
- cloud-users
Where the groups member attribute is optional. I'm having a hard time achiving the result also because Terraform does not like mixed objects. Is there a way to achive something similar?
Thanks for helping, and let me know how to improve my question.
Related
I have multiple logstash pipelines set up on server that feeds data in Index. Every pipelines adds bunch of fields at the first level of Index along with their nested level.
I already have kpi1 and kpi2 values inside Metrics => data with Metrics being nested array. And I have a requirement to add a new pipeline that will feed the value of kpi3. Here is my filter section in the new pipeline that I created:
filter {
ruby {
code => "
event.set('kpi3', event.get('scoreinvitation'))
"
}
mutate {
# Rename the properties according to the document schema.
rename => {"kpi3" => "[metrics][data][kpi3]"}
}
}
It overwrites the Metrics section ( may be because it is an array??). Here is my mapping :
"metrics" : {
"type" : "nested",
"properties" : {
"data" : {
"properties" : {
"kpi1" : {
....
}
}
}
"name" : {
"type" : "text",
....
}
}
}
How can I keep the existing fields (and values) and still add the new fields inside Metrics => Data ? Any help is appeciated.
The Logstash pipeline looks good, however your mapping doesn't make much sense to me if I'm understanding your requirement correctly.
The metrics property doesn't have to be of type nested. In fact, the metrics property is just a json namespace that contains sub-fields / -objects.
Try the following mapping instead
"metrics": {
"properties": {
"data": {
"properties": {
"kpi1": {
# if you want to assign a value to the kpi1 field, it must have a type
}
}
},
"name": {
"type": "text"
}
}
}
I am trying to create a predefined set of IAM roles.
locals {
default_iam_roles = {
group1 = {
name = "group:group1-group#mydomain.com"
roles = toset([
"roles/viewer"
])
}
group2 = {
name = "group:group2-group#mydomain.com"
roles = toset([
"roles/owner"
])
}
}
formatted_iam_roles = [ for member in local.default_iam_roles : setproduct([member.name], member.roles) ]
}
If I print local.formatted_iam_roles I get the following:
[
toset([
[
"group:group1-group#mydomain.com",
"roles/viewer",
],
]),
toset([
[
"group:group2-group#mydomain.com",
"roles/owner",
],
]),
]
Now I want to create a single set which contains all of the combinations contained in the list, so that I can feed it to a resource on a for_each statement, but I am not able to find the logic for this.
The expected output would be:
toset([
[
"group:group1-group#mydomain.com",
"roles/viewer",
],
[
"group:group2-group#mydomain.com",
"roles/owner",
]
])
The operation of taking multiple sets and deriving a new set which contains all of the elements across all of the input sets is called union and so Terraform's function for it is called setunion to follow that.
locals {
all_iam_roles = setunion(local.formatted_iam_roles...)
}
The ... symbol after the argument tells Terraform that it should use each element of local.formatted_iam_roles as a separate argument to setunion, because that function is defined as taking an arbitrary number of arguments that are all sets, rather than as taking a list of sets.
It might be helpful to think of setunion as being a similar sort of function as concat. The concat function joins multiple lists together, preserving each list's element order and the order the lists are given. Set elements don't have any particular order and contain each distinct value only once, and so the behavior of setunion is different but its purpose is related.
Your solution in the question was close, you just have to apply a flatten function to the setproduct output.
locals {
# ...
formatted_iam_roles = [for member in local.default_iam_roles : flatten(setproduct([member.name], member.roles))]
}
The output will be the following:
formatted_iam_roles = [
[
"group:group1-group#mydomain.com",
"roles/viewer",
],
[
"group:group2-group#mydomain.com",
"roles/owner",
],
]
Now, if you really want it to make a set for the final result, you can use toset, most like it will be pointless:
formatted_iam_roles = toset([for member in local.default_iam_roles : flatten(setproduct([member.name], member.roles))])
I finally faced the problem using maps instead of lists, because I needed to use a for_each argument and it doesn't accept a set of tuples as value.
If anyone knows a better approach, post it and I will be pleased of marking it as correct.
locals {
formatted_iam_roles = merge([ for member in local.default_iam_roles :
{ for pair in setproduct([member.name], member.roles) :
"${pair[0]}${pair[1]}" => { "name": pair[0], "role": pair[1] }
}
]...)
}
resource "google_project_iam_member" "team_access" {
for_each = local.formatted_iam_roles
project = var.project_id
member = each.value["name"]
role = each.value["role"]
}
I am trying to dynamically create a list of objects within a Terraform module so I dont need to hard code unnecessary repeated values. I found a module on the Terraform Registry that is the basis of what I am doing. The module is located at https://github.com/cloudposse/terraform-aws-sso. In the examples/complete/main.tf in module "sso_account_assignments", they duplicate the AdministratorAccess permission set for different AWS accounts. My problem is I have nearly 30 accounts where I want to assign the same permission set but I dont want to duplicate entries in the code with just the account number being different. I am experienced with Python and the way I would write it with Python would be something like the following:
If I Were to Write It In Python
account_list = ['11111111111', '22222222222', '33333333333']
account_assignments = []
for acct in account_list:
obj = {
"account": acct,
"permission_set_arn": "Some value......",
"permission_set_name": "AdministratorAccess",
"principal_type": "GROUP",
"principal_name": "Administrators"
}
account_assignments.append(obj)
print(account_assignments)
Output
[
{
"account":"11111111111",
"permission_set_arn":"Some value......",
"permission_set_name":"AdministratorAccess",
"principal_type":"GROUP",
"principal_name":"Administrators"
},
{
"account":"22222222222",
"permission_set_arn":"Some value......",
"permission_set_name":"AdministratorAccess",
"principal_type":"GROUP",
"principal_name":"Administrators"
},
{
"account":"33333333333",
"permission_set_arn":"Some value......",
"permission_set_name":"AdministratorAccess",
"principal_type":"GROUP",
"principal_name":"Administrators"
}
]
Basically I am having trouble figuring out how to dynamically build the list of objects in Terraform. I am sure it can be solved with a for_each or for loop but not figuring it out. Hopefully this makes sense.
Tried writing the code but it is not working and is erroring. I looked at HashiCorp's documentation but no luck.
You can accomplish this with a simple for loop:
variable "account_list" {
default = ["11111111111", "22222222222", "33333333333"]
}
locals {
account_assignments = [for account_id in var.account_list : {
"account" : account_id,
"permission_set_arn" : "Some value......",
"permission_set_name" : "AdministratorAccess",
"principal_type" : "GROUP",
"principal_name" : "Administrators"
}]
}
output "account_assignments" {
value = local.account_assignments
}
I'm working on an tf plan what builds a json template and out of a map variable and I'm not quite sure how to use the existing looping, type, list functions to do the work. I know that I cannot pass lists or map to a data "template_file" so my thought was to build the string in a locals or null resource block and then pass that to the template
Variable
variable "boostrap_servers" {
type = map
default = {
"env01" : [
"k01.env01",
"k02.env01"
],
"env02" : [
"k01.env02"
]
}
Desired text
"connections": {
"env01": {
"properties": {
"bootstrap.servers": "k01.env01,k02.env01"
}
},
"env02": {
"properties": {
"bootstrap.servers": "k01.env02"
}
},
You may simply use the jsonencode function and list comprehension for this:
locals {
connections = jsonencode({
for cluster, servers in local.bootstrap_servers :
cluster => {
properties = {
"bootstrap.servers" = join(",", servers)
}
}
})
}
Ok, so the following works but there's a better question: why not just use the jsonencode function to build the json
locals {
clusters = [
for cluster, servers in var.boostrap_servers :
"{\"${cluster}\":{\"properties\":{\"bootstrap.servers\":\"${join(" ,", servers)}\"}}"]
connections = join(",", local.clusters)
}
I have below terraform configuration for cognito client:
data "aws_cognito_user_pools" "re_user_pool" {
name = "${var.cognito_user_pool_name}"
}
resource "aws_cognito_user_pool_client" "app_client" {
name = "re-app-client"
user_pool_id = data.aws_cognito_user_pools.re_user_pool.id
depends_on = [data.aws_cognito_user_pools.re_user_pool]
explicit_auth_flows = ["USER_PASSWORD_AUTH"]
prevent_user_existence_errors = "ENABLED"
allowed_oauth_flows_user_pool_client = true
allowed_oauth_flows = ["code"]
allowed_oauth_scopes = ["phone", "openid", "email", "profile", "aws.cognito.signin.user.admin"]
supported_identity_providers = ["COGNITO", "Google"]
callback_urls = ["https://scnothzsf0.execute-api.ap-southeast-2.amazonaws.com/staging/signup"]
}
I references the cognito user pool which already exists on AWS. The error happens on the line user_pool_id = data.aws_cognito_user_pools.re_user_pool.id when it uses the user pool id in aws_cognito_user_pool_client.
I will get the error
Error: Error creating Cognito User Pool Client: InvalidParameterException: 1 validation error detected: Value 're-user' at 'userPoolId' failed to satisfy constraint: Member must satisfy regular expression pattern: [\w-]+_[0-9a-zA-Z]+
on infra/cognito.tf line 5, in resource "aws_cognito_user_pool_client" "app_client":
5: resource "aws_cognito_user_pool_client" "app_client" {`
It seems the format of the ID is not correct. I have read this document https://www.terraform.io/docs/providers/aws/d/cognito_user_pools.html and it has a reference attribute ids - The list of cognito user pool ids.. I wonder why it gives a list of user pool id. How can I reference this ID?
I also tried to reference it as user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0] but got an error:
Error: Invalid index
on infra/cognito.tf line 8, in resource "aws_cognito_user_pool_client" "app_client":
8: user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0]
This value does not have any indices.
The re_user_pool referenced above is defined here:
resource "aws_cognito_user_pool" "re_user_pool" {
name = "re-user"
}
I came across your question while working through this same problem. I see the question is several months old, but I'm still going to add an answer for anyone else that ends up here like I did.
First, the solution is to convert the ids value from a set to a list via the tolist function and then access it as you would any terraform list.
Caveat: In my case, I have ensured I only have one user pool for a given name, but you could get multiple user pools if you haven't followed this convention. This solution will not be a complete solution for that situation, but perhaps it will still point in the right direction.
Example code:
data "aws_cognito_user_pools" "test" {
name = "a_name"
}
output "test" {
value = "${tolist(data.aws_cognito_user_pools.test.ids)[0]"
}
Second, how I arrived at it:
I added an output block so I could see what I was working with and I commented out the problematic lines in my terraform file so I could successfully execute terraform apply. Next I ran terraform apply followed by terraform output --json (note: the apply must be successful for output to have the latest values).
Example temporary output block:
output "test" {
value = "${data.aws_cognito_user_pools.test}" // output top-level object for debugging
}
Relevant terraform apply output:
test = {
"arns" = [
"<redacted>",
]
"id" = "a_name"
"ids" = [
"us-east-1_<redacted>",
]
"name" = "a_name"
}
Relevant terraform output --json output:
"test": {
"sensitive": false,
"type": [
"object",
{
"arns": [
"set",
"string"
],
"id": "string",
"ids": [
"set",
"string"
],
"name": "string"
}
],
"value": {
"arns": [
"<redacted>"
],
"id": "a_name",
"ids": [
"us-east-1_<redacted>"
],
"name": "a_name"
}
}
As you can see, the ids portion is a set of type string. I decided to try converting ids to a list to see if I could then access the 0 index and it worked. I feel like this could be a terraform bug, but I haven't filed an issue yet.