I have 2 eks cluster as part of our upgrade. I want to handle assume policy such that it has access to both eks cluster. Both the cluster in same AWS account.
i want my policy to look like the below policy. such that the we are not updating any roles, but only the assume policy to handle both clusters.
locals.tf
eks_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::11111111111:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/yyyyyyyyyyyyyyyyyy",
"Federated": "arn:aws:iam::11111111111:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/yyyyyyyyyyyyyyyyyy"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/xxxxxxxxxxxxxx:sub": "system:serviceaccount:%s:%s",
"oidc.eks.us-east-1.amazonaws.com/id/xxxxxxxxxxxxx:sub": "system:serviceaccount:%s:%s"
}
}
}
]
}
EOF
Launcher = "job-Launcher"
Role.tf
resource "aws_iam_role" "launcher" {
name = local.Launcher
assume_role_policy = format(local.eks_policy, "my-namepsace", local.Launcher)
tags = {
terraform = "true"
owner = "stg"
}
}
So i tried like this in locals.tf
count = length(var.federated)
eks_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::11111111111:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/${join(",",${element(var.federated, count.index)})}",
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/${join(",", ${element(var.federated, count.index)})}:sub": "system:serviceaccount:%s:%s"
}
}
}
]
}
But i'm getting an error as count cannot be used within locals.tf,
Can someone pls help me.
Update2:
How do we get something like this
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/xxxxxxxxxxxxx:sub": "system:serviceaccount:ihr-system:ihr-system-external-dns18",
"oidc.eks.us-east-1.amazonaws.com/id/yyyyyyyyyyyyy:sub": "system:serviceaccount:ihr-system:ihr-system-external-dns"
}
}
I tried this ,
federated = [
"xxxxxxxxxxxxxxxxxxxxxx",
"yyyyyyyyyyyyyyyyyyyyyy"
]
Condition : {
"StringEquals" : {
join("",[for oidc in local.federated:"oidc.eks.us-east-1.amazonaws.com/id/${oidc}:sub:","system:serviceaccount:%s:%s"])
}
getting syntax error near in , local expected and another error got
',' or '}' expected got '"system:serviceaccount.."'
for oidc in local.federated
Terraform format function expects an argument per each placeholder. From the documentation:
The specification is a string that includes formatting verbs that are introduced with the % character. The function call must then have one additional argument for each verb sequence in the specification. The verbs are matched with consecutive arguments and formatted as directed, as long as each given argument is convertible to the type required by the format verb.
With that said, you need to provide four arguments, even though it's the same local variables in your case:
format(local.eks_policy, "my-namepsace", local.Launcher, "my-namepsace", local.Launcher)
Depending on your use case, you might also consider defining a list of objects with configuration and build the policy statement using loop in order to prepare final string.
Update 1
Example with dynamic generation might look like this, where role could be assumed by any account from the variable local.params:
locals {
# key = account ID, value could be whatever
params = {
"1111" = { foo = "bar" },
"2222" = { x = "y" }
}
assume_role_str = jsonencode({
# skipped beginning for brevity
Effect = "Allow",
Principal = {
Federated: [ for account in keys(local.params): "arn:aws:iam::11111111111:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/:${account}" ]
}
})
}
Related
THe problem I am facing now is this. I am trying to make my policy more flexible. So I shifted them into a file instead of using EOF.
How to make the template file recognise a number value?
"${max_untagged_images}" and "${max_tagged_images}" are suppose to be numbers.
Aws lifecycle policy:
resource "aws_ecr_lifecycle_policy" "lifecycle" {
count = length(aws_ecr_repository.repo)
repository = aws_ecr_repository.repo[count.index].name
depends_on = [aws_ecr_repository.repo]
policy = var.policy_type == "app" ? data.template_file.lifecycle_policy_app.rendered : data.template_file.lifecycle_policy_infra.rendered
}
Data template:
data "template_file" "lifecycle_policy_app" {
template = file("lifecyclePolicyApp.json")
vars = {
max_untagged_images = var.max_untagged_images
max_tagged_images = var.max_tagged_images
env = var.env
}
}
Policy:
{
"rules": [
{
"rulePriority": 1,
"description": "Expire untagged images older than ${max_untagged_images} days",
"selection": {
"tagStatus": "untagged",
"countType": "sinceImagePushed",
"countUnit": "days",
"countNumber": "${max_untagged_images}"
},
"action": {
"type": "expire"
}
},
{
"rulePriority": 2,
"description": "Expire tagged images of ${env}, older than ${max_tagged_images} days",
"selection": {
"tagStatus": "tagged",
"countType": "imageCountMoreThan",
"countNumber": "${max_tagged_images}",
"tagPrefixList": [
"${env}"
]
},
"action": {
"type": "expire"
}
}
]
}
I would try the following 2 steps:
Remove the double quotes that around the "${max_tagged_images}"
Use terraform function called tonumber in order to convert it to a number:
tonumber("1")
(Follow the official documentation: https://www.terraform.io/docs/configuration/functions/tonumber.html)
I have below terraform configuration for cognito client:
data "aws_cognito_user_pools" "re_user_pool" {
name = "${var.cognito_user_pool_name}"
}
resource "aws_cognito_user_pool_client" "app_client" {
name = "re-app-client"
user_pool_id = data.aws_cognito_user_pools.re_user_pool.id
depends_on = [data.aws_cognito_user_pools.re_user_pool]
explicit_auth_flows = ["USER_PASSWORD_AUTH"]
prevent_user_existence_errors = "ENABLED"
allowed_oauth_flows_user_pool_client = true
allowed_oauth_flows = ["code"]
allowed_oauth_scopes = ["phone", "openid", "email", "profile", "aws.cognito.signin.user.admin"]
supported_identity_providers = ["COGNITO", "Google"]
callback_urls = ["https://scnothzsf0.execute-api.ap-southeast-2.amazonaws.com/staging/signup"]
}
I references the cognito user pool which already exists on AWS. The error happens on the line user_pool_id = data.aws_cognito_user_pools.re_user_pool.id when it uses the user pool id in aws_cognito_user_pool_client.
I will get the error
Error: Error creating Cognito User Pool Client: InvalidParameterException: 1 validation error detected: Value 're-user' at 'userPoolId' failed to satisfy constraint: Member must satisfy regular expression pattern: [\w-]+_[0-9a-zA-Z]+
on infra/cognito.tf line 5, in resource "aws_cognito_user_pool_client" "app_client":
5: resource "aws_cognito_user_pool_client" "app_client" {`
It seems the format of the ID is not correct. I have read this document https://www.terraform.io/docs/providers/aws/d/cognito_user_pools.html and it has a reference attribute ids - The list of cognito user pool ids.. I wonder why it gives a list of user pool id. How can I reference this ID?
I also tried to reference it as user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0] but got an error:
Error: Invalid index
on infra/cognito.tf line 8, in resource "aws_cognito_user_pool_client" "app_client":
8: user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0]
This value does not have any indices.
The re_user_pool referenced above is defined here:
resource "aws_cognito_user_pool" "re_user_pool" {
name = "re-user"
}
I came across your question while working through this same problem. I see the question is several months old, but I'm still going to add an answer for anyone else that ends up here like I did.
First, the solution is to convert the ids value from a set to a list via the tolist function and then access it as you would any terraform list.
Caveat: In my case, I have ensured I only have one user pool for a given name, but you could get multiple user pools if you haven't followed this convention. This solution will not be a complete solution for that situation, but perhaps it will still point in the right direction.
Example code:
data "aws_cognito_user_pools" "test" {
name = "a_name"
}
output "test" {
value = "${tolist(data.aws_cognito_user_pools.test.ids)[0]"
}
Second, how I arrived at it:
I added an output block so I could see what I was working with and I commented out the problematic lines in my terraform file so I could successfully execute terraform apply. Next I ran terraform apply followed by terraform output --json (note: the apply must be successful for output to have the latest values).
Example temporary output block:
output "test" {
value = "${data.aws_cognito_user_pools.test}" // output top-level object for debugging
}
Relevant terraform apply output:
test = {
"arns" = [
"<redacted>",
]
"id" = "a_name"
"ids" = [
"us-east-1_<redacted>",
]
"name" = "a_name"
}
Relevant terraform output --json output:
"test": {
"sensitive": false,
"type": [
"object",
{
"arns": [
"set",
"string"
],
"id": "string",
"ids": [
"set",
"string"
],
"name": "string"
}
],
"value": {
"arns": [
"<redacted>"
],
"id": "a_name",
"ids": [
"us-east-1_<redacted>"
],
"name": "a_name"
}
}
As you can see, the ids portion is a set of type string. I decided to try converting ids to a list to see if I could then access the 0 index and it worked. I feel like this could be a terraform bug, but I haven't filed an issue yet.
I came across a terraform example code regarding template_file in the below link
https://blog.james-carr.org/using-templates-in-terraform-17bb8f4a0aac
policies/s3_bucket_readonly.json.tpl
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::${bucket_name}",
"arn:aws:s3:::${bucket_name}/${key_prefix}"
],
"Effect": "Allow"
}
]
}
Terraform script
data "template_file" "cloud-trail-logs-s3-readonly" {
template = "${file("policies/s3_bucket_readonly.json.tpl")}"
vars {
bucket_name = "${aws_s3_bucket.cloudtrail-logs.bucket}"
key_prefix = "AWSLogs/*"
}
}
resource "aws_s3_bucket" "cloudtrail-logs" {
bucket = "cloudtrail-logs"
acl = "private"
lifecycle_rule {
enabled = true
noncurrent_version_expiration {
days = 30
}
}
}
resource "aws_iam_policy" "cloudtrail-logs-readonly" {
name = "prod-cloudtrail-logs-s3-readonly"
path = "/production/"
description = "Readonly access to cloudtrail-logs bucket"
policy = "${data.template_file.cloud-trail-logs-s3-readonly.rendered}"
}
Can someone explains what the ${data.template_file.cloud-trail-logs-s3-readonly.rendered} actually does ?? Does it simply apply the template variable values to the policies/s3_bucket_readonly.json.tpl and add the same as policy ?? If yes, then what does ".rendered" mean or stand for ??
Yes you are right, even if your wording is inaccurate. It's the result of the template policies/s3_bucket_readonly.json.tpl after variables are applied.
Have a look at the docs (attributes reference) https://www.terraform.io/docs/providers/template/d/file.html#attributes-reference
rendered - The final rendered template.
The value of the policy will be the content of the template after terraform renders it.
After the HCL is interpreted (the terraform language is called HCL - Hashicorp Configuration Language), "${bucket_name}" will be equivalent to "${aws_s3_bucket.cloudtrail-logs.bucket}" as passed in the template resource in the vars block, and "${key_prefix}" to AWSLogs/*
I think "${aws_s3_bucket.cloudtrail-logs.bucket}" will refer to the string "cloudtrail-logs" (value of the bucket attribute in the aws_s3_bucket resource)
So the policy value should be:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::cloudtrail-logs",
"arn:aws:s3:::cloudtrail-logs/AWSLogs/*"
],
"Effect": "Allow"
}
]
}
I have a terraform script that provides a lambda function on aws to send emails. I pieced this terraform script from tutorials and templates on the web to use AWS SES, Api Gateway, Lambda and Cloudwatch services.
To get permissions to work though, I had to run the script and then, separately, build a policy in the AWS console and apply it to the lambda function so that it could fully access the SES and Cloudwatch services. But it's not at all not clear to me how to take that working policy and adapt it to my terraform script. Could anyone please provide or point to guidance on this matter?
The limited/inadequate but otherwise working role in my terraform script looks like this:
resource "aws_iam_role" "iam_for_lambda" {
name = "${var.role_name}"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Effect": "Allow",
"Sid": ""
}
]
} EOF
}
... and the working policy generated in the console (by combining two roles together for all-Cloudwatch and all-SES access):
{
"permissionsBoundary": {},
"roleName": "las_role_new",
"policies": [
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"autoscaling:Describe*",
"cloudwatch:*",
"logs:*",
"sns:*",
"iam:GetPolicy",
"iam:GetPolicyVersion",
"iam:GetRole"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/events.amazonaws.com/AWSServiceRoleForCloudWatchEvents*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "events.amazonaws.com"
}
}
}
]
},
"name": "CloudWatchFullAccess",
"id": "ANPAIKEABORKUXN6DEAZU",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/CloudWatchFullAccess"
},
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ses:*"
],
"Resource": "*"
}
]
},
"name": "AmazonSESFullAccess",
"id": "ANPAJ2P4NXCHAT7NDPNR4",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/AmazonSESFullAccess"
}
],
"trustedEntities": [
"lambda.amazonaws.com"
]
}
There are fields
So my question in summary, and put most generally, is this:
given a "policy" built in the aws console (by selecting a bunch of roles, etc. as in ), how do you convert that to a "role" as required for the terraform script?
To anyone else who might struggle to understand terraform-aws-policy matters, here's my understanding after some grappling. The game here is to carefully distinguish the various similar-sounding terraform structures (aws_iam_role, aws_iam_role_policy, aws_iam_role, assume_role_policy, etc.) and to work out how these black-box structures fit together.
First, the point of an aws role is to collect together policies (i.e. permissions to do stuff). By assigning such a role to a service (e.g. lambda), you thereby give that service the permissions described by those policies. A role must have at least one policy sort of built-in to it: the 'assume-role' policy that specifies which service(s) can use ('assume') that role. This assume-role policy is relatively simple and so 'might as well' be included in the terraform script explicitly (using the <<EOF ... EOF syntax above).
Secondly, if you want to now let that service with the (basic) role do anything to other services, then you need to somehow associate additional policies with that role. I've learned that there are several ways to do this but, in order to answer my question most succinctly, I'll now describe the most elegant way I have found to incorporate multiple template policies offered in the AWS console into one's terraform script.
The code is:
# Define variable for name of lambda function
variable "role_name" {
description = "Name for the Lambda role."
default = "las-role"
}
# Create role with basic policy enabling lambda service to use it
resource "aws_iam_role" "iam_for_lambda" {
name = "${var.role_name}"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": [ "lambda.amazonaws.com" ]
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
# Define a list of policy arn's given in the AWS console
variable "iam_policy_arn_list" {
type = list(string)
description = "IAM Policies to be attached to role"
default = ["arn:aws:iam::aws:policy/CloudWatchFullAccess", "arn:aws:iam::aws:policy/AmazonSESFullAccess"]
}
# Create attachment of the policies for the above arn's to our named role
# The count syntax has the effect of looping over the items in the list
resource "aws_iam_role_policy_attachment" "role-policy-attachment" {
role = var.role_name
count = length(var.iam_policy_arn_list)
policy_arn = var.iam_policy_arn_list[count.index]
depends_on = [aws_iam_role.iam_for_lambda]
}
As you can see, the template policies are included here using the arns which can be found in the AWS console. For example, here's the view for finding the arn for full access to Amazon SES via the AWS Management Console:
When you succesfully deploy your lambda to AWS using terraform, it will pull down these policies from the arns and generate a permission json for your lambda function (which you can view in the lambda-service section of the aws console) that looks a lot like the json I posted in the question.
In the terraform documentation we are provided with the markup as attached below. I cant find any mention of what the purpose of the field "b" is and how it should be used in general.
resource "aws_s3_bucket" "b" {
bucket = "my_tf_test_bucket"
}
resource "aws_s3_bucket_policy" "b" {
bucket = "${aws_s3_bucket.b.id}"
policy =<<POLICY
{
"Version": "2012-10-17",
"Id": "MYBUCKETPOLICY",
"Statement": [
{
"Sid": "IPAllow",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::my_tf_test_bucket/*",
"Condition": {
"IpAddress": {"aws:SourceIp": "8.8.8.8/32"}
}
}
]
}
POLICY
}
"b" is simply the NAME of the resource that you are creating.
Terraform resources are defined in "blocks", and each resource block creates a resource of the given TYPE (first parameter) and NAME (second parameter). The combination of the type and name must be unique.
So, in your example, you are creating a resource of TYPE aws_s3_bucket and NAME b.
Each resource defined has an id, that you can use to refer this resource in other resources, using a syntax like TYPE.NAME.id, for example ${aws_s3_bucket.b.id}
You can find further information in the doc here: https://www.terraform.io/docs/configuration/resources.html