Original Attempt
data "aws_iam_policy_document" "lambda_read_secrets" {
statement {
actions = [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
]
effect = "Allow"
resources = [
"${var.enable_test_users == true ? aws_secretsmanager_secret.test_user[0].arn : ""}",
"${var.enable_prod_users == true ? aws_secretsmanager_secret.prod_user[0].arn : ""}"
]
}
statement {
effect = "Allow"
actions = ["secretsmanager:ListSecrets"]
resources = ["*"]
}
}
The issue is that this runs into
Error: error creating IAM policy test-lambda-logging20211011172058509500000003: MalformedPolicyDocument: Resource must be in ARN format or "*".
status code: 400, request id: c5c62446-eba7-450d-b97d-505be530ba2d
on ../../../../module/lambda/iam.tf line 58, in resource "aws_iam_policy" "lambda_read_secrets":
58: resource "aws_iam_policy" "lambda_read_secrets" {
because of the empty string.
Current Solution
Create a data "aws_iam_policy_document" "dev_lambda_read_secrets" and data "aws_iam_policy_document" "prod-lambda_read_secrets" and do if statements on which environment we're deploying to.
My primary issue with this solution is that it requires me to essentially double declare the same policy with a tweaked set of resources. I would love to just be able to have a single policy declaration with only the resources changing.
Terraform has the compact function. This lets us declare
data "aws_iam_policy_document" "lambda_read_secrets" {
statement {
actions = [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
]
effect = "Allow"
resources = compact([
"${var.enable_test_users == true ? aws_secretsmanager_secret.test_user[0].arn : ""}",
"${var.enable_prod_users == true ? aws_secretsmanager_secret.prod_user[0].arn : ""}"
])
}
statement {
effect = "Allow"
actions = ["secretsmanager:ListSecrets"]
resources = ["*"]
}
}
Related
I am trying to leverage precondition hook to check for an input to a module creation in terraform.
module "groups" {
source = "../path/to/groups"
for_each = var.groups.groups
name = each.key
type = each.value.type
policies = each.value.policies
depends_on = [
module.policies
]
lifecycle {
precondition {
condition = alltrue([ for item in self.policies :
alltrue([ for p in item : contains(locals.policies_list, p) ]) ] )
error_message = format("Attempt to create a group with a non existing policy")
}
}
}
However, although:
terraform --version --json
{
"terraform_version": "1.3.7",
"platform": "linux_amd64",
"provider_selections": {},
"terraform_outdated": false
}
This fails:
The block type name "lifecycle" is reserved for use by Terraform in a future version.
Is this because the specific functionality is not available in terraform for module creation? Is there a way around making my module creation fail based on the above condition?
Unfortunately the lifecycle block is not available for a module.
A way around this is the following
module "groups" {
source = "../path/to/groups"
for_each = var.groups.groups
name = each.key
type = each.value.type
policies = each.value.policies
depends_on = [
null_resource.group_check,
module.policies
]
}
resource "null_resource" "group_check" {
for_each = var.groups.groups
lifecycle {
precondition {
condition = alltrue([for p in each.value.policies : contains(local.policies_list, p)])
error_message = format("Attempt to create a group with a non existing policy")
}
}
depends_on = [
module.policies
]
}
We are using Terraform to store secrets inside AWS secrets manager. We would like to expand our Terraform to add resource access policy to each secret to only allow certain IAM roles or user access the secret and get it is value. We are defining each secret and it is metadata using YAML. Terraform will then decode the yaml and store all the contents in a map. We then have a for_each to iterate through each map and create the secrets. Below is the yaml definition for a secret
nonprod:
- name: my-super-secret
metadata:
description: my-super-secret
value: somesecret
policy: true #This is the feature we trying to add. It will tell TF to add resource access policy
iam_roles:
- "arn:aws:iam::account-id:role/sagemaker"
- "arn:aws:iam::account-id:user/jon.doe"
- "arn:aws:iam::account-id:role/test"
tags:
purpose: sagemaker
The YAML is decoded and then stored in a var.secrets map. Using TF console this is what TF store in var.secrets after decoding YAML.
{
"metadata" = {
"description" = "my-super-secret"
}
"name" = "my-super-secret"
"policy" = true
"iam_roles" = [
"arn:aws:iam::account-id:role/sagemaker",
"arn:aws:iam::account-id:user/jane.doe",
"arn:aws:iam::account-id:role/test",
]
"tags" = {
"purpose" = "sagemkaer"
}
"value" = "somesecret"
}
on the main.tf file, I added the following code for the IAM policy document:
data "aws_iam_policy_document" "example" {
for_each = { for item in var.secrets : item.name => item }
statement {
sid = "EnableAccessFor${each.value.name}"
principals {
type = "AWS"
identifiers = [lookup(each.value, "iam_roles")]
}
actions = [
"secretsmanager:GetSecretValue",
]
resources = [
"*",
]
}
}
then I am passing the policy document to aws_secretsmanager_secret_policy resource to attach the policy to the secret that has policy set as true
resource "aws_secretsmanager_secret_policy" "policy" {
depends_on = [aws_secretsmanager_secret.secret]
for_each = { for item in var.secrets : item.name => item }
secret_arn = each.key
policy = data.aws_iam_policy_document.example.json
}
no matter what way I use, I always get errors when I run a plan. I have used the following functions with no luck:
join, concat, toset, splat, jsonencode and for expressions
I get the following errors:
using for expression identifiers = [for r in lookup(each.value, "iam_roles") : r] produces this error: Invalid value for "inputMap" parameter: the given object has no attribute "iam_roles"
using splat identifiers = "${each.value[*].iam_roles}" produces this error: Inappropriate value for attribute "identifiers": element 0: string required.
using toset with lookup identifiers = "${toset(lookup(each.value, "iam_roles", ""))}" produces this error Invalid value for "v" parameter: cannot convert string to set of any single type.
using join with lookup identifiers = ["${join(", ", lookup(each.value, "iam_roles", ""))}"] produces this error Invalid value for "lists" parameter: list of string required.
using jsonencode identifiers = [jsonencode(each.value.iam_roles)] produces this error This object does not have an attribute named "iam_roles".
using just each.value without lookup identifiers = [each.value.iam_roles] produces this error Inappropriate value for attribute "identifiers": element 0: string required.
Any idea?
Update
I got rid of the iam_policy_document and instead opt-in to use the aws_secretsmanager_secret_policy resource with a json-policy. See example below:
resource "aws_secretsmanager_secret_policy" "policy" {
depends_on = [aws_secretsmanager_secret.secret]
for_each = { for item in var.secrets : item.name => item if try(item.policy, false)}
secret_arn = aws_secretsmanager_secret.secret[each.key].arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAccessFor${each.value.name}",
"Effect": "Allow",
"Principal": {
"AWS": [
"${join(",", formatlist("\"%s\"", each.value.iam_roles))}"
]
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
The part I am having issue is with the each.value.iam_roles as that is a tuple vs. a string. I tried multiple ways to convert that into string but it is not working. Perhaps someone can help me with that.
issue was resolved with encoding the entire policy to JSON using the `jsonencode function in Terraform.
I have sample code below which creates an IAM role, a policy document, attachment of policy document and then the attachment of that policy to role.
resource "aws_iam_role" "aws_snsANDsqsTeam" {
name = "aws_snsANDsqsTeam"
assume_role_policy = data.aws_iam_policy_document.production-okta-trust-relationship.json
}
data "aws_iam_policy_document" "sns-and-sqs-policy" {
statement {
sid = "AllowToPublishToSns"
effect = "Allow"
actions = [
"sns:Publish",
]
resources = [
data.resource.arn,
]
}
statement {
sid = "AllowToSubscribeFromSqs"
effect = "Allow"
actions = [
"sqs:changeMessageVisibility*",
"sqs:SendMessage",
"sqs:ReceiveMessage",
"sqs:GetQueue*",
"sqs:DeleteMessage",
]
resources = [
data.resource.arn,
]
}
}
resource "aws_iam_policy" "sns-and-sqs" {
name = "sns-and-sqs-policy"
policy = data.aws_iam_policy_document.sns-and-sqs-policy.json
}
resource "aws_iam_role_policy_attachment" "sns-and-sqs-role" {
role = "aws_snsANDsqsTeam"
policy_arn = aws_iam_policy.sns-and-sqs.arn
}
Now below is the directory tree that I am trying to get
Now I want the policy document and policy code to be moved to the developer.tf file under shared/iam folder so it will look like this
data "aws_iam_policy_document" "sns-and-sqs-policy" {
statement {
sid = "AllowToPublishToSns"
effect = "Allow"
actions = [
"sns:Publish",
]
resources = [
data.resource.arn,
]
}
statement {
sid = "AllowToSubscribeFromSqs"
effect = "Allow"
actions = [
"sqs:changeMessageVisibility*",
"sqs:SendMessage",
"sqs:ReceiveMessage",
"sqs:GetQueue*",
"sqs:DeleteMessage",
]
resources = [
data.resource.arn,
]
}
}
resource "aws_iam_policy" "sns-and-sqs" {
name = "sns-and-sqs-policy"
policy = data.aws_iam_policy_document.sns-and-sqs-policy.json
}
and have the role creation and policy attachment code in main.tf file under iam-platform-security folder, so the code will look like this:
resource "aws_iam_role" "aws_snsANDsqsTeam" {
name = "aws_snsANDsqsTeam"
assume_role_policy = data.aws_iam_policy_document.production-okta-trust-relationship.json
}
resource "aws_iam_role_policy_attachment" "sns-and-sqs-role" {
role = "aws_snsANDsqsTeam"
policy_arn = aws_iam_policy.sns-and-sqs.arn
}
My Question is how can I reference a policy which is under shared/iam folder to attach it to a role I created in main.tf file under the folder iam-platform-security. The goal is to create policies separately in the shared/iam folder and roles under team/sub-team folders ( like iam-platform-security, iam-platform-architecture,iam-platform-debug etc etc) and then create attachments so policies remains separately as standalone.
Can somebody help me on this.
How can I reference the policy document in main.tf file in different directory.
You have to use modules so that you can separate your parent TF code from other code, such as your IAM related code in a different folder.
I deployed using Terraform an IAM Role to be used in EMR:
data "aws_iam_policy_document" "emr_assume_role" {
statement {
sid = "EMRAssume"
effect = "Allow"
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = [
"elasticmapreduce.amazonaws.com"
]
}
}
}
resource "aws_iam_role" "my_emr_ec2_instance_role" {
name = "my_emr_ec2_instance_role"
assume_role_policy = data.aws_iam_policy_document.emr_assume_role.json
}
resource "aws_iam_policy" "emr_ec2_instances_policy" {
name = "emr_ec2_instances_policy"
policy = file("${path.module}/my/path/my_emr_instance_role_policy.json")
}
resource "aws_iam_role_policy_attachment" "policy_attachment" {
role = aws_iam_role.my_emr_ec2_instance_role.name
policy_arn = aws_iam_policy.emr_ec2_instances_policy.arn
}
Then when I try to run run_job_flow() method from boto3 like this:
client.run_job_flow(
Name="EMR",
LogUri=logs_uri,
ReleaseLabel='emr-6.2.0',
Instances=instances,
VisibleToAllUsers=True,
Steps=steps,
BootstrapActions=ba,
Applications=[{'Name': 'Spark'}],
ServiceRole='my_service_role_emr',
JobFlowRole='my_emr_ec2_instance_role',
Tags=tags)
But I straight-away receive the following error message:
{ClientError}An error occurred (ValidationException) when calling the RunJobFlow operation: Invalid InstanceProfile my_emr_ec2_instance_role
How to resolve?
I'm sharing my experience hoping to help someone else, please share yours if different.
In my case a first mistake was with the identifiers field, which should have had "ec2.amazonaws.com" as value, so the aws_iam_policy_document block would get:
data "aws_iam_policy_document" "emr_assume_role" {
statement {
sid = "EMRAssume"
effect = "Allow"
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = [
"ec2.amazonaws.com"
]
}
}
}
Another issue is relative to the Instance Profile which would have been automatically created if the Role would have been generated from the AWS Console, but Terraform doesn't provide it automatically. So in Terraform this block of code should fix the problem:
resource "aws_iam_instance_profile" "emr_ec2_instance_profile" {
name = aws_iam_role.my_emr_ec2_instance_role.name
role = aws_iam_role.my_emr_ec2_instance_role.name
}
I am trying to apply different actions for different IAM users, through Terraform, using the aws_iam_policy_document data source.
Let's take as an example the following KMS Key policy statement:
data "aws_iam_policy_document" "kms_key_policy" {
statement {
sid = "Allow use of the key"
principals {
type = "AWS"
identifiers = var.A == true ? [ARN1, ARN2] : [ARN1]
}
actions = [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
]
resources = ["*"]
}
}
In the policy above I want to restrict the first two actions to the ARN2, but keep the ARN1 with all the actions that are originally in the actions block. Of course that I could just add another statement and separate both logics (as shown below) but I was trying to keep all the logic into the same statement and avoid repeating code:
statement {
sid = "Allow ARN1 use of the key"
principals {
type = "AWS"
identifiers = [ARN1]
}
actions = [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
]
resources = ["*"]
}
statement {
sid = "Allow ARN2 use of the key"
principals {
type = "AWS"
identifiers = var.A == true ? [ARN2] : []
}
actions = [
"kms:Encrypt",
"kms:Decrypt"
]
resources = ["*"]
}
I've already tried to add a condition similar to what is being used to check for the presence of ARN2 (in case var A is defined) but I was restricting ARN1 actions with the ARN2 one's (if ARN2 was present), as can be observed below:
actions = var.A == true ? ["kms:Encrypt", "kms:Decrypt"] : ["kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey"]
How could I separate the actions for the different principal identifiers ARNs inside the same statement?
What you're describing is not possible, as you can't make the list of actions a function of the principal during policy evaluation. That's not a Terraform limitation but how these AWS statements work. The conditionals you write here are evaluated when you apply the Terraform code, not during policy evaluation.
You never want to allow the same list of actions for both principals, regardless of the value of var.A, so you will need always need at least two statements. I hope this make sense to you.
I'd propose something like this:
statement {
sid = "AllowEncryptDecrypt"
principals {
type = "AWS"
identifiers = var.A == true ? [ARN1, ARN2] : [ARN1]
}
actions = [
"kms:Encrypt",
"kms:Decrypt",
]
resources = ["*"]
}
statement {
sid = "AllowOtherKeyUse"
principals {
type = "AWS"
identifiers = [ARN1]
}
actions = [
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey",
]
resources = ["*"]
}
I'd argue there's no real repetition here, as there's no overlap between the lists of actions.