Provisioning Teams in GitHub using Terraform - team hierarchy - terraform

I am attempting to use the github terraform provider to simplify how we manage teams in our GitHub Organization. To reduce the amount of changes to the Terraform code I wanted to use a solution like https://learn.hashicorp.com/tutorials/terraform/github-user-teams where contributors would only need to change values in a csv file.
The example does not include a solution for having teams be members of teams. Looking at the provider documentation https://registry.terraform.io/providers/integrations/github/latest/docs/resources/team, it requires parent_team_id be provided.
My current approach is to have two CSV files, one with parent team name and one without, and build a local mapping. The issue is that when I apply I get the error
The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.
Can someone point out where I am going wrong, it will be something simple but I cannot see what.
NOTE: I am aware (after posting) that I am not actually using the parent_team_id in the second team resource, thats a simple fix once I get the local working.
Resources
resource "github_team" "l0" {
for_each = {
for team in csvdecode(file("teams.csv")) :
team.name => team
}
name = each.value.name
description = each.value.description
privacy = each.value.privacy
create_default_maintainer = true
}
resource "github_team" "l1" {
for_each = {
for team in local.l1_mapping : team.name => team
}
name = each.value.name
description = each.value.description
privacy = each.value.privacy
create_default_maintainer = true
depends_on = [
github_team.l0
]
}
Locals
# Create local values to retrieve items from CSVs
locals {
l1_team_plus = {
for file2 in csvdecode(file("l1_teams.csv")) :
file2.name => file2
}
l1_mapping = flatten([
for team2 in local.l1_team_plus : [
for t in github_team.l0 : {
id = t.id
slug = t.slug
team2 = team2
} if t.slug == team2.name
]
])
}

Related

MySQL firewall rules from an app service's outbound IPs on Azure using Terraform

I'm using Terraform to deploy an app to Azure, including a MySQL server and an App Service, and want to restrict database access to only the app service. The app service has a list of outbound IPs, so I think I need to create firewall rules for these on the database. I've found that in Terraform, I can't use count or for_each to dynamically create these rules, as the value isn't known in advance.
We've also considered hard coding the count but the Azure docs don't confirm the number of IPs. With this, and after seeing different numbers in stackoverflow comments, I'm worried that the number could change at some point and break future deployments.
The output error suggests using -target as a workaround, but the Terraform docs explicitly advise against this due to potential risks.
Any suggestions for a solution? Is there a workaround, or is there another approach that would be better suited?
Non-functional code I'm using so far to give a better idea of what I'm trying to do:
...
locals {
appIps = split(",", azurerm_app_service.appService.outbound_ip_addresses)
}
resource "azurerm_mysql_firewall_rule" "appFirewallRule" {
count = length(appIps)
depends_on = [azurerm_app_service.appService]
name = "appService-${count.index}"
resource_group_name = "myResourceGroup"
server_name = azurerm_mysql_server.databaseServer.name
start_ip_address = local.appIps[count.index]
end_ip_address = local.appIps[count.index]
}
...
This returns the error:
Error: Invalid count argument
on main.tf line 331, in resource "azurerm_mysql_firewall_rule" "appFirewallRule":
331: count = length(local.appIps)
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
I dig deeper there and I think I have a solution which works at least for me :)
The core of the problem here is in the necessity to do the whole thing in 2 steps (we can't have not yet known values as arguments for count and for_each). It can be solved with explicit imperative logic or actions (like using -targetonce or commenting out and then uncommenting). Besides, it's not declarative it's also not suitable for automation via CI/CD (I am using Terraform Cloud and not the local environment).
So I am doing it just with Terraform resources, the only "imperative" pattern is to trigger the pipeline (or local run) twice.
Check my snippet:
data "azurerm_resources" "web_apps_filter" {
resource_group_name = var.rg_system_name
type = "Microsoft.Web/sites"
required_tags = {
ProvisionedWith = "Terraform"
}
}
data "azurerm_app_service" "web_apps" {
count = length(data.azurerm_resources.web_apps_filter.resources)
resource_group_name = var.rg_system_name
name = data.azurerm_resources.web_apps_filter.resources[count.index].name
}
data "azurerm_resources" "func_apps_filter" {
resource_group_name = var.rg_storage_name
type = "Microsoft.Web/sites"
required_tags = {
ProvisionedWith = "Terraform"
}
}
data "azurerm_app_service" "func_apps" {
count = length(data.azurerm_resources.func_apps_filter.resources)
resource_group_name = var.rg_storage_name
name = data.azurerm_resources.func_apps_filter.resources[count.index].name
}
locals {
# flatten ensures that this local value is a flat list of IPs, rather
# than a list of lists of IPs.
# distinct ensures that we have only uniq IPs
web_ips = distinct(flatten([
for app in data.azurerm_app_service.web_apps : [
split(",", app.possible_outbound_ip_addresses)
]
]))
func_ips = distinct(flatten([
for app in data.azurerm_app_service.func_apps : [
split(",", app.possible_outbound_ip_addresses)
]
]))
}
resource "azurerm_postgresql_firewall_rule" "pgfr_func" {
for_each = toset(local.web_ips)
name = "web_app_ip_${replace(each.value, ".", "_")}"
resource_group_name = var.rg_storage_name
server_name = "${var.project_abbrev}-pgdb-${local.region_abbrev}-${local.environment_abbrev}"
start_ip_address = each.value
end_ip_address = each.value
}
resource "azurerm_postgresql_firewall_rule" "pgfr_web" {
for_each = toset(local.func_ips)
name = "func_app_ip_${replace(each.value, ".", "_")}"
resource_group_name = var.rg_storage_name
server_name = "${var.project_abbrev}-pgdb-${local.region_abbrev}-${local.environment_abbrev}"
start_ip_address = each.value
end_ip_address = each.value
}
The most important piece there is azurerm_resources resource - I am using it to do filtering on what web apps are already existing in my resource group (and managed by automation). I am doing DB firewall rules on that list, next terraform run, when newly created web app is there, it will also whitelist the lastly created web app.
An interesting thing is also the filtering of IPs - a lot of them are duplicated.
At the moment, using -target as a workaround is a better choice. Because with how Terraform works at present, it considers this sort of configuration to be incorrect. Using resource computed outputs as arguments to count and for_each is not recommended. Instead, using variables or derived local values which are known at plan time is the preferred approach. If you choose to go ahead with using computed values for count/for_each, this will sometimes require you to work around this using -target as illustrated above. For more details, please refer to here
Besides, the bug will be fixed in the pre-release 0.14 code. For more details, please

terraform code for creating bulk IAM Users and attaching multiple user policy to the same

My goal is to create multiple IAM users and also attach multiple policies to each user in a single piece of code but I am not able to achieve this.
Could you please help me with the code.
Below is the code I am trying to run but it is failing.
resource "aws_iam_user" "example" {
count = "${length(var.aws_iam_user)}"
name = "${element(var.aws_iam_user,count.index )}"
}
resource "aws_iam_user_policy_attachment" "user-policy-attachment" {
aws_iam_user = "${var.aws_iam_user}"
count = "${length(var.iam_policy_arn)}"
policy_arn = "${var.iam_policy_arn[count.index]}"
}
Can anyone please help me with the correct code to achieve this task?
In the code we should be able to create bulk users and also attach policy to each user. The policy can be the same or different for each user.
If you're using terraform 0.12 or newer you can use for_each to interact over a map and generate a map, if you use count and delete a user, all users defined after that position will be recreated.
Here is a quick example.
terraform.tfvars
users = [
"user1": [ "policy1ARN", "policy3ARN" ]
"user2": [ "policy1ARN" ],
"user3": [ "police2ARN" ]
]
example.tf
resource "aws_iam_user" "list" {
for_each = var.users
name = each.key
force_destroy = true
}
resource "aws_iam_user_policy_attachment" "list" {
for_each = var.users
user = aws_iam_user[each.key].name
policy_arn = each.value
}
Just an example to show how to use for_each based on how you want to implement, but as a best practice, it's better if you create IAM groups with their respective policies and instead of attaching policies to each user, you add users to groups.

Terraform Digitalocean: move resources under project

I want to create N droplets on DigitalOcean and assign them to a DigitalOcean project (that is not yet existing).
First I'm creating a project and I assign the droplets to the project using the resources item. I'm also creating the two droplets.
resource "digitalocean_project" "project" {
name = "playground"
count = "2"
description = "Description"
purpose = "Description purposes"
environment = "Development"
resources = [
digitalocean_droplet.myserver[count.index].urn
]
}
resource "digitalocean_droplet" "myserver" {
count = "2"
name = "server-${count.index}"
image = "ubuntu-18-04-x64"
size = "1gb"
region = "${var.region}"
}
The droplets are created successfully. One droplet is moved to the newly created project, while the other droplet remains in my default project.
The error message is below is clear. It tries to create a second project with the same name.
Error: Error creating Project: POST
https://api.digitalocean.com/v2/projects: 409 name is already in use
(duplicate)
on create_server.tf line 1, in resource "digitalocean_project"
"project": 1: resource "digitalocean_project" "project" {
How can I assign the two droplets to my project (which I want to create dynamically)?
If you want one project with multiple resources then you need to only create a single project and assign the list of resources to it. To do this you'll want to remove the count parameter from the digitalocean_project resource (this would create multiple projects) and then use the splat expression of the digitalocean_droplet resources to pass a list of the resources to the project.
So you want something that looks a little like this:
resource "digitalocean_project" "project" {
name = "playground"
description = "Description"
purpose = "Description purposes"
environment = "Development"
resources = digitalocean_droplet.myserver[*].urn
}
resource "digitalocean_droplet" "myserver" {
count = "2"
name = "server-${count.index}"
image = "ubuntu-18-04-x64"
size = "1gb"
region = var.region
}

AWS Codepipeline with Terraform - How to dynamically create stages

I designed an AWS codepipeline module using terraform module, I have multiple actual codepipelines using the codepipeline module. I use module as design pattern because all the codepipelines look similar, except that some of the codepipelines need approval stages, some do not need. How do I design the codepipeline module approval stages so that the actual codepipelines can be created based on different needs?
I tried to use count = 0 or 1 to control the stage but it does not work because the stage is not resource-level. Is there any tricky way or workaround?
I feel this link asked the similar question but I cannot figure out what is the answer:
Terraform & AWS CodePipeline - Dynamically define actions on a stage
Here is my codepipeline terraform module:
resource "aws_codepipeline" "dev" {
name = "my_codepipeline"
role_arn = ...
...
stage {
name = "Source"
...
}
stage {
name = "test"
...
}
stage {
# count = 0 # or 1. it does not work
name = "Approval"
action {
name = "Approval"
owner = "AWS"
category = "Approval"
provider = "Manual"
version = "1"
configuration {
NotificationArn = "..."
CustomData = "..."
ExternalEntityLink = "..."
}
}
}
stage {
name = "prod"
...
}
}
To dynamically add a stage (and not just an action) you could do the following:
dynamic "stage" {
for_each = var.production_approval ? [1] : []
content {
name = "Approve"
action {
configuration = {
NotificationArn = var.approve_sns_arn
CustomData = var.approve_comment
}
name = "Production-Approval"
category = "Approval"
owner = "AWS"
provider = "Manual"
version = "1"
}
}
}
When going through your use case, I have the feeling that it is very suitable with new terraform feature in v0.12.x
Below is a sample on how to use for_each to set dynamic target regions, you should be fine to do the same for stages.
dynamic "target_region" {
for_each = var.target_image_regions
content {
name = target_region.value
regional_replica_count = 1
}
}
let me know if this works for you or not.
Reference: https://www.hashicorp.com/blog/announcing-terraform-0-12
I figured that you can get this working in Terraform 0.12+ as BMW said, but only if you have number of block greater than 0.
At least 1 "action" blocks are required.
Unfortunately, my (and your) use case required 0/1 actions depending on the environment so we have to manage it manually for a while.
Cheers.
dynamic "action" {
for_each = local.production_approval # e.g. [] || [true]
content {
category = "Approval"
configuration = {}
input_artifacts = []
name = "Production-Approval"
output_artifacts = []
owner = "AWS"
provider = "Manual"
run_order = 1
version = "1"
}
}

How to iterate multiple resources over the same list?

New to Terraform here. I'm trying to create multiple projects (in Google Cloud) using Terraform. The problem is I've to execute multiple resources to completely set up a project. I tried count, but how can I tie multiple resources sequentially using count? Here are the following resources I need to execute per project:
Create project using resource "google_project"
Enable API service using resource "google_project_service"
Attach the service project to a host project using resource "google_compute_shared_vpc_service_project" (I'm using shared VPC)
This works if I want to create a single project. But, if I pass a list of projects as input, how can I execute all the above resources for each project in that list sequentially?
Eg.
Input
project_list=["proj-1","proj-2"]
Execute the following sequentially:
resource "google-project" for "proj-1"
resource "google_project_service" for "proj-1"
resource "google_compute_shared_vpc_service_project" for "proj-1"
resource "google-project" for "proj-2"
resource "google_project_service" for "proj-2"
resource "google_compute_shared_vpc_service_project" for "proj-2"
I'm using Terraform version 0.11 which does not support for loops
In Terraform, you can accomplish this using count and the two interpolation functions, element() and length().
First, you'll give your module an input variable:
variable "project_list" {
type = "list"
}
Then, you'll have something like:
resource "google_project" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
resource "google_project_service" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
resource "google_compute_shared_vpc_service_project" {
count = "${length(var.project_list)}"
name = "${element(var.project_list, count.index)}"
}
And of course you'll have your other configuration in those resource declarations as well.
Note that this pattern is described in Terraform Up and Running, Chapter 5, and there are other examples of using count.index in the docs here.
A small update to this question/answer (terraform 0.13 and above). The count or length is not advisable to use anymore due to the way that terraforms works, let's imagine the next scenario:
Suppose you have an array with 3 elements: project_list=["proj-1","proj-2","proj-3"], once you apply that if you want to delete the "proj-2" item from your array once you run the plan, terraform will modify your second element to "proj-3" instead of removing It from the list (more info in this good post). The solution to get the proper behavior is to use the for_each function as follow:
variable "project_list" {
type = list(string)
}
resource "google_project" {
for_each = toset(var.project_list)
name = each.value
}
resource "google_project_service" {
for_each = toset(var.project_list)
name = each.value
}
resource "google_compute_shared_vpc_service_project" {
for_each = toset(var.project_list)
name = each.value
}
Hope this helps! 👍

Resources