Why can't I loop twice in an Azure resource module? - azure

I am trying to set up alerts in Azure bicep resources/modules. The goal is to have 3 files, one where the initial resource structure is created and parameterized:
param metricAlertsName string
param description string
param severity int
param enabled bool = true
param scopes array = []
param evaluationFrequency string
param windowSize string
param targetResourceRegion string = resourceGroup().location
param allOf array = []
param actionGroupName string
var actionGroupId = resourceId(resourceGroup().name, 'microsoft.insights/actionGroups', actionGroupName)
resource dealsMetricAlerts 'Microsoft.Insights/metricAlerts#2018-03-01' = [for appService in appServices : {
name: metricAlertsName
location: 'global'
tags: {}
properties: {
description: description
severity: severity
enabled: enabled
scopes: scopes
evaluationFrequency: evaluationFrequency
windowSize: windowSize
targetResourceRegion: targetResourceRegion
criteria: {
'odata.type': 'Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria'
allOf: allOf
}
actions: [
{
actionGroupId: actionGroupId
}
]
}
}]
A second file, main.bicep where all of my resources are being consumed as modules:
module dealsMetricAlerts '../modules/management/metric-alert.bicep' = [for (alert, index) in alertConfig.metricAlerts: {
name: alert.name
params: {
metricAlertsName: alert.params.metricAlertsName
actionGroupName: actionGroupName
description: alert.params.description
evaluationFrequency: alert.params.evaluationFrequency
severity: alert.params.severity
windowSize: alert.params.windowSize
allOf: alert.params.allOf
scopes: [
resourceId(resourceGroup().name,'Microsoft.Web/sites', 'gbt-${env1}-${env}-${location}-webapp')
]
}
}]
And the third file which is a parameter file in which I can loop over, enter its values into the module in main.bicep to create multiple resources without having to write more than one module:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertConfig": {
"value": {
"metricAlerts": [
{
"name": "403ErrorTest",
"params": {
"metricAlertsName": "AppService403ErrorTest",
"actionGroupName": "actionGroupName",
"description": "Alert fires whenever the App Service throws a 403 error",
"evaluationFrequency": "PT5M",
"severity": 2,
"windowSize": "PT15M",
"allOf": [
{
"name": "403errorTest",
"metricName": "Http403",
"dimensions": [],
"operator": "GreaterThan",
"threshold": 0,
"timeAggregation": "Count"
}
]
}
},
{
"name": "5XXErrorTest",
"params": {
// Values for 500 error
}
}
]
}
}
}
}
I have a total of 5 app services, 1 web app and 4 function apps. If I was to hard code the name of the app service like above then that one web app is deployed with the 2 errors exactly as it should.
The issue that I'm having is to try and use another loop on the scopes param to have the 2 errors attached to all 5 app services.
I tried something like:
// Created an array with the names of the app services
var appServices = [
'dda-webApp'
'dealservice-fnapp'
'itemdata-fnapp'
'refdata-fnapp'
'userprofile-fnapp'
]
// Update the scopes param to loop through the names\
scopes: [for appService in appServices: [
resourceId(resourceGroup().name,'Microsoft.Web/sites', 'gbt-${env1}-${env}-${location}-${appService}')
]]
This most often gives me an error saying: Unexpected token array in scopes[0].
I believe what's happening is when it's trying to look up the resource by name in resourceId it's looping through all of the appServices names and attaching the entire array onto the resource name making it invalid.
Is there another way to loop through these scopes in order to attach the 403 and 500 alerts to each of my 5 app services?

Looking at the documentation, you can't do inner loops:
Using loops in Bicep has these limitations:
Loop iterations can't be a negative number or exceed 800 iterations.
Can't loop a resource with nested child resources. Change the child resources to top-level resources. See Iteration for a child resource.
Can't loop on multiple levels of properties.
In you case, you can define a variable and create the array outside of the module loop:
// Created an array with the names of the app services
var appServices = [
'dda-webApp'
'dealservice-fnapp'
'itemdata-fnapp'
'refdata-fnapp'
'userprofile-fnapp'
]
// Create an array of app service resource ids
var appServiceIds = [for appService in appServices: resourceId(resourceGroup().name,'Microsoft.Web/sites', 'gbt-${env1}-${env}-${location}-${appService}')]
// Set the scope param with this variable
module dealsMetricAlerts '../modules/management/metric-alert.bicep' = [for (alert, index) in alertConfig.metricAlerts: {
name: alert.name
params: {
...
scopes: appServiceIds
}
}]

Related

Azure generate URL for a standard Logic app with connection to CosmosDB

I have a workflow in a standard logic app, that have HTTP trigger. When the workflow is trigged, the workflow, retrieve some data from a CosmosDB. Something like:
The previous method will require to have an API connection. I have already created and deployed a 'V2' API connection. Let's call it myCosmosCon
Also in the ARM template for my logic app I have already added a connectionRuntimeUrl of my connection API (to myCosmosCon) to appSettings (configuration):
....
"siteConfig": {
"appSettings": [
{
"name": "subscriptionId",
"value": "[subscription().subscriptionId]"
},
{
"name": "resourceGroup_name",
"value": "[resourceGroup().name]"
},
{
"name": "location_name",
"value": "[resourceGroup().location]"
},
{
"name": "connectionRuntimeUrl",
"value": "[reference(resourceId('Microsoft.Web/connections', parameters('connection_name')),'2016-06-01', 'full').properties.connectionRuntimeUrl]"
},
.....
]
},
Then I wrote the following in the connections.json:
{
"managedApiConnections": {
"documentdb": {
"api": {
"id": "/subscriptions/#appsetting('subscriptionId')/providers/Microsoft.Web/locations/#appsetting('location_name')/managedApis/documentdb"
},
"connection": {
"id": "/subscriptions/#appsetting('subscriptionId')/resourceGroups/#appsetting('resourceGroup_name')/providers/Microsoft.Web/connections/myCosmosCon"
},
"connectionRuntimeUrl": "#appsetting('connection_runtimeUrl')",
"authentication": {
"type": "ManagedServiceIdentity"
}
}
}
}
Now, when I deploy the ARM template of my Logic app, workflow, ... etc. I see no errors, the workflow looks also good. The only problem is the URL link to the HTTP trigger is not generated, I can't run the program.
However, if I change the connection_runtimeUrl in the connections.json file to have the actual value; to look something like:
"connectionRuntimeUrl": "https://xxxxxxxxxxxxx.xx.common.logic-norwayeast.azure-apihub.net/apim/myCosmosCon/xxxxxxxxxxxxxxxxxxxxxxxx/",
The URL is generated directly and I can simply run the workflow. AFTER that, if I return the connection_runtimeUrl as it was (a call to appsettings()), it still working!! the link also stay there.
It looks like the when I deploy the Logic app and the workflow that the connections.json, do not compile or make the call, so Azure think that there is an error and do not generate the link.
Any idea about how to solve the problem??
Thanks!
Not sure but could be the issue:
When you create a connection api for a logic app standard, you also need to create an access policy at the connection api level for the system assigned identity running the logic app standard.
param location string = resourceGroup().location
param cosmosDbAccountName string
param connectorName string = '${cosmosDbAccountName}-connector'
// The principalid of the logic app standard system assigned identity
param principalId string
// get a reference to the cosmos db account
resource cosmosDbAccount 'Microsoft.DocumentDB/databaseAccounts#2021-06-15' existing = {
name: cosmosDbAccountName
}
// create the related connection api
resource cosmosDbConnector 'Microsoft.Web/connections#2016-06-01' = {
name: connectorName
location: location
kind: 'V2'
properties: {
displayName: connectorName
parameterValues: {
databaseAccount: cosmosDbAccount.name
accessKey: listKeys(cosmosDbAccount.id, cosmosDbAccount.apiVersion).primaryMasterKey
}
api: {
id: 'subscriptions/${subscription().subscriptionId}/providers/Microsoft.Web/locations/${location}/managedApis/documentdb'
}
}
}
// Grant permission to the logic app standard to access the connection api
resource cosmosDbConnectorAccessPolicy 'Microsoft.Web/connections/accessPolicies#2016-06-01' = {
name: '${cosmosDbConnector.name}/${principalId}'
location: location
properties: {
principal: {
type: 'ActiveDirectory'
identity: {
tenantId: subscription().tenantId
objectId: principalId
}
}
}
}
output connectionRuntimeUrl string = reference(cosmosDbConnector.id, cosmosDbConnector.apiVersion, 'full').properties.connectionRuntimeUrl
I'm having trouble with the exact same issue/bug. The only work around as I see it is to deploy the workflow twice. First time with an actual URL pointing to a dummy connection and the second time with the appsetting reference.

locals.tf file - parsing jsonencode body

Wondering if anyone has ran tackled it. So, I need to be able to generate list of egress CIDR blocks that is currently available for listing over an API. Sample output is the following:
[
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.1.191",
"cidr": "192.168.1.128/26",
"ip": "192.168.1.128",
"ip_range": {
"start": "192.168.1.128",
"end": "192.168.1.191"
},
"netmask": "255.255.255.192",
"network": "192.168.1.128",
"prefix": "26",
"size": "64"
}
},
{
"description": "blahnet-public-acl",
"metadata": {
"broadcast": "192.168.160.127",
"cidr": "192.168.160.0/25",
"ip": "192.168.160.0",
"ip_range": {
"start": "192.168.160.0",
"end": "192.168.160.127"
},
"netmask": "255.255.255.128",
"network": "192.168.160.0",
"prefix": "25",
"size": "128"
}
}
]
So, I need convert it to Azure Firewall
###############################################################################
# Firewall Rules - Allow Access To TEST VMs
###############################################################################
resource "azurerm_firewall_network_rule_collection" "azure-firewall-azure-test-access" {
for_each = local.egress_ips
name = "azure-firewall-azure-test-rule"
azure_firewall_name = azurerm_firewall.public_to_test.name
resource_group_name = var.resource_group_name
priority = 105
action = "Allow"
rule {
name = "test-access"
source_addresses = local.egress_ips[each.key]
destination_ports = ["43043"]
destination_addresses = ["172.16.0.*"]
protocols = [ "TCP"]
}
}
So, bottom line is that allowed IP addresses have to be a list of strings for the "source_addresses" parameter, such as this:
["192.168.44.0/24","192.168.7.0/27","192.168.196.0/24","192.168.229.0/24","192.168.138.0/25",]
I configured data_sources.tf file:
data "http" "allowed_networks_v1" {
url = "https://testapiserver.com/api/allowed/networks/v1"
}
...and in locals.tf, I need to configure
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
egress_ips = ...
}
...and that's where I am stuck. How can parse that data in locals.tf file so I can reference it from within TF ?
Thanks a metric ton!!
I'm assuming that the list of string you are referring to are the objects under: metadata.cidr we can extract that with a for loop in a local, and also do a distinct just in case we get duplicates.
Here is a sample code
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/heldersepu/hs-scripts/master/json/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}
and here is the output of a plan on that:
terraform plan
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
Terraform will perform the following actions:
Plan: 0 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ data = [
+ "192.168.1.128/26",
+ "192.168.160.0/25",
]
Here is the code for your second sample:
data "http" "allowed_networks_v1" {
url = "https://raw.githack.com/akamalov/testfile/master/networks.json"
}
locals {
allowed_networks_json = jsondecode(data.http.allowed_networks_v1.body)
distinct_cidrs = distinct(flatten([
for key, value in local.allowed_networks_json.egress_nat_ranges : [
value.metadata.cidr
]
]))
}
output "data" {
value = local.distinct_cidrs
}

How do I make my data template file recognise a number in terraform

THe problem I am facing now is this. I am trying to make my policy more flexible. So I shifted them into a file instead of using EOF.
How to make the template file recognise a number value?
"${max_untagged_images}" and "${max_tagged_images}" are suppose to be numbers.
Aws lifecycle policy:
resource "aws_ecr_lifecycle_policy" "lifecycle" {
count = length(aws_ecr_repository.repo)
repository = aws_ecr_repository.repo[count.index].name
depends_on = [aws_ecr_repository.repo]
policy = var.policy_type == "app" ? data.template_file.lifecycle_policy_app.rendered : data.template_file.lifecycle_policy_infra.rendered
}
Data template:
data "template_file" "lifecycle_policy_app" {
template = file("lifecyclePolicyApp.json")
vars = {
max_untagged_images = var.max_untagged_images
max_tagged_images = var.max_tagged_images
env = var.env
}
}
Policy:
{
"rules": [
{
"rulePriority": 1,
"description": "Expire untagged images older than ${max_untagged_images} days",
"selection": {
"tagStatus": "untagged",
"countType": "sinceImagePushed",
"countUnit": "days",
"countNumber": "${max_untagged_images}"
},
"action": {
"type": "expire"
}
},
{
"rulePriority": 2,
"description": "Expire tagged images of ${env}, older than ${max_tagged_images} days",
"selection": {
"tagStatus": "tagged",
"countType": "imageCountMoreThan",
"countNumber": "${max_tagged_images}",
"tagPrefixList": [
"${env}"
]
},
"action": {
"type": "expire"
}
}
]
}
I would try the following 2 steps:
Remove the double quotes that around the "${max_tagged_images}"
Use terraform function called tonumber in order to convert it to a number:
tonumber("1")
(Follow the official documentation: https://www.terraform.io/docs/configuration/functions/tonumber.html)

How can I reference a aws_cognito_user_pools id in `Terraform`?

I have below terraform configuration for cognito client:
data "aws_cognito_user_pools" "re_user_pool" {
name = "${var.cognito_user_pool_name}"
}
resource "aws_cognito_user_pool_client" "app_client" {
name = "re-app-client"
user_pool_id = data.aws_cognito_user_pools.re_user_pool.id
depends_on = [data.aws_cognito_user_pools.re_user_pool]
explicit_auth_flows = ["USER_PASSWORD_AUTH"]
prevent_user_existence_errors = "ENABLED"
allowed_oauth_flows_user_pool_client = true
allowed_oauth_flows = ["code"]
allowed_oauth_scopes = ["phone", "openid", "email", "profile", "aws.cognito.signin.user.admin"]
supported_identity_providers = ["COGNITO", "Google"]
callback_urls = ["https://scnothzsf0.execute-api.ap-southeast-2.amazonaws.com/staging/signup"]
}
I references the cognito user pool which already exists on AWS. The error happens on the line user_pool_id = data.aws_cognito_user_pools.re_user_pool.id when it uses the user pool id in aws_cognito_user_pool_client.
I will get the error
Error: Error creating Cognito User Pool Client: InvalidParameterException: 1 validation error detected: Value 're-user' at 'userPoolId' failed to satisfy constraint: Member must satisfy regular expression pattern: [\w-]+_[0-9a-zA-Z]+
on infra/cognito.tf line 5, in resource "aws_cognito_user_pool_client" "app_client":
5: resource "aws_cognito_user_pool_client" "app_client" {`
It seems the format of the ID is not correct. I have read this document https://www.terraform.io/docs/providers/aws/d/cognito_user_pools.html and it has a reference attribute ids - The list of cognito user pool ids.. I wonder why it gives a list of user pool id. How can I reference this ID?
I also tried to reference it as user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0] but got an error:
Error: Invalid index
on infra/cognito.tf line 8, in resource "aws_cognito_user_pool_client" "app_client":
8: user_pool_id = data.aws_cognito_user_pools.re_user_pool.ids[0]
This value does not have any indices.
The re_user_pool referenced above is defined here:
resource "aws_cognito_user_pool" "re_user_pool" {
name = "re-user"
}
I came across your question while working through this same problem. I see the question is several months old, but I'm still going to add an answer for anyone else that ends up here like I did.
First, the solution is to convert the ids value from a set to a list via the tolist function and then access it as you would any terraform list.
Caveat: In my case, I have ensured I only have one user pool for a given name, but you could get multiple user pools if you haven't followed this convention. This solution will not be a complete solution for that situation, but perhaps it will still point in the right direction.
Example code:
data "aws_cognito_user_pools" "test" {
name = "a_name"
}
output "test" {
value = "${tolist(data.aws_cognito_user_pools.test.ids)[0]"
}
Second, how I arrived at it:
I added an output block so I could see what I was working with and I commented out the problematic lines in my terraform file so I could successfully execute terraform apply. Next I ran terraform apply followed by terraform output --json (note: the apply must be successful for output to have the latest values).
Example temporary output block:
output "test" {
value = "${data.aws_cognito_user_pools.test}" // output top-level object for debugging
}
Relevant terraform apply output:
test = {
"arns" = [
"<redacted>",
]
"id" = "a_name"
"ids" = [
"us-east-1_<redacted>",
]
"name" = "a_name"
}
Relevant terraform output --json output:
"test": {
"sensitive": false,
"type": [
"object",
{
"arns": [
"set",
"string"
],
"id": "string",
"ids": [
"set",
"string"
],
"name": "string"
}
],
"value": {
"arns": [
"<redacted>"
],
"id": "a_name",
"ids": [
"us-east-1_<redacted>"
],
"name": "a_name"
}
}
As you can see, the ids portion is a set of type string. I decided to try converting ids to a list to see if I could then access the 0 index and it worked. I feel like this could be a terraform bug, but I haven't filed an issue yet.

Terraform - Passing type Object as a parameter to Azure Template Deployment

I am tying to to provision Azure AD Domain Service using Terraform by giving Terraform the Azure ARM template, this is because Terrafrom does not support provisioning Azure AD Domain Service natively.
I have exported the ARM Template and it's parameters, one of the parameters is called "notificationSettings" which is a type Object and looks like below :
"notificationSettings": {
"value": {
"notifyGlobalAdmins": "Enabled",
"notifyDcAdmins": "Enabled",
"additionalRecipients": []
}
}
Other parameters are all strings and I can pass them without any issue, for example:
"apiVersion" = "2017-06-01"
I have tried passing this object to parameters like below :
"notificationSettings" = [{
"notifyGlobalAdmins" = "Enabled"
"notifyDcAdmins" ="Enabled"
"additionalRecipients" = []
}]
However, when I execute terrafrom apply, terrafrom complains and say:
Inappropriate value for attribute "parameters": element
"notificationSettings": string required.
How do I pass parameters type of Object to template body?
I have also tried giving the entire ARM json parameter as a file to terrafrom by using parameters_body option like below :
parameters_body = "${file("${path.module}/temp/params.json")}"
however, I am getting the followig error when executing the terrafrom script:
The request content was invalid and could not be deserialized: 'Error
converting value
"https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#"
to type
'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Data.Definitions.DeploymentParameterDefinition'.
Path 'properties.parameters.$schema', line 1, position 2952.'.
Below is the params.json file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"apiVersion": {
"value": "2017-06-01"
},
"sku": {
"value": "Standard"
"location": {
"value": "westus"
},
"notificationSettings": {
"value": {
"notifyGlobalAdmins": "Enabled",
"notifyDcAdmins": "Enabled",
"additionalRecipients": []
}
},
"subnetName": {
"value": "xxxx"
},
"vnetName": {
"value": "xxxx"
},
"vnetAddressPrefixes": {
"value": [
"10.0.1.0/24"
]
},
"subnetAddressPrefix": {
"value": "10.0.1.0/24"
},
"nsgName": {
"value": "xxxxx"
}
}
}
There is a way to pass arbitrary data structures from Terraform to ARM.
There are two ways to pass data to the ARM template within the azure_template_deployment provider
use the parameters block, which is limited to string parameters only
use the parameters_body block, which is pretty much arbitrary JSON.
I find the easiest way to use the parameters block is to create a local variable with the structure I require, then call jsonencode on it. I also like to keep the ARM template in a separate file and pull it in via a file() call, reducing the complexity of the terraform.
locals {
location = "string"
members = [
"array",
"of",
"members"
]
enabled = true
tags = {
"key" = "value",
"simple" = "store"
}
# this is the format required by ARM templates
parameters_body = {
location = {
value = "${local.location}"
},
properties = {
value = {
users = {
members = "${local.members}"
}
boolparameter = "${local.enabled}"
}
}
tags = {
value = "${module.global.tags}"
}
}
}
resource "azurerm_template_deployment" "sample" {
name = "sample"
resource_group_name = "rg"
deployment_mode = "Incremental"
template_body = "${file("${path.module}/arm/sample_arm.json")}"
parameter_body = "${jsonencode(local.parameters_body)}"
}
The only caveat I've found is that the bool parameters pass as a string, so declare them as a string in the ARM parameters section, then use a ARM function to convert to bool
"parameters: {
"boolParameter": {
"type": "string"
}
},
"variables": {
"boolVariable": "[bool(parameters('boolParameter'))]"
},
"resources": [
...
"boolArm": "[variables('boolVariable')]",
...
]

Resources