Working on a code to apply CIS policies set via code to track changes. What I am trying to do is create a custom policy set that contains the policies within CIS Microsoft Azure Foundation Benchmark v1.4.0 initiative definition. I am using the Terraform’s azurerm_policy_set_definition but repeatedly encounter a listofAllowedLocations issue.
Code:
data "azurerm_management_group" "Standard" {
display_name = "Standard"
}
resource "azurerm_policy_set_definition" "cis_benchmark" {
name = var.cis_policy_name
policy_type = "Custom"
display_name = var.cis_display_name
lifecycle {
create_before_destroy = true
}
parameters = local.parameters
metadata = local.metadata
policy_definition_reference {
policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/e765b5de-1225-4ba3-bd56-1ac6695af988"
parameter_values = <<VALUE
{
"allowedLocation": {"value": ["eastus"]}
}
VALUE
}
}
Error:
Error: creating Policy Set Definition "CIS Benchmark v1.4.0": policy.SetDefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="MissingPolicyParameter" Message="The policy set definition 'CIS Benchmark v1.4.0' is missing the parameter(s) 'listOfAllowedLocations' as defined in the policy definition 'e765b5de-1225-4ba3-bd56-1ac6695af988'."
│
│ with azurerm_policy_set_definition.cis_benchmark,│ on cisbenchmark.tf line 22, in resource "azurerm_policy_set_definition" "cis_benchmark":
│ 22: resource "azurerm_policy_set_definition" "cis_benchmark" {
│
You misspelled the parameter name allowedLocation, which should be listOfAllowedLocations. Take a close look at the example below, this should solve your issue. I agree it is a bit confusing that the names differ.
resource "azurerm_policy_set_definition" "example" {
name = "testPolicySet"
policy_type = "Custom"
display_name = "Test Policy Set"
parameters = <<PARAMETERS
{
"allowedLocations": {
"type": "Array",
"metadata": {
"description": "The list of allowed locations for resources.",
"displayName": "Allowed locations",
"strongType": "location"
}
}
}
PARAMETERS
policy_definition_reference {
policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/e765b5de-1225-4ba3-bd56-1ac6695af988"
parameter_values = <<VALUE
{
"listOfAllowedLocations": {"value": "[parameters('allowedLocations')]"}
}
VALUE
}
}
I got this example from the official policy_set_definition documentation.
Related
I’m creating a new Alias and Subscription for an Enrollment Account by using the following terraform code:
data "azurerm_billing_enrollment_account_scope" "example" {
billing_account_name = "1234567890"
enrollment_account_name = "0123456"
}
resource "azurerm_subscription" "example" {
subscription_name = "My Example EA Subscription"
billing_scope_id = data.azurerm_billing_enrollment_account_scope.example.id
}
Next, manages a policy set definition and subscription policy assignment by using the following TF code:
data "azurerm_subscriptions" "availablesubscriptions" {
display_name_prefix = var.subscription_name
}
resource "azurerm_policy_set_definition" "tag_definition" {
name = "${var.subscription_name}-tag-def"
display_name = "${var.subscription_name}-tag-def"
description = "Append the default tags definition"
policy_type = "Custom"
policy_definitions = templatefile("${path.module}/templates/tag_definitions.json",
{ environmentType = var.environment_Type})
metadata = <<METADATA
{ "category" : "Tags" }
METADATA
lifecycle { ignore_changes = [metadata] }
}
resource "azurerm_subscription_policy_assignment" "tag_assignment" {
name = "${var.subscription_name}-tag-assignment"
display_name = "${var.subscription_name}-tag-assignment"
description = "Append the default tags assignment"
subscription_id = data.azurerm_subscriptions.availablesubscriptions.subscriptions[0].subscription_id
policy_definition_id = azurerm_policy_set_definition.tag_definition.id
}
I have configured the release pipeline in Azure DevOps, for creating above infrastructure. But in the release, terraform apply task giving the following error:
Note: The service connection/Active Directory application having the Owner access.
Error: creating Policy Set Definition "XXXX-tag-def": policy.SetDefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'XXXXXXXXXXXXX' with object id 'XXXXXXXXXX' does not have authorization to perform action 'Microsoft.Authorization/policySetDefinitions/write' over scope '/subscriptions/***' or the scope is invalid. If access was recently granted, please refresh your credentials."
I am trying to build a custom seccomp template for Azure Policy using Terraform and keep running into errors when adding multiple paramaters similar to how the templates are built. If I build these into Azure manually, I have no problems.
My Terraform is below, the error I keep getting in this example is
╷
│ Error: creating/updating Policy Definition "k8s_seccomp_governance": policy.DefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPolicyRuleEffectDetails" Message="The policy definition 'k8s_seccomp_governance' rule is invalid. The policy effect 'details' property could not be parsed."
│
│ with azurerm_policy_definition.k8s_seccomp_governance,
│ on policy_definitions.tf line 1, in resource "azurerm_policy_definition" "k8s_seccomp_governance":
│ 1: resource "azurerm_policy_definition" "k8s_seccomp_governance" {
│
╵
Code:
resource "azurerm_policy_definition" "k8s_seccomp_governance" {
name = "k8s_seccomp_governance"
description = "Kubernetes cluster containers should only use allowed seccomp profiles"
policy_type = "Custom"
mode = "All"
display_name = "AMPS K8s Seccomp Governance"
metadata = <<METADATA
{
"category": "Kubernetes",
"version": "1.0.0"
}
METADATA
policy_rule = <<POLICY_RULE
{
"if": {
"field": "type",
"in": [
"AKS Engine",
"Microsoft.Kubernetes/connectedClusters",
"Microsoft.ContainerService/managedClusters"
]
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"constraintTemplate": "https://store.policy.core.windows.net/kubernetes/allowed-seccomp-profiles/v2/template.yaml",
"constraint": "https://store.policy.core.windows.net/kubernetes/allowed-seccomp-profiles/v2/constraint.yaml",
"excludedNamespaces": "[parameters('excludedNamespaces')]"
}
}
}
POLICY_RULE
parameters = <<PARAMETERS
{
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "'audit' allows a non-compliant resource to be created or updated, but flags it as non-compliant. 'deny' blocks the non-compliant resource creation or update. 'disabled' turns off the policy."
},
"allowedValues": ["audit", "deny","disabled"],
"defaultValue": "audit"
},
"excludedNamespaces": {
"type": "Array",
"metadata": {
"displayName": "Namespace exclusions",
"description": "List of Kubernetes namespaces to exclude from policy evaluation."
},
"defaultValue": ["kube-system", "gatekeeper-system", "azure-arc"]
}
}
PARAMETERS
}
To add,
If I don't include description, then I get this error:
╷
│ Error: creating/updating Policy Definition "k8s_seccomp_governance": policy.DefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="UnusedPolicyParameters" Message="The policy 'k8s_seccomp_governance' has defined parameters 'excludedNamespaces' which are not used in the policy rule. Please either remove these parameters from the definition or ensure that they are used in the policy rule."
│
│ with azurerm_policy_definition.k8s_seccomp_governance,
│ on policy_definitions.tf line 1, in resource "azurerm_policy_definition" "k8s_seccomp_governance":
│ 1: resource "azurerm_policy_definition" "k8s_seccomp_governance" {
│
╵
I was able to resolve this, the problem was that I was using mode: "All" and needed to change it to mode = "Microsoft.Kubernetes.Data" for these to work
I am trying to code an Azure Data Factory in Terraform, but I am not sure how to code this REST dataset:
{
"name": "RestResource1",
"properties": {
"linkedServiceName": {
"referenceName": "API_Connection",
"type": "LinkedServiceReference"
},
"annotations": [],
"type": "RestResource",
"schema": []
},
"type": "Microsoft.DataFactory/factories/datasets"
}
I don't see one in the azurerm documentation. Can one instead use an azurerm_data_factory_dataset_http resource instead?
azurerm_data_factory_linked_service_rest - Does not currently exist.
azurerm_data_factory_linked_service_web - This only support a web
table and not a REST API endpoint and can't be used with the Azure
integrated runtime.
As I tried to create the linked service using rest and http it always redirected to create a web table using terraform. Hence, for now the fix for this is to use azurerm_data_factory_linked_custom_service.
Here, is the example: How to create a Custom Linked service :
provider "azurerm" {
features{}
}
data "azurerm_resource_group" "example" {
name = "Your Resource Group"
}
data "azurerm_data_factory" "example" {
name = "vipdashadf"
resource_group_name = data.azurerm_resource_group.example.name
}
resource "azurerm_data_factory_linked_custom_service" "example" {
name = "ipdashlinkedservice"
data_factory_id = data.azurerm_data_factory.example.id
type = "RestService"
description = "test for rest linked"
type_properties_json = <<JSON
{
"url": "http://www.bing.com",
"enableServerCertificateValidation": false,
"authenticationType": "Anonymous"
}
JSON
annotations = []
}
resource "azurerm_data_factory_dataset_http" "example" {
name = "apidataset"
resource_group_name = data.azurerm_resource_group.example.name
data_factory_name = data.azurerm_data_factory.example.name
linked_service_name = azurerm_data_factory_linked_custom_service.example.name
relative_url = "http://www.bing.com"
request_body = "foo=bar"
request_method = "POST"
}
Outputs:
Linked Service - ipdashlinkservice type Rest Connector
Dataset: apidataset
You could find the same stated in the GitHub discussion: Support for Azure Data Factory Linked Service for REST API #9431
I am attempting to create an Isolated App Service Environment (ASE) in Azure using Terraform. I have succeeded once and have an ASE running. Attempts to create a second ASE in the same subscription, but within a separate resource group, fail. The only error message available is "An error has occurred". Investigation within Azure Monitoring reveals that the create request which was delivered via an ARM template has encountered a 500 error (Internal Server Error) somewhere in the process of creating the ASE. However, no details are available in the log message to indicate where/when/how Azure encountered the 500 error.
Environment:
azure cli (2.26.1)
terraform (0.14.11)
hashicorp/azurerm provider (2.67.0)
Details
I am creating multiple environments for my project: dev, test, and staging. They are each in separate resource groups within the same Azure subscription. All resources in my terraform are constructed with names unique to the environment/resource group. The intended lifecycle is for infrastructure changes to be deployed to dev, then test, then staging (and eventually a prod environment in a separate subscription). The initial configuration and deployment to dev has succeeded. Attempts to deploy to test, or to deploy a different ASE to dev, fail abjectly with very little feedback.
The original dev ASE is a v1 ASE. I have attempted to create a second ASE in test using the same terraform code. I have also tried creating a v3 ASE in dev (because the v3 will be cheaper). If the v3 ASE deploys successfully I will cut over to it in dev and will use it as the basis for test and stage instead of the v1 ASE. Regardless whether I try to deploy a v1 ASE to a separate resource group, or whether I try to deploy a v3 ASE to the same resource group as the v1 ASE, I get the same error.
This is the Terraform for the v1 ASE, including the subnet which will host it:
resource "azurerm_subnet" "subnet" {
name = "${local.prefix}-subnet"
resource_group_name = var.resource_group_name
virtual_network_name = var.vnet_name
address_prefixes = var.cidrs
enforce_private_link_endpoint_network_policies = var.enforce_private_link_endpoint_network_policies
enforce_private_link_service_network_policies = var.enforce_private_link_service_network_policies
dynamic "delegation" {
for_each = var.delegations
content {
name = "${local.prefix}-delegation-${delegation.key}"
service_delegation {
name = delegation.value.name
actions = delegation.value.actions
}
}
}
// List of Service endpoints to associate with the subnet.
service_endpoints = var.service_endpoints
}
resource "azurerm_network_security_group" "nsg" {
name = "${local.prefix}-nsg"
location = var.resource_group_location
resource_group_name = var.resource_group_name
tags = merge(map("Name", "${local.prefix}-nsg"), local.tags)
}
resource "azurerm_subnet_network_security_group_association" "nsg_assoc" {
subnet_id = azurerm_subnet.subnet.id
network_security_group_id = azurerm_network_security_group.nsg.id
}
resource "azurerm_network_security_rule" "ase_mgmt" {
name = "${local.prefix}-ase-mgmt"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
source_address_prefix = "AppServiceManagement"
destination_port_range = "454-455"
destination_address_prefix = var.subnet_cidr
resource_group_name = var.resource_group_name
network_security_group_name = azurerm_network_security_group.nsg.name
}
resource "azurerm_network_security_rule" "ingress" {
for_each = {
for idx, cidr in var.ingress_cidrs : idx => cidr
}
name = "${local.prefix}-ingress-${each.key}"
priority = 200 + each.key
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
source_address_prefix = each.value
destination_port_range = "*"
destination_address_prefix = var.subnet_cidr
resource_group_name = var.resource_group_name
network_security_group_name = azurerm_network_security_group.nsg.name
}
resource "azurerm_app_service_environment" "env" {
name = "${local.prefix}-ase"
subnet_id = azurerm_subnet.subnet.id
pricing_tier = var.pricing_tier
front_end_scale_factor = var.front_scale_factor
internal_load_balancing_mode = "Web, Publishing"
allowed_user_ip_cidrs = var.allowed_user_ip_cidrs
cluster_setting {
name = "DisableTls1.0"
value = "1"
}
depends_on = [
azurerm_network_security_rule.ase_mgmt
]
}
The v3 ASE is configured identically, except for azurerm_app_service_environment.env, which is replaced with:
resource "azurerm_app_service_environment_v3" "env" {
name = "${local.prefix}-ase-v3"
resource_group_name = var.resource_group_name
subnet_id = azurerm_subnet.subnet.id
cluster_setting {
name = "DisableTls1.0"
value = "1"
}
depends_on = [
azurerm_network_security_rule.ase_mgmt
]
}
Results
Terraform generates this ARM request (identifiers have been redacted):
2021/07/19 09:07:44 [TRACE] dag/walk: vertex "root" is waiting for "meta.count-boundary (EachMode fixup)"
2021-07-19T09:07:45.121-0700 [DEBUG] plugin.terraform-provider-azurerm_v2.67.0_x5: AzureRM Request:
PUT /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Web/hostingEnvironments/xxxxxxxx-dev-ase-v3?api-version=2020-06-01 HTTP/1.1
Host: management.azure.com
User-Agent: Go/go1.16.3 (amd64-darwin) go-autorest/v14.2.1 Azure-SDK-For-Go/v55.4.0 web/2020-06-01 HashiCorp Terraform/0.14.11 (+https://www.terraform.io) Terraform Plugin SDK/2.7.0 terraform-provider-azurerm/2.67.0 pid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Content-Length: 382
Content-Type: application/json; charset=utf-8
X-Ms-Correlation-Request-Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Accept-Encoding: gzip
{
"kind":"ASEV3",
"location":"centralus",
"properties":
{
"clusterSettings":[{
"name":"DisableTls1.0",
"value":"1"
}],
"name":"xxxxxxxx-dev-ase-v3",
"virtualNetwork":{
"id":"/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Network/virtualNetworks/xxxxxxxx-dev-vnet/subnets/xxxxxxxx-dev-ase-v3-ase-subnet",
"subnet":"xxxxxxxx-dev-ase-v3-ase-subnet"
}
},
"tags":{}
}
The error eventually reported by Terraform looks like this in the debug output:
2021/07/19 09:13:53 [DEBUG] azurerm_app_service_environment_v3.env: apply errored, but we're indicating that via the Error pointer rather than returning it: creating App Service Environment: (Hosting Environment Name "xxxxxxxx-dev-ase-v3" / Resource Group "xxxxxxxx-dev-rg"): web.AppServiceEnvironmentsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="Failed" Message="The async operation failed." AdditionalInfo=[{"Message":"An error has occurred."}]: creating App Service Environment: (Hosting Environment Name "xxxxxxxx-dev-ase-v3" / Resource Group "xxxxxxxx-dev-rg"): web.AppServiceEnvironmentsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="Failed" Message="The async operation failed." AdditionalInfo=[{"Message":"An error has occurred."}]
Reviewing the logs within Azure Monitor, I find a similarly vague error message. The message is summarized as InternalServerError. The JSON detail is included here for reference:
{
"authorization": {
"action": "Microsoft.Web/hostingEnvironments/write",
"scope": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Web/hostingEnvironments/xxxxxxxx-dev-ase-v3"
},
"caller": "duffy.gillman#presencepg.com",
"channels": "Operation",
"claims": {
//REDACTED
},
"correlationId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
"description": "",
"eventDataId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
"eventName": {
"value": "EndRequest",
"localizedValue": "End request"
},
"category": {
"value": "Administrative",
"localizedValue": "Administrative"
},
"eventTimestamp": "2021-07-19T15:51:45.4835627Z",
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Web/hostingEnvironments/xxxxxxxx-dev-ase-v3/events/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/ticks/637623067054835627",
"level": "Error",
"operationId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
"operationName": {
"value": "Microsoft.Web/hostingEnvironments/write",
"localizedValue": "Create or Update App Service Environment"
},
"resourceGroupName": "xxxxxxxx-dev-rg",
"resourceProviderName": {
"value": "Microsoft.Web",
"localizedValue": "Azure Web Sites"
},
"resourceType": {
"value": "Microsoft.Web/hostingEnvironments",
"localizedValue": "Microsoft.Web/hostingEnvironments"
},
"resourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Web/hostingEnvironments/xxxxxxxx-dev-ase-v3",
"status": {
"value": "Failed",
"localizedValue": "Failed"
},
"subStatus": {
"value": "InternalServerError",
"localizedValue": "Internal Server Error (HTTP Status Code: 500)"
},
"submissionTimestamp": "2021-07-19T15:52:29.177138Z",
"subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
"tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
"properties": {
"statusCode": "InternalServerError",
"serviceRequestId": null,
"statusMessage": "{\"Message\":\"An error has occurred.\"}",
"eventCategory": "Administrative",
"entity": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/xxxxxxxx-dev-rg/providers/Microsoft.Web/hostingEnvironments/xxxxxxxx-dev-ase-v3",
"message": "Microsoft.Web/hostingEnvironments/write",
"hierarchy": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
},
"relatedEvents": []
}
This to me looks like you have hit some subscription limitations. If you try create the same ASE via the Azure Portal does it provision for you? It will be good to know if you get an error trying to do the same thing via UI, if you did hit the same error in the GUI it will give you a better error message.
I have a custom policy initiative I would like to assign using terraform, one of the policies does have parameters.
I know that for single policy assignment, we could use something like this, and pass variable as parameter.
parameters = jsonencode({
"operationName": {
"value": var.allowed_values,
}
})
however, I tried this with initiative, this doesn't seem to like it keeps complaining parameter is missing.
Infact, I tried to hardcode parameter into the body of azurerm_policy_assignment, I don't think even this is working.
resource "azurerm_policy_assignment" "example" {
name = "mcs_governance_policy_assignment"
scope = var.scopes
policy_definition_id = azurerm_policy_set_definition.iam.id
description = "example"
display_name = "Governance Policy"
metadata = <<METADATA
{
"category": "General"
}
METADATA
parameters = <<PARAMETERS
{
"operationName": {
"value": "Microsoft.Authorization/policyAssignments/write"
}
}
PARAMETERS
}
the policy initiative definition policy is as following, it uses built in policy, I have not included all the policies in this initiative, just for the one that complaining about the parameter
resource "azurerm_policy_set_definition" "iam" {
name = "mcs_iam"
policy_type = "Custom"
display_name = "MCS IAM Governance Policy Set"
description = "Contains MCS Governance policies"
metadata = <<METADATA
{
"category": "General"
}
METADATA
policy_definition_reference {
# An activity log alert should exist for specific Policy operations
policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/c5447c04-a4d7-4ba8-a263-c9ee321a6858"
reference_id = "MCS IAM 3.1"
}
As I see, the azurerm_policy_set_definition iam only reference the policy definition, but it does not set the parameters in the policy_definition_reference block for the property parameter_values. You don't show how the policy definition configured, so you need to check how do the parameters setting in the policy definition and then add the parameter setting in the policy set definition.