When i am trying to create cloud endpoint from terraform script in azure i am getting following error,
Error: waiting for creation of Storage Sync Cloud Endpoint: (Cloud Endpoint Name “azbackup001zscallerc-file-sync-grp-CE” / Sync Group Name “azbackup001zscallerc-file-sync-grp” / Storage Sync Service Name “azbackup001zscallerc-file-sync” / Resource Group “RG”): Code=“-2134364065” Message=“Unable to read specified storage account. Please check the permissions and try again after some time.”
however when i am creating the same from azure portal i am able to create without any issues. I have checked all my permissions and even from global admin account as well, i am unable to do so. Please assist the possible solution
Please assist on checking permission issue as i can do same thing from az cli as well as powershell.
As it is even having issues with global admin account,Check When creation of Cloud Endpoint setup permission to that storage sync service that cloud sync is dependent on.
See Storage Sync service errors : make sure Azure File Sync has access to the storage account.
resource "azurerm_storage_sync" "example" {
name = "kaexample-ss"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
}
resource "azurerm_storage_sync_group" "example" {
name = "kaexample-ss-group"
storage_sync_id = azurerm_storage_sync.example.id
}
resource "azurerm_storage_account" "example" {
name = "kaaexample"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_share" "example" {
name = "kaexample-share"
storage_account_name = azurerm_storage_account.example.name
quota = 50
acl {
id = "GhostedRecall"
access_policy {
permissions = "r"
}
}
}
resource "azurerm_storage_sync_cloud_endpoint" "example" {
name = "example-ss-ce"
storage_sync_group_id = azurerm_storage_sync_group.example.id
file_share_name = azurerm_storage_share.example.name
storage_account_id = azurerm_storage_account.example.id
}
Please check this Az.StorageSync: Cloud endpoint creation access rigths failure issue · GitHub
Related
It seems my question is related to this post but since there is no answer I will ask again.
I have an Azure Devops project which I use to deploy static content into a container inside a Storage Account via Pipelines. I've recently decided to deploy my infrastructure using Terraform as well as my code but I'm running into an issue. I managed to create all my infrastructure with Terraform inside my Pipeline except for the Role Assignment.
I basically need to add a new Role Assignment to my Storage Account, through Azure it goes :
Go to my Storage Account
Go to Access Control (IAM)
Add a new Role Assignments
Select Storage Blob Data Contributor
Click on Select members
Select my Azure Devops Project
Review + assign
From what I understand in the Terraform documentation I should do something like this :
resource "azurerm_resource_group" "resource_group" {
name = var.resource_group_name
location = var.location
}
resource "azurerm_storage_account" "storage_account" {
name = var.storage_account_name
resource_group_name = azurerm_resource_group.resource_group.name
location = azurerm_resource_group.resource_group.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_role_assignment" "role_assignment" {
scope = azurerm_storage_account.storage_account.id
role_definition_id = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe" # Which is the Storage Blob Data Contributor role if I'm not mistaken.
principal_id = "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy" # Which should be the Application ID ?
}
Except it doesn't work, when I try to run it in local without the Azure Pipeline to check if this works, the process is stuck in the "Still creating..." state for more than 10 minutes, which seems weird since when you do it manually it only takes up to a few seconds. I don't have any error I just end up canceling the command.
What am I missing / doing wrong here ?
I've found what was the issue. For the principal_id you need to put the Object_ID of your Service Principal and not your Application_ID. You end up with something like :
main.tf
...
locals {
sub = "/subscription"
permission_storage_blob_data_contributor = "providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe"
}
data "azurerm_subscription" "primary" { }
resource "azurerm_resource_group" "resource_group" {
name = var.resource_group_name
location = var.location
}
resource "azurerm_storage_account" "storage_account" {
name = var.storage_account_name
resource_group_name = azurerm_resource_group.resource_group.name
location = azurerm_resource_group.resource_group.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_role_assignment" "role_assignment" {
scope = azurerm_storage_account.storage_account.id
role_definition_id = join("/", [local.sub, data.azurerm_subscription.primary.subscription_id, local.permission_storage_blob_data_contributor])
principal_id = var.devops_project_object_id
}
...
variables.tf
...
variable "location" {
type = string
description = "Location for the deployment"
default = "West Europe"
}
variable "resource_group_name" {
type = string
description = "Resource Group Name"
}
variable "storage_account_name" {
type = string
description = "Storage Account Name"
}
# yyyyyyyy-yyyy-yyyy-yyyyyyyyyyyy format
variable "devops_project_object_id" {
type = string
description = "Object ID (principal_id) for the Devops Project linked to the Azure Subscription in the Azure Active Directory."
}
...
Role assignment can be simplified to this call:
resource "azurerm_role_assignment" "blob_contributor" {
scope = azurerm_storage_account.storage_account.id
role_definition_name = "Storage Blob Data Contributor"
principal_id = var.devops_project_object_id
}
I'm trying to create a Synapse Workspace using Terraform. The workspace deploys successfully, but when testing the connection to the WorkSpaceDefaultStorage in Synapse studio I receive the following error:
ADLS Gen2 operation failed for: Storage operation '' on container 'testconnection' get failed with 'Operation returned an invalid status code 'Forbidden''.
The code for the Synapse workspace deployment:
resource "azurerm_storage_account" "sa" {
name = var.storage_account_name
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
account_tier = "Standard"
account_replication_type = "GRS"
account_kind = "StorageV2"
is_hns_enabled = true
}
resource "azurerm_storage_data_lake_gen2_filesystem" "adlfs" {
name = var.azure_data_lake_name
storage_account_id = azurerm_storage_account.sa.id
}
resource "azurerm_synapse_workspace" "synapseworkspace" {
name = var.synapse_workspace_name
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.adlfs.id
sql_administrator_login = var.synapse_sql_admin_user
sql_administrator_login_password = var.synapse_sql_admin_password
managed_resource_group_name = var.synapse_managed_resource_group_name
aad_admin {
login = var.azure_ad_admin_login
object_id = data.azurerm_client_config.current.object_id
tenant_id = data.azurerm_client_config.current.tenant_id
}
identity {
type = "SystemAssigned"
}
# Add tags
tags = {
source = "terraform"
}
}
resource "azurerm_synapse_firewall_rule" "synapsefirewall" {
name = "AllowAll"
synapse_workspace_id = azurerm_synapse_workspace.synapseworkspace.id
start_ip_address = "0.0.0.0"
end_ip_address = "255.255.255.255"
}
I am assuming the error can easily be fixed using role management in the Azure portal, but solving it using Terraform would be the best option.
Try the following?
I found it's not enough for the app and account to be added as owners.
I would go into your storage account > IAM > Add role assignment, and add the special permissions for this type of request:
Storage Blob Data Contributor
Storage Queue Data Contributor
Cited from: Azure Blob Storage "Authorization Permission Mismatch" error for get request with AD token
With the terraform module azurerm_role_assignment you can assigns a given Principal (User or Group) to a given Role. https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment
For Azure Synapse Workspace specifically, the principal identity can be retrieved using
azurerm_synapse_workspace.synapseworkspace.identity[0].principal_id
And assigned to a role:
# Create storage account
resource "azurerm_storage_account" "sa" {
...
}
# Create synapse workspace
resource "azurerm_synapse_workspace" "synapseworkspace" {
...
}
# Grant Synapse Workspace access to storage as Storage Blob Data Contributor
resource "azurerm_role_assignment" "synapsedatacontributor" {
role_definition_name = "Storage Blob Data Contributor"
scope = azurerm_storage_account.sa.id
principal_id = azurerm_synapse_workspace.synapseworkspace.identity[0].principal_id
}
I am using Azure Blob Storage as a state backend, due to new security requirements, I now need to access the azure storage accounts using SSL. This however fails with the following:
module.core_infra.data.terraform_remote_state.mccp_core_infra:
data.terraform_remote_state.mccp_core_infra: storage: service returned
error: StatusCode=403, ErrorCode=AuthenticationFailed,
ErrorMessage=Server failed to authenticate the request. Make sure the
value of Authorization header is formed correctly including the
signature.
Here’s an example configuration:
resource "azurerm_storage_account" "terraform_state_account" {
name = "${lower(replace(var.azure_tenant_name, "/\\W|_/", ""))}tfstate"
resource_group_name = "${azurerm_resource_group.main.name}"
location = "${var.azure_location}"
account_tier = "Standard"
account_replication_type = "LRS"
enable_https_traffic_only = true
network_rules {
ip_rules = ["masked/24"]
virtual_network_subnet_ids = ["${azurerm_subnet.mccp_vnet_subnet.id}"]
}
tags = {
environment = "${var.azure_tenant_name} terraform state account"
}
}
data "terraform_remote_state" "mccp_core_infra" {
backend = "azurerm"
config = {
storage_account_name = "${lower(replace(var.azure_tenant_name, "/\\W|_/", ""))}tfstate"
container_name = "mccp-core-infra-tf-state"
key = "terraform.tfstate"
access_key = "${var.azure_mccp_storage_account_key}"
}
}
I am using Terraform 0.11.11 with azurerm provider 1.33.0. This works just fine without the enable_https_traffic_only flag. What am I missing here?
The enable_https_traffic_only feature would not affect on that error. This works fine with enable_https_traffic_only flag in the Terraform v0.12.9
+ provider.azurerm v1.35.0 on my side.
It looks like a credential issue. I can reproduce your issue when the access_key is invalid in the data source. You could verify if you could access that storage account blob with that access key or you are getting references from a correct storage account name that hosts the .tfstate.
You could also try to delete the local .terraform folder and try again as it is mentioned in this post.
Following this article you can link Azure API Management to Users/Groups in Azure Active Directory.
At the moment I am creating the APIM instance with Terraform
resource "azurerm_api_management" "test" {
name = "example-apim"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
publisher_name = "My Company"
publisher_email = "company#terraform.io"
sku {
name = "Developer"
capacity = 1
}
}
How do I add the Active Directory Identity Provider to this?
Terraform added support for this in December 2019
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/api_management_identity_provider_aad
You can now link it with:
resource "azurerm_api_management_identity_provider_aad" "example" {
resource_group_name = azurerm_resource_group.example.name
api_management_name = azurerm_api_management.example.name
client_id = "00000000-0000-0000-0000-000000000000"
client_secret = "00000000000000000000000000000000"
allowed_tenants = ["00000000-0000-0000-0000-000000000000"]
}
This doesn't seem to be possible with terraform, however, it can be added by calling the REST API from the Azure CLI.
az rest -m put -u "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/my-resource-group/providers/Microsoft.ApiManagement/service/my-apim/identityProviders/aad?api-version=2019-01-01" -b "{'properties':{'clientId':'xxxxx-xxx-xxxx-xxxx-xxxxxxxxxx','clientSecret':'super-secret-password','allowedTenants':['mysite.com']}}"
The body -b is json that has been formatted to a single line.
You need to look up the clientId from active directory and know what the clientSecret is.
You can embedd this command in terraform if you wish:
resource "null_resource" "add-ad-identity-provider" {
provisioner "local-exec" {
command = "az rest -m put -u \"https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/my-resource-group/providers/Microsoft.ApiManagement/service/my-apim/identityProviders/aad?api-version=2019-01-01\" -b \"{'properties':{'clientId':'xxxxx-xxx-xxxx-xxxx-xxxxxxxxxx','clientSecret':'super-secret-password','allowedTenants':['mysite.com']}}\""
}
depends_on = ["azurerm_api_management.test"]
}
the original answer from March 4th mostly works. However, a piece is missing. You also need to set up an app registration via https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-aad
That supplies the answers you need (Other than the allowed tenants, which is the tenant-id's to allow).
And that is also missing a piece, which is to, when configurating the app registration, to also go to API Permissions, add a new permission for Azure Active Directory Graph (in supported legacy APIs), create an Application permission, and add Directory.Read.All. Then grant admin consent.
If you combine resources from both azurerm and azuread providers, you can now automate the process of deploying APIM with an app registration and AAD authentication on the developer portal. It covers these two guides from Microsoft:
Quickstart: Create a new Azure API Management service instance by using the Azure portal
Authorize developer accounts by using Azure Active Directory in Azure API Management
Terraform code example:
terraform {
required_version = ">=1.0.9"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.81.0"
}
azuread = {
source = "hashicorp/azuread"
version = "=2.7.0"
}
}
backend "azurerm" {}
}
provider "azurerm" {
features {}
}
provider "azuread" {}
resource "azurerm_api_management" "api_management" {
name = var.api_management_name
location = var.location
resource_group_name = var.resource_group_name
publisher_name = var.publisher_name
publisher_email = var.publisher_email
sku_name = var.api_management_sku
identity {
type = "SystemAssigned"
}
}
resource "azuread_application" "application" {
display_name = var.application_name
web {
redirect_uris = ["${azurerm_api_management.api_management.developer_portal_url}/"]
}
}
resource "azuread_application_password" "password" {
application_object_id = azuread_application.application.object_id
}
resource "azurerm_api_management_identity_provider_aad" "identity_provider_aad" {
resource_group_name = var.resource_group_name
api_management_name = azurerm_api_management.api_management.name
client_id = azuread_application.application.application_id
client_secret = azuread_application_password.password.value
allowed_tenants = var.id_provider_allowed_tenants
}
I'm trying to execute a sample terraform plan given below.
# Configure the Microsoft Azure Provider
provider "azurerm" {
subscription_id = "..."
client_id = "..."
client_secret = "..."
tenant_id = "..."
}
# Create a resource group
resource "azurerm_resource_group" "production" {
name = "production"
location = "West US"
}
# Create a virtual network in the web_servers resource group
resource "azurerm_virtual_network" "network" {
name = "productionNetwork"
address_space = ["10.0.0.0/16"]
location = "West US"
resource_group_name = "${azurerm_resource_group.production.name}"
subnet {
name = "subnet1"
address_prefix = "10.0.1.0/24"
}
subnet {
name = "subnet2"
address_prefix = "10.0.2.0/24"
}
subnet {
name = "subnet3"
address_prefix = "10.0.3.0/24"
}
}`enter code here`
I followed [1] to generate credentials via creating Active Directory application and used the correct subscription_id, client_id, client_secret, tenant_id in the above plan and executed 'terraform plan' against it. But I'm getting below error.
Error refreshing state: 1 error(s) occurred:
Credentials for acessing the Azure Resource Manager API are likely to be incorrect, or
the service principal does not have permission to use the Azure Service Management
API.
[1] https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/
Any idea on this?
It seems like in terraform documentation, they haven't included the step of assigning role to the service principal. Follow these steps and it works.
1) Create the service principal through Azure CLI by following this link https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal-cli/ which assigns the role as well to the service principal
2) Go to Azure RM portal-->Active Directory -->App registration --> Create the key
3) Use the appropriate values from above in .tf file.
Then run the command terraform plan.