Error while assigning admin role to SCIM provisioned AAD groups in Databricks - databricks

We use Azure databricks and managing via terraform. We have configured SCIM connector provisioner(AAD Enterprise app) to sync users and groups from AAD to Databricks. This works good. I can able to assign job or cluster permissions to these SCIM synced groups but when I try to assign admin role(entire workspace admin) to SCIM synced group the terraform error shows "API is not available for this worspace". Sorry, I don't what it means, Is it related to terraform provider or Am I putting something wrong? Please suggest me what should I use or correct. Please find below code 'principal_id' argument accepts user id or group id or service principal id as per terraform documentation here https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/permission_assignment#principal_id
Provider configuration:
terraform {
required_version = ">= 1.1.4"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 3.8.0"
}
databricks = {
source = "databricks/databricks"
version = ">= 1.6.3"
}
}
}
provider "databricks" { #Assign databricks workspace id to provider
azure_workspace_resource_id =
data.azurerm_databricks_workspace.adb_ws.id
}
Resource Block:
resource "databricks_permission_assignment" "assign_scim_admingroup" {
principal_id = data.databricks_group.dbricks_admin_group.id
permissions = ["ADMIN"]
}
Error in terraform:
│ Error: cannot create permission assignment: Permission assignment APIs are not available for this workspace.
│
│ with databricks_permission_assignment.assign_scim_admingroup,
│ on Dbricks-permission.tf line 104, in resource "databricks_permission_assignment" "assign_scim_admingroup":
│ 43: resource "databricks_permission_assignment" "assign_scim_admingroup" {
My expectation is Databricks group synced with AAD via SCIM connecter provisioner groups should be assigned as "ADMIN" role using terraform.

I tried to reproduce the same in my environment:
Code:
resource "azuread_group" "example" {
display_name = "kavyaMyGroup"
owners = [data.azuread_client_config.current.object_id]
security_enabled = true
members = [
azuread_user.example.object_id,
# more users
]
}
resource "databricks_user" "me" {
// user_name = "testuser#databricks.com"
user_name = azuread_user.example.user_principal_name
display_name = "Test User"
}
resource "databricks_group" "this" {
// display_name = "vsakaSomeGroup"
display_name = azuread_group.example.display_name
allow_cluster_create = true
allow_instance_pool_create = true
workspace_access = true
databricks_sql_access = true
}
resource "databricks_group_member" "vip_member" {
group_id = databricks_group.this.id
member_id = databricks_user.me.id
}
I received the same error:
│ Error: cannot create permission assignment: Permission assignment APIs are not available for this workspace.
│
│ with databricks_permission_assignment.assign_scim_admingroup,
│ on main.tf line 145, in resource "databricks_permission_assignment" "assign_scim_admingroup":
Please note :
The admins group is a reserved group in Azure Databricks and cannot be
removed. Note that Workspace-local groups cannot be granted
access to data in a Unity Catalog metastore or assigned to other
workspaces .
To add groups to a workspace using the account console,
the workspace must be enabled for identity federation. Only account-level groups are assignable.
Manage groups - Azure Databricks | Microsoft Learn
If I checked my environment, the group created is local and not
account level , so I did not have permissions to assign admin role to
it.
The account admins can assign them using ,the principal ID which can be retrieved using the SCIM API.
resource "databricks_permission_assignment" "assign_scim_admingroup" {
/principal_id = databricks_group.this.id
permissions = ["ADMIN"]
}
Make sure to enable identity federation , to assign group roles and have premium plan in order to manage the assignment of users to workspaces
You can also check Automate SCIM provisioning using Microsoft Graph from
Reference: Configure SCIM provisioning using Microsoft Azure Active Directory - Azure Databricks | Microsoft Learn

Related

How to setup an emergency access accounts in Azure AD using Terraform?

I have configured the emergency access accounts in Azure AD following the article
https://learn.microsoft.com/en-us/azure/active-directory/roles/security-emergency-access however I want this to be configured using the Terraform script.
How to setup an emergency access accounts in Azure AD using Terraform?
First You need a create a App Registration, add the "Owner" role assignment for the App Registration. Use tenant_id, client_id, client_secret in the provider block.
Follow the link to create a App Registration: https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app
Provide Required Api Permissions to the App Reg. Once you Create the App Registration and client secret(Store in a Key Vault Secret as it will be shown only one time). Open Subscription>Access control (IAM)>Add Role assignments>Owner> Select members(select your app registration)> Next>finish.
API Permissions Screenshot :
[App Registration API Permissions][1]
It's fine if you have already created a App Registration. App Registration must have owner role assignments and domain, directory role assignment read write permissions.
Use the below code you can able to create a User and Assign a role to the user.
terraform {
required_version = ">= 1.0.0"
required_providers {
azuread = {
source = "hashicorp/azuread"
version = "~> 2.0"
}
}
}
provider "azuread" {
#Configuration options
tenant_id = "tenant id of your"
client_id = "use client ID of your"
client_secret = "use client Secret of your"
}
resource "azuread_user" "aduser" {
user_principal_name = "azure.terraform#gmohankumar0341hotmail.onmicrosoft.com"
display_name = "azterraform"
mail_nickname = "aztf"
password = "Terraform#123"
country = "India"
}
resource "azuread_directory_role" "ad-role" {
display_name = "Global administrator"
depends_on = [
azuread_user.aduser
]
}
resource "azuread_directory_role_assignment" "aduser" {
role_id = azuread_directory_role.ad-role.template_id
principal_object_id = azuread_user.aduser.object_id
depends_on = [
azuread_user.aduser
]
}
[terraform apply command Screenshot][2]
[User created Reference Screenshot][3]
[Assigned roles Screenshot][4]
[1]: https://i.stack.imgur.com/KBr7y.png
[2]: https://i.stack.imgur.com/9zhP3.png
[3]: https://i.stack.imgur.com/6tlCe.png
[4]: https://i.stack.imgur.com/MScgE.png

Azure Terraform Build Generic Components using Infrastructure as Code

I am new to Terraform and Azure. I am trying to build a Resource Group / Resources using Terraform. Below is the design for the same.
I have written Terraform code to build Log Analytics workspace and Automation account.
Now below are my questions :
Cost Mgmt / Azure Monitor / Network Watcher / Defender for Cloud ? Can I build all these using Terraform code in this resource group or they need to manually built from Azure portal. When we create any resource on the left hand side options like Cost estimator / management are already available. Does that mean they can be easily selected from there on usage and no need to build from Terraform code ?
How does we apply Role Entitlement / Policy Assignment from Terraform code ?
Here is my code what I have written to build Automation account / Log Analytics
terraform {
required_version = ">=0.12"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~>2.0"
}
}
}
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "management" {
# Mandatory resource attributes
name = "k8s-log-analytics-test"
location = "eastus"
}
resource "random_id" "workspace" {
keepers = {
# Generate a new id each time we switch to a new resource group
group_name = azurerm_resource_group.management.name
}
byte_length = 8
}
resource "azurerm_log_analytics_workspace" "management" {
# Mandatory resource attributes
name = "k8s-workspace-${random_id.workspace.hex}"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
# Optional resource attributes
retention_in_days = 30
sku = "PerGB2018"
}
resource "azurerm_log_analytics_solution" "management" {
# Mandatory resource attributes
solution_name = "mgmyloganalytsolution"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
workspace_resource_id = azurerm_log_analytics_workspace.management.id
workspace_name = azurerm_log_analytics_workspace.management.name
plan {
publisher = "Microsoft"
product = "OMSGallery/ContainerInsights"
}
}
resource "azurerm_automation_account" "management" {
# Mandatory resource attributes
name = "mgmtautomationaccount"
location = azurerm_resource_group.management.location
resource_group_name = azurerm_resource_group.management.name
sku_name = "Basic"
}
resource "azurerm_log_analytics_linked_service" "management" {
# Mandatory resource attributes
resource_group_name = azurerm_resource_group.management.name
workspace_id = azurerm_log_analytics_workspace.management.id
read_access_id = azurerm_automation_account.management.id
}
Cost Mgmt / Azure Monitor / Network Watcher / Defender for Cloud ? Can
I build all these using Terraform code in this resource group or they
need to manually built from Azure portal. When we create any resource
on the left hand side options like Cost estimator / management are
already available. Does that mean they can be easily selected from
there on usage and no need to build from Terraform code ?
Yes , you can create Network Watcher , Azure Monitor resources & Cost Management using terraform resource blocks as azurerm_network_watcher , azurerm_network_watcher_flow_log ,azurerm_monitor_metric_alert ... , azurerm_resource_group_cost_management_export, azurerm_consumption_budget_resource_group etc. Defender for Cloud can't be built from terraform . Yes you are correct , cost management ,monitoring etc are also available on portal but there is a need for its resources to be created like budget alert etc. for simplification it has been added as a blade in portal.
How does we apply Role Entitlement / Policy Assignment from Terraform
code ?
You can use azurerm_role_assignment to assign built-in roles and use azurerm_role_definition to create a custom role and then assign it . For Policy assignment you can use this azurerm_resource_policy_assignment and remediate using azurerm_policy_insights_remediation.
For all the azure resource block you can refer the Official Registry Documentation of Terraform AzureRM Provider & Terraform AzureAD Provider.

Bootstraping an Azure service account in Terraform

I am trying to write the Terraform to create an Azure "service account" and am getting quite confused by the distinction between what Azure AD calls "Applications" and "Service Principals". Effectively, I'm trying to mimic the following Azure CLI call:
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/${subscription_id}"
The idea would be for a human administrator to run the Terraform, to set it up once, then those credentials could later be used to authenticate for the remaining IaC. (i.e., It's a bootstrapping exercise.)
I wish to do it in Terraform, rather than a Bash script, as it seems more explicit and fits with the rest of my IaC. This is what I have so far:
data "azurerm_subscription" "current" {}
data "azuread_client_config" "current" {}
resource "azuread_application" "terraform" {
display_name = "Terraform"
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_password" "terraform" {
application_object_id = azuread_application.terraform.object_id
}
# resource "azuread_service_principal" "terraform" {
# application_id = azuread_application.terraform.application_id
# owners = [data.azuread_client_config.current.object_id]
# }
#
# resource "azuread_service_principal_password" "terraform" {
# service_principal_id = azuread_service_principal.terraform.object_id
# }
resource "local_file" "azurerc" {
filename = ".azurerc"
file_permission = "0600"
content = <<EOF
export ARM_ENVIRONMENT="public"
export ARM_SUBSCRIPTION_ID="${data.azurerm_subscription.current.subscription_id}"
export ARM_TENANT_ID="${data.azuread_client_config.current.tenant_id}"
export ARM_CLIENT_ID="${azuread_application.terraform.application_id}"
export ARM_CLIENT_SECRET="${azuread_application_password.terraform.value}"
EOF
}
This runs, but later authenticating with the generated credentials gives an authentication error. Specifically, Terraforms says:
If you are accessing as application please make sure service principal is properly created in the tenant.
Clearly I haven't done that -- it's commented out in the above snippet -- but that's because this is where my understanding starts to break down. Why do I need both? Why do both the application and the service principal have password resources? If I generate passwords for both, which is the ARM_CLIENT_SECRET (I think the application password is the right one)? Then there's the role assignment: I see there's an azuread_app_role_assignment resource, but I'm having trouble unpicking it.
I am trying to write the Terraform to create an Azure "service
account" and am getting quite confused by the distinction between what
Azure AD calls "Applications" and "Service Principals".
Applications can be seen from Azure AD App registrations blade Where as Service Principals are other wise know as Enterprise Applications. The difference is well documented in this Microsoft Documentation.
This runs, but later authenticating with the generated credentials
gives an authentication error. Specifically, Terraforms says:
If you are accessing as application please make sure service principal
is properly created in the tenant.
This is because you have no associated service principal to that azure ad application which you have created from Terraform. The association is needed to access the application or authenticating to the azure environment with contributor role. When a App registration is created from portal it by default creates an association of AD app and Service principal , which by default results in creating a service principal for that app registration. It also applies the same concept when we use az ad sp create-for-rbac.
Effectively, I'm trying to mimic the following Azure CLI call:
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/${subscription_id}"
You can use the below to mimic the above command :
provider "azurerm" {
features{}
}
provider "azuread" {}
data "azurerm_subscription" "current" {}
data "azuread_client_config" "current" {}
resource "azuread_application" "terraform" {
display_name = "Ansumantest"
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_password" "terraform" {
application_object_id = azuread_application.terraform.object_id
}
resource "azuread_service_principal" "terraform" {
application_id = azuread_application.terraform.application_id
owners = [data.azuread_client_config.current.object_id]
}
resource "azurerm_role_assignment" "example" {
scope = data.azurerm_subscription.current.id
role_definition_name = "Contributor"
principal_id = azuread_service_principal.terraform.object_id
}
resource "local_file" "azurerc" {
filename = ".azurerc"
file_permission = "0600"
content = <<EOF
export ARM_ENVIRONMENT="public"
export ARM_SUBSCRIPTION_ID="${data.azurerm_subscription.current.subscription_id}"
export ARM_TENANT_ID="${data.azuread_client_config.current.tenant_id}"
export ARM_CLIENT_ID="${azuread_application.terraform.application_id}"
export ARM_CLIENT_SECRET="${azuread_application_password.terraform.value}"
EOF
}
Output :
Using the above details I created a reosurce group in the subscription :
provider "azurerm" {
features{}
subscription_id = "88073b30-cadd-459e-b90b-8442c93573ae"
tenant_id = "ab078f81-xxxx-xxxx-xxxx-620b694ded30"
client_id = "c022ec46-xxxx-xxxx-xxxx-c72a9b82f429"
client_secret = "wdV7Q~8Grxxxxxxxxxxxxxx~SCwbRrKIq9"
}
resource "azurerm_resource_group" "name" {
name = "testansterraform"
location = "west us 2"
}

How can I create Azure AD group memberships using Terraform?

I'm trying to create Group Memberships in an Azure AD group for the purpose of inheriting the other group's access
I can do this in the Azure portal here:
Azure Portal Group Membership Blade
But I can't figure out how to do this with powershell/azure-cli or terraform
Is this even possible ? If not, is there a workaround for it ?
You can give it a try using the Terraform Azure AD provider. We use it for user membership but I see there's support for a group object as well.
data "azuread_user" "example" {
user_principal_name = "jdoe#hashicorp.com"
}
resource "azuread_group" "example" {
name = "my_group"
}
resource "azuread_group_member" "example" {
group_object_id = azuread_group.example.id
member_object_id = data.azuread_user.example.id
}
member_object_id - (Required) The Object ID of the Azure AD Object you want to add as a Member to the Group. Supported Object types are Users, Groups or Service Principals. Changing this forces a new resource to be created.
So you would use something like this instead:
data "azuread_group" "example" {
display_name = "MyGroupName"
security_enabled = true
}
resource "azuread_group_member" "example" {
group_object_id = azuread_group.example.id
member_object_id = data.azuread_group.example.id
}
Azure support got back to me with a solution:
There's 2 ways to do this with powershell:
With the msonline module (this does not directly work in powershell core)
> Add-MsolGroupMember -GroupObjectId $groupid -GroupMemberType User -GroupMemberObjectId $userid
With the azuread module (this works directly in powershell core)
> Add-AzADGroupMember -MemberObjectId $targetGroupIdValue --TargetGroupObjectId $groupIdValue
Note, this solution adds a member to the other group for no apparent reason, might be a bug

How can I use Terraform to create a service principal and use that principal in a provider?

I have read the write-ups online but they dont seem to cover this topic completely and was hoping someone who has done it may have some direction for me.
We are setting up a complicated Terraform template to satisfy our IaC requirements relating to our SaaS offering. In doing so we want the template to use the user's credentials at launch to create a new service principal in Azure AD (This part I have no problem doing). Then in the next portion of the template we are using that service principal as the provider. Problem is that it throws errors in the plan/apply because the service principal doesnt exist (aka the id is non existent due to the service provider section not running yet).
So is there a way that I can do this? Create a service principal and then us it in a provider alias that uses that service principal without splitting this into multiple templates?
In the end I want this template to create the service provider using the local user's permissions or MSI, give it RBAC to a subscription, then use that service provider to create assets in that subscription.
main.ts (root)
provider "azurerm" {
alias = "ActiveDirectory"
subscription_id = "${var.subscriptionNucleus}"
}
provider "azurerm" {
alias = "Infrastructure"
subscription_id = "${var.subscriptionInfrastructure}"
}
module "activedirectory" {
providers = { azurerm = "azurerm.ActiveDirectory"
}
source = "./modules/activedirectory"
subscription_id_infrastructure = "${var.subscriptionInfrastructure}"
}
module "infrastructure" {
providers = { azurerm = "azurerm.Infrastructure"}
source = "./modules/infrastructure"
location = "${var.location}"
application_id =
"${module.activedirectory.service_principal_application_id}"
subscription_id = "${var.subscriptionInfrastructure}"
prefix = "${var.prefix}"
}
main.ts (./modules/infrastructure)
data "azurerm_azuread_service_principal" "serviceprincipal" {
application_id = "${var.application_id}"
}
provider "azurerm" {
alias = "InfrastructureSP"
subscription_id = "${var.subscription_id}"
client_id = "${var.application_id}"
client_secret = "secret"
tenant_id =
"${data.azurerm_client_config.clientconfig.tenant_id}"
}
For Azure Service Principal, there are two ways to use the service principal.
First: If you already have a service principal and want to use it in the Terraform. You can make use of the Terraform Data and the test like this:
data "azurerm_azuread_service_principal" "sp" {
application_id = "21f3e1de-54e2-4951-9743-c280ad7bd74a"
}
output "test" {
value = "${data.azurerm_azuread_service_principal.sp.id}"
}
The screenshot of the result is here:
Second: You don't have the service principal and you can just create a service principal in the Terraform like this:
resource "azurerm_azuread_service_principal" "test" {
application_id = "${azurerm_azuread_application.test.application_id}"
}
resource "azurerm_azuread_service_principal_password" "test" {
service_principal_id = "${azurerm_azuread_service_principal.test.id}"
value = "your pasword"
end_date = "2020-01-01T01:02:03Z"
}
Then, no matter which way you choose, there is an important step you should do for most resources. The step is that you need to create the role to give the permission and then assign it to the resource which needs. You can do that like this:
resource "azurerm_role_assignment" "test" {
scope = "yourScope" # the resource id
role_definition_name = "the Role In need" # such as "Contributor"
principal_id = "your service principal id"
}
Hope this will help you.
There is currently no working "depends_on" that works with modules that is not a hack (null_reference). This means that if you are breaking your template into modules(separating concerns) your order of operation is required to be correct to complete this successfully as one module will not know that the data source of service provider has to wait on the previous module to complete. I have had to break this into 2 separate templates where the first creates the service principal and the second has the modular separation that can then use a data source of azurerm_azuread_service_principal.
Once Hashicorp can implement the module depends_on, this will become easier.

Resources