How to create a new metastore? - databricks

I want to configure Unity Catalog and one step is creating a metastore in the region where I create databricks workspace (I am on Azure).
I created a workspace with a premium pricing tier and I am the admin.
Following the documentation, I should go to the Data tab to create metastore.
However, when I open the Data tab, I don't see "Create Metastore" button.
The same in SQL persona:
Could you guide me how to make a new metastore?
If a metastore is already created in the region, how can I find it?

In order to do this sort of management, you should access the Databricks account portal at the tenant level:
Databricks Account
From there, you can create and manage the metastores, as well as assign a metastore with a Databricks Workspace, which is what you have created.
Take into account that for most of what you have described, you must be an account admin for the Databricks Account.
As per the official docs (source):
The first Azure Databricks account admin must be an Azure Active Directory Global Administrator at the time that they first log in to the Azure Databricks account console. Upon first login, that user becomes an Azure Databricks account admin and no longer needs the Azure Active Directory Global Administrator role to access the Azure Databricks account. The first account admin can assign users in the Azure Active Directory tenant as additional account admins (who can themselves assign more account admins). Additional account admins do not require specific roles in Azure Active Directory.

Configure your Unity Catalog Metastore
Go to + New add click on new notebook and open.
If you already have catalogs with data .then use below command to check,
# Show all catalogs in the metastore.
display(spark.sql("SHOW CATALOGS"))
If you don't have catalog . create utility catalog :
# Create a catalog.
spark.sql("CREATE CATALOG IF NOT EXISTS catalog_name")
# Set the current catalog.
spark.sql("USE CATALOG catalog_name")
for more information refer this offical_document and Notebook.

You must be an Azure Databricks account admin to getting started using Unity Catalog this can be done for first time using Azure Active Directory Global Administrator of your subscription.
As per official documentation:
The first Azure Databricks account admin must be an Azure Active
Directory Global Administrator at the time that they first log in to
the Azure Databricks account console. Upon first login, that user
becomes an Azure Databricks account admin and no longer needs the
Azure Active Directory Global Administrator role to access the Azure
Databricks account. The first account admin can assign users in the
Azure Active Directory tenant as additional account admins (who can
themselves assign more account admins). Additional account admins do
not require specific roles in Azure Active Directory.
How to identify your Microsoft Azure global administrators for your subscriptions?
The global administrator has access to all administrative features. By default, the person who signs up for an Azure subscription is assigned the global administrator role for the directory. Only global administrators can assign other administrator roles.
Login into the Azure Databricks account console via Global admin and then account admin can assign users in the Azure Active Directory tenant.
For more details, refer to Azure Databricks - Get started using Unity Catalog and also refer to MS Q&A thread - How to access Azure Databricks account admin? addressing similar issue.

Related

Azure Databricks Unity Catalogue Create metastore button unavailable

Trying to create a Metastore for manage identity incorporating in Azure Databricks but the data tab only shows create table.
Per the documentation, it should be there. Also, I have created the databricks service and have azure contributor role.
I am an admin to the Databricks workspace. Is it unavailable on Azure?
Well, you don't give details about your environment, so I just can give some ideas about what is missing.
First, change the environment to "SQL" (click on "Data Science & Engineering" menu at the top left)
Second, do you have all the requirements? The requirements are here: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started#requirements
I think do you missing this permission here:
*You must be an Azure Databricks account admin.
The first Azure Databricks account admin must be an Azure Active Directory Global Administrator at the time that they first log in to the Azure Databricks account console. Upon first login, that user becomes an Azure Databricks account admin and no longer needs the Azure Active Directory Global Administrator role to access the Azure Databricks account. The first account admin can assign users in the Azure Active Directory tenant as additional account admins (who can themselves assign more account admins). Additional account admins do not require specific roles in Azure Active Directory.*
To check if you are an Azure Databricks account admin you can access:
https://accounts.azuredatabricks.net/login?next_url=%2Flogin%2F
and verify if you have the access to the Databricks administration screen

Using Azure SQL Migration extension on Azure Data Studio

I have connected my azure account in Data Studio and I am using Azure SQL migration extension (v0.1.12) to migrate on-prem SQL to Azure Managed Instance.
However my subscription details are not getting fetched.
Screen Shot Attached Here
When I manually add Azure Subscription details I am getting following error
Manually Entered Details
And the error message Error
The issue seems to be more of access level issues.
Below are the type of access levels that you need to have for creating Azure Migrate Appliance project
Contributor or Owner permissions in the Azure subscription.
Permissions to register Azure Active Directory (Azure AD) apps.
Owner or Contributor and User Access Administrator permissions in the Azure subscription to create an instance of Azure Key Vault, which is used during agentless server migration.
Below are the steps to set contributor or Owner permissions
From Azure Subscriptions panel select the subscription
Move to Access Control IAM and select Add role Assignment
Assign the following roles.
For complete information check the Microsoft Document on providing access.

How to add an account to Azure Databricks workspace region?

I spun up an instance of Azure Databricks. When I try to login, I get the following error.
The workspace you are trying to access does not exist in this Azure region, or your account vikrambhatt1405#outlook.com does not belong to any Databricks workspace in the region. Please ask your administrator to add you as a user, or click here to logout of Azure Active Directory and login with a different user.
I already have a role assigned to me as Service Administrator, which means access to all resources.
How do I fix this?
Please follow the process below to add users to the Azure Databricks workspace :
1.As an administrator go to the Admin Console.
2.On the Users tab, click Add User.
3.Provide the user email ID.
You can add any user who belongs to the Azure Active Directory tenant
of your Azure Databricks workspace.
4.If cluster access control is enabled, the user is added without cluster creation permission.
please see:here.
Update:
According to your error message:
The workspace you are trying to access does not exist in this Azure region, or your account vikrambhatt1405#outlook.com does not belong to any Databricks workspace in the region.
So,I think your account does not belong In this workspace,you should try to add your account as a user following the instructions of the error message.
The instructions in the document are indeed not very clear, you can follow this process to operate, go to Azure portal>Azure Databricks:
The previous answer and comments led me there, but because it is so awkward I figured it would be good to post as a separate answer:
Instead of clicking on the 'url', click the blue button called 'Launch workspace' to access the cluster.

Can an Azure "group" be used as Azure Sql Database's "Active Directory Admin"?

Azure SQL Database lets you set a user as an Active Directory Admin. This allows you to log into the database using your microsoft account credentials. I would like to use an Azure security group instead of a single user so that multiple people in our domain can be Azure SQL DB admins.
I've tried creating an Azure security group "TestGroup" and using it in the Azure configuration. I can successfully set "TestGroup" as the Active Direcory Admin in the Azure portal for the DB, but when I try to log into the DB using SQL Server Management Studio, it doesn't let me in. I confirmed my user is in this group.
This is the error it returns:
From what you described you did assign the Active Directory admin as group correctly.
I suspect that your problem lies with the logging in method, in order to log in by AAD, you need to select "Azure Active Directory - Universal with MFA" option, and under user name provide email from within the AAD security group. Then the azure login window should pop up.
Azure "groups" can be used as "Active Directory administrators" for Azure Sql databases.
When using Azure Active Directory authentication, put Azure Active
Directory users into an Azure Active Directory security group. Create
a contained database user for the group. Place one or more database
users into a custom database role with specific permissions
appropriate to that group of users.
When using SQL authentication, create contained database users in the
database. Place one or more database users into a custom database role
with specific permissions appropriate to that group of users.
I hope this article will help you here.

Azure SQL - How to add AD users to SQL Pool Databases

Problem: I am trying to add active directory users & groups (synced from on-prem to Azure AD) to SQL databases that exist in a SQL elastic pool on Azure in the same tenant. I do not want to use SQL Server Authentication (integrated). It seems that I am not able to log in with an AD account (required) to add additional AD accounts.
The on premise directory is synced to Azure AD. Tenant is configured and working with O365 email and we now have several SQL databases that exist in our tenant in an elastic pool.
Note: this is not a managed SQL instance. When I manage the elastic pool or database directly, I do not have the menu option Active Directory admin available, as outlined here.
My AD administrator account is added to elastic pool Access control (IAM) as an owner.
Using SSMS to connect to database in cloud, it is possible to connect to database using SQL Server Authentication. When I try to CREATE USER [username#mytenant.com] FROM EXTERNAL PROVIDER I receive error Principal 'username#mytenant.com' could not be created. Only connections established with Active Directory accounts can create other Active Directory users.
The reason for this, also outlined here, is "Users that are not based on an Azure AD account (including the Azure SQL server administrator account), cannot create Azure AD-based users, because they do not have permission to validate proposed database users with the Azure AD."
Using SSMS again and connecting with Active Directory - Universal with MFA support as username#mytenant.com, on login attempt I receive error Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'. (Microsoft SQL Server, Error: 18456). Same results when I configure Connect to database as DBName and AD domain name or tenant ID as mytenant.onmicrosoft.com. I have the same outcome even when using the initial tenant administrator account that we created when setting up environment.
So my question at the end of the day - How do I add an AD user synced from on prem to an Azure SQL database (or elastic pool) so that I can login with that user and begin adding additional AD users to the database?
I am clearly missing something fundamental and just need a point in the right direction. Thanks for any assistance provided.
First create an Active Directory Admin for the pool.
Select your existing Elastic Pool in the portal
From “Overview”, click on your “Server Name”
Select “Active Directory Admin” -> “Set Admin”. You can set a single user or an AD Group as the Administrator.
You can also do this from the CLI:
> az sql server ad-admin create --object-id <Object-Id-OfUserOrGroup>
-s <Database-Name>
-g <Resource-Group>
-u <NameOrEmailAkaDisplayName>
Now, you should now be able to log in to a database in the Elastic Pool with your AD Admin login via SSMS. In this case you can log in via “Active Directory Password Authentication”, but you can also use “Universal Authentication” or “Integrated Authentication”.
Once you’re logged in as the AD Administrator, you can create a new SQL Server USER that corresponds to an AD Group. Please consider I’ve already created an AD group called “SQL Developers” so I can map it to an Azure SQL Server database and add it to db_datareader fixed role like this:
CREATE USER [SQL Developers] FROM EXTERNAL PROVIDER
ALTER ROLE db_datareader ADD MEMBER [SQL Developers]

Resources