I'm trying to figure out if I'm a member of a specific group on Databricks using either the website or Databricks CLI.
I found a reference to managing groups via an account console, but this requires an admin account. I also found the Databricks groups CLI, but the documentation says you need an admin account to use the API.
(Answering my own question.) It turns out that the Databricks groups CLI documentation is incorrect and you can use it to view your own groups as a user without admin access.
The command is:
databricks groups list-parents --user-name <your-user-name>
However, I can't view parents of other users, and I can't access the other databricks groups commands without getting an error message about lacking valid scope.
Related
I have azure databricks workspace and I added service principal in that workspace using databricks cli. I have been trying to create cluster using service principal and not able to figure it. Can any help me?
I am able to create cluster using my account but I want to create using Service Principal and want it to be the owner of the cluster not me.
Also, it there a way I can transfer the ownership of my cluster to Service Principal?
First, answering the second question - no, you can't change the owner of the cluster.
To create a cluster that will have Service Principal as owner you need to execute creation operation under its identity. To do this you need to perform following steps:
Prepare a JSON file with cluster definition as described in the documentation
Set DATABRICKS_HOST environment variable to an address of your workspace:
export DATABRICKS_HOST=https://adb-....azuredatabricks.net
Generate AAD token for Service principal as described in documentation and assign its value to DATABRICKS_TOKEN or DATABRICKS_AAD_TOKEN environment variables (see docs).
Create Databricks cluster using databricks-cli providing name of JSON file with cluster specification (docs):
databricks clusters create --json-file create-cluster.json
P.S. Another approach (really recommended) is to use Databricks Terraform provider to script your Databricks infrastructure - it's used by significant number of Databricks customers, and much easier to use compared with command-line tools.
I'm trying to obtain the owner of a list of jobs on databricks using CLI. The issue is, the command databricks jobs list doesn't have any information related with that, any suggestions?
Thanks in advance!
You need to use Permissions API for that. Specifically the Get Job Permissions Levels API. Just do a GET request against endpoint https://<databricks-instance>/api/2.0/preview/permissions/jobs/{job_id}/permissionLevels - replace {job_id} with actual Job ID
I have two separate Azure accounts.
One for each project in which I am involved, these accounts are totally independent, that is, they do not share any type of resource and do not have the same domain. They are from two totally different companies.
I find that both accounts respond to me at the time of login from PowerShell and I can access those resources.
Both work with Kubernetes (kubectl) but only one of the two accounts is shown. Whatever you do always shows the content of co-owners of one and not the other.
I have the Azure CLI (v.2.0.76) and the PS version is (5.1)
someone know how to I can do?
EDIT with pictures -
Although the account is default, I am not able to access the kubernetes of the same
PS Default Account
Services from the other account..not the default account
services from other cluster
I just found the solution.
When we access from PS with Az Login and select the account, it allows us to access all the resources of that account (the one that is predetermined)
What I have done is basically see the
kubectl config view
This returns the result of all the clusters that it finds with its context. The next thing we have to do is tell kubectl what CONTEXT we want to work with in the following way:
kubectl config use-context "CONTEXT NAME"
And thats it.
I need to generate Azure Databricks token using Powershell script.
I am done with creation of Azure Databricks using ARM template , now i am looking to generate Databricks token using powershell script .
Kindly let me know how to create Databricks token using Powershell script
The only way to generate a new token is via the api which requires you to have a token in the first place.
Or use the Web ui manually.
There is no official powershell commands for databricks, there are some unofficial ones but they still require you to generate a token manually first.
https://github.com/DataThirstLtd/azure.databricks.cicd.tools
Disclaimer I'm the author of these.
UPDATE: these powershell commands can now authenticate using a service principal instead of a bearer token (or can generate a bearer token for you).
so right now there is no way to use the API directly after deploying an Azure Databricks Workspace. I assume that you want to use it as part of an CI/CD pipeline - right? Reason is that you first need to manually create an API token which you can then use for all subsequent API requests.
But I will investigate and keep you updated here!
another option is to create it via terraform.
https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/token
mind you, it creates the token as whomever you az login'd as. so if you az login as yourself (when it spawns a browser asking who to log in as), that's who the token will be created as (assuming that user has permissions in the databricks workspace) and contributor (or custom read role, reader role doesn't grant the right permissions) permissions into the resource group that houses the workspace.
you can always use az login -u username#email.com -p to log in as someone else, assuming that user doesn't have MFA then run the terraform init/plan/apply. mind you, if you have a backend storage, that user also has to have permissions to that backend storage as well so it can create/update any tfstate files stored there.
I am trying to understand how can I run a python application inside azure container that can spin up various resources like VM, route-table inside resource group of Azure.
http://azure-sdk-for-python.readthedocs.io/en/latest/
Looks like the examples provided in azure python sdk sets authentication stuff like credentials, subscription ID, etc.
Is there a way to avoid setting up of this information by using Role Based Access Control ? What if I provide container an Contributor access to the resource group, in that case, would i able to avoid setting up authentication variables and still achieve creating resources or atleast read/access resources in resource group ?
No, how do you think that would work? In order to be able to create\read\modify resources you need permissions (else anyone would be able to do that). There is no way to do that without some sort of Auth.