Azure-quantum authentication is near to impossible with EnvironmentCredential - azure

The code below to solve a simple problem on azure quantum never works. Microsoft, please help me with authentication.
from azure.quantum.optimization import Problem, ProblemType, Term
from typing import List
from azure.quantum.optimization import Term
from azure.quantum import Workspace
from azure.identity import EnvironmentCredential
from azure.quantum.optimization import ParallelTempering
problem = Problem(name="My First Problem", problem_type=ProblemType.ising)
workspace = Workspace (
subscription_id = "my-subscription-id", # Add your subscription_id
resource_group = "AzureQuantum", # Add your resource_group
name = "my-workspace-name", # Add your workspace name
location = "my-workspace-location" , # Add your workspace location (for example, "westus")
credential = EnvironmentCredential(AZURE_USERNAME="my-email-id", AZURE_PASSWORD="my-microsoft-password")
# credential = ManagedIdentityCredential()
)
terms = [
Term(c=-9, indices=[0]),
Term(c=-3, indices=[1,0]),
Term(c=5, indices=[2,0]),
Term(c=9, indices=[2,1]),
Term(c=2, indices=[3,0]),
Term(c=-4, indices=[3,1]),
Term(c=4, indices=[3,2])
]
problem.add_terms(terms=terms)
solver = ParallelTempering(workspace, timeout=100)
result = solver.optimize(problem)
print(result)
The above code throws the error:
EnvironmentCredential.get_token failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.
---------------------------------------------------------------------------
CredentialUnavailableError Traceback (most recent call last)
[<ipython-input-19-90cd448f8194>](https://localhost:8080/#) in <module>()
3 solver = ParallelTempering(workspace, timeout=100)
4
----> 5 result = solver.optimize(problem)
6 print(result)
19 frames
[/usr/local/lib/python3.7/dist-packages/azure/identity/_credentials/environment.py](https://localhost:8080/#) in get_token(self, *scopes, **kwargs)
141 "this issue."
142 )
--> 143 raise CredentialUnavailableError(message=message)
144 return self._credential.get_token(*scopes, **kwargs)
CredentialUnavailableError: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
Details
The above code works perfectly fine when I do not pass any credentials to the workspace, but this pops up a window for authentication. I do not want to click manually on the browser every time I run something to autheticate. I just want to pass the credentials in code with ease without having to deal with all the complicated things defined for the authentication in the docs.
Note: I'm passing my email and password in the EnvironmentCredential (I'm obviously not writing confidential info here like subscription id passed in workspace)

The EnvironmentCredential takes its parameters from environment variables -not from arguments in the constructor-, so in order to avoid getting prompted for credentials you would need to set the corresponding environment variables with the correct values, then run your program. Something like:
set AZURE_USERNAME="my-email-id"
set AZURE_PASSWORD="my-microsoft-password"
python myprogram.py
in your code you would then do something like:
workspace = Workspace (
subscription_id = "my-subscription-id", # Add your subscription_id
resource_group = "AzureQuantum", # Add your resource_group
name = "my-workspace-name", # Add your workspace name
location = "my-workspace-location" , # Add your workspace location (for example, "westus")
credential = EnvironmentCredential()
)
That being said, an easier way to not get prompted for credentials is to install the Azure CLI and login using az login; that will prompt once for credentials and then persist them locally in your machine so you don't have to login again.
Optionally, if you are using VS Code you can install the Azure Account extension.
For all these options, you don't need to provide any credential in the AzureQuantum constructor. It will automatically try to discover if you've set the environment variables, used the CLI or the extension to log in and used that. It only defaults to prompt on the browser if it can't find anything else.

Related

How to retrieve the star counts in GitLab Python API?

I try to request the number of stars and commits of a public software repository in GitLab using its Python client. However, I keep getting GitlabHttpError 503 if executing the following script.
import gitlab
import requests
url = 'https://gitlab.com/juliensimon/huggingface-demos'
private_token = 'xxxxxxxx'
gl = gitlab.Gitlab(url, private_token=private_token)
all_projects = gl.projects.list(all=True)
I read the previous posts but none of them works for me: [1], [2], and [3]. People mentioned:
Retrying later usually works [I tried in different periods but still got the same error.]
Setting an environment variable for no_proxy [Not sure what it means for me? I do not set the proxy explicitly.]
The 503 response is telling you something - your base URL is off. You only need to provide the base GitLab URL so the client makes requests against its api/v4/ endpoint.
Either use https://gitlab.com only, so that the client will correctly call https://gitlab.com/api/v4 endpoints (instead of trying https://gitlab.com/juliensimon/huggingface-demos/api/v4 as it does now), or skip it entirely when using GitLab.com if you're on python-gitlab 3.0.0 or later.
# Explicit gitlab.com
url = 'https://gitlab.com'
gl = gitlab.Gitlab(url, private_token=private_token)
# Or just use the default for GitLab.com (python-gitlab 3.0.0+ required)
gl = gitlab.Gitlab(private_token=private_token)
Edit: The original question was about the 503, but the comment to my answer is a follow-up on how to get project details. Here's the full snippet, which also fetches the token from the environment instead:
import os
import gitlab
private_token = os.getenv("GITLAB_PRIVATE_TOKEN")
gl = gitlab.Gitlab(private_token=private_token)
project = gl.projects.get("juliensimon/huggingface-demos")
print(project.forks_count)
print(project.star_count)

Python : Implementing Feature Flags based on Environment (dev, prod)

I would like the features to be based on the environment. For example a feature is being worked or tested so I could have it on in DEV , but it's not ready for the public, so it's turned off in PROD.
Do I need to implement a custom strategy or can I use one of the existing strategies in a creative way?
If there any concise example that would be most helpful.
The easiest way I’ve found to implement feature flags at the environment level is to use a third-party hosted management system. Many feature flag services allow you to control which environment a flag will be enabled in. In the example below, I used DevCycle’s Python SDK, referencing feature flags and environments I had created in the DevCycle dashboard.
First, I installed the SDK from the command line:
$ pip install devcycle-python-server-sdk
Then I initialized the SDK it with the SDK key that corresponded to my desired environment. In this case, I used the key for my Dev environment. DevCycle provides SDK keys for each environment you set up.
from __future__ import print_function
from devcycle_python_sdk import Configuration, DVCClient
from devcycle_python_sdk.rest import ApiException
configuration = Configuration()
# Set up authorization
configuration.api_key['Authorization'] = 'SDK_KEY_FOR_DEV_ENV'
# Create an instance of the API class
dvc = DVCClient(configuration)
# Create user object. All functions require user data to be an instance of the UserData class
user = UserData(
user_id='test'
)
key = 'enable-feature-flag' # feature flag key created in the DevCycle Dashboard
try:
# Fetch variable values using the identifier key, with a default value and user object
# The default value can be of type string, boolean, number, or JSON
flag = dvc.variable(user, key, False)
# Use received value of feature flag.
if flag.value:
# Put feature code here, or launch feature from here
else:
# Put existing functionality here
except ApiException as e:
print("Exception when calling DVCClient->variable: %s" %e)
By passing in the SDK key for my Dev environment, 'SDK_KEY_FOR_DEV_ENV', it gives my program access to only the features enabled in Dev. You can choose which environment(s) a feature is enabled in directly from the DevCycle dashboard. So if 'enable-feature-flag' was set to true for your Dev environment, you would see your feature. Likewise, you could set 'enable-feature-flag' to false in your Prod environment, and replace 'SDK_KEY_FOR_DEV_ENV' with the key for your Prod environment. This would disable the new functionality from Prod.
Full disclosure: My name is Sandrine and I am a Developer Advocate for DevCycle. I hope this answer helps you get started on environment-specific feature flags.

How to get reference to AzureML Workspace Class in scoring script?

My scoring function needs to refer to an Azure ML Registered Dataset for which I need a reference to the AzureML Workspace object. When including this in the init() function of the scoring script it gives the following error:
"code": "ScoreInitRestart",
"message": "Your scoring file's init() function restarts frequently. You can address the error by increasing the value of memory_gb in deployment_config."
On debugging the issue is:
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code [REDACTED] to authenticate.
How can I resolve this issue without exposing Service Principal Credentials in the scoring script?
I found a workaround to reference the workspace in the scoring script. Below is a code snippet of how one can do that -
My deploy script looks like this :
from azureml.core import Environment
from azureml.core.model import InferenceConfig
#Add python dependencies for the models
scoringenv = Environment.from_conda_specification(
name = "scoringenv",
file_path="config_files/scoring_env.yml"
)
#Create a dictionary to set-up the env variables
env_variables={'tenant_id':tenant_id,
'subscription_id':subscription_id,
'resource_group':resource_group,
'client_id':client_id,
'client_secret':client_secret
}
scoringenv.environment_variables=env_variables
# Configure the scoring environment
inference_config = InferenceConfig(
entry_script='score.py',
source_directory='scripts/',
environment=scoringenv
)
What I am doing here is creating an image with the python dependencies(in the scoring_env.yml) and passing a dictionary of the secrets as environment variables. I have the secrets stored in the key-vault.
You may define and pass native python datatype variables.
Now, In my score.py, I reference these environment variables in the init() like this -
tenant_id = os.environ.get('tenant_id')
client_id = os.environ.get('client_id')
client_secret = os.environ.get('client_secret')
subscription_id = os.environ.get('subscription_id')
resource_group = os.environ.get('resource_group')
Once you have these variables, you may create a workspace object using Service Principal authentication like #Anders Swanson mentioned in his reply.
Another way to resolve this may be by using managed identities for AKS. I did not explore that option.
Hope this helps! Please let me know if you found a better way of solving this.
Thanks!
Does your score.py include a Workspace.get() with auth=InteractiveAuthentication call? You should swap it to ServicePrincipalAuthentication (docs) to which you pass your credentials ideally through environment variables.
import os
from azureml.core.authentication import ServicePrincipalAuthentication
svc_pr_password = os.environ.get("AZUREML_PASSWORD")
svc_pr = ServicePrincipalAuthentication(
tenant_id="my-tenant-id",
service_principal_id="my-application-id",
service_principal_password=svc_pr_password)
ws = Workspace(
subscription_id="my-subscription-id",
resource_group="my-ml-rg",
workspace_name="my-ml-workspace",
auth=svc_pr
)
print("Found workspace {} at location {}".format(ws.name, ws.location))
You can get the workspace object directly from your run.
from azureml.core.run import Run
ws = Run.get_context().experiment.workspace
I came across the same challenge. As you are mentioning AML Datasets, I assume an AML Batch Endpoint is suitable to your scenario. The scoring script for a batch endpoint is meant to receive a list of files as input. When invoking the batch endpoint, you can pass (among the others) AML Datasets (consider that an endpoint is deployed in the context of an AML workspace). Have a look to this.
When Running on AML Compute CLuster use the following code
from azureml.pipeline.core import PipelineRun as Run
run = Run.get_context()
run.experiment.workspace
ws = run.experiment.workspace
Note: This works only when you run on AML Cluster
Run.get_context() gets the conext of total AML cluster from that object we can extract workspace context which allows you to authenticate AML workspace with AML cluster

Give write access to deploy key for new project created via gitpython

I have created a script using gitpython/python3 that creates a new project in a group and assigns our deploy key to the project. This all works perfectly, but I am not sure how to give the assigned key write access from the script. The relevant parts are below:
def add_deploy_key(project_id):
home = os.path.expanduser('~')
gl = gitlab.Gitlab.from_config('redacted', ['%s/.python-gitlab.cfg' % home])
project = gl.projects.get(project_id)
DEPLOY_KEY_ID = redacted
try:
project.keys.enable(DEPLOY_KEY_ID)
print("Your project has been created.")
except:
print("Failed to assign deploy key.")
Again, this works but does not give the assigned key write access to the new project. Also please excuse my sloppy error handling - not done yet.
Looked through gitpython documentation with no luck.
No error as the existing code works, but write access is not enabled.

Get list of application packages available for a batch account of Azure Batch

I'm making a python app that launches a batch.
I want, via user inputs, to create a pool.
For simplicity, I'll just add all the applications present in the batch account to the pool.
I'm not able to get the list of available application packages.
This is the portion of code:
import azure.batch.batch_service_client as batch
from azure.common.credentials import ServicePrincipalCredentials
credentials = ServicePrincipalCredentials(
client_id='xxxxx',
secret='xxxxx',
tenant='xxxx',
resource="https://batch.core.windows.net/"
)
batch_client = batch.BatchServiceClient(
credentials,
base_url=self.AppData['CloudSettings'][0]['BatchAccountURL'])
# Get list of applications
batchApps = batch_client.application.list()
I can create a pool, so credentials are good and there are applications but the returned list is empty.
Can anybody help me with this?
Thank you,
Guido
Update:
I tried:
import azure.batch.batch_service_client as batch
batchApps = batch.ApplicationOperations.list(batch_client)
and
import azure.batch.operations as batch_operations
batchApps = batch_operations.ApplicationOperations.list(batch_client)
but they don't seem to work. batchApps is always empty.
I don't think it's an authentication issue since I'd get an error otherwise.
At this point I wonder if it just a bug in the python SDK?
The SDK version I'm using is:
azure.batch: 4.1.3
azure: 4.0.0
This is a screenshot of the empty batchApps var:
Is this the link you are looking for:
Understanding the application package concept here: https://learn.microsoft.com/en-us/azure/batch/batch-application-packages
Since its python SDK in action here: https://learn.microsoft.com/en-us/python/api/azure-batch/azure.batch.operations.applicationoperations?view=azure-python
list operation and here is get
hope this helps.
I haven't tried lately using the Azure Python SDK but the way I solved this was to use the Azure REST API:
https://learn.microsoft.com/en-us/rest/api/batchservice/application/list
For the authorization, I had to create an application and give it access to the Batch services and the I programmatically generated the token with the following request:
data = {'grant_type': 'client_credentials',
'client_id': clientId,
'client_secret': clientSecret,
'resource': 'https://batch.core.windows.net/'}
postReply = requests.post('https://login.microsoftonline.com/' + tenantId + '/oauth2/token', data)

Resources