I am trying to use the Azure Identity package to access Key Vault secrets. I am using AzureML and it has its own system assigned managed identity ("Identity" in the left-hand blade).
This system assigned managed identity yields me an Object (principal) ID. It also allows me to set Azure role assignments. This managed identity is a: (1) contributor, (2) administrator, and (3) key vaults secret user for the key vault service I want to use with my secrets.
Inside AzureML, on a Python notebook, if I run:
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
# load key vault data
credential = DefaultAzureCredential()
secret_client = SecretClient(vault_url="--KV URL--", credential=credential)
secret_client.get_secret("secret-i-want")
# fails
EnvironmentCredential.get_token failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Fair enough, it tells me to basically create a service principal and put its details into the environment. But, I don't want to. I want to use the Managed Identity which is apparently best in class for security because I am not storing secrets anywhere.
Looking through documentation, there exists a way to allegedly use the Managed Identity, but I cannot make it work. https://azuresdkdocs.blob.core.windows.net/$web/python/azure-identity/1.6.0/azure.identity.html#azure.identity.ManagedIdentityCredential
I am uncertain how to get a client ID for AzureML or its managed identity.
from azure.identity import ManagedIdentityCredential
credential2 = ManagedIdentityCredential(identity_config={"object_id": "principal-id-for-aml-managed-identity"})
secret_client = SecretClient(vault_url="--KV URL--", credential=credential2)
# hangs with no error
I have tested in my environment.
You can use AzureCliCrendential instead of ManagedIdentityCredential.
In the terminal, login using below command to login using System Assigned Identiy
az login --identity
Now, use the below pyhton notebook script :
from azure.identity import AzureCliCredential
from azure.keyvault.secrets import SecretClient
managed_identity = AzureCliCredential()
secret_client = SecretClient(vault_url="https://radapakv.vault.azure.net/", credential=managed_identity)
value = secret_client.get_secret("test")
Related
I have a setup as below.
Azure App Service connects to Azure Key Vault using Azure-Identity Python SDK. Azure KV's data plane has access policy enabled to include App Service's managed identity. I have used the following code to connect to KV from app service webjobs.
import os
import json
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient
keyVaultName = os.getenv('KeyVaultNm')
KVUri = f"https://{keyVaultName}.vault.azure.net"
credential = DefaultAzureCredential()
_sc = SecretClient(vault_url=KVUri, credential=credential)
srv = _sc.get_secret("server-nm").value
print(f'KV Secret: {srv}')
This setup had been working without any issues for almost a year to allow App service extract secrets from KV but over the last weekend, it started failing with the following error.
[08/29/2022 09:14:14 > 6f77be: ERR ] ManagedIdentityCredential.get_token failed: <urllib3.connection.HTTPConnection object at 0x00000234348D8AC8>: Failed to establish a new connection:
[WinError 10049] The requested address is not valid in its context
[08/29/2022 09:14:14 > 6f77be: ERR ] DefaultAzureCredential failed to retrieve a token from the included credentials.
[08/29/2022 09:14:14 > 6f77be: ERR ] Attempted credentials:
Any idea what's this error is regarding? Will this happen due to any firewall issue?
PS: The same code works well inside a VM using its managed identity and retrieves secrets from KV. The issue happens only when the code is run from Webjobs.
I am following the Microsoft documentation to retrieve secrets from a key vault using python sdk.
The code and explanation offered by Microsoft leads to this code:
import os
import cmd
from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential
keyvault_name = f'https://<Keyvaultname>.vault.azure.net/'
KeyVaultName = "<Keyvaultname>"
credential = DefaultAzureCredential()
client = SecretClient(vault_url=keyvault_name, credential=credential)
print(" done.")
print(f"Retrieving your secret from {KeyVaultName}.")
retrieved_secret = client.get_secret("test")
print(f"Your secret is '{retrieved_secret.value}'.")
According to my understanding, the DefaultCredentials are the one configured in the az login which is fine, my code runs just fine but I keep getting this message in the terminal.
done.
Retrieving your secret from <KeyvaultName>.
EnvironmentCredential.get_token failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
ImdsCredential.get_token failed: ManagedIdentityCredential authentication unavailable, no managed identity endpoint found.
ManagedIdentityCredential.get_token failed: ManagedIdentityCredential authentication unavailable, no managed identity endpoint found.
SharedTokenCacheCredential.get_token failed: SharedTokenCacheCredential authentication unavailable. Multiple accounts
were found in the cache. Use username and tenant id to disambiguate.
I presume that this warnings are due the fact that I have multiple subscription in my azure portal.
I was wondering, how can I get rid of those and set the credentials for only a single subscription?
Thank you so much for any help and explanation you can offer me.
Generally speaking, I would not worry about this warning. When you use DefaultAzureCredential, SDK tries the following credential options in that order (Reference):
EnvironmentCredential
ManagedIdentityCredential
SharedTokenCacheCredential
VisualStudioCredential
VisualStudioCodeCredential
AzureCliCredential
AzurePowerShellCredential
InteractiveBrowserCredential
SDK moves from one credential options to another if that credential option fails. The warning message is just a way for the SDK to tell you what all credential options it has tried.
However if you still want to get rid of this message, there are a few options available to you:
Exclude the credential options that you do not want SDK to try when using DefaultAzureCredential. You can specify those via exclude_xxx_credential option in the constructor. For example, if you want to exclude EnvironmentCredential, you would specify exclude_environment_credential=True in the DefaultAzureCredential constructor. SDK will skip those credential methods. Please see this link for all constructor options.
Use specific credential option. For example, if you always want to use Azure CLI credentials, then instead of using DefaultAzureCredential you can use AzureCliCredential.
I am on an Azure VM with a dynamic IP adress. When I am logged in, I am able to retrieve secrets using the following python code without any issues;
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
credential = DefaultAzureCredential()
secret_client = SecretClient(vault_url="https://xxxx/", credential=credential)
secret = secret_client.get_secret("testSecret")
I need to retrieve the secrets when the VM is on but when I am not logged to enable other processes to run. I noticed the code above was failing when I am logged off. The system admin gave me the AZURE_CLIENT_ID, AZURE_CLIENT_SECRET,AZURE_TENANT_ID and VAULT_URL for me to set them as EnvironmentCredentials.
I set them in the CMD as follows;
SETX AZURE_CLIENT_ID "pppp"
SETX AZURE_CLIENT_SECRET "mmmm"
SETX AZURE_TENANT_ID "kkkk"
SETX VAULT_URL "xxxx"
When I check the system environment settings, I can see they have been set
I tried retrieving my secret using this code,
from azure.keyvault.secrets import SecretClient
VAULT_URL = os.environ["VAULT_URL"]
credential = EnvironmentCredential()
client = SecretClient(vault_url=VAULT_URL, credential=credential)
password = client.get_secret("testSecret").value
I got this error
raise HttpResponseError(response=response, model=error)
azure.core.exceptions.HttpResponseError: (Forbidden) The user, group or application 'pppp;iss=https://sts.windows.net/kkkk/' does not have secrets get permission on key vault 'name of my vault-vault;location=australiasoutheast'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
Question
The system admin confirms the credentials issued are the service principal's correct details.
How can correct this or what am I doing wrong?
Is there a way for me to print DefaultAzureCredentials so that I set the same as EnvironmentCredential because I believe why I recover secrets when I am logged in is that the credentials are cached when I sign in?
Your help will highly be appreciated.
How can correct this or what am I doing wrong?
The error means your service principal does not have the correct secret permission in your keyvault -> Access policies, to solve the issue, add the application(service principal) mentioned in the error message to the Access policies with the Get secret permission in your keyvault in the azure portal. If it still not work, please try to set the environment variables in the System variables instead of User variables for xxx as shown in your screenshot.
Is there a way for me to print DefaultAzureCredentials so that I set the same as EnvironmentCredential because I believe why I recover secrets when I am logged in is that the credentials are cached when I sign in?
No need to do this, the DefaultAzureCredential attempts to authenticate via the following mechanisms in this order, see here. If you didn't set the environment variables before, it should use the managed identity of your VM to authenticate.
I have experimented trying to access Azure Blob Storage using service principal credentials through Python SDK & have some confusions I thought the community could help with.
#1 azure.common.credentials vs azure.identity-------------------------------------------------
I have noticed two different python packages in Azure having credential classes.
- azure.common.credentials
- azure.identity
What is the difference between the two, and when should one be used against other? More specifically, when attempting to work with Azure service principals,
**azure.identity** provides both **ClientSecretCredential & CertificateCredential** so we can use either shared secret, or SSL certificate.
**azure.common.credentials** package provides only the **ServicePrincipalCredentials** class that needs a shared secret, and there is no counterpart for working with certificate credentials.
Am I missing something? I am looking to use certificate based service principal.
#2 ServicePrincipalCredentials works, but ClientSecretCredential fails ------------------------------------------------
My test code to access Azure storage works successfully with ServicePrincipalCredentials class.
But fails with ClientSecretCredential class with Exception message: 'ClientSecretCredential' object has no attribute 'signed_session'"
Appreciate any help with understanding why. There is no difference in the code apart from instantiating the credentials to be one of the two classes above.
The #2 issue above is important mainly because of #1. I am looking to use certificate based Auth, but can't find a supporting class under azure.common.credentials.
Python Environ details:
>python3 --version
Python 3.6.9
>pip3 freeze | grep -i azure
azure-common==1.1.25
azure-core==1.5.0
azure-identity==1.3.1
azure-mgmt-resource==9.0.0
azure-mgmt-storage==10.0.0
azure-storage-blob==12.3.1
msrestazure==0.6.3
snippets from my code:
# for credential classes
from azure.identity import ClientSecretCredential
from azure.identity import CertificateCredential
# for storage & other resource mgmt classes
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.storage import StorageManagementClient
tenant_id = params['tenant-id']
client_id = params['client-id']
client_secret = params['secret']
subscription_id = params['subscription-id']
creds = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret)
# create resource group
resource_client = ResourceManagementClient(creds, subscription_id)
# create storage group, access keys etc
storage_client = StorageManagementClient(creds, subscription_id)
When attempting with certificate rather than secret, here is the code snippet for creating credential instance; rest of code is same.
client_keycert_path = params['cert-path']
creds = CertificateCredential(tenant_id =tenant_id, client_id = client_id, certificate_path = client_keycert_path)
The current situation is misleading, I admit, here's a few details as of today (monitor the situation here https://github.com/Azure/azure-sdk-for-python/issues/9310):
For azure-storage-blob, azure.common is used for storage SDK <= 2.x, and azure-identity is used for storage SDK >= v12.x.
For any package starting with azure-mgmt-xxx, azure-common is still the official way. Check this issue for workaround on how to write mgmt code that uses azure-identity (https://github.com/Azure/azure-sdk-for-python/issues/9310)
This will change SOON, by summer 2020 mgmt SDKs should support azure-identity out of the box.
Hope this helps, feel free to open an issue on Github too if there is further questions:
https://github.com/Azure/azure-sdk-for-python/issues
(I work in the Azure SDK team at MS)
We have a multi-tenant application in our Azure AD tenant. It is authorized in some other tenants (we know which ones). And it has multiple certificates registered to it to be used as client credentials.
We want to remove the certificates from the local stores and use a certificate in the key vault to request a token for one of the external tenant. According to the documentation this is one of the use cases.
Our tenant (id: xxxx):
Has app registration (app id: abcd-xxx-xxxx-xxx)
has keyvault
has managed service principal (with access to the key vault)
other tenant (id: yyyy):
Executed Admin consent for our application.
Question 1:
How do I create a certificate in the Key vault that is connected to an existing application (app id: abcd-xxx-xxxx-xxx)? It is important to note that since the application is already approved by several third party admins, it cannot be recreated. Same counts for creating a new certificate after it would be expired.
Question 2:
How to I setup the Microsoft.Azure.Services.AppAuthentication library to:
Use the managed identity to access the key vault in our tenant (xxxx).
Use the certificate in the key vault to request a token for our app (abcd-xxx-xxxx-xxx) in other companies tenant (yyyy)
Answer 1:
You could use az ad sp credential reset command like below. If you don't want to overwrite the existing certificate of the App, please pass the --append parameter.
az ad sp credential reset --name '<application-id>' --keyvault joykeyvault --cert cer136 --create-cert --append
Answer 2:
1.To use the MSI access the keyvault in your tenant, just use the code below.
No code changes are required, when you run your code on an Azure App Service or an Azure VM with a managed identity enabled, the library automatically uses the managed identity, see this link.
The environment variable AzureServicesAuthConnectionString has to be set to any credential with access to the keyvault. RunAs=Developer; DeveloperTool=AzureCli for dev or RunAs=App; for managed service identity (automatically in azure).
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Azure.KeyVault;
// Instantiate a new KeyVaultClient object, with an access token to Key Vault
var azureServiceTokenProvider1 = new AzureServiceTokenProvider();
var kv = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider1.KeyVaultTokenCallback));
2.If you want to use the service principal along with its certificate stored in the keyvault to get the token for the resources in another tenant, the connection string on the AzureServiceTokenProvider has to be set to RunAs=App;AppId={TestAppId};KeyVaultCertificateSecretIdentifier={KeyVaultCertificateSecretIdentifier} then you can get tokens for other tenants like.
const string appWithCertConnection = "RunAs=App;AppId={TestAppId};KeyVaultCertificateSecretIdentifier=https://myKeyVault.vault.azure.net/secrets/myCert";
Then use the code to get the token, e.g. for the resource https://management.azure.com/.
var azureServiceTokenProvider2 = new AzureServiceTokenProvider(appWithCertConnection);
string accessToken = await azureServiceTokenProvider2.GetAccessTokenAsync("https://management.azure.com/", "tenant-id-of-thridh-party-tenant").ConfigureAwait(false);