Local azure function runtime fails to retrieve a token from the included credentials despite being logged in - azure

I'd like to read blobs in a storage container, based off this Quickstart. I've already been assigned the role of Storage Account Contributor. Using VS code, I followed the tutorial (in VS code) by starting with az login, then start my environment localy, and finally executing the function. I get the following warning:
DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Failed to invoke Azure CLI
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
my code, following the tutorial, looks like:
import azure.functions as func
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from azure.identity import DefaultAzureCredential
def main(req=None) -> func.HttpResponse:
account_url = "https://<name>.blob.core.windows.net"
default_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(account_url, credential=default_credential)
container_client = blob_service_client.get_container_client('<name>')
blob_list = container_client.list_blobs()
for b in blob_list:
return func.HttpResponse(f'{b.name}')
Not sure what else I should do if I've already done the az login part.

I tried in my environment and got below results:
DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Failed to invoke Azure CLI
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
The above error suggest you need to define at least one of the credential like environmentcredential,ManagedIdentitycredential,sharedtokencachecredential,AzureCliCredential. to retrieve token.
You can follow this document to retrieve the token with credential.
I followed the document in terminal I tried az login for signing in and used the below code to get the list of blobs using Http trigger function.
Code:
import azure.functions as func
import logging
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from azure.identity import DefaultAzureCredential
def main(req=None) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
account_url = "https://venkat123.blob.core.windows.net"
default_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(account_url, credential=default_credential)
container_name="test"
container_client = blob_service_client.get_container_client(container_name)
#blob_list = container_client.list_blobs()
blobs=[]
for b in container_client.list_blobs():
blobs.append(b.name)
logging.info(b.name)
return func.HttpResponse(f"Listed blobs in container: {blobs}")
The above code executed successfully and listed blobs from the storage container.
Console:
Browser:

Related

Azure App Service to Key Vault - Managed Identity Access Issue using Python

I have a setup as below.
Azure App Service connects to Azure Key Vault using Azure-Identity Python SDK. Azure KV's data plane has access policy enabled to include App Service's managed identity. I have used the following code to connect to KV from app service webjobs.
import os
import json
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient
keyVaultName = os.getenv('KeyVaultNm')
KVUri = f"https://{keyVaultName}.vault.azure.net"
credential = DefaultAzureCredential()
_sc = SecretClient(vault_url=KVUri, credential=credential)
srv = _sc.get_secret("server-nm").value
print(f'KV Secret: {srv}')
This setup had been working without any issues for almost a year to allow App service extract secrets from KV but over the last weekend, it started failing with the following error.
[08/29/2022 09:14:14 > 6f77be: ERR ] ManagedIdentityCredential.get_token failed: <urllib3.connection.HTTPConnection object at 0x00000234348D8AC8>: Failed to establish a new connection:
[WinError 10049] The requested address is not valid in its context
[08/29/2022 09:14:14 > 6f77be: ERR ] DefaultAzureCredential failed to retrieve a token from the included credentials.
[08/29/2022 09:14:14 > 6f77be: ERR ] Attempted credentials:
Any idea what's this error is regarding? Will this happen due to any firewall issue?
PS: The same code works well inside a VM using its managed identity and retrieves secrets from KV. The issue happens only when the code is run from Webjobs.

Azure python sdk authentication using SAS url [duplicate]

We got error of Authentication fail, when we try to create an azure blob client from connection string, using python v12 sdk with Azure Blob Storage v12.5.0, and Azure core 1.8.2.
I used
azure-storate-blob == 12.5.0
azure-core == 1.8.2
I tried to access my blob storage account using connection string with Python v12 SDK and received the error above. The environment I'm running in is python venv in NixShell.
The code for calling the blob_upload is as following:
blob_service_client = BlobServiceClient(account_url=<>,credential=<>)
blob_client = blob_service_client.get_blob_client(container=container_name,
blob=file)
I printed out blob_client, and it looks normal. But the next line of upload_blob gives error.
with open(os.path.join(root,file), "rb") as data:
blob_client.upload_blob(data)
The error message is as follows
File "<local_address>/.venv/lib/python3.8/site-packages/azure/storage/blob/_upload_helpers.py", in upload_block_blob
return client.upload(
File "<local_address>/.venv/lib/python3.8/site-packages/azure/storage/blob/_generated/operations/_block_blob_operations.py", in upload
raise models.StorageErrorException(response, self._deserialize)
azure.storage.blob._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'
So I printed out the http put request to azure blob storage, and get the response value of [403]
I can work the following code well with the version the same as yours.
from azure.storage.blob import BlobServiceClient
blob=BlobServiceClient.from_connection_string(conn_str="your connect string in Access Keys")
with open("./SampleSource.txt", "rb") as data:
blob.upload_blob(data)
Please check your connect-string, and check your PC's time.
There is a similar issue about the error: AzureStorage Blob Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature
UPDATE:
I tried with this code, and get the same error:
from azure.storage.blob import BlobServiceClient
from azure.identity import DefaultAzureCredential
token_credential = DefaultAzureCredential()
blob_service_client = BlobServiceClient(account_url="https://pamelastorage123.blob.core.windows.net/",credential=token_credential)
blob_client = blob_service_client.get_blob_client(container="pamelac", blob="New Text Document.txt")
with open("D:/demo/python/New Text Document.txt", "rb") as data:
blob_client.upload_blob(data)
Then I use AzureCliCredential() instead of DefaultAzureCredential(). I authenticate via the Azure CLI with az login. And it works.
If you use environment credential, you need to set the variables. Anyway, I recommend you to use the specific credentials instead DefaultAzureCredential.
For more details about Azure Identity, see here.

Azure Retrieve Secret from key vault

I am following the Microsoft documentation to retrieve secrets from a key vault using python sdk.
The code and explanation offered by Microsoft leads to this code:
import os
import cmd
from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential
keyvault_name = f'https://<Keyvaultname>.vault.azure.net/'
KeyVaultName = "<Keyvaultname>"
credential = DefaultAzureCredential()
client = SecretClient(vault_url=keyvault_name, credential=credential)
print(" done.")
print(f"Retrieving your secret from {KeyVaultName}.")
retrieved_secret = client.get_secret("test")
print(f"Your secret is '{retrieved_secret.value}'.")
According to my understanding, the DefaultCredentials are the one configured in the az login which is fine, my code runs just fine but I keep getting this message in the terminal.
done.
Retrieving your secret from <KeyvaultName>.
EnvironmentCredential.get_token failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
ImdsCredential.get_token failed: ManagedIdentityCredential authentication unavailable, no managed identity endpoint found.
ManagedIdentityCredential.get_token failed: ManagedIdentityCredential authentication unavailable, no managed identity endpoint found.
SharedTokenCacheCredential.get_token failed: SharedTokenCacheCredential authentication unavailable. Multiple accounts
were found in the cache. Use username and tenant id to disambiguate.
I presume that this warnings are due the fact that I have multiple subscription in my azure portal.
I was wondering, how can I get rid of those and set the credentials for only a single subscription?
Thank you so much for any help and explanation you can offer me.
Generally speaking, I would not worry about this warning. When you use DefaultAzureCredential, SDK tries the following credential options in that order (Reference):
EnvironmentCredential
ManagedIdentityCredential
SharedTokenCacheCredential
VisualStudioCredential
VisualStudioCodeCredential
AzureCliCredential
AzurePowerShellCredential
InteractiveBrowserCredential
SDK moves from one credential options to another if that credential option fails. The warning message is just a way for the SDK to tell you what all credential options it has tried.
However if you still want to get rid of this message, there are a few options available to you:
Exclude the credential options that you do not want SDK to try when using DefaultAzureCredential. You can specify those via exclude_xxx_credential option in the constructor. For example, if you want to exclude EnvironmentCredential, you would specify exclude_environment_credential=True in the DefaultAzureCredential constructor. SDK will skip those credential methods. Please see this link for all constructor options.
Use specific credential option. For example, if you always want to use Azure CLI credentials, then instead of using DefaultAzureCredential you can use AzureCliCredential.

Accessing Azure Storage via certificate based service principal credentials; azure.identity vs azure.common.credentials

I have experimented trying to access Azure Blob Storage using service principal credentials through Python SDK & have some confusions I thought the community could help with.
#1 azure.common.credentials vs azure.identity-------------------------------------------------
I have noticed two different python packages in Azure having credential classes.
- azure.common.credentials
- azure.identity
What is the difference between the two, and when should one be used against other? More specifically, when attempting to work with Azure service principals,
**azure.identity** provides both **ClientSecretCredential & CertificateCredential** so we can use either shared secret, or SSL certificate.
**azure.common.credentials** package provides only the **ServicePrincipalCredentials** class that needs a shared secret, and there is no counterpart for working with certificate credentials.
Am I missing something? I am looking to use certificate based service principal.
#2 ServicePrincipalCredentials works, but ClientSecretCredential fails ------------------------------------------------
My test code to access Azure storage works successfully with ServicePrincipalCredentials class.
But fails with ClientSecretCredential class with Exception message: 'ClientSecretCredential' object has no attribute 'signed_session'"
Appreciate any help with understanding why. There is no difference in the code apart from instantiating the credentials to be one of the two classes above.
The #2 issue above is important mainly because of #1. I am looking to use certificate based Auth, but can't find a supporting class under azure.common.credentials.
Python Environ details:
>python3 --version
Python 3.6.9
>pip3 freeze | grep -i azure
azure-common==1.1.25
azure-core==1.5.0
azure-identity==1.3.1
azure-mgmt-resource==9.0.0
azure-mgmt-storage==10.0.0
azure-storage-blob==12.3.1
msrestazure==0.6.3
snippets from my code:
# for credential classes
from azure.identity import ClientSecretCredential
from azure.identity import CertificateCredential
# for storage & other resource mgmt classes
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.storage import StorageManagementClient
tenant_id = params['tenant-id']
client_id = params['client-id']
client_secret = params['secret']
subscription_id = params['subscription-id']
creds = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret)
# create resource group
resource_client = ResourceManagementClient(creds, subscription_id)
# create storage group, access keys etc
storage_client = StorageManagementClient(creds, subscription_id)
When attempting with certificate rather than secret, here is the code snippet for creating credential instance; rest of code is same.
client_keycert_path = params['cert-path']
creds = CertificateCredential(tenant_id =tenant_id, client_id = client_id, certificate_path = client_keycert_path)
The current situation is misleading, I admit, here's a few details as of today (monitor the situation here https://github.com/Azure/azure-sdk-for-python/issues/9310):
For azure-storage-blob, azure.common is used for storage SDK <= 2.x, and azure-identity is used for storage SDK >= v12.x.
For any package starting with azure-mgmt-xxx, azure-common is still the official way. Check this issue for workaround on how to write mgmt code that uses azure-identity (https://github.com/Azure/azure-sdk-for-python/issues/9310)
This will change SOON, by summer 2020 mgmt SDKs should support azure-identity out of the box.
Hope this helps, feel free to open an issue on Github too if there is further questions:
https://github.com/Azure/azure-sdk-for-python/issues
(I work in the Azure SDK team at MS)

Authenticate calls to Google Cloud Functions programmatically

I am trying to authenticate to Google Cloud Functions from SAP CPI to fetch some data from a database. To push data, we use pub/sub, with a service account access token, and it works perfectly. But for the functions, it needs an identity token instead of an access token. We get the previous token with a groovy script (No Jenkins). Is it possible to authenticate to the functions also with an access token? Or to get the identity token without building a whole IAP layer?
You have to call your Cloud Functions (or Cloud Run, it's the same) with a signed identity token.
So you can use a groovy script for generating a signed identity token. Here an example
import com.google.api.client.http.GenericUrl
import com.google.api.client.http.HttpRequest
import com.google.api.client.http.HttpRequestFactory
import com.google.api.client.http.HttpResponse
import com.google.api.client.http.javanet.NetHttpTransport
import com.google.auth.Credentials
import com.google.auth.http.HttpCredentialsAdapter
import com.google.auth.oauth2.IdTokenCredentials
import com.google.auth.oauth2.IdTokenProvider
import com.google.auth.oauth2.ServiceAccountCredentials
import com.google.common.base.Charsets
import com.google.common.io.CharStreams
String myUri = "YOUR_URL";
Credentials credentials = ServiceAccountCredentials
.fromStream(new FileInputStream(new File("YOUR_SERVICE_ACCOUNT_KEY_FILE"))).createScoped("https://www.googleapis.com/auth/cloud-platform");
String token = ((IdTokenProvider) credentials).idTokenWithAudience(myUri, Collections.EMPTY_LIST).getTokenValue();
System.out.println(token);
IdTokenCredentials idTokenCredentials = IdTokenCredentials.newBuilder()
.setIdTokenProvider((ServiceAccountCredentials) credentials)
.setTargetAudience(myUri).build();
HttpRequestFactory factory = new NetHttpTransport().createRequestFactory(new HttpCredentialsAdapter(idTokenCredentials));
HttpRequest request = factory.buildGetRequest(new GenericUrl(myUri));
HttpResponse httpResponse = request.execute();
System.out.println(CharStreams.toString(new InputStreamReader(httpResponse.getContent(), Charsets.UTF_8)));
Service account key file is required only if you are outside GCP. Else, the default service account is enough, but must be a service account. Your personal user account won't work
Add this dependency (here in Maven)
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<version>0.20.0</version>
</dependency>
Or you can use a tool that I wrote and open sourced. I also wrote a Medium article for explaining the use cases
You can only access your secured cloud function using Identity token.
1.Create a service account with roles/cloudfunctions.invoker
2.Create a cloud function that allows only authenticated requests
https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
target_audience = 'https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME'
creds = service_account.IDTokenCredentials.from_service_account_file(
'/path/to/svc.json', target_audience=target_audience)
authed_session = AuthorizedSession(creds)
# make authenticated request and print the response, status_code
resp = authed_session.get(target_audience)
print(resp.status_code)
print(resp.text)

Resources