I've used the python code sample below from Microsoft to try and decode access and identity tokens (JWT) from Microsoft AD. I've tried every method I can find online for doing this and no matter what I keep getting this error:
File "C:\Users\Connor Johnson\AppData\Local\Programs\Python\Python37\lib\site-packages\jwt\api_jwt.py", line 92, in decode
jwt, key=key, algorithms=algorithms, options=options, **kwargs
File "C:\Users\Connor Johnson\AppData\Local\Programs\Python\Python37\lib\site-packages\jwt\api_jws.py", line 156, in decode
key, algorithms)
File "C:\Users\Connor Johnson\AppData\Local\Programs\Python\Python37\lib\site-packages\jwt\api_jws.py", line 223, in _verify_signature
raise InvalidSignatureError('Signature verification failed')
jwt.exceptions.InvalidSignatureError: Signature verification failed
I've tried different encoding-centered solutions to no avail, and at this point all I can figure is that it's an AD configuration issue. If I need to provide specific AD settings, let me know.
import jwt
import sys
import requests
from cryptography.x509 import load_pem_x509_certificate
from cryptography.hazmat.backends import default_backend
PEMSTART = '-----BEGIN CERTIFICATE-----\n'
PEMEND = '\n-----END CERTIFICATE-----'
# get Microsoft Azure public key
def get_public_key_for_token(kid):
response = requests.get(
'https://login.microsoftonline.com/common/.well-known/openid-configuration',
).json()
jwt_uri = response['jwks_uri']
response_keys = requests.get(jwt_uri).json()
pubkeys = response_keys['keys']
public_key = ''
for key in pubkeys:
# found the key that matching the kid in the token header
if key['kid'] == kid:
# construct the public key object
mspubkey = str(key['x5c'][0])
cert_str = PEMSTART + mspubkey + PEMEND
cert_obj = load_pem_x509_certificate(str.encode(cert_str), default_backend())
public_key = cert_obj.public_key()
return public_key
# decode the given Azure AD access token
def aad_access_token_decoder(access_token):
header = jwt.get_unverified_header(access_token)
print(header['kid'])
public_key = get_public_key_for_token(header['kid'])
# the value of the databricks_resource_id is as defined above
databricks_resource_id=<APP ID>
decoded=jwt.decode(access_token, key = public_key, algorithms = 'RS256',
audience = databricks_resource_id)
for key in decoded.keys():
print(key + ': ' + str(decoded[key]))
aad_access_token_decoder(<JWT BEARER TOKEN IN STRING>)
You have a misconfiguration on your app registration. You issuer url, audience, or something, is set up wrong. I can't tell you more without knowing a lot more about the app registration and code setup.
I'm pretty sure that "common" in the oidc config url only works for multi-tenant apps. If your app is not multi tenant you might try substituting your tenant id right there.
Related
I want to pull emails by Graph API from client inbox using python.
I started with a tutorial and successfully experimented over my personal inbox.
My problem,
Every time my code generates an authorization URL.
I have to browse through it (using web browser library) , sign in using my credentials and copy paste the authorization code for generating access token.
Which is a lot of manual work every time.
Question :
Is there a way to automate the whole process of token generation ?
Such that my client only shares his application id and client secret, and email is pulled without his sign in credentials ?
My code is attached below -
import msal
from msal import PublicClientApplication
import webbrowser
import requests
import pandas as pd
APPLICATION_ID="app id"
CLIENT_SECRET="client secret"
authority_url='https://login.microsoftonline.com/common/'
base_url = 'https://graph.microsoft.com/v1.0/'
endpoint_url = base_url+'me'
SCOPES = ['Mail.Read','Mail.ReadBasic']
client_instance = msal.ConfidentialClientApplication(client_id = APPLICATION_ID,client_credential = CLIENT_SECRET,authority = authority_url)
authorization_request_url=client_instance.get_authorization_request_url(SCOPES)
#print(authorization_request_url)
# browsing authorization request URL for retrieving authorization code.
webbrowser.open(authorization_request_url,new=True)
# Manually pasting authorization code.
authorization_code='authorization code from authorization URL'
access_token = client_instance.acquire_token_by_authorization_code(code=authorization_code,scopes=SCOPES)
access_token_id=access_token['access_token']
# Rest of the codes are for hitting the end point and retrieving the messages
Any help with code suggestions will be much appreciated.
Thanks in advance
If you would like to authenticate only with a clientId and clientSecret, without any user context, you should leverage a client credentials flow.
You can check this official MS sample that uses the same MSAL library to handle the client credentials flow. It is quite straightforward, as you can see below:
import sys # For simplicity, we'll read config file from 1st CLI param sys.argv[1]
import json
import logging
import requests
import msal
# Optional logging
# logging.basicConfig(level=logging.DEBUG)
config = json.load(open(sys.argv[1]))
# Create a preferably long-lived app instance which maintains a token cache.
app = msal.ConfidentialClientApplication(
config["client_id"], authority=config["authority"],
client_credential=config["secret"],
# token_cache=... # Default cache is in memory only.
# You can learn how to use SerializableTokenCache from
# https://msal-python.rtfd.io/en/latest/#msal.SerializableTokenCache
)
# The pattern to acquire a token looks like this.
result = None
# Firstly, looks up a token from cache
# Since we are looking for token for the current app, NOT for an end user,
# notice we give account parameter as None.
result = app.acquire_token_silent(config["scope"], account=None)
if not result:
logging.info("No suitable token exists in cache. Let's get a new one from AAD.")
result = app.acquire_token_for_client(scopes=config["scope"])
if "access_token" in result:
# Calling graph using the access token
graph_data = requests.get( # Use token to call downstream service
config["endpoint"],
headers={'Authorization': 'Bearer ' + result['access_token']}, ).json()
print("Graph API call result: ")
print(json.dumps(graph_data, indent=2))
else:
print(result.get("error"))
print(result.get("error_description"))
print(result.get("correlation_id")) # You may need this when reporting a bug
The sample is retrieving a list of users from MS Graph, but it should be just a matter of adapting it to retrieve the list of emails of a specific user by changing the "endpoint" parameter in the parameters.json file to:
"endpoint": "https://graph.microsoft.com/v1.0/users//users/{id | userPrincipalName}/messages"
You can check here more information regarding the MS Graph request to list emails.
register your app
get your tenant id from azure portal and disable mfa
application_id = "xxxxxxxxxx"
client_secret = "xxxxxxxxxxxxx"
#authority_url = "xxxxxxxxxxx"
authority_url = 'xxxxxxxxxxxxxxxxxxxx'
base_url = "https://graph.microsoft.com/v1.0/"
endpoint = base_url+"me"
scopes = ["User.Read"]
tenant_id = "xxxxxxxxxxxx"
token_url = 'https://login.microsoftonline.com/'+tenant_id+'/oauth2/token'
token_data = {
'grant_type': 'password',
'client_id': application_id,
'client_secret': client_secret,
'resource': 'https://graph.microsoft.com',
'scope':'https://graph.microsoft.com',
'username':'xxxxxxxxxxxxxxxx', # Account with no 2MFA
'password':'xxxxxxxxxxxxxxxx',
}
token_r = requests.post(token_url, data=token_data)
token = token_r.json().get('access_token')
print(token)
I want to access the listed websites data in the Google Search Console using the Google Sign-In access_token (that one can get as the response when using Google Sign-In).
But, the thing is I can access that data only by using the authorization_code that can be copied from the OAuth2-Consent screen by going to the generated authorize_url and signing in using the registered Google account.
Here's the minimum reproducible version of the code:
from oauth2client.client import OAuth2WebServerFlow
import httplib2
from apiclient.discovery import build
CLIENT_ID = 'YOUR_CLIENT_ID'
CLIENT_SECRET = 'YOUR_CLIENT_SECRET'
OAUTH_SCOPE = 'https://www.googleapis.com/auth/webmasters.readonly'
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, redirect_uri=REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print ('Go to the following link in your browser: ' + authorize_url)
code = input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
http = httplib2.Http()
http = credentials.authorize(http)
webmasters_service = build('webmasters', 'v3', http=http)
def get_property_list(webmasters_service):
'''
Get a list of validated properties from GSC
'''
site_list = webmasters_service.sites().list().execute()
# Filter for verified websites
verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry']
if s['permissionLevel'] != 'siteUnverifiedUser'
and s['siteUrl'][:4] == 'http']
return verified_sites_urls
print({"available_websites": get_property_list(webmasters_service)})
Consider that I'll be provided with the Google Sign-In access-token as the request-parameter from another server which has implemented Google Sign-In feature.
So, again my question is how can I access the same data using that token instead of manually getting the auth_code from the OAuth2 consent screen ?
I have followed the documentation shared by DaImTo in the comments above. And modified the code as shown below:
from oauth2client.client import OAuth2WebServerFlow
import httplib2
from apiclient.discovery import build
from oauth2client import tools, file
CLIENT_ID = 'YOUR_CLIENT_ID'
CLIENT_SECRET = 'YOUR_CLIENT_SECRET'
OAUTH_SCOPE = 'https://www.googleapis.com/auth/webmasters.readonly'
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
# Acquire and store oauth token.
storage = file.Storage('token.json')
credentials = storage.get()
if credentials is None or credentials.invalid:
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, redirect_uri=REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
credentials = tools.run_flow(flow, storage)
http = httplib2.Http()
http = credentials.authorize(http)
webmasters_service = build('webmasters', 'v3', http=http)
def get_property_list(webmasters_service):
'''
Get a list of validated properties from GSC
'''
site_list = webmasters_service.sites().list().execute()
# Filter for verified websites
verified_sites_urls = [s['siteUrl'] for s in site_list['siteEntry']
if s['permissionLevel'] != 'siteUnverifiedUser'
and s['siteUrl'][:4] == 'http']
return verified_sites_urls
print({"available_websites": get_property_list(webmasters_service)})
It's working fine now, without any manual interaction for copying and pasting the authorization_code from the OAuth2-Consent screen.
Building Microservices With Django and pyJWT
Let me explain this in summarised manor, maybe someone can help me out been at this for too long
I have three microservices i.e.
Authentication : Which creates the User and makes the token for a user using RS256 algorithm
Student : which is basically a user type that needs to be verified using JWT
Investor : which is also a user type that needs to be verified using JWT
[I have tried to make a generic create token function in authentication service something like]
data = {"id": user.uuid.hex, "type": user.user_type}
encoded = jwt.encode(data, private_key, algorithm="RS256")
return encoded
It is generating a correct token i have verified using JWT.io
In my student service i have created a middleware something like this
class JWTAuthenticationMiddleware(object):
#Simple JWT token based authentication.
#Clients should authenticate by passing the token key in the "Authorization"
#HTTP header, prepended with the string "Bearer ". For example:
# Authorization: Bearer <UNIQUE_TOKEN>
keyword = "Bearer"
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
auth = request.META.get("AUTHORIZATION", None)
splitted = auth.split() if auth else []
if len(splitted) == 2 and splitted[0] == self.keyword:
if not splitted or splitted[0].lower() != self.keyword.lower().encode():
return None
if len(splitted) == 0:
msg = _("Invalid token header. No credentials provided.")
raise exceptions.AuthenticationFailed(msg)
elif len(splitted) > 2:
msg = _("Invalid token header. Token string should not contain spaces.")
raise exceptions.AuthenticationFailed(msg)
user = get_user(request)
decoded = verify_token(auth)
try:
student = Student.objects.get(user_id=decoded.get("id"))
except Student.DoesNotExist:
student = Student.objects.update(user_id=decoded.get("id"))
response = self.get_response(request)
return response
My verify_token function looks like
def verify_token(token):
decoded = jwt.decode(token, JWT_AUTH["PUBLIC_KEY"], algorithms=["RS256"])
return decoded
I have also added my middleware in my settings just below authentication middleware
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"corsheaders.middleware.CorsMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.locale.LocaleMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"config.middleware.authentication.JWTAuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.common.BrokenLinkEmailsMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
]
I am just not able to decode my token and assign the payload to respective user type, Can someone help me out where i am going wrong? I have also tried using Authentication backend something like this, it doesn't works, have implemented simple_jwt_djangorestframework package but it doesn't decode my payload on student service and says invalid token so i don't want to add it for increasing unnecessary code too.
My viewset looks like this
class StudentViewset(viewsets.ViewSet):
queryset = Student.objects.filter(is_active=True)
serializer_class = StudentSerializer
lookup_field = "uuid"
permission_classes = [IsAuthenticated]
My error is always saying when i am using isAuthenticated as permission class
"message": "Authentication credentials were not provided.",
Maybe validating raw token should work :
decoded = verify_token(splitted[1])
Actually you don't have to implement it. There is simple jwt package for it. Install the package by its documentation and if it wouldn't work provide errors to help you.
To answer this question myself for future references
class JWTAuthenticationMiddleware(object):
#Simple JWT token based authentication.
#Clients should authenticate by passing the token key in the "Authorization"
#HTTP header, prepended with the string "Bearer ". For example:
# Authorization: Bearer <UNIQUE_TOKEN>
keyword = "Bearer"
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
auth = request.META.get("AUTHORIZATION", None)
splitted = auth.split() if auth else []
if len(splitted) == 2 and splitted[0] == self.keyword:
if not splitted or splitted[0].lower() != self.keyword.lower().encode():
return None
if len(splitted) == 0:
msg = _("Invalid token header. No credentials provided.")
return JsonResponse(data=msg, status=403, safe=False)
elif len(splitted) > 2:
msg = _("Invalid token header. Token string should not contain spaces.")
return JsonResponse(data=msg, status=403, safe=False)
user = get_user(request)
decoded = verify_token(auth)
try:
student = Student.objects.get(user_id=decoded.get("id"))
return self.get_response(request)
except Student.DoesNotExist:
msg = _("Invalid User Not found")
return JsonResponse(data=msg, status=403, safe=False)
couple of things that i was doing wrong was not passing a response when i found a student that's why middleware was not bypassed at that time, also exceptions cannot be used at middleware level middleware only returns HTTPResponses, thus i thought of using JsonResponse.
I am running this code to obtain a Bearer token from the InteractiveBrowserCreedentail and log in to azure blob storage:
cred = InteractiveBrowserCredential(authority="login.microsoftonline.com", tenant_id="**", client_id="**")
token = cred.get_token()
print(token)
blobber = BlobServiceClient(account_url="https://**.blob.core.windows.net", credential=cred)
blobs = blobber.list_containers()
for b in blobs:
print(b)
This works well.
I am trying to reuse the token in another call, this time a direct rest interaction:
import requests
auth_header = ("Authorization", "Bearer " + "***")
version = ("x-ms-version", "2017-11-09")
response = requests.get("https://***.blob.core.windows.net/?comp=list", headers=dict([auth_header, version]))
I get a 403 response saying:
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
According to official documentation, this should be working.
What am I missing?
According to my research, when you request AD access token and call Azure blob storage, the scope must contain https://storage.azure.com/user_impersonation or https://storage.azure.com/.default. For more details, please refer to the document. In other words, the request url should be like
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?client_id=<>
&scope=https://storage.azure.com/user_impersonation
&...
But when I run the cred.get_token(), the request url just be like below. The scope does not contain https://storage.azure.com/user_impersonation or https://storage.azure.com/.default. So you cannot call Azure Blob rest api with the token.
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?
client_id=<>
&scope=offline_access openid profile&state=204238ac-4fcd-44f2-9eed-528ab4d9c37
&...
Meanwhile, I do test, if we run the code blob_service_client = BlobServiceClient(account_url="https://blobstorage0516.blob.core.windows.net/", credential=cred), the request url is
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?
client_id=<>
&scope=https://storage.azure.com/.default offline_access openid profile&state=204238ac-4fcd-44f2-9eed-528ab4d9c37
&...
That is my solution:
from azure.identity import InteractiveBrowserCredential
class InteractiveAuthentication():
def __init__(self):
self.tenant_id: str = ""
self.authority: str = ""
self.client_id: str = ""
self.resource_id: str = ""
self.scope: str = f"{self.resource_id}/.default"
self.token: str = ""
def get_access_token(self):
credential = DeviceCodeCredential(
exclude_interactive_browser_credential=True,
disable_automatic_authentication=True,
tenant_id=self.tenant_id,
authority=self.authority,
client_id=self.client_id
)
self.token = credential._request_token(self.scope)
return self.token['access_token']
I'm trying to create a pool using a custom image I created from VM with azure python sdk. The location and resource group match.
Here's my code:
import azure.batch as batch
from azure.batch import BatchServiceClient
from azure.batch.batch_auth import SharedKeyCredentials
from azure.batch import models
account = 'mybatch'
key = 'Adgfdj1hhsdfqATc/K2fgxdfg/asYgKRP2pUdfglBce7mgmSBdfgdhC7f3Zdfgrcgkdgh/dfglA=='
batch_url = 'https://mybatch.westeurope.batch.azure.com'
creds = SharedKeyCredentials(account, key)
batch_client = BatchServiceClient(creds, base_url = batch_url)
pool_id_base = 'mypool'
idx = 1
pool_id = pool_id_base + str( idx )
while batch_client.pool.exists( pool_id ):
idx += 1
pool_id = pool_id_base + str( idx )
print( 'pool_id ' + pool_id )
sku_to_use = 'batch.node.ubuntu 18.04'
#
# image_ref_to_use = models.ImageReference(
# offer = 'UbuntuServer',
# publisher = 'Canonical',
# sku = '18.04-LTS',
# version = 'latest'
# )
image_ref_to_use = models.ImageReference(
virtual_machine_image_id = '/subscriptions/1834572sd-34sd409a-sdfb-sc345csdfesourceGroups/resource-group-1/providers/Microsoft.Compute/images/my-image-1'
)
vm_size = 'Standard_D3_v2'
vmc = models.VirtualMachineConfiguration(
image_reference = image_ref_to_use,
node_agent_sku_id = sku_to_use
)
new_pool = models.PoolAddParameter(
id = pool_id,
vm_size = vm_size,
virtual_machine_configuration = vmc,
target_dedicated_nodes = 1
)
batch_client.pool.add(new_pool)
According to the docs I should be able to use either virtual_machine_image_id other provide marketplace image parameters.
I can create a pool of standard marketplace images, but I get an error when I'm trying to use an id of my custom image.
Traceback (most recent call last): File "create_pool.py", line 60, in <module>
batch_client.pool.add(new_pool) File "/root/miniconda/lib/python3.6/site-packages/azure/batch/operations/pool_operations.py", line 312, in add
raise models.BatchErrorException(self._deserialize, response) azure.batch.models.batch_error.BatchErrorException: {'additional_properties': {}, 'lang': 'en-US', 'value': 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:0dfdf9c1-edad-4b72-8e8f-f8dbcfd0abbdf\nTime:2018-12-06T10:51:21.9417222Z'}
How can I resolve this issue?
UPDATE
I tried to use ServicePrincipalCredentials with the following:
CLIENT_ID: I created a new application in Defaut Directiry -> Add registration and got it's Application ID.
SECRET: A created a key for the new application and used its value.
TENANT_ID: az account show in the cloud shell.
RESOURCE: Used 'https://batch.core.windows.net/'.
Updated my code like this:
from azure.common.credentials import ServicePrincipalCredentials
creds = ServicePrincipalCredentials(
client_id=CLIENT_ID,
secret=SECRET,
tenant=TENANT_ID,
resource=RESOURCE
)
And I get another error:
Keyring cache token has failed: No recommended backend was available. Install the keyrings.alt package if you want to use the non-recommended backends. See README.rst for details.
Traceback (most recent call last):
File "create_pool.py", line 41, in <module>
while batch_client.pool.exists( pool_id ):
File "/root/miniconda/lib/python3.6/site-packages/azure/batch/operations/pool_operations.py", line 624, in exists
raise models.BatchErrorException(self._deserialize, response)
azure.batch.models.batch_error.BatchErrorException: Operation returned an invalid status code 'Server failed to authorize the request.'
Try using Service Principal Credentials instead of the Shared Key Credentials
credentials = ServicePrincipalCredentials(
client_id=CLIENT_ID,
secret=SECRET,
tenant=TENANT_ID,
resource=RESOURCE
)
There seems to be an error with Shared Key Credentials.
Documentation Link: https://learn.microsoft.com/en-us/azure/batch/batch-aad-auth
Issue Link: https://github.com/Azure/azure-sdk-for-python/issues/1668
Note: Please remove your account details as it can be accessed by anyone. Replace the account name and key with ****.
Update
If the Service Principal Credentials are not working, try using the User credentials and see if that works.
from azure.common.credentials import UserPassCredentials
import azure.batch.batch_service_client as batch
credentials = UserPassCredentials(
azure_user,
azure_pass
)
batch_client = batch.BatchServiceClient(credentials, base_url = batch_url)