Using Google Cloud APIs and Oauth2, I am trying to list down projects and display IAM Policies for each project using my Python Desktop app. Sample code is below:
appflow = flow.InstalledAppFlow.from_client_secrets_file("client_secrets.json",
scopes=["https://www.googleapis.com/auth/cloud-platform"])
appflow.run_console()
credentials = appflow.credentials
service = googleapiclient.discovery.build(
'cloudresourcemanager', 'v1', credentials=credentials)
operation1 = service.projects().list().execute()
jason=json.dumps(
operation1,
sort_keys=True,
indent=3)
data = json.loads(jason)
#Gets the list of projects in a Python List object [Proj]
proj=[]
for mem in data['projects']:
print(mem['projectId'])
proj.append(mem['projectId'])
for prj in proj:
resource = 'projects/' + prj
response1 = service.projects().testIamPermissions(resource=resource, body=None, x__xgafv=None).execute()
response2 = service.projects().listOrgPolicies(resource=resource, body=None, x__xgafv=None).execute()
response3 = service.projects().getIamPolicy(resource=resource, body=None, x__xgafv=None).execute()
I get the similar error for all the 3 calls:
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://cloudresourcemanager.googleapis.com/v1/projects/projects%2Fproject-name:testIamPermissions?alt=json returned "Request contains an invalid argument.". Details: "Request contains an invalid argument.">
Arguments appear to be correct. Does the service(cloudresourcemanager) version v1/v3 make a difference? Am I missing something? Thanks in Advance.
I think you should not need to parse the resource with projects/ like the HTTPS example because you are using the library that should be abstract this for you, so if you remove resource = 'projects/' + prj and try the call directly with the project id instead
response1 = service.projects().testIamPermissions(resource=prj, body=None, x__xgafv=None).execute()
response2 = service.projects().listOrgPolicies(resource=prj, body=None, x__xgafv=None).execute()
response3 = service.projects().getIamPolicy(resource=prj, body=None, x__xgafv=None).execute()
If it worked, you should no longer get error 400, but rather 403 "permission denied" because you are missing some of the scopes for those API calls(based on your code example).
The example google provided
testIamPermissions
listOrgPolicies
getIamPolicy
Related
I am trying to find all repos based on a keyword 'TERRAGRUNT_VERSION' using github search api and its returning the below error. Am I missing something in the url?
https://docs.github.com/en/rest/reference/search#search-repositories
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 3 column 1 (char 2)
Code:
import requests
token = "access_token=" + "23456789087"
base_api_url = 'https://api.github.com/'
search_final_url = base_api_url + 'search/repositories?q=TERRAGRUNT_VERSIONin:env/Makefile' + '&' + token
try:
response = requests.get(search_final_url).json()
for item in response['items']:
repo_name = item['name']
print(repo_name)
except Exception as e:
print(e)
The issue is I'm getting a 400 response.
I tried the below
response = requests.get("https://api.github.com/search/repositories?q=TERRAGRUNT_VERSIONin:env/Makefile&access_token=456744")
print(response)
Output:
<Response [400]>
{'message': 'Must specify access token via Authorization header. https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param', 'documentation_url': 'https://docs.github.com/v3/#oauth2-token-sent-in-a-header'}
In short, you have to pass access_token not as the url param but through the request header
If you try to search your link with browser, you would get the answer
{
"message": "Must specify access token via Authorization header. https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param",
"documentation_url": "https://docs.github.com/v3/#oauth2-token-sent-in-a-header"
}
Full description is on the link from response
I have a Django app that I have been working on. When I run it locally it runs perfectly. When I run it in a container using Cloud Run I get the following error:
'Credentials' object has no attribute 'id_token'
Here is the offending code (payload is a dictionary object):
def ProcessPayload(payload):
# Get authorized session credentials
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
credentials.refresh(Request(session))
# Process post request
headers = {'Authorization': f'Bearer {credentials.id_token}'}
response = requests.post(URL, json=payload, headers=headers)
In my local environment, the refresh properly loads credentials with the correct id_toled for the needed header, but for some reason when the code is deployed to Cloud Run this does not work. I have the Cloud run instance set to use a service account so it should be able to get credentials from it. How do I make this work? I have googled until my fingers hurt and have found no viable solutions.
When executing code under a Compute Service (Compute Engine, Cloud Run, Cloud Functions), call the metadata service to obtain an OIDC Identity Token.
import requests
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/' \
'instance/service-accounts/default/identity?' \
'audience={}'
def fetch_identity_token(audience):
# Construct a URL with the audience and format.
url = METADATA_URL.format(audience)
# Request a token from the metadata server.
r = requests.get(url, headers=METADATA_HEADERS)
r.raise_for_status()
return r.text
def ProcessPayload(payload):
id_token = fetch_identity_token('replace_with_service_url')
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(URL, json=payload, headers=headers)
The equivalent curl command to fetch an Identity Token looks like this. You can test from a Compute Engine instance:
curl -H "metadata-flavor: Google" \
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=URL
where URL is the URL of the service you are calling.
Authentication service-to-service
I have seen this metadata URL shortcut (for Cloud Run), but I have not verified it:
http://metadata/instance/service-accounts/default/identity?audience=URL
So, after much playing around I found a solution that works in both places. Many thanks to Paul Bonser for coming up with this simple method!
import google.auth
from google.auth.transport.requests import AuthorizedSession, Request
from google.oauth2.id_token import fetch_id_token
import requests
def GetIdToken(audience):
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
request = Request(session)
credentials.refresh(request)
if hasattr(credentials, "id_token"):
return credentials.id_token
return fetch_id_token(request, audience)
def ProcessPayload(url, payload):
# Get the ID Token
id_token = GetIdToken(url)
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(url, json=payload, headers=headers)
I am running this code to obtain a Bearer token from the InteractiveBrowserCreedentail and log in to azure blob storage:
cred = InteractiveBrowserCredential(authority="login.microsoftonline.com", tenant_id="**", client_id="**")
token = cred.get_token()
print(token)
blobber = BlobServiceClient(account_url="https://**.blob.core.windows.net", credential=cred)
blobs = blobber.list_containers()
for b in blobs:
print(b)
This works well.
I am trying to reuse the token in another call, this time a direct rest interaction:
import requests
auth_header = ("Authorization", "Bearer " + "***")
version = ("x-ms-version", "2017-11-09")
response = requests.get("https://***.blob.core.windows.net/?comp=list", headers=dict([auth_header, version]))
I get a 403 response saying:
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
According to official documentation, this should be working.
What am I missing?
According to my research, when you request AD access token and call Azure blob storage, the scope must contain https://storage.azure.com/user_impersonation or https://storage.azure.com/.default. For more details, please refer to the document. In other words, the request url should be like
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?client_id=<>
&scope=https://storage.azure.com/user_impersonation
&...
But when I run the cred.get_token(), the request url just be like below. The scope does not contain https://storage.azure.com/user_impersonation or https://storage.azure.com/.default. So you cannot call Azure Blob rest api with the token.
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?
client_id=<>
&scope=offline_access openid profile&state=204238ac-4fcd-44f2-9eed-528ab4d9c37
&...
Meanwhile, I do test, if we run the code blob_service_client = BlobServiceClient(account_url="https://blobstorage0516.blob.core.windows.net/", credential=cred), the request url is
https://login.microsoftonline.com/<tenat id>/oauth2/v2.0/authorize?
client_id=<>
&scope=https://storage.azure.com/.default offline_access openid profile&state=204238ac-4fcd-44f2-9eed-528ab4d9c37
&...
That is my solution:
from azure.identity import InteractiveBrowserCredential
class InteractiveAuthentication():
def __init__(self):
self.tenant_id: str = ""
self.authority: str = ""
self.client_id: str = ""
self.resource_id: str = ""
self.scope: str = f"{self.resource_id}/.default"
self.token: str = ""
def get_access_token(self):
credential = DeviceCodeCredential(
exclude_interactive_browser_credential=True,
disable_automatic_authentication=True,
tenant_id=self.tenant_id,
authority=self.authority,
client_id=self.client_id
)
self.token = credential._request_token(self.scope)
return self.token['access_token']
I am trying to use requests in a Python3 script to update a deploy key in a gitlab project with write access. Unfortunately, I am receiving a 404 error when trying to connect via the requests module. The code is below:
project_url = str(url)+('/deploy_keys/')+str(DEPLOY_KEY_ID)
headers = {"PRIVATE-TOKEN" : "REDACTED"}
payload = {"can_push" : "true"}
r = requests.put(project_url, headers=headers, json=payload)
print(r)
Is there something that I am doing wrong where in the syntax of my Private Key/headers?
I have gone through gitlab api and requests documentation. I have also confirmed that my Private Token is working outside of the script.
I am expecting it to update the deploy key with write access, but am receiving a upon exit, making me think the issue is with the headers/auth.
This is resolved, it was actually not an auth problem. You must use the project ID instead of url, for example:
project_url = f'https://git.REDACTED.com/api/v4/projects/{project_id}/deploy_keys/{DEPLOY_KEY_ID}'
headers = {"PRIVATE-TOKEN" : "REDACTED"}
payload = {'can_push' : 'true'}
try:
r = requests.put(project_url, headers=headers, data=payload)
I tried to use partnercenter api to get the billing details. But API call return 401 error code. the details provided on partnercpi are correct one and its working fine If I give it as input for Subscription API.
URL : https://api.partnercenter.microsoft.com/v1/customers/<Customer-tenant-ID>/subscriptions/<Subscription-ID>/usagerecords/resources
Can you suggest what can be the reason for the error.
Reference link : https://msftstack.wordpress.com/2016/01/05/azure-resource-manager-authentication-with-python/
Here is my python code.
import adal
import requests
application_secret = 'asdq23Y+tRgEspFxQDPasd'
application_id = '523129-1267b-123'
authentication_endpoint = 'https://login.microsoftonline.com/'
resource = 'https://management.core.windows.net/'
context = adal.AuthenticationContext(authentication_endpoint + tenant_id)
token_response = context.acquire_token_with_client_credentials(resource, application_id, application_secret)
access_token = token_response.get('accessToken')
endpoint = "https://api.partnercenter.microsoft.com/v1/customers/<customer tenant ID>/subscriptions/<sub-id>/usagerecords/resources"
headers = {"Authorization": 'Bearer ' + access_token}
json_output = requests.get(endpoint,headers=headers)
print json_output
Output Response:
<Response [401]>
Is this method not fit for partnerapi usage collection ?. If not pls suggest alternate option. Thanks in advance.
401 is because partner center does not work with just the token from Azure AD.
The actual Auth workflow
Get a Token from Azure AD.
Use that token to get an access token from PartnerCenter.
Use this token to communicate with the partner center REST APIs.
More Info Here