I am trying to use requests in a Python3 script to update a deploy key in a gitlab project with write access. Unfortunately, I am receiving a 404 error when trying to connect via the requests module. The code is below:
project_url = str(url)+('/deploy_keys/')+str(DEPLOY_KEY_ID)
headers = {"PRIVATE-TOKEN" : "REDACTED"}
payload = {"can_push" : "true"}
r = requests.put(project_url, headers=headers, json=payload)
print(r)
Is there something that I am doing wrong where in the syntax of my Private Key/headers?
I have gone through gitlab api and requests documentation. I have also confirmed that my Private Token is working outside of the script.
I am expecting it to update the deploy key with write access, but am receiving a upon exit, making me think the issue is with the headers/auth.
This is resolved, it was actually not an auth problem. You must use the project ID instead of url, for example:
project_url = f'https://git.REDACTED.com/api/v4/projects/{project_id}/deploy_keys/{DEPLOY_KEY_ID}'
headers = {"PRIVATE-TOKEN" : "REDACTED"}
payload = {'can_push' : 'true'}
try:
r = requests.put(project_url, headers=headers, data=payload)
Related
I have a Django app that I have been working on. When I run it locally it runs perfectly. When I run it in a container using Cloud Run I get the following error:
'Credentials' object has no attribute 'id_token'
Here is the offending code (payload is a dictionary object):
def ProcessPayload(payload):
# Get authorized session credentials
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
credentials.refresh(Request(session))
# Process post request
headers = {'Authorization': f'Bearer {credentials.id_token}'}
response = requests.post(URL, json=payload, headers=headers)
In my local environment, the refresh properly loads credentials with the correct id_toled for the needed header, but for some reason when the code is deployed to Cloud Run this does not work. I have the Cloud run instance set to use a service account so it should be able to get credentials from it. How do I make this work? I have googled until my fingers hurt and have found no viable solutions.
When executing code under a Compute Service (Compute Engine, Cloud Run, Cloud Functions), call the metadata service to obtain an OIDC Identity Token.
import requests
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/' \
'instance/service-accounts/default/identity?' \
'audience={}'
def fetch_identity_token(audience):
# Construct a URL with the audience and format.
url = METADATA_URL.format(audience)
# Request a token from the metadata server.
r = requests.get(url, headers=METADATA_HEADERS)
r.raise_for_status()
return r.text
def ProcessPayload(payload):
id_token = fetch_identity_token('replace_with_service_url')
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(URL, json=payload, headers=headers)
The equivalent curl command to fetch an Identity Token looks like this. You can test from a Compute Engine instance:
curl -H "metadata-flavor: Google" \
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=URL
where URL is the URL of the service you are calling.
Authentication service-to-service
I have seen this metadata URL shortcut (for Cloud Run), but I have not verified it:
http://metadata/instance/service-accounts/default/identity?audience=URL
So, after much playing around I found a solution that works in both places. Many thanks to Paul Bonser for coming up with this simple method!
import google.auth
from google.auth.transport.requests import AuthorizedSession, Request
from google.oauth2.id_token import fetch_id_token
import requests
def GetIdToken(audience):
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
request = Request(session)
credentials.refresh(request)
if hasattr(credentials, "id_token"):
return credentials.id_token
return fetch_id_token(request, audience)
def ProcessPayload(url, payload):
# Get the ID Token
id_token = GetIdToken(url)
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(url, json=payload, headers=headers)
I wish to make a post request to add an attachment utilising urllib3 in python without success. I have confirmed the API itself is working in postman but cannot work out how to convert this request to python. Appreciating I'm mixing object types I just don't know how to avoid it.
Python code:
import urllib3
import json
api_key = "secret_key"
header = {"X-API-KEY": api_key, "ACCEPT": "application/json", "content-type": "multipart/form-data"}
url = "https://secret_url.com/api/"
http = urllib3.PoolManager()
with open("invoice.html", 'rb') as f:
file_data = f.read()
payload = {
"attchment": {
"file": file_data
}
}
payload = json.dumps(payload)
r = http.request('post', url, headers = header, fields = payload)
print(r.status)
print(r.data)
Postman - which works and properly sends file-name through also (I'm guessing it splits the bytes and filename up?)
Edit: I've also tried the requests library as I'm more familiar with this (but can't use it as the script will be running in AWS lambda). Removing the attachment element form the dict allows it to run but the API endpoint gives 401 presumably because it's missing the "attachement" part to the data structure as per postman below... but when I put this in I get runtime errors.
r = requests.post(url, headers = header, files={"file": open("invoice.html", 'rb')})
For anyone who stumbles upon this from Dr google a few points:
I was completely mis-interpreting the structure of the element. It's actually a string "attachment[file]" not a dict like object.
Postman has the ability to output python code in urllib/request syntax albeit not 100% what I was after. Note: the chrome version (depreciated) outputs gibberish code that only half works so the client version should be used. A short bit of work below shows it working as expected:
http = urllib3.PoolManager()
with open("invoice.html", "rb") as f:
file = f.read()
payload={
'attachment[file]':('invoice.html',file,'text/html')
}
r = http.request('post', url, headers = header, fields = payload)
I'm trying to get a POST requests through python. Server side dev provided me with an auth key ie: short124 and a long key. They were generated by Mohawk library. Dev from server side gave me a code example:
import requests
from mohawk import Sender
creds = {'id': short,'key':very_long_key,'algorithm':'sha256'}
auth_session = Sender( credentials = creds, url = url, method = 'POST',always_hash_content=False)
headers = {'Authorization' : auth_session.request_header}
r = requests.post(url,data = input, headers = headers, verify = False)
print(r)
Unfortunately I get 401 error (unauthorized) with the credentials provided,
Can anyone give me some advide?
The way it's being used seem correct. I would double check the credentials and the url.
Also consider using Black it will make code formatting a breeze.
Below returns all the info for the dashboards belonging to vhost 'test'
import requests
url = 'https://abcde/api/dashboards
headers = {'vhost' : 'test'}
r = requests.get(url, headers=headers, auth=('user' , '1234'))
print(r.text)
Now I would like to delete all the dashboards but what I have below hangs for minutes until I break out of it. The dashboards are still there viewed from the UI. I have no baseline as to how long a delete should take.
import requests
url = 'https://abcde/api/dashboards
headers = {'vhost' : 'test'}
r = requests.delete(url, headers=headers, auth=('test' , '1234'))
print(r.text)
Thank you
Using Postman, I received a 500 error
While the server was building the response a unexpected internal server error occured which forced the server to abort the request
I'm a student and only a few weeks into Python, so bear with me. I found a good answer in this link for initially working with Yelp's v3 API to at least get it to successfully make a request by business_id: How to use Yelp's new API
However, I can't seem to figure out how to search by anything other than reviews using the code that someone provided above (copy-pasted here as the below does work, just not what I need it for:
import requests
import yelp
from config import api_key
API_KEY = api_key
API_HOST = 'https://api.yelp.com'
BUSINESS_PATH = '/v3/businesses/'
def get_business(business_id):
business_path = BUSINESS_PATH + business_id
url = API_HOST + business_path + '/reviews'
headers = {'Authorization': f"Bearer {API_KEY}"}
response = requests.get(url, headers=headers)
return response.json()
results = get_business('the-white-horse-pub-kansas-city')
pprint(results)
Again, the code does work if you're only looking up one place by name. But when I try something other than "/reviews" in the url function, such as "search" or "term" or something else going off of the Yelp Fusion API documentation (https://yelp.com/developers/documentation/v3/business_search), I can't get anything to pull. My intent is to pull a bunch of breweries in the local area and then eventually put them in a dataframe, but I can't figure out what parameters or code to use, other than 'review'.
I believe I found an answer where you can at least look for type if you add the first item to the dependencies and the lower items as a separate function:
BUSINESS_PATH_CAT = '/v3/categories/'
def get_all_categories(alias):
url = API_HOST + BUSINESS_PATH_CAT + alias
headers = {'Authorization': f"Bearer {API_KEY}"}
response = requests.get(url, headers=headers)
return response.json()
results = get_all_categories('brewpubs')
pprint(results)
Aside from this though, on the authentication guide for Yelp Fusion, it discusses using Postman. If you're a fellow noob, setting this up can be a lifesaver as this allowed me to actually see the HTTP wording and how it's split up for search terms and how to add them compared to the API documentation.