I'm having issues w/ authenticating to pkgs.org api, a token was produced, by support mentioned it needs to be passed as a cookie. I've never worked with cookies before.
import requests
import json
import base64
import urllib3
import sys
import re
import os
token=('super-secret')
#s = requests.Session()
head = {'Accept':'application/json'}
r = requests.get('https://api.pkgs.org/v1/distributions', auth=(token), headers=head)
print(r)
print(r.connection)
print(r.cookies)
I tried to use the request.session method, to handle the cookie, but i honestly don't know syntax on how to ever 1 create a cookie, let alone pass the cookie.
If I read the API documentation correctly you should set acces_token cookie:
import requests
token = "super-secret"
cookies = {"access_token": token}
headers = {"Accept": "application/json"}
r = requests.get(
"https://api.pkgs.org/v1/distributions", cookies=cookies, headers=headers
)
Related
I'm trying to make requests to the Google API to create source repositories using a service account and his JSON key file.
Since there are no client libraries for this product, I am using the queries with Python using this documentation
https://cloud.google.com/source-repositories/docs/reference/rest
I already used a similar code to invoke my cloud-functions with success, but this time I'm block for these requests at the 401 error. I set up the GOOGLE_APPLICATION_CREDENTIALS with the JSON of my service account, give the service-account the permissions of Source Repository Administrator, but still return 401.
Here's my code
import urllib.request
import json
import urllib
import google.auth.transport.requests
import google.oauth2.id_token
body = { "name" : "projects/$my_project_name/repos/$name_repo"}
jsondata = json.dumps(body).encode("utf8")
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos')
req.add_header('Content-Type', 'application/json; charset=utf-8')
auth_req = google.auth.transport.requests.Request()
id_token = google.oauth2.id_token.fetch_id_token(auth_req, 'https://www.googleapis.com/auth/cloud-platform')
req.add_header("Authorization", f"Bearer {id_token}")
response = urllib.request.urlopen(req, jsondata)
print (response.read().decode())
I tried also using the with an API-KEY at the end of the url like this
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos?key=$my-api-key')
Thank you
I tried also using the with an API-KEY at the end of the url like this
API Keys are not supported.
Your code is using an OIDC Identity Token instead of an OAuth Acess Token.
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(
'/path/to/key.json',
scopes=['https://www.googleapis.com/auth/cloud-platform'])
request = google.auth.transport.requests.Request()
credentials.refresh(request)
// Use the following code to add the access token:
req.add_header("Authorization", f"Bearer {credentials.token}")
I am unable to access an API.
This is what I am doing:
import os, re
import requests
import logger
from requests_oauthlib import OAuth1
oauth_consumer_key = "123abc"
oauth_secret = "aaabbbccc"
url = "https://example_testing/v1/<ID_VALUE>/reports?"
oauth = OAuth1(oauth_consumer_key, oauth_secret)
output = requests.get(url,auth=oauth)
I keep getting a 500 error.
What am I doing wrong?
I didn't know this but apparently I needed to set values for headers and the payload. This solved my problem.
headers = {"Content-Type": "application/json"}
payload = json.dumps({
"report_format":"csv",
<other stuff>
}
output = requests.get(url=url, auth=oauth, headers=headers, data=payload)
Problem Solved
I am trying to get cookies from the browser using aiohttp. From the docs and googling I have only found articles about setting cookies in aiohttp.
In flask I would get the cookies as simply as
cookie = request.cookies.get('name_of_cookie')
# do something with cookie
Is there a simple way to fetch the cookie from browser using aiohttp?
Is there a simple way to fetch the cookie from the browser using aiohttp?
Not sure about whether this is simple but there is a way:
import asyncio
import aiohttp
async def main():
urls = [
'http://httpbin.org/cookies/set?test=ok',
]
for url in urls:
async with aiohttp.ClientSession(cookie_jar=aiohttp.CookieJar()) as s:
async with s.get(url) as r:
print('JSON', await r.json())
cookies = s.cookie_jar.filter_cookies('http://httpbin.org')
for key, cookie in cookies.items():
print('Key: "%s", Value: "%s"' % (cookie.key, cookie.value))
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
The program generates the following output:
JSON: {'cookies': {'test': 'ok'}}
Key: "test", Value: "ok"
Example adapted from https://aiohttp.readthedocs.io/en/stable/client_advanced.html#custom-cookies + https://docs.aiohttp.org/en/stable/client_advanced.html#cookie-jar
Now if you want to do a request using a previously set cookie:
import asyncio
import aiohttp
url = 'http://example.com'
# Filtering for the cookie, saving it into a varibale
async with aiohttp.ClientSession(cookie_jar=aiohttp.CookieJar()) as s:
cookies = s.cookie_jar.filter_cookies('http://example.com')
for key, cookie in cookies.items():
if key == 'test':
cookie_value = cookie.value
# Using the cookie value to do anything you want:
# e.g. sending a weird request containing the cookie in the header instead.
headers = {"Authorization": "Basic f'{cookie_value}'"}
async with s.get(url, headers=headers) as r:
print(await r.json())
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
For testing urls containing a host part made up by an IP address use aiohttp.ClientSession(cookie_jar=aiohttp.CookieJar(unsafe=True)), according to https://github.com/aio-libs/aiohttp/issues/1183#issuecomment-247788489
Yes, the cookies are stored in request.cookies as a dict, just like in flask, so request.cookies.get('name_of_cookie') works the same.
In the examples section of the aiohttp repository there is a file, web_cookies.py that shows how to retrieve, set, and delete a cookie. Here's the section from that script that reads the cookies and returns it to the template as a preformatted string:
from pprint import pformat
from aiohttp import web
tmpl = '''\
<html>
<body>
Login<br/>
Logout<br/>
<pre>{}</pre>
</body>
</html>'''
async def root(request):
resp = web.Response(content_type='text/html')
resp.text = tmpl.format(pformat(request.cookies))
return resp
You can get the cookie value, domain, path etc, without having to loop thru all cookies.
s.cookie_jar._cookies
gives you all the cookies in a defaultdict with the domains as keys and their respective cookies as values. aiohttp uses SimpleCookie
So, to get the value of a cookie
s.cookie_jar._cookies.get("https://httpbin.org")["cookie_name"].value
for domain, path:
s.cookie_jar._cookies.get("https://httpbin.org")["cookie_name"]["domain"]
s.cookie_jar._cookies.get("https://httpbin.org")["cookie_name"]["path"]
more info can be found here: https://docs.python.org/3/library/http.cookies.html
I am using the Flask JWT Extended extension for flask and have built a login application successfully using JWT. I have gone through the tutorial on JWT in Cookies on the JWT extended documentation site using CSRF protection and everything.
What I can't seem to figure out is when using the set_access_cookies() and set_refresh_cookies() methods the JWTs are not saved in a httponly cookie named using the JWT Extended default configuration setting.
app.config.setdefault('JWT_ACCESS_COOKIE_NAME', 'access_token_cookie')
app.config.setdefault('JWT_REFRESH_COOKIE_NAME', 'refresh_token_cookie')
Instead when I debug the return back from the auth call the cookies are saved in the base Flask default configuration instead.
'SESSION_COOKIE_NAME': 'session',
Shouldn't the set_access_cookies() and set_refresh_cookies() methods override the base Flask default configurations as long as make sure to register my app in the JWTManager()?
uscc_login_app = Flask(__name__)
jwt = JWTManager(uscc_login_app)
Or is there something else I missed in the base Flask JWT Extended documentation to ensure that its configuration defaults are used when appropriate?
Updated code via request.
The code is pretty spread out but here is my best shot to include what I think will help.
In init.py:
from flask import Flask, url_for
from flask_restful import Api
from flask_jwt_extended import JWTManager
from resources.auth import Authenticate
from resources.refresh import Refresh
from temp_app.temp import TempApp
from uscc_login.uscc_app_login import *
uscc_login_app = Flask(__name__)
uscc_login_app.config.from_object(os.environ.get('FLASK_ENV'))
jwt = JWTManager(uscc_login_app)
api = Api(uscc_login_app, prefix='/v1')
# Add resources via the add_resource method
api.add_resource(Authenticate, '/login')
api.add_resource(Refresh, '/refresh_token')
login_view = Login.as_view(name='uscc_login')
uscc_login_app.add_url_rule('/login', view_func=login_view, methods=['POST', 'GET'])
In my app.py:
from uscc_login import uscc_login_app
if __name__ == '__main__':
uscc_login_app.run(debug=uscc_login_app.config.get('DEBUG'), threaded=uscc_login_app.config.get('THREADED'),
port=uscc_login_app.config.get('PORT'), host=uscc_login_app.config.get('HOST'))
In my config.py since I am using the Flask config.from_objects
import os
import datetime
uscc_login_app_dir = os.path.abspath(os.path.dirname(__file__))
class BaseConfig:
SECRET_KEY = os.environ.get('USCC_SECRET_KEY') or 'you-will-never-guess'
JWT_SECRET_KEY = os.environ.get('USCC_JWT_KEY') or 'super-secret'
JWT_TOKEN_LOCATION = ['cookies']
JWT_COOKIE_CSRF_PROTECT = True
JWT_HEADER_TYPE = 'JWT'
PROPAGATE_EXCEPTIONS = True
THREADED = True
class DevelopmentConfig(BaseConfig):
DEBUG = True
PORT = 5000 if os.environ.get("PORT") is None else int(os.environ.get("PORT"))
HOST = os.environ.get('HOST') or 'localhost'
if os.environ.get('access_token_expiration') is not None:
JWT_ACCESS_TOKEN_EXPIRES = datetime.timedelta(seconds=int(os.environ.get('access_token_expiration')))
if os.environ.get('refresh_token_expiration') is not None:
JWT_REFRESH_TOKEN_EXPIRES = datetime.timedelta(seconds=int(os.environ.get('refresh_token_expiration')))
So then in my Flask MethodView that contains my login authorization POST I have the following:
auth.py
import sys
import os
from flask import jsonify, request
from flask_restful import Resource
from flask_jwt_extended import create_access_token, create_refresh_token, jwt_refresh_token_required, get_jwt_identity, \
set_access_cookies, set_refresh_cookies
from utilities import Common
class Authenticate(Resource):
#staticmethod
def post():
"""
:return:
"""
api_cred_path = os.environ.get('api_cred_path')
if api_cred_path is None:
response = jsonify({"msg": "Environment Variable 'api_cred_path' is not set."})
response.status_code = 500
return response
if not request.is_json:
response = jsonify({'msg': 'Missing JSON in request'})
response.status_code = 400
return response
params = request.get_json()
user_name = params.get('username')
user_password = params.get('password')
if not user_name:
response = jsonify({'msg': 'Missing username parameter'})
response.status_code = 400
return response
if not user_password:
response = jsonify({'msg': 'Missing password parameter'})
response.status_code = 400
return response
if Common.check_path_exists(api_cred_path):
with open(api_cred_path) as afh:
for line in afh:
file_userid, file_password = line.split('=')
if file_userid == user_name and file_password.strip('\n') == user_password:
access_token = create_access_token(identity=user_name)
refresh_token = create_refresh_token(identity=user_name)
response = jsonify({'login': True})
set_access_cookies(response, access_token)
set_refresh_cookies(response, refresh_token)
# # Identity can be any data that is json serializable
# art = {
# 'access_token': create_access_token(identity=user_name),
# 'refresh_token': create_refresh_token(identity=user_name)}
# response = jsonify(art)
response.status_code = 200
return response
else:
response = jsonify({"msg": "api_cred_path invalid."})
response.status_code = 500
return response
response = jsonify({'msg': 'Bad username or password'})
response.status_code = 401
return response
Could you provide some code to duplicate what you are seeing? When I try running the example token in jwt code (https://github.com/vimalloc/flask-jwt-extended/blob/master/examples/jwt_in_cookie.py) I see the expected cookie values when I login:
$ http :5000/token/auth username=test password=test
...
Set-Cookie: access_token_cookie=<jwt>; HttpOnly; Path=/api/
Set-Cookie: refresh_token_cookie=<jwt>; HttpOnly; Path=/token/refresh
...
So I realized my mistake in this. I was trying to get the access_token_cookie variable to be set from my auth.py which serves as my RESTFUL based microservice of which my login app calls to do the authorization. Realized it won't be available after redirecting back to the caller from the login apps POST method since the cookie was related to the login app UI frontend. So I just base the access and refresh tokens back from the auth.py POST method to the login POST method and let it set the cookies so they are available to the end client.
This was more of design problem than a code problem.
I'm logged in on some page in Firefox and I want to take the cookie and try to browse webpage with python-requests. Problem is that after importing cookie to the requests session nothing happen (like there is no cookie at all). Structure of the cookie made by requests differ from the one from Firefox as well.
Is such it possible to load FF cookie and use it in requests session?
My code so far:
import sys
import sqlite3
import http.cookiejar as cookielib
import requests
from requests.utils import dict_from_cookiejar
def get_cookies(final_cookie, firefox_cookies):
con = sqlite3.connect(firefox_cookies)
cur = con.cursor()
cur.execute("SELECT host, path, isSecure, expiry, name, value FROM moz_cookies")
for item in cur.fetchall():
if item[0].find("mydomain.com") == -1:
continue
c = cookielib.Cookie(0, item[4], item[5],
None, False,
item[0], item[0].startswith('.'), item[0].startswith('.'),
item[1], False,
item[2],
item[3], item[3]=="",
None, None, {})
final_cookie.set_cookie(c)
cookie = cookielib.CookieJar()
input_file = ~/.mozilla/firefox/myprofile.default/cookies.sqlite
get_cookies(cookie, input_file)
#print cookie given from firefox
cookies = dict_from_cookiejar(cookie)
for key, value in cookies.items():
print(key, value)
s = requests.Session()
payload = {
"lang" : "en",
'destination': '/auth',
'credential_0': sys.argv[1],
'credential_1': sys.argv[2],
'credential_2': '86400',
}
r = s.get("mydomain.com/login", data = payload)
#print cookie from requests
cookies = dict_from_cookiejar(s.cookies)
for key, value in cookies.items():
print(key, value)
Structure of cookies from firefox is:
_gid GA1.3.2145214.241324
_ga GA1.3.125598754.422212
_gat_is4u 1
Structure of cookies from requests is:
UISTestAuth tesskMpA8JJ23V43a%2FoFtdesrtsszpw
After all, when trying to assign cookies from FF to session.cookies, requests works as I import nothing.
It looks like there are two type of cookies in the Firefox - request and response. It could be seen while Page inspector > Network > login (post) > Cookies:
Response cookies:
UISAuth
httpOnly true
path /
secure true
value tesskMpA8JJ23V43a%2FoFtdesrtsszpw
Request cookies:
_ga GA1.3.125598754.422212
_gat_is4u 1
_gid GA1.3.2145214.241324
The request cookies are stored in the cookies.sqlite file in the
~/.mozilla/firefox/*.default/cookies.sqlite
and can be load to the python object in more ways, for example:
import sqlite3
import http.cookiejar
def get_cookies(cj, ff_cookies):
con = sqlite3.connect(ff_cookies)
cur = con.cursor()
cur.execute("SELECT host, path, isSecure, expiry, name, value FROM moz_cookies")
for item in cur.fetchall():
c = cookielib.Cookie(0, item[4], item[5],
None, False,
item[0], item[0].startswith('.'), item[0].startswith('.'),
item[1], False,
item[2],
item[3], item[3]=="",
None, None, {})
print c
cj.set_cookie(c)
where cj is CookieJar object and ff_cookies is path to Firefox cookies.sqlite. Taken from this page.
The whole code to load cookies and import to the python requests using session would looks like:
import requests
import sys
cj = http.cookiejar.CookieJar()
ff_cookies = sys.argv[1] #pass path to the cookies.sqlite as an argument to the script
get_cookies(cj, ff_cookies)
s = requests.Session()
s.cookies = cj
Response cookie is basically session ID, which usualy expires at the end of the session (or some timeout), so they are not stored.
There is a package on PyPi, browser-cookie3, which does exactly this.
import browser_cookie3
import requests
cookiejar = browser_cookie3.firefox(domain_name='signed-in-website.tld')
resp = requests.get('https://signed-in-website.tld/path/', cookies=cookiejar)
print(resp.content)
browser_cookie3.firefox() retrieves Firefox cookies as a cookiejar with a domain name as optional argument.