Flask - b' text appears before request.data results - python-3.x

So I for a while I was running my application on Python's Localhost for debugging reasons, but now I desperately want to make my Flask application run on my Apache Localhost. I have configured Ubuntu on Vagrant to run the application on Apache and the only thing that does not work right now is the Facebook Sign-In. While my Google Sign-in method works fine on both Python's Localhost and on Apache, my Facebook Sign-in method only works on Python's Localhost and NOT on Apache's Localhost for some reason.
Specifically, in my fbconnect() method that preforms the fb sign-in functionality, when the code reaches my 4th print( print("4. FB http request results: %s" % result) ) of the given code block (scroll down for the block of code), the message given off by the 4th print is this error:
4. FB http request results: b'{"error":{"message":"Invalid OAuth access token.","type":"OAuthException","code":190,"fbtrace_id":"AjWGMxq54MULV0sCpjpW2DT"}}'
I don't know what the b' is doing there (it appears right at the start of the error message) and how to remove it, but it also appears in the 1st print( _print("1. Access token: %s " % access_token)_ ) :
1. Access token: b'EAAE7dgXeMUup...'
and it does NOT appear in the 2nd or the 3rd print:
2. App ID: 3425...
3. App secret: 75a2...
I think that the problem is caused by those b' since they do not appear in my prints when I run the app on my Python Localhost and they also do not appear in the 2nd or 3rd prints on Apache, but I'm not sure as they might be appearing because the print output is somehow changed when I write out the messages in a 'print.log' file, since Apache doesn't actually print out messages to the terminal like Python's Localhost does.
Here's my fbconnect() method:
def fbconnect():
''' Validate state token, by confirming that the token that the client sends
to the server matches the token that the server sent to the client. This
round-trip verification helps ensure that the user is making the request
not a malicious script. Using the request.args.get() method, we examine
the state token passed in by the ajax request and compare with the state
of the login_session. If these 2 do NOT match, we create a response of an
invalid state token and return this message to the client. '''
if request.args.get('state') != login_session['state']:
response = make_response(json.dumps('Invalid state parameter.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
access_token = request.data
print("1. Access token: %s " % access_token)
# Get Facebook app ID.
app_id = json.loads(open('/vagrant/Inhinito/fb_client_secrets.json', 'r').read())['web']['app_id']
print("2. App ID: %s " % app_id)
# Get Facebook app secret.
app_secret = json.loads(open('/vagrant/Inhinito/fb_client_secrets.json', 'r').read())['web']['app_secret']
print("3. App secret: %s " % app_secret)
# Make http request to the Facebook API.
url = "https://graph.facebook.com/oauth/access_token?client_id=%s" % (app_id)
url += "&client_secret=%s&grant_type=fb_exchange_token" % (app_secret)
url += "&fb_exchange_token=%s" % (access_token)
http = httplib2.Http()
result = http.request(url, 'GET')[1]
print("4. FB http request results: %s" % result)
....
Here's the correct output that I get from Python's Localhost:
1. Access token: EAgE7...
4. FB http request results: {"access_token":"EAgE7...","token_type":"bearer","expires_in":5183999}

As Doobeh mentioned in the comments the solution is to decode the data sent by the client-side to UTF-8. This is a Python2 vs Python3 issue explained in detail here.
The b' ' apparently is used to indicate the string is binary, as opposed to Unicode.
The solution is to use
access_token = request.data.decode('UTF-8')
Instead of
access_token = request.data and
http.request(url, 'GET')[1].decode('UTF-8') instead of http.request(url, 'GET')[1].

Related

Python3 how to resume at item in for loop when an exception was triggered

I'm somehow unable to came up with a solution after spending already few days on this. What I need to do:
Iterate trough a large list of hosts in a .txt file to perform a request to the devices/queries/devices/v1 API endpoint for CrowdStrike to get a so called aid (agent id).
Once I have the aid (there can be multiple aid's for a single host) I need to again iterate but this time over the aid's to the devices/entities/devices/v1 API endpoint to retrieve information such as hostname, last seen plus need to make another call, this time to the policy/combined/reveal-uninstall-token/v1 API endpoint, to get the uninstall token for each of the aid's.
The problem is that the bearer token that I'm generating initially is only valid for 30 minutes, and it needs to by provided with each subsequent request. My idea was to use the try except construct + I'm raising exceptions with Response raise for status as described in correct-way-to-try-except-using-python-requests-module. So I'm hoping that at any point I'm making a request, if it fails due to an HTTP error like 401 Client Error: Unauthorized for url: it would indicate that the token has expired and would call the login() function token = login() within except to make a call to request new a new token and assign it to the token variable and that would allow my to proceed with my for loop since the new token would be provided. The problem I now see, is that for the given host that I'm iterating over, when the token expires, it get's skipped right after the new token is generated (the loop basically continues for the next item within the list). How could I workaround this ?
PS. I know the code is far from perfect, but it's my first steps and would be really nice I someone could lead me here. Was looking and continue and break but continue seams to actually be doing exactly what I'm seeing as behavior with my code and break is not applicable for what I want to do.
for host in host_list:
try:
'''
1.Get agent id (aid) for each host (NOTE:can return multiple aid's for single host).
If the returned list is empty, that means the host is not in CS and skip the
processing of the remaining code.
'''
query_aid = requests.get(base_url+ 'devices/queries/devices/v1?limit=5000&filter=hostname:' + "'" + host + "'", headers = {"accept": "application/json", "authorization": "Bearer " + token} )
query_aid.raise_for_status()
aux = json.loads(query_aid.text)
aux2 = aux["resources"]
if len(aux2) == 0:
print(host + ' ; ' + 'not in CS')
else:
'''
for whatever count of aid's returned iterate through each and make a call to the
devices/entities/devices/v1 endpoint to pull hostname and last_seen information,
and make for each aid a call to the policy/combined/reveal-uninstall-token/v1
endpoint to pull the uninstall token for each
'''
for aid in aux2:
host_details_request = requests.get(base_url + 'devices/entities/devices/v1?ids=' + aid , headers = {"accept": "application/json", "authorization": "Bearer " + token})
host_details_request.raise_for_status()
aux = json.loads(host_details_request.text)
last_seen = re.search(r'\d{4}-\d{2}-\d{2}', aux["resources"][0]["last_seen"])
request_uninstall_token = requests.post(base_url + 'policy/combined/reveal-uninstall-token/v1', headers = {'Content-type': 'application/json', "authorization": "Bearer " + token }, json = { "audit_message": "test api ", "device_id": aid })
request_uninstall_token.raise_for_status()
aux3 = json.loads(request_uninstall_token.text)
print(aux["resources"][0]["hostname"] + 'uninstall token: ' + aux3["resources"][0]["uninstall_token"] + ';' + last_seen.group(0))
except requests.exceptions.HTTPError as err:
print(err)
token = login()
Create a function that handles your try code. Then call this function within your expect after the login call
for host in host_list:
try:
handle_host(host)
except requests.exceptions.HTTPError as err:
print(err)
token = login()
handle_host(host)

Python authorization

I need to write a script that is included in the personal account of my Internet provider and transmits information about the current balance.
At the moment I am stuck at the time of authorization. I found and edited such a script for myself:
import requests
url = 'https://bill.tomtel.ru/login.html'
USERNAME, PASSWORD, = 'mylogin', 'mypass'
resp = requests.get(url, auth=(USERNAME, PASSWORD))
r = requests.post(url)
print(r.content)
But this does not help to pass authorization...
I can enter this link through a browser and go to a page of this type:
https://bill.tomtel.ru/fastcom/!w3_p_main.showform?FORMNAME=QFRAME&CONFIG=CONTRACT&SID=BLABLABLA&NLS=WR
I can go through browser authorization through both links, but why can't I do this through a script?
Please help with this.
Your browser probably has a session token/cookie stored and that is why you can access it through the browser. There are a couple issues here:
It looks like you need to login to the site first -- through a POST method, not a GET. The GET is what loads the page. But once you submit the form it's going to do a POST request.
Actually, using requests to login to a site is not as easy as it looks. Usually you have to find the url it's posting to (examine the developer toolbar to see the url), and you often have to pass information in addition to your username/password, such as a csrf token, a cookie, or something else.
I would suggest using a browser-automator for this, perhaps something like selenium Webdriver. It makes logging into a site much simpler than using HTTP in a raw request, as it emulates a browser. I would suggest this -- it's much simpler and faster!
Another thing to note: auth=(USERNAME, PASSWORD) is not quite the username/password in the form (it's something else) but I don't think understanding that is too relevant to what you're trying to do.
Here is the url and required form data to log in:
I think you should try this:
import requests
url = 'https://bill.tomtel.ru/signin.php'
USERNAME = input('Enter your username: ')
PASSWORD = input('Enter your password: ')
d = {
'USERNAME' : USERNAME,
'PASSWORD' : PASSWORD,
'FORMNAME' : 'QFRAME'}
session = requests.Session()
resp = session.post(url, data=d).text
if not '<TITLE>' in resp:
print('Incorrect username or password!')
quit()
print('Logging in ... ')
for line in resp.split('\n'):
if 'location' in line:
red = 'https://bill.tomtel.ru/fastcom/!w3_p_main.showform%s' % line.replace(' if (P>0) self.location.replace("', '').replace('");', '')
if not red:
print('An error has occured')
quit()
print('Redirecting to %s' % red)
page = session.get(red).text
print('')
print(' MAIN PAGE')
print(page)

Set a passthrough using browsermob proxy?

So I'm using browsermob proxy to login selenium tests to get passed IAP for Google Cloud. But that just gets the user to the site, they still need to login to the site using some firebase login form. IAP has me adding Authorization header through browsermob so you can get to the site itself but the when you try to login through the firebase form you get a 401 error message: "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential..
I thought I could get around this using the whitelist or blacklist feature to just not apply those headers to urls related to the firebase login, but it seems that whitelist and blacklist just return status codes and empty responses for calls that match the regex.
Is there a way to just passthrough certain calls based on the host? Or on the off chance I'm doing something wrong, let me know. Code below:
class ExampleTest(unittest.TestCase):
def setUp(self):
server = Server("env/bin/browsermob-proxy/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()
bearer_header = {}
bearer_header['Authorization'] = 'Bearer xxxxxxxxexamplexxxxxxxx'
proxy.headers({"Authorization": bearer_header["Authorization"]})
profile = webdriver.FirefoxProfile()
proxy_info = proxy.selenium_proxy()
profile.set_proxy(proxy_info)
proxy.whitelist('.*googleapis.*, .*google.com.*', 200) # This fakes 200 from urls on regex match
# proxy.blacklist('.*googleapis.*', 200) # This fakes 200 from urls if not regex match
self.driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("file-example")
def test_wait(self):
self.driver.get("https://example.com/login/")
time.sleep(3)
def tearDown(self):
self.driver.close()
Figured this out a bit later. There isn't anything built into the BrowserMob proxy/client to do this. But you can achieve it through webdriver's proxy settings.
Chrome
self.chrome_options = webdriver.ChromeOptions()
proxy_address = '{}:{}'.format(server.host, proxy.port)
self.chrome_options.add_argument('--proxy-server=%s' % proxy_address)
no_proxy_string = ''
for item in no_proxy:
no_proxy_string += '*' + item + ';'
self.chrome_options.add_argument('--proxy-bypass-list=%s' % no_proxy_string)
self.desired_capabilities = webdriver.DesiredCapabilities.CHROME
self.desired_capabilities['acceptInsecureCerts'] = True
Firefox
self.desired_capabilities = webdriver.DesiredCapabilities.FIREFOX
proxy_address = '{}:{}'.format(server.host, proxy.port)
self.desired_capabilities['proxy'] = {
'proxyType': "MANUAL",
'httpProxy': proxy_address,
'sslProxy': proxy_address,
'noProxy': ['google.com', 'example.com']
}
self.desired_capabilities['acceptInsecureCerts'] = True

Connect to API and send back a message with Flask app

I have a flask app deployed to Heroku and would like to receive text from Chatfuel (bot building platform) and send back texts in return.
Now, what I did is to use my heroku app as a web-hook, so that Chatfuel can make a simple GET or POST query to my API. The problem is that I have no experience with Flask or APIs, so I am not sure about how my app can receive information (in json format) and send it back to chatfuel.
This is what I wrote so far:
import os
import sys
import json
import requests
from flask import Flask, jsonify, render_template, request
app = Flask(__name__)
#app.route('/', methods=['GET'])
def verify():
# when the endpoint is registered as a webhook, it must echo back
# the 'hub.challenge' value it receives in the query arguments
if request.args.get("hub.mode") == "subscribe" and request.args.get("hub.challenge"):
if not request.args.get("hub.verify_token") == os.environ["VERIFY_TOKEN"]:
return "Verification token mismatch", 403
return request.args["hub.challenge"], 200
return "Hello world", 200
#app.route("/json", methods=['GET','POST'])
def json():
url = "chatfuel_api"
data = json.load(urllib2.urlopen(url))
if request.json:
mydata = request.json
return "Thanks",200
else:
return "no json received"
#app.route('/hello', methods = ['GET','POST'])
def api_echo():
if request.method == 'GET':
return "ECHO: GET\n",200
if __name__ == '__main__':
app.run(debug=True)
The verify() function works, as I see an 'Hello world' message if I run the app locally. However, both json() and api_echo() don't work, and when my server receives a get or post request from chatfuel, it returns a 404 error.
As you can see, I really have a lot of confusion, and your help would be really invaluable.
Thanks
You need to make sure you have registered the proper webhook url with Chatfuel. For the code you currently have, to hit the json endpoint the url would be https://www.your_server.com/json
The verify route looks like the hub challenge FB sends, so you would have to register the root of your site (that is, with your current code) with FB to hit the verify function. That url would look like this https://www.your_site.com/

How to get saved tracks for a specific user, or change the current user?

The Spotify API has an endpoint "Get a Users's Saved Tracks" GET https://api.spotify.com/v1/me/tracks but as you can see from me in the url, and in the documentation, this is only for the current user. How can I access information about a non current user, or change the current user?
For example, userA logs in, I get an access and refresh token for userA. userB logs in, replacing userA as the current user, I get userB's tokens. How can I now make make requests for information about userA?
You need to store the tokens you get from authenticating users.
Say you're using user sessions:
User A logs in.
You get the access and refresh tokens for user A.
You save these tokens to User A's session.
User B logs in.
You get the access and refresh tokens for user B.
You save these tokens to User B's session.
You'd do this the same way that you have already implemented user sessions.
And so when a user lands on your redirect URI, you save the tokens you received to their session.
And then when you need to use the Spotify API you use the tokens saved in the users session.
If you however want to do this for one end-user, then with a web server things get a little harder.
But with a CLI app things can be a little easier.
What you want to do is log user A and B into your application, manually saving both tokens independently.
This is as easy as making an authentication function that you call twice and save the results to two variables.
After this you can then call the API with the saved tokens.
And use user A's token when you want to get user A's saved tracks.
Here's a low-level example implementation in Python 3 using Requests, of getting the users tracks and user information, using different scopes. Where the comments are the part the code's at in the authorization code flow:
import time
import urllib.parse as parse
import webbrowser
import requests
from requests.auth import HTTPBasicAuth
OAUTH_AUTHORIZE_URL = 'https://accounts.spotify.com/authorize'
OAUTH_TOKEN_URL = 'https://accounts.spotify.com/api/token'
# Change to your application settings
class Settings:
client_id = ''
client_secret = ''
redirect_uri = ''
def authenticate(scope=None):
'''Implement OAuth 2 Spotify authentication'''
# Application: Request authorization to access data
payload = {'client_id': Settings.client_id,
'response_type': 'code',
'redirect_uri': Settings.redirect_uri,
'show_dialog': 'true'} # allow second account to login
if scope:
payload['scope'] = scope
auth_url = '{}?{}'.format(OAUTH_AUTHORIZE_URL, parse.urlencode(payload))
# Spotify: Displays scopes & prompts user to login (if required)
# User: Logs in, authorizes access
webbrowser.open(auth_url)
response = input('Enter the URL you were redirected to: ')
code = parse.parse_qs(parse.urlparse(response).query)['code'][0]
payload = {'redirect_uri': Settings.redirect_uri,
'code': code,
'grant_type': 'authorization_code'}
if scope:
payload['scope'] = scope
# Application: Request access and refresh tokens
# Spotify: Returns access and refresh tokens
auth = HTTPBasicAuth(Settings.client_id, Settings.client_secret)
response = requests.post(OAUTH_TOKEN_URL, data=payload, auth=auth)
if response.status_code != 200:
response.raise_for_status()
token_info = response.json()
token_info['expires_at'] = int(time.time()) + token_info['expires_in']
token_info['scope'] = scope
return token_info
if __name__ == '__main__':
user_a = authenticate(scope='user-library-read')
user_b = authenticate(scope='user-read-email user-read-private user-read-birthdate')
print('user_a', user_a)
print('user_b', user_b)
for url in ['https://api.spotify.com/v1/me/tracks',
'https://api.spotify.com/v1/me']:
for user in [user_a, user_b]:
token = 'Bearer ' + user['access_token']
# Application: Uses access token in requests to Web API
# Spotify: Returns request data
r = requests.get(url, headers={'authorization': token})
if r.status_code != 200:
print(r.text)
else:
print([
'{}: {}'.format(key, str(value)[:20])
for key, value in r.json().items()
])

Resources