Im using python 3 to scrape facebook page of "nytimes".
I tried to create my own Api first so i can get my app_id && app_secret.
when i tried to ping NYT's Facebook page to verify that the access_token works and the page_id is valid i got this ERROR.
HTTPError: HTTP Error 400: Bad Request
My code is Below
First connect to my API via id and secret code.
`
import urllib
import datetime
import json
import time
import urllib.request
id = "111111111111"
secret ="123123123123123113"
token = id +"|"+ secret`
Second To ping the page i did:
page_id = 'nytimes'
Then
def testFacebookPageData(page_id, token):
# construct the URL string
base = "https://graph.facebook.com/v2.4"
node = "/" + page_id
parameters = "/?token=%s" % token
url = base + node + parameters
# retrieve data
req = urllib.request.Request(url)
response = urllib.request.urlopen(req)
data = json.loads(response.read())
print (json.dumps(data, indent=4, sort_keys=True))
testFacebookPageData(page_id, token)
Hope that im clear..
I need your help
thank You
According to Facebooks docs for Graph API, the parameter should be named access_token, and not token.
parameters = "/?access_token=%s" % token
Related
I'm trying to make requests to the Google API to create source repositories using a service account and his JSON key file.
Since there are no client libraries for this product, I am using the queries with Python using this documentation
https://cloud.google.com/source-repositories/docs/reference/rest
I already used a similar code to invoke my cloud-functions with success, but this time I'm block for these requests at the 401 error. I set up the GOOGLE_APPLICATION_CREDENTIALS with the JSON of my service account, give the service-account the permissions of Source Repository Administrator, but still return 401.
Here's my code
import urllib.request
import json
import urllib
import google.auth.transport.requests
import google.oauth2.id_token
body = { "name" : "projects/$my_project_name/repos/$name_repo"}
jsondata = json.dumps(body).encode("utf8")
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos')
req.add_header('Content-Type', 'application/json; charset=utf-8')
auth_req = google.auth.transport.requests.Request()
id_token = google.oauth2.id_token.fetch_id_token(auth_req, 'https://www.googleapis.com/auth/cloud-platform')
req.add_header("Authorization", f"Bearer {id_token}")
response = urllib.request.urlopen(req, jsondata)
print (response.read().decode())
I tried also using the with an API-KEY at the end of the url like this
req = urllib.request.Request('https://sourcerepo.googleapis.com/v1/projects/$my_project_name/repos?key=$my-api-key')
Thank you
I tried also using the with an API-KEY at the end of the url like this
API Keys are not supported.
Your code is using an OIDC Identity Token instead of an OAuth Acess Token.
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(
'/path/to/key.json',
scopes=['https://www.googleapis.com/auth/cloud-platform'])
request = google.auth.transport.requests.Request()
credentials.refresh(request)
// Use the following code to add the access token:
req.add_header("Authorization", f"Bearer {credentials.token}")
I have a Django app that I have been working on. When I run it locally it runs perfectly. When I run it in a container using Cloud Run I get the following error:
'Credentials' object has no attribute 'id_token'
Here is the offending code (payload is a dictionary object):
def ProcessPayload(payload):
# Get authorized session credentials
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
credentials.refresh(Request(session))
# Process post request
headers = {'Authorization': f'Bearer {credentials.id_token}'}
response = requests.post(URL, json=payload, headers=headers)
In my local environment, the refresh properly loads credentials with the correct id_toled for the needed header, but for some reason when the code is deployed to Cloud Run this does not work. I have the Cloud run instance set to use a service account so it should be able to get credentials from it. How do I make this work? I have googled until my fingers hurt and have found no viable solutions.
When executing code under a Compute Service (Compute Engine, Cloud Run, Cloud Functions), call the metadata service to obtain an OIDC Identity Token.
import requests
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/' \
'instance/service-accounts/default/identity?' \
'audience={}'
def fetch_identity_token(audience):
# Construct a URL with the audience and format.
url = METADATA_URL.format(audience)
# Request a token from the metadata server.
r = requests.get(url, headers=METADATA_HEADERS)
r.raise_for_status()
return r.text
def ProcessPayload(payload):
id_token = fetch_identity_token('replace_with_service_url')
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(URL, json=payload, headers=headers)
The equivalent curl command to fetch an Identity Token looks like this. You can test from a Compute Engine instance:
curl -H "metadata-flavor: Google" \
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=URL
where URL is the URL of the service you are calling.
Authentication service-to-service
I have seen this metadata URL shortcut (for Cloud Run), but I have not verified it:
http://metadata/instance/service-accounts/default/identity?audience=URL
So, after much playing around I found a solution that works in both places. Many thanks to Paul Bonser for coming up with this simple method!
import google.auth
from google.auth.transport.requests import AuthorizedSession, Request
from google.oauth2.id_token import fetch_id_token
import requests
def GetIdToken(audience):
credentials, _ = google.auth.default()
session = AuthorizedSession(credentials)
request = Request(session)
credentials.refresh(request)
if hasattr(credentials, "id_token"):
return credentials.id_token
return fetch_id_token(request, audience)
def ProcessPayload(url, payload):
# Get the ID Token
id_token = GetIdToken(url)
# Process post request
headers = {'Authorization': f'Bearer {id_token}'}
response = requests.post(url, json=payload, headers=headers)
I am trying to get the tweets in a 10 miles radius around Twitter's headquarters in San Francisco location using Tweepy API. I have authenitication information (api key, etc.) in an external .json file.
Below is my code:
import tweepy as tw
import json
import pandas as pd
import datetime
secrets = json.loads(open('secrets.json').read())
api_key = secrets['api_key']
api_secret_key = secrets['api_secret_key']
access_token = secrets['access_token']
access_token_secret = secrets['access_token_secret']
auth = tw.OAuthHandler(api_key, api_secret_key)
auth.set_access_token(access_token, access_token_secret)
api = tw.API(auth)
try:
api.verify_credentials()
print('Authenticated')
except:
print("Error authenticating")
# After running the above, I get status 'Authenitcated'
start = '2020-09-01'
end = '2020-09-26'
location = '37.7749, 122.4149, 10miles'
query = "*"
max_tweets = 1000
tweet_search_results = [status for status in tw.Cursor(api.search, query=query, geocode=location, since=start, until=end).items(max_tweets)]
After running the above, I get this error:
TweepError: Twitter error response: status code = 403
This status code means access to the requested resource is forbidden for some reason. The server understood the request, but will not fulfill due to the client-related issues.
Within the same code I am able to get the list of tweets for a specific user name, but without geolocation data.
Why am I getting 403 when trying to get geolocation tweets?
I'm trying to generate a user authentication token by posting the user name, password in a URL-encoded form, and received a JWT token.
I have tried using payload instead of header and both, but nothing works. I'm using Python 3.7.
import requests
headers = {'username':'username', 'password':'password'}
r = requests.post("https://sso.xxxxxx.com/sso-api/v1/token", headers=headers)
print (r)
I expected the output Response 200 Token generated successfully, but I'm receiving the error Response 404
I was able to to do it the the tips and help from xbound. Instead of header, I had to pass it as data. Then I confirmed that the response was ok and extracted the token.
import requests
payload = {'username':'*******#*********.com', 'password':'**********', 'grant_type':'password', 'scope':'openid'}
r = requests.post("https://***.******.com/***/**/token", data = payload)
##Confirm that reponse is OK!
r.status_code
print(r.status_code == requests.codes.ok)
##Decode json reponse - token
data = r.json()
##Select token
my_token = data['id_token']
I've got a list of ~3,000 URLs I'm trying to create Google shortened links of, the idea is this CSV has a list of links and I want my code to output the shortened links in the column next to the original URLs.
I've been trying to modify the code found on this site here but I'm not skilled enough to get it to work.
Here's my code (I would not normally post an API key but the original person who asked this already posted it publicly on this site) :
import json
import pandas as pd
df = pd.read_csv('Links_Test.csv')
def shorternUrl(my_URL):
API_KEY = "AIzaSyCvhcU63u5OTnUsdYaCFtDkcutNm6lIEpw"
apiUrl = 'https://www.googleapis.com/urlshortener/v1/url'
longUrl = my_URL
headers = {"Content-type": "application/json"}
data = {"longUrl": longUrl}
h = httplib2.Http('.cache')
headers, response = h.request(apiUrl, "POST", json.dumps(data), headers)
return response
for url in df['URL']:
x = shorternUrl(url)
# Then I want it to write x into the column next to the original URL
But I only get errors at this point, before I even started figuring out how to write the new URLs to the CSV file.
Here's some sample data:
URL
www.apple.com
www.google.com
www.microsoft.com
www.linux.org
Thank you for your help,
Me
I think the issue is that you didnot include the API key in the request. By the way, the certifi package allows you to secure a connection to a link. You can get it using pip install certifi or pip urllib3[secure].
Here I create my own API key, so you might want to replace it with yours.
from urllib3 import PoolManager
import json
import certifi
sampleURL = 'http://www.apple.com'
APIkey = 'AIzaSyD8F41CL3nJBpEf0avqdQELKO2n962VXpA'
APIurl = 'https://www.googleapis.com/urlshortener/v1/url?key=' + APIkey
http = PoolManager(cert_reqs = 'CERT_REQUIRED', ca_certs=certifi.where())
def shortenURL(url):
data = {'key': APIkey, 'longUrl' : url}
response = http.request("POST", APIurl, body=json.dumps(data), headers= {'Content-Type' : 'application/json'}).data.decode('utf-8')
r = json.loads(response)
return (r['id'])
The decoding part converts the response object into a string so that we can convert it to a JSON and retrieve data.
From there on, you can store the data into another column and so on.
For the sampleUrl, I got back https(goo.gl/nujb) from the function.
I found a solution here:
https://pypi.python.org/pypi/pyshorteners
Example copied from linked page:
from pyshorteners import Shortener
url = 'http://www.google.com'
api_key = 'YOUR_API_KEY'
shortener = Shortener('Google', api_key=api_key)
print "My short url is {}".format(shortener.short(url))