How can I access Airtable's pagination by including a given offset in either the url or query?
My attempt below at including in the url has failed with the error "KeyError: 'offset'".
I've been able to access the 1st offset per the function at_page(). For simplicity's sake, I took the result from the function and hardcoded it to the variable offset_01.
Thank you!
import requests
import pandas as pd
offset_01 = 'itrCG3VDp2c34x1j1/recKTJGYMICe13iA8'
url = 'https://api.airtable.com/v0/PRIVATETABLEKEY/accounts' + \
'&offset=' + offset_01
headers = {'Authorization': 'Bearer PRIVATEAPIKEY'}
# Calling API
response = requests.get(
url,
headers=headers,
)
json_response = response.json()
# Finding AT page offset
def at_page(at_json):
df_initial = pd.DataFrame(at_json)
at_offset = df_initial['offset'][0]
return at_offset
offset = at_page(json_response)
Figured it out:
It’s the params={‘offset’: ‘itrXXXXXX/recXXXXXX’} query that allows access to the next 100 records:
# Calling API
response = requests.get(
url,
headers=headers,
params={'offset': 'itrXXXXXX/recXXXXXX'}
)
json_response = response.json()
Hope this helps someone. Cheers!
Related
I am programming an API in python to query a server if it has endpoint agents in it. The server and the endpoint belong to Apex central SaaS trend micro.
The error I get is that I'm putting the wrong parameters but I don't think the problem is there.
The code I have for the query is as follows:
import base64
import jwt
import hashlib
import requests
import time
import json
import urllib.parse
def create_checksum(http_method, raw_url, headers, request_body):
string_to_hash = http_method.upper() + '|' + raw_url.lower() + '|' + headers + '|' + request_body
base64_string = base64.b64encode(hashlib.sha256(str.encode(string_to_hash)).digest()).decode('utf-8')
return base64_string
def create_jwt_token(appication_id, api_key, http_method, raw_url, headers, request_body,
iat=time.time(), algorithm='HS256', version='V1'):
payload = {'appid': appication_id,
'iat': iat,
'version': version,
'checksum': create_checksum(http_method, raw_url, headers, request_body)}
token = jwt.encode(payload, api_key, algorithm=algorithm)
return token
# Use this region to setup the call info of the Apex Central server (server url, application id, api key)
# server info
use_url_base = 'https://arct3w.manage.trendmicro.com'
use_application_id = '52EB0005-B6DA-4249-9764-62AE3BFCDBB1'
use_api_key = 'B3FE1D91-5D05-490C-B45C-26A9EFF6C363'
productAgentAPIPath = '/WebApp/API/AgentResource/ProductAgents'
canonicalRequestHeaders = ''
useQueryString=''
payload = {
'computerId':'e34a13a1-1d0f-47bc-96e0-ae4db4288940'
}
useRequestBody = json.dumps(payload)
jwt_token = create_jwt_token(use_application_id, use_api_key, 'POST',
productAgentAPIPath + useQueryString,
canonicalRequestHeaders, useRequestBody, iat=time.time())
headers = {'Authorization': 'Bearer ' + jwt_token , 'Content-Type': 'application/json;charset=utf-8'}
#Choose by call type.
r = requests.post(use_url_base + productAgentAPIPath + useQueryString, data=useRequestBody, headers=headers, verify=False)
print(r.status_code)
if 'application/json' in r.headers.get('Content-Type', '') and len(r.content):
print(json.dumps(r.json(), indent=4))
else:
print(r.text)
I tried to do something similar to an api belonging to Vision One but it didn't work:
https://automation.trendmicro.com/xdr/api-v2#tag/Search/paths/~1v2.0~1xdr~1eiqs~1query~1endpointInfo/post
the original code of the query is :
https://automation.trendmicro.com/apex-central/api#tag/Security-Agents/operation/AgentResource_GetProductAgentsV2
I am trying to send a get request to Airtable to get metadata but I keep getting errors.
I have tried the below.
base_schema_url = 'https://api.airtable.com/v0/meta/bases/BaseId(replaced this)/tables'
secret_meta_token = '123'
#attempt 1
r = requests.get(base_schema_url, headers={'Authorization': secret_meta_token})
print(r.text)
#attempt 2
r = requests.get(base_schema_url,
headers={'Content-Type':'application/json',
'Authorization': 'Bearer {}'.format(secret_meta_token)})
print(r.text)
#attempt 3
header = {'PRIVATE-TOKEN': secret_meta_token}
r = requests.get(base_schema_url, headers=header)
print(r.text)
I have a feeling it's because I need to pass in the user API + the Meta token but I'm not too sure how to do it.
The metadata API uses a separate key that you need to send a formal request to the Airtable team by registering at the link provided at the top of the page.
import requests
user_account_api = xxx
meta_api = xxx
base_id = xxx
headers = {
"Authorization":f"Bearer {user_account_api}",
"x-airtable-client-secret": meta_api
}
response = requests.get(f"https://api.airtable.com/v0/meta/bases/{base_id}/tables", headers=headers)
print(response.text)
I've tried to do a POST request using python's request library, which looked something like below:
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url,data='Datasets/rtHereTest.csv')
The response I've been getting a code 400
{'faultCode': '16a6f70f-1fa3-4b57-9ef3-a0a440f8a42e',
'responseCode': '400 Bad Request',
'message': 'Column LATITUDE missing'}
However, in my dataset, here I have all the headings that's required from the HERE API documentation to be able to make a call.
Is there something I'm doing wrong, I don't quite understand the POST call or the requirement as the HERE documentation doesn't explicitly give many examples.
The data field of your post request should contain the actual data, not just a filename. Try loading the file first:
f = open('Datasets/rtHereTest.csv', 'r')
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url, data=f.read())
f.close()
Here's what I use in my own code, with the coordinates defined before:
query = 'https://rme.api.here.com/2/matchroute.json?routemode=car&app_id={id}&app_code={code}'.format(id=app_id, code=app_code)
coord_strings = ['{:.5f},{:.5f}'.format(coord[0], coord[1]) for coord in coords]
data = 'latitude,longitude\n' + '\n'.join(coord_strings)
result = requests.post(query, data=data)
You can try to post the data using below format.
import requests
url = "https://rme.api.here.com/2/matchroute.json"
querystring = {"routemode":"car","app_id":"{app_id}","app_code":"{app_code}","filetype":"CSV"}
data=open('path of CSV file','r')
headers = {'Content-Type': "Content-Type: text/csv;charset=utf-8",
'Accept-Encoding': "gzip, deflate",
}
response = requests.request("POST", url, data=data.read().encode('utf-8'), params=querystring,headers=headers)
print(response.status_code)
Im using python 3 to scrape facebook page of "nytimes".
I tried to create my own Api first so i can get my app_id && app_secret.
when i tried to ping NYT's Facebook page to verify that the access_token works and the page_id is valid i got this ERROR.
HTTPError: HTTP Error 400: Bad Request
My code is Below
First connect to my API via id and secret code.
`
import urllib
import datetime
import json
import time
import urllib.request
id = "111111111111"
secret ="123123123123123113"
token = id +"|"+ secret`
Second To ping the page i did:
page_id = 'nytimes'
Then
def testFacebookPageData(page_id, token):
# construct the URL string
base = "https://graph.facebook.com/v2.4"
node = "/" + page_id
parameters = "/?token=%s" % token
url = base + node + parameters
# retrieve data
req = urllib.request.Request(url)
response = urllib.request.urlopen(req)
data = json.loads(response.read())
print (json.dumps(data, indent=4, sort_keys=True))
testFacebookPageData(page_id, token)
Hope that im clear..
I need your help
thank You
According to Facebooks docs for Graph API, the parameter should be named access_token, and not token.
parameters = "/?access_token=%s" % token
I've got a list of ~3,000 URLs I'm trying to create Google shortened links of, the idea is this CSV has a list of links and I want my code to output the shortened links in the column next to the original URLs.
I've been trying to modify the code found on this site here but I'm not skilled enough to get it to work.
Here's my code (I would not normally post an API key but the original person who asked this already posted it publicly on this site) :
import json
import pandas as pd
df = pd.read_csv('Links_Test.csv')
def shorternUrl(my_URL):
API_KEY = "AIzaSyCvhcU63u5OTnUsdYaCFtDkcutNm6lIEpw"
apiUrl = 'https://www.googleapis.com/urlshortener/v1/url'
longUrl = my_URL
headers = {"Content-type": "application/json"}
data = {"longUrl": longUrl}
h = httplib2.Http('.cache')
headers, response = h.request(apiUrl, "POST", json.dumps(data), headers)
return response
for url in df['URL']:
x = shorternUrl(url)
# Then I want it to write x into the column next to the original URL
But I only get errors at this point, before I even started figuring out how to write the new URLs to the CSV file.
Here's some sample data:
URL
www.apple.com
www.google.com
www.microsoft.com
www.linux.org
Thank you for your help,
Me
I think the issue is that you didnot include the API key in the request. By the way, the certifi package allows you to secure a connection to a link. You can get it using pip install certifi or pip urllib3[secure].
Here I create my own API key, so you might want to replace it with yours.
from urllib3 import PoolManager
import json
import certifi
sampleURL = 'http://www.apple.com'
APIkey = 'AIzaSyD8F41CL3nJBpEf0avqdQELKO2n962VXpA'
APIurl = 'https://www.googleapis.com/urlshortener/v1/url?key=' + APIkey
http = PoolManager(cert_reqs = 'CERT_REQUIRED', ca_certs=certifi.where())
def shortenURL(url):
data = {'key': APIkey, 'longUrl' : url}
response = http.request("POST", APIurl, body=json.dumps(data), headers= {'Content-Type' : 'application/json'}).data.decode('utf-8')
r = json.loads(response)
return (r['id'])
The decoding part converts the response object into a string so that we can convert it to a JSON and retrieve data.
From there on, you can store the data into another column and so on.
For the sampleUrl, I got back https(goo.gl/nujb) from the function.
I found a solution here:
https://pypi.python.org/pypi/pyshorteners
Example copied from linked page:
from pyshorteners import Shortener
url = 'http://www.google.com'
api_key = 'YOUR_API_KEY'
shortener = Shortener('Google', api_key=api_key)
print "My short url is {}".format(shortener.short(url))