KeyError when trying to send body to API via function parameter - python-3.x

Attempting to send a POST request to an API via a python function, and I'm unable to iterate through a list of strings and pass strings into the function.
Successfully tested this out in Postman (the request sends a string to the API in the form of "raw Body" as shown in Postman). Copied the code over from Postman to Python, and verified that also works. The problem is that if I change the static string to the function's parameter, I get a KeyError, however if I simply replace the parameter (which is a value, not a key), to a string, then the keyerror goes away.
This works...
payload = "{\"ids\":[\"cb5f9a97c0e749ab67409e78b4fcb11d\"]}" works
But none of these work (note the error code to the right); especially the first two that are exactly the same as above...
payload = "{\"ids\":[\"{0}\"]}".format(id) #gives KeyError: '"ids"'
payload = "{\"ids\":[{0}]}".format(id) #gives KeyError: '"ids"'
payload = "{'ids':'[{0}]'}".format(id) #gives KeyError: "'ids'"
payload = "{ids:[\"{0}\"]}".format(id) #gives KeyError: 'ids'
I also tried to modify the key ('ids') within the key/value pair, which resulted in NameErrors. Since this deviates from the known working example above, I don't think the below attempts are worth continuing to try...
payload = {ids:"[{0}]".format(id)} #gives NameError: name 'ids' is not defined
payload = {ids:"{0}".format(id)} #gives NameError: name 'ids' is not defined
payload = {ids:id} #gives NameError: name 'ids' is not defined
I even verified that the string produced from the list is in fact a string.
Full (relevant) code below:
def cs_delete(id):
print(id)
url = "https://api.crowdstrike.com/devices/entities/devices-actions/v2"
querystring = {"action_name":"hide_host"}
payload = "{'ids':['{0}']}".format(id)
headers = {
'Content-Type': "application/json",
'Authorization': "Bearer " + cs_auth,
'Accept': "*/*",
'Cache-Control': "no-cache",
'Host': "api.crowdstrike.com",
'Accept-Encoding': "gzip, deflate",
'Content-Length': "83",
'Connection': "keep-alive",
'cache-control': "no-cache"
}
response = requests.request("POST", url, data=payload, headers=headers, params=querystring)
print(response.text)
for host in dfList:
print(host)
cs_delete(host)
And for completeness, dfList shows as:
['2a9cf64988e6464f7d2ba7f305a612f3', '5ba4654e1dbe418f7b6361582e3d8f47', '7c6fc20572c241f664813f48bb36c340', 'ccbaf1ebe52042fc6b8269bf86732676']

double the outer curly bracket to escape it or format() will expect the input value should be applied for outer curly bracket rather than {0}
>>> id = 'cb5f9a97c0e749ab67409e78b4fcb11d'
>>> "{{'ids':[{0}]}}".format(id)
"{'ids':[cb5f9a97c0e749ab67409e78b4fcb11d]}"

Related

Cloud Functions with Requests return 'Could not parse JSON' error

I'm running a cloud function in python to return some data from an api. The function is not executed and I have the error {'code': 400, 'message': 'Could not parse JSON'}.
Here is my code:
import requests
import json
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX',
'content-type': 'application/json'}
data = '{"dateRangeStart":"2020-05-10T00:00:00.000","dateRangeEnd":"2020-05-16T23:59:59.000","amountShown": "HIDE_AMOUNT","detailedFilter":{ "page":"1","pageSize":"50"}}'
#req = requests.post(url, headers=headers, json=data)
req = requests.post(url, headers=headers, data=json.dumps(data))
print(req.json())
how should I format my data variable?
Just give your dict as your json argument, you don't need to specify the content-type headers requests will do it for you.
import requests
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX', }
data = {"dateRangeStart": "2020-05-10T00:00:00.000", "dateRangeEnd": "2020-05-16T23:59:59.000", "amountShown": "HIDE_AMOUNT", "detailedFilter": { "page": "1", "pageSize": "50", }, }
req = requests.post(url, headers=headers, json=data)
print(req.json())
If you do not set a content-type header will try to set it for you:
When using the keyword argument json: it will set it to application/json
When using the keyword argument data (and the value passed respects some criteria, most of the time you don't have to worry about it): it will set it to application/x-www-form-urlencoded
If both kwargs are present data takes priority for the header so it will be set to application/x-www-form-urlencoded
I have not detailed the behaviour when the kwarg files is used as it would be really lengthy and is out of scope here.
Here's the source code.

requests.post makes a get request

I call requests.post but it ends up making a GET request.
post_body="""
{
...
}
"""
headers = {'Content-type': 'application/json', 'Accept': 'application/json'}
post_response = requests.post("https://...", data=post_body, headers=headers)
print(post_response.request.method)
The last print statement prints "GET". I expected to see "POST".
To debug this further, I changed the code like so:
req = requests.Request('POST', "https://...", data=booking_body, headers=headers)
prepared = req.prepare
print(prepared.method) // "POST"
s = requests.Session()
post_response = s.send(prepared)
print(post_response.request.method) // "GET"
The print statements print "POST" and "GET". What am I doing wrong?
PS:
$ python3 -V
Python 3.7.0
As stated in the comments, the issue was in redirect. The call was initially been made to http://... then redirected to https://.... Hence, the last method was GET.
Once the initial call was made to https://..., the issue was resolved.

How to get a POST request using python on HERE route matching API?

I've tried to do a POST request using python's request library, which looked something like below:
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url,data='Datasets/rtHereTest.csv')
The response I've been getting a code 400
{'faultCode': '16a6f70f-1fa3-4b57-9ef3-a0a440f8a42e',
'responseCode': '400 Bad Request',
'message': 'Column LATITUDE missing'}
However, in my dataset, here I have all the headings that's required from the HERE API documentation to be able to make a call.
Is there something I'm doing wrong, I don't quite understand the POST call or the requirement as the HERE documentation doesn't explicitly give many examples.
The data field of your post request should contain the actual data, not just a filename. Try loading the file first:
f = open('Datasets/rtHereTest.csv', 'r')
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url, data=f.read())
f.close()
Here's what I use in my own code, with the coordinates defined before:
query = 'https://rme.api.here.com/2/matchroute.json?routemode=car&app_id={id}&app_code={code}'.format(id=app_id, code=app_code)
coord_strings = ['{:.5f},{:.5f}'.format(coord[0], coord[1]) for coord in coords]
data = 'latitude,longitude\n' + '\n'.join(coord_strings)
result = requests.post(query, data=data)
You can try to post the data using below format.
import requests
url = "https://rme.api.here.com/2/matchroute.json"
querystring = {"routemode":"car","app_id":"{app_id}","app_code":"{app_code}","filetype":"CSV"}
data=open('path of CSV file','r')
headers = {'Content-Type': "Content-Type: text/csv;charset=utf-8",
'Accept-Encoding': "gzip, deflate",
}
response = requests.request("POST", url, data=data.read().encode('utf-8'), params=querystring,headers=headers)
print(response.status_code)

Requests.get gives error Microsoft.SharePoint.Client.InvalidClientQueryException

I am calling sharepoint from my python requests.get code to pull sharepoint list items. However i am getting below error.
{"odata.error":{"code":"-1, Microsoft.SharePoint.Client.InvalidClientQueryException","message":{"lang":"en-US","value":"The expression \"web/lists/GetByTitle(Useful Links)/items\" is not valid."}}}
This works fine when i call equivalent code from my angular code.
I tried to encode the url as it has spaces, brackets but it did not help.
i verified the url works fine by testing it from angular code and also postman tool.
#app.route('/getList',methods=['GET'])
def getList():
try:
api_token = request.args.get('api_token', '')
headers = {'Accept': 'application/json; odata=nometadata',
'Authorization': 'Bearer {0}'.format(api_token)}
api_url = '//<my company name>.sharepoint.com/teams/EIMGDS/_api/web/lists/GetByTitle(Useful Links)/items'
response = requests.get("https:"+urllib.parse.quote(api_url),params={},headers=headers, verify=False)
return response.text
except Exception as ex:
return jsonify({'message' : 'Error in Get List', 'detailedMessage': traceback.format_exc()}), 500
Expected result:
JSON having the sharepoint list details
Actual result:
{"odata.error":{"code":"-1, Microsoft.SharePoint.Client.InvalidClientQueryException","message":{"lang":"en-US","value":"The expression \"web/lists/GetByTitle(Useful Links)/items\" is not valid."}}}

Multipart form post, csv and param python

Im having difficulty with the following code:
payload = {
'file' : ('csvtest.csv', open('csvtest.csv', 'rb')),
'parser' : '{"name":"CSV","displayName":null,"description":"Supports delimited text files with a field delimiter and optional escape and quote characters.","shortDescription":null,"properties":[{"name":"Auto Detect?","displayName":"Auto Detect?","value":"true","values":null,"placeholder":"","type":"select","hint":"Auto detect will attempt to infer delimiter from the sample file.","objectProperty":"autoDetect","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":1,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_0"},{"name":"Header?","displayName":"Header?","value":"true","values":null,"placeholder":"","type":"select","hint":"Whether file has a header.","objectProperty":"headerRow","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":2,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_1"},{"name":"Delimiter Char","displayName":"Delimiter Char","value":",","values":null,"placeholder":"","type":"string","hint":"Character separating fields","objectProperty":"separatorChar","selectableValues":[],"required":false,"group":"","groupOrder":3,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_2"},{"name":"Quote Char","displayName":"Quote Char","value":"\'","values":null,"placeholder":"","type":"string","hint":"Character enclosing a quoted string","objectProperty":"quoteChar","selectableValues":[],"required":false,"group":"","groupOrder":4,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_3"},{"name":"Escape Char","displayName":"Escape Char","value":"\\\\","values":null,"placeholder":"","type":"string","hint":"Escape character","objectProperty":"escapeChar","selectableValues":[],"required":false,"group":"","groupOrder":5,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_4"}],"objectClassType":"com.thinkbiganalytics.discovery.parsers.csv.CSVFileSchemaParser","objectShortClassType":"CSVFileSchemaParser","propertyValuesDisplayString":null,"supportsBinary":false,"generatesHiveSerde":true,"tags":["CSV","TSV"],"clientHelper":null,"allowSkipHeader":true,"groups":[{"group":"","layout":"column","properties":[{"name":"Auto Detect?","displayName":"Auto Detect?","value":"true","values":null,"placeholder":"","type":"select","hint":"Auto detect will attempt to infer delimiter from the sample file.","objectProperty":"autoDetect","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":1,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_0"},{"name":"Header?","displayName":"Header?","value":"true","values":null,"placeholder":"","type":"select","hint":"Whether file has a header.","objectProperty":"headerRow","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":2,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_1"},{"name":"Delimiter Char","displayName":"Delimiter Char","value":",","values":null,"placeholder":"","type":"string","hint":"Character separating fields","objectProperty":"separatorChar","selectableValues":[],"required":false,"group":"","groupOrder":3,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_2"},{"name":"Quote Char","displayName":"Quote Char","value":"\'","values":null,"placeholder":"","type":"string","hint":"Character enclosing a quoted string","objectProperty":"quoteChar","selectableValues":[],"required":false,"group":"","groupOrder":4,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_3"},{"name":"Escape Char","displayName":"Escape Char","value":"\\\\","values":null,"placeholder":"","type":"string","hint":"Escape character","objectProperty":"escapeChar","selectableValues":[],"required":false,"group":"","groupOrder":5,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_4"}]}],"editable":true}'
}
headers = {
'accept': "application/json",
'authorization': "Basic ZGxhZG1pbjp0aGlua2JpZw==",
'cache-control': "no-cache",
}
url = "http://localhost:8400/proxy/v1/schema-discovery/hive/sample-file"
req = requests.post(url, data=payload, headers=headers)
print(req.text)
For some reason im getting the error "HTTP 415 Unsupported Media Type".
I'm trying to send a csv, along with some json in a multipart form. I originally had this working with http.client but the way I was doing it it was adding extra line breaks and carriage return literals into the multipart message body.
Any help greatly appreciated!
The issue I couldnt originally see was with declaring the file type in the file tuple, shown below as 'text/csv' in the solution.
import http.client
file = { 'file' : ('csvtest.csv', open('csvtest.csv', 'rt'), 'text/csv') }
payload = { 'parser' : '{object trimmed for comment}' }
headers = { 'accept': "application/json", 'authorization': "Basic ZGxhZG1pbjp0aGlua2JpZw==", }
url = "http://localhost:8400/proxy/v1/schema-discovery/hive/sample‌​-file"
req = requests.post(url, data=payload, files=file, headers=headers)
pprint(req.text)

Resources