I'm running a cloud function in python to return some data from an api. The function is not executed and I have the error {'code': 400, 'message': 'Could not parse JSON'}.
Here is my code:
import requests
import json
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX',
'content-type': 'application/json'}
data = '{"dateRangeStart":"2020-05-10T00:00:00.000","dateRangeEnd":"2020-05-16T23:59:59.000","amountShown": "HIDE_AMOUNT","detailedFilter":{ "page":"1","pageSize":"50"}}'
#req = requests.post(url, headers=headers, json=data)
req = requests.post(url, headers=headers, data=json.dumps(data))
print(req.json())
how should I format my data variable?
Just give your dict as your json argument, you don't need to specify the content-type headers requests will do it for you.
import requests
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX', }
data = {"dateRangeStart": "2020-05-10T00:00:00.000", "dateRangeEnd": "2020-05-16T23:59:59.000", "amountShown": "HIDE_AMOUNT", "detailedFilter": { "page": "1", "pageSize": "50", }, }
req = requests.post(url, headers=headers, json=data)
print(req.json())
If you do not set a content-type header will try to set it for you:
When using the keyword argument json: it will set it to application/json
When using the keyword argument data (and the value passed respects some criteria, most of the time you don't have to worry about it): it will set it to application/x-www-form-urlencoded
If both kwargs are present data takes priority for the header so it will be set to application/x-www-form-urlencoded
I have not detailed the behaviour when the kwarg files is used as it would be really lengthy and is out of scope here.
Here's the source code.
I'm trying to generate an access token from an API using the following required information:
Authorization endpoint: https://api.paylocity.com/IdentityServer/connect/token
Authorization Header
The request is expected to be in the form of a basic authentication request, with the "Authorization" header containing the client-id and client-secret. This means the standard base-64 encoded user:password, prefixed with "Basic" as the value for the Authorization header, where user is the client-id and password is the client-secret.
Content-Type Header
The "Content-Type" header is required to be "application/x-www-form-urlencoded".
Other Values
The request must post the following form encoded values within the request body:
grant_type = client_credentials
scope = WebLinkAPI
I've tried using the Python 'requests' package without success, see below:
import requests
url = "https://api.paylocity.com/IdentityServer/connect/token"
headers = {
'Authorization': "Basic **********:**********",
'Content-Type': "application/x-www-form/urlencoded"
}
body = {
'grant_type': "client_credentials",
'scope': 'WebLinkAPI'
}
response = requests.request("POST", url, headers=headers, params=body)
print(response.text)
Instead of an access token, I receive an error message:
{"error":"invalid_client"}
I've verified that my base64 encoded username and password are correct, it looks something like "G/RaNdOm:MoReRaNdOm==" and when I add them to postman it works so my credentials are correct. What could be wrong?
I resolved this. There were two issues, the Authorization header was not using ASCII as the destination charset with base64 encoding. The other issue was trying to add the additional values to the parameters instead of the body in the requests package. The solution was to replace "params" with "data". Here was the solution:
url = "https://api.paylocity.com/IdentityServer/connect/token"
body = "grant_type=client_credentials&scope=WebLinkAPI"
headers = {
'Content-Type': "application/x-www-form-urlencoded",
'Authorization': "Basic ***(base64ASCII)username:password***",
}
response = requests.request("POST", url, data=body, headers=headers)
print(response.text)
Im having difficulty with the following code:
payload = {
'file' : ('csvtest.csv', open('csvtest.csv', 'rb')),
'parser' : '{"name":"CSV","displayName":null,"description":"Supports delimited text files with a field delimiter and optional escape and quote characters.","shortDescription":null,"properties":[{"name":"Auto Detect?","displayName":"Auto Detect?","value":"true","values":null,"placeholder":"","type":"select","hint":"Auto detect will attempt to infer delimiter from the sample file.","objectProperty":"autoDetect","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":1,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_0"},{"name":"Header?","displayName":"Header?","value":"true","values":null,"placeholder":"","type":"select","hint":"Whether file has a header.","objectProperty":"headerRow","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":2,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_1"},{"name":"Delimiter Char","displayName":"Delimiter Char","value":",","values":null,"placeholder":"","type":"string","hint":"Character separating fields","objectProperty":"separatorChar","selectableValues":[],"required":false,"group":"","groupOrder":3,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_2"},{"name":"Quote Char","displayName":"Quote Char","value":"\'","values":null,"placeholder":"","type":"string","hint":"Character enclosing a quoted string","objectProperty":"quoteChar","selectableValues":[],"required":false,"group":"","groupOrder":4,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_3"},{"name":"Escape Char","displayName":"Escape Char","value":"\\\\","values":null,"placeholder":"","type":"string","hint":"Escape character","objectProperty":"escapeChar","selectableValues":[],"required":false,"group":"","groupOrder":5,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_4"}],"objectClassType":"com.thinkbiganalytics.discovery.parsers.csv.CSVFileSchemaParser","objectShortClassType":"CSVFileSchemaParser","propertyValuesDisplayString":null,"supportsBinary":false,"generatesHiveSerde":true,"tags":["CSV","TSV"],"clientHelper":null,"allowSkipHeader":true,"groups":[{"group":"","layout":"column","properties":[{"name":"Auto Detect?","displayName":"Auto Detect?","value":"true","values":null,"placeholder":"","type":"select","hint":"Auto detect will attempt to infer delimiter from the sample file.","objectProperty":"autoDetect","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":1,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_0"},{"name":"Header?","displayName":"Header?","value":"true","values":null,"placeholder":"","type":"select","hint":"Whether file has a header.","objectProperty":"headerRow","selectableValues":[{"label":"true","value":"true","hint":null},{"label":"false","value":"false","hint":null}],"required":false,"group":"","groupOrder":2,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_1"},{"name":"Delimiter Char","displayName":"Delimiter Char","value":",","values":null,"placeholder":"","type":"string","hint":"Character separating fields","objectProperty":"separatorChar","selectableValues":[],"required":false,"group":"","groupOrder":3,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_2"},{"name":"Quote Char","displayName":"Quote Char","value":"\'","values":null,"placeholder":"","type":"string","hint":"Character enclosing a quoted string","objectProperty":"quoteChar","selectableValues":[],"required":false,"group":"","groupOrder":4,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_3"},{"name":"Escape Char","displayName":"Escape Char","value":"\\\\","values":null,"placeholder":"","type":"string","hint":"Escape character","objectProperty":"escapeChar","selectableValues":[],"required":false,"group":"","groupOrder":5,"layout":"column","hidden":false,"pattern":"","patternInvalidMessage":"Invalid Input","formKey":"property_4"}]}],"editable":true}'
}
headers = {
'accept': "application/json",
'authorization': "Basic ZGxhZG1pbjp0aGlua2JpZw==",
'cache-control': "no-cache",
}
url = "http://localhost:8400/proxy/v1/schema-discovery/hive/sample-file"
req = requests.post(url, data=payload, headers=headers)
print(req.text)
For some reason im getting the error "HTTP 415 Unsupported Media Type".
I'm trying to send a csv, along with some json in a multipart form. I originally had this working with http.client but the way I was doing it it was adding extra line breaks and carriage return literals into the multipart message body.
Any help greatly appreciated!
The issue I couldnt originally see was with declaring the file type in the file tuple, shown below as 'text/csv' in the solution.
import http.client
file = { 'file' : ('csvtest.csv', open('csvtest.csv', 'rt'), 'text/csv') }
payload = { 'parser' : '{object trimmed for comment}' }
headers = { 'accept': "application/json", 'authorization': "Basic ZGxhZG1pbjp0aGlua2JpZw==", }
url = "http://localhost:8400/proxy/v1/schema-discovery/hive/sample-file"
req = requests.post(url, data=payload, files=file, headers=headers)
pprint(req.text)