I have successfully used the requests module to upload a binary file (jpg), with something like the following:
upload_url = 'http:10.1.1.1:8080/api/media/photo'
headers = {'Authorization': token_string, 'Content-Type': 'image/jpg'}
data = open('photo.JPG', 'rb')
params = {'name': 'photo.JPG'}
r = requests.post(upload_url, params=params, data=data, headers=headers)
Now trying to do this with aiohttp client. This is what I have so far:
def upload_photos(token):
upload_url = '10.0.1.1:8080/api/media/photo'
headers = {'Authorization': token, 'Content-Type': 'image/jpg'}
data = {'file': open('photo.JPG', 'rb')}
params = {'name': 'photo.JPG'}
r = yield from aiohttp.request('post', upload_url, params=params, data=data, headers=headers)
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(upload_photos(token))
But I am getting a 400 back, with {"detail": "Specified value is invalid: Invalid Content-Length specified"}.
It's as if it's not properly sending the photo.jpg. The aiohttp docs mentione multi-part encoded files and streaming, but that's not whant I want here.
How do I form a similar POST w/ binary file like in requests, but with aiohttp?
Thank you for a bug report. As workaround I guess to use chunked transfer encoding:
headers['Transfer-Encoding'] = 'chunked'
r = yield from aiohttp.request('post', upload_url, params=params, data=data, headers=headers, chunked=1024)
The recipe should work, at least we have very simular code in our test suite: https://github.com/KeepSafe/aiohttp/blob/master/tests/test_client_functional.py#L322
I'll try to fix Content-Length calculation for your uploading way in next aiohttp release.
Related
I'm running a cloud function in python to return some data from an api. The function is not executed and I have the error {'code': 400, 'message': 'Could not parse JSON'}.
Here is my code:
import requests
import json
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX',
'content-type': 'application/json'}
data = '{"dateRangeStart":"2020-05-10T00:00:00.000","dateRangeEnd":"2020-05-16T23:59:59.000","amountShown": "HIDE_AMOUNT","detailedFilter":{ "page":"1","pageSize":"50"}}'
#req = requests.post(url, headers=headers, json=data)
req = requests.post(url, headers=headers, data=json.dumps(data))
print(req.json())
how should I format my data variable?
Just give your dict as your json argument, you don't need to specify the content-type headers requests will do it for you.
import requests
def my_function(request):
url = 'https://blablabla/detailed'
headers = {'X-Api-Key': 'XXXXXXXX', }
data = {"dateRangeStart": "2020-05-10T00:00:00.000", "dateRangeEnd": "2020-05-16T23:59:59.000", "amountShown": "HIDE_AMOUNT", "detailedFilter": { "page": "1", "pageSize": "50", }, }
req = requests.post(url, headers=headers, json=data)
print(req.json())
If you do not set a content-type header will try to set it for you:
When using the keyword argument json: it will set it to application/json
When using the keyword argument data (and the value passed respects some criteria, most of the time you don't have to worry about it): it will set it to application/x-www-form-urlencoded
If both kwargs are present data takes priority for the header so it will be set to application/x-www-form-urlencoded
I have not detailed the behaviour when the kwarg files is used as it would be really lengthy and is out of scope here.
Here's the source code.
I am sending a file as an object via postman POST or PUT API like below:
How can I in Python -
get this file object
read and save
If you have a working request in Postman, you could copy autogenerated Code Snippet in Python - Requests format:
It might look like this:
import requests
url = "localhost:8080"
payload="<file contents here>"
headers = {
'Content-Type': 'application/octet-stream'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
Answer to this question which i finally implemented without url -
#app.route('/uploadFIle', methods=['PUT'])
def uploadFile():
chunk_size = 4096
with open("/Users/xyz/Documents/filename", 'wb') as f:
while True:
chunk = request.stream.read(chunk_size)
if len(chunk) == 0:
break
f.write(chunk)
return jsonify({"success":"File transfer initiated"})
I would like to add content type explicity for multipart form data before sending post request
Below is my sample code i managed to add conten type for file data but couldn't figure out how to add content type correctly for json data, i would like to add "application/json; charset=utf-8" for json data
import requests
import json
import traceback
def uploadLogs(fileName):
f = open(fileName, 'rb')
payload = { "var1":"this", "var2" : "that"
}
files = {'file': ('current', f, "text/plain; charset=us-ascii")}
data = {'info': json.dumps(payload)}
headers = {'type': 'myReport', "Keep-Alive": "timeout=100"}
try:
url = "http://localhost:8009/upload"
response = requests.post(url, data=data, files=files, headers=headers)
print(response.request.body)
print(response.request.headers)
print(response.status_code)
if (response != None and (response.status_code == 200 or
response.status_code == 201)):
return True
except:
traceback.print_exc()
return False
filename = "C:\\sample.txt"
print(uploadLogs(filename))
If someone knows how to do please suggest
Running a Django App locally, i can use Postman to upload a zip file along with some dict data. Breaking the application in 'def Post()' i can see that Postman's successful upload:
request.data = <QueryDict: {'request_id': ['44'], 'status': [' Ready For Review'], 'is_analyzed': ['True'], 'project': ['test value'], 'plate': ['Plate_R0'], 'antigen': ['tuna'], 'experiment_type': ['test'], 'raw_file': [<TemporaryUploadedFile: testfile.zip (application/zip)>]}>
Postman offers the following python code to replicate these results in my python script:
import requests
url = "http://127.0.0.1:8000/api/upload/"
payload = {'request_id': '44',
'status': ' Ready For Review',
'is_analyzed': 'True',
'project': 'test value',
'plate': 'Plate_R0',
'antigen': 'tuna',
'experiment_type': 'test'}
files = [
('raw_file', open(r'C:/testfile.zip','rb'))
]
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
response = requests.request("POST", url, headers=headers, data = payload, files = files)
print(response.text.encode('utf8'))
running this code directly and retrieving the request.data (server side) i see the binary representation of the xlsx file is in the object and the payload data is not there (this is the error in the response).
How do i get my python script to produce the same server-side object as postman? Specifically how do i upload my data such that the file is represented as: <TemporaryUploadedFile: testfile.zip (application/zip)>
Thanks.
Turns out, inspection of the object posted by Postman shows that it was using multipart-form upload. Searching around i found this answer to a related question to describe posting as multi-part: https://stackoverflow.com/a/50595768/2917170 .
The working python is as follows:
from requests_toolbelt import MultipartEncoder
url = "http://127.0.0.1:8000/api/upload/"
zip_file = r'C:/testfile.zip'
m = MultipartEncoder([('raw_file', (os.path.basename(zip_file), open(zip_file, 'rb'))),
('request_id', '44'),
('status', 'Ready For Review'),
('is_analyzed', 'True'),
('project', 'test value'),
('plate', 'Plate_R0'),
('antigen', 'tuna'),
('experiment_type', 'test')])
header = {'content-type': m.content_type}
response = requests.post(url, headers=header, data=m, verify=True)
I need to upload a local picture to an URL.
My curl request looks like : "curl -T 'my_picture' 'the_url' ".
My purpose is to do it in Python, I saw, that I have to use : requests.put().
Everything I did before works nice, but this function give me a lot of trouble.
Solution is probably really easy, but I'm lost.
Thanks for any help.
def apithree(urlupload):
url = urlupload
picture = Image.open("test.png")
headers = {"content-type": "multipart/form-data"}
response = requests.put(url, headers=headers, data=picture)
response = response.json()
print(response)
Other Code I tried
def apithree(urlupload):
url = urlupload
files = {'image': open('test.png', 'rb')}
response = requests.post(url, data=files)
response = response.json()
print(response)
If the command works,the output should be empty, but I always have error messages.
I can add error messages if necessary.
Thanks for any help.
If server accepts multipart/form-data images, you probably need files= parameter for your request:
import requests
files = {'file_name': open(r'C:\data\...\image.png', 'rb')}
r = requests.post(YOUR_URL, files=files) # or requests.put(...)
print(r.status_code, r.json())