Urllib3 POST request with attachment in python - python-3.x

I wish to make a post request to add an attachment utilising urllib3 in python without success. I have confirmed the API itself is working in postman but cannot work out how to convert this request to python. Appreciating I'm mixing object types I just don't know how to avoid it.
Python code:
import urllib3
import json
api_key = "secret_key"
header = {"X-API-KEY": api_key, "ACCEPT": "application/json", "content-type": "multipart/form-data"}
url = "https://secret_url.com/api/"
http = urllib3.PoolManager()
with open("invoice.html", 'rb') as f:
file_data = f.read()
payload = {
"attchment": {
"file": file_data
}
}
payload = json.dumps(payload)
r = http.request('post', url, headers = header, fields = payload)
print(r.status)
print(r.data)
Postman - which works and properly sends file-name through also (I'm guessing it splits the bytes and filename up?)
Edit: I've also tried the requests library as I'm more familiar with this (but can't use it as the script will be running in AWS lambda). Removing the attachment element form the dict allows it to run but the API endpoint gives 401 presumably because it's missing the "attachement" part to the data structure as per postman below... but when I put this in I get runtime errors.
r = requests.post(url, headers = header, files={"file": open("invoice.html", 'rb')})

For anyone who stumbles upon this from Dr google a few points:
I was completely mis-interpreting the structure of the element. It's actually a string "attachment[file]" not a dict like object.
Postman has the ability to output python code in urllib/request syntax albeit not 100% what I was after. Note: the chrome version (depreciated) outputs gibberish code that only half works so the client version should be used. A short bit of work below shows it working as expected:
http = urllib3.PoolManager()
with open("invoice.html", "rb") as f:
file = f.read()
payload={
'attachment[file]':('invoice.html',file,'text/html')
}
r = http.request('post', url, headers = header, fields = payload)

Related

How to retrieve file object in python sent via Postman without any url

I am sending a file as an object via postman POST or PUT API like below:
How can I in Python -
get this file object
read and save
If you have a working request in Postman, you could copy autogenerated Code Snippet in Python - Requests format:
It might look like this:
import requests
url = "localhost:8080"
payload="<file contents here>"
headers = {
'Content-Type': 'application/octet-stream'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
Answer to this question which i finally implemented without url -
#app.route('/uploadFIle', methods=['PUT'])
def uploadFile():
chunk_size = 4096
with open("/Users/xyz/Documents/filename", 'wb') as f:
while True:
chunk = request.stream.read(chunk_size)
if len(chunk) == 0:
break
f.write(chunk)
return jsonify({"success":"File transfer initiated"})

How to get a POST request using python on HERE route matching API?

I've tried to do a POST request using python's request library, which looked something like below:
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url,data='Datasets/rtHereTest.csv')
The response I've been getting a code 400
{'faultCode': '16a6f70f-1fa3-4b57-9ef3-a0a440f8a42e',
'responseCode': '400 Bad Request',
'message': 'Column LATITUDE missing'}
However, in my dataset, here I have all the headings that's required from the HERE API documentation to be able to make a call.
Is there something I'm doing wrong, I don't quite understand the POST call or the requirement as the HERE documentation doesn't explicitly give many examples.
The data field of your post request should contain the actual data, not just a filename. Try loading the file first:
f = open('Datasets/rtHereTest.csv', 'r')
url = "https://rme.api.here.com/2/matchroute.json?routemode=car&filetype=CSV&app_id={id}&app_code={code}"
response = requests.post(url, data=f.read())
f.close()
Here's what I use in my own code, with the coordinates defined before:
query = 'https://rme.api.here.com/2/matchroute.json?routemode=car&app_id={id}&app_code={code}'.format(id=app_id, code=app_code)
coord_strings = ['{:.5f},{:.5f}'.format(coord[0], coord[1]) for coord in coords]
data = 'latitude,longitude\n' + '\n'.join(coord_strings)
result = requests.post(query, data=data)
You can try to post the data using below format.
import requests
url = "https://rme.api.here.com/2/matchroute.json"
querystring = {"routemode":"car","app_id":"{app_id}","app_code":"{app_code}","filetype":"CSV"}
data=open('path of CSV file','r')
headers = {'Content-Type': "Content-Type: text/csv;charset=utf-8",
'Accept-Encoding': "gzip, deflate",
}
response = requests.request("POST", url, data=data.read().encode('utf-8'), params=querystring,headers=headers)
print(response.status_code)

MissingSchema - Invalid URL Error with Requests module

Trying to get data from the eBay API using GetItem. But requests isn't reading or getting the URL properly, any ideas? Getting this error:
requests.exceptions.MissingSchema: Invalid URL '<urllib.request.Request object at 0x00E86210>': No schema supplied. Perhaps you meant http://<urllib.request.Request object at 0x00E86210>?
I swear I had this code working before but now it's not, so I'm not sure why.
My code:
from urllib import request as urllib
import requests
url = 'https://api.ebay.com/ws/api.dll'
data = '''<?xml version="1.0" encoding="utf-8"?>
<GetItemRequest xmlns="urn:ebay:apis:eBLBaseComponents">
<RequesterCredentials>
<eBayAuthToken>my-auth-token</eBayAuthToken>
</RequesterCredentials>
<ItemID>any-item-id</ItemID>
</GetItemRequest>'''
headers = {
'Content-Type' : 'text/xml',
'X-EBAY-API-COMPATIBILITY-LEVEL' : 719,
'X-EBAY-API-DEV-NAME' : 'dev-name',
'X-EBAY-API-APP-NAME' : 'app-name',
'X-EBAY-API-CERT-NAME' : 'cert-name',
'X-EBAY-API-SITEID' : 3,
'X-EBAY-API-CALL-NAME' : 'GetItem',
}
req = urllib.Request(url, data, headers)
resp = requests.get(req)
content = resp.read()
print(content)
Thank you in advance. Any good reading material for urllib would be great, too.
You are mixing the urllib and the requests library. They are different libraries that can both do HTTP-requests in Python. I'd suggest you use only the requests library.
Remove the line req = urllib.Request(url, data, headers) and replace the resp = ... line with:
r = requests.post(url, data=data, headers=headers)
Print the response body like this:
print(r.text)
Check out the Requests Quickstart here for more examples: https://2.python-requests.org//en/master/user/quickstart/

python3 -- getting POST request text from requests.post?

I'm using python3. For debugging purposes, I'd like to get the exact text that is sent to the remote server via a requests.post command. The requests.post object seems to contain the response text, but I'm looking for the request text.
For example ...
import requests
import json
headers = {'User-Agent': 'Mozilla/5.0'}
payload = json.dumps({'abc':'def','ghi':'jkl'})
url = 'http://example.com/site.php'
r = requests.post(url, headers=headers, data=payload)
How do I get the text of the exact request that is sent to url via this POST command?

Create Google Shortened URLs, Update My CSV File

I've got a list of ~3,000 URLs I'm trying to create Google shortened links of, the idea is this CSV has a list of links and I want my code to output the shortened links in the column next to the original URLs.
I've been trying to modify the code found on this site here but I'm not skilled enough to get it to work.
Here's my code (I would not normally post an API key but the original person who asked this already posted it publicly on this site) :
import json
import pandas as pd
df = pd.read_csv('Links_Test.csv')
def shorternUrl(my_URL):
API_KEY = "AIzaSyCvhcU63u5OTnUsdYaCFtDkcutNm6lIEpw"
apiUrl = 'https://www.googleapis.com/urlshortener/v1/url'
longUrl = my_URL
headers = {"Content-type": "application/json"}
data = {"longUrl": longUrl}
h = httplib2.Http('.cache')
headers, response = h.request(apiUrl, "POST", json.dumps(data), headers)
return response
for url in df['URL']:
x = shorternUrl(url)
# Then I want it to write x into the column next to the original URL
But I only get errors at this point, before I even started figuring out how to write the new URLs to the CSV file.
Here's some sample data:
URL
www.apple.com
www.google.com
www.microsoft.com
www.linux.org
Thank you for your help,
Me
I think the issue is that you didnot include the API key in the request. By the way, the certifi package allows you to secure a connection to a link. You can get it using pip install certifi or pip urllib3[secure].
Here I create my own API key, so you might want to replace it with yours.
from urllib3 import PoolManager
import json
import certifi
sampleURL = 'http://www.apple.com'
APIkey = 'AIzaSyD8F41CL3nJBpEf0avqdQELKO2n962VXpA'
APIurl = 'https://www.googleapis.com/urlshortener/v1/url?key=' + APIkey
http = PoolManager(cert_reqs = 'CERT_REQUIRED', ca_certs=certifi.where())
def shortenURL(url):
data = {'key': APIkey, 'longUrl' : url}
response = http.request("POST", APIurl, body=json.dumps(data), headers= {'Content-Type' : 'application/json'}).data.decode('utf-8')
r = json.loads(response)
return (r['id'])
The decoding part converts the response object into a string so that we can convert it to a JSON and retrieve data.
From there on, you can store the data into another column and so on.
For the sampleUrl, I got back https(goo.gl/nujb) from the function.
I found a solution here:
https://pypi.python.org/pypi/pyshorteners
Example copied from linked page:
from pyshorteners import Shortener
url = 'http://www.google.com'
api_key = 'YOUR_API_KEY'
shortener = Shortener('Google', api_key=api_key)
print "My short url is {}".format(shortener.short(url))

Resources