Trying to use the Python Requests module to post to a form:
using Chrome/inspect/network tab after I search for something is
shows:
Request
URL:https://www.fbo.gov/index?s=opportunity&mode=list&tab=search
Status Code:302 Found
The Request Payload is:
... begin snippet ....
procurement_notice
------WebKitFormBoundary9stOUHDiQLr2yrEB Content-Disposition: form-data; name="dnf_class_values[procurement_notice][notice_id]"
8045295a672345856701e8ff0ab87a4c
------WebKitFormBoundary9stOUHDiQLr2yrEB Content-Disposition: form-data;
name="dnf_class_values[procurement_notice][_so_agent_save_agent]"
------WebKitFormBoundary9stOUHDiQLr2yrEB Content-Disposition: form-data;
name="dnf_class_values[procurement_notice][custom_response_date]"
------WebKitFormBoundary9stOUHDiQLr2yrEB Content-Disposition: form-data;
name="dnf_class_values[procurement_notice][custom_posted_date]"
30
------WebKitFormBoundary9stOUHDiQLr2yrEB Content-Disposition: form-data; name="dnf_class_values[procurement_notice][keywords]"
Colorado
... end snippet ....
The code, I am trying is:
import requests
headers ={'X-Requested-With': 'XMLHttpRequest',
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 GTB7.1 (.NET CLR 3.5.30729)'}
url = 'https://www.fbo.gov/index?s=opportunity&mode=list&tab=search'
search_data={"dnf_class_values[procurement_notice][keywords]":Colorado}
r = requests.post(url, data=search_data, headers=headers)
it doesn't give me the r.content as if I passed in the search term 'Colorado'.
So my question is does the 'Code:302', which is a re-direct, mean they don't accept a POST request and also how do I deal with the 'WebKitFormBoundary' ...'Content-Disposition: form-data...' as it doesn't look like a normal dictionary element I can pass in.
Related
I am trying to use concord technologies cloud fax tool. The goal is to set up incoming and outgoing fax service from a nodejs express server.
This is an example of the 'raw text format' from concord's website. I am not sure how I can convert this to a cal using axios.
POST /api/intake/fax-notify HTTP/1.1
Host: https://nextstep.concord.net
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="FaxMessageId"
bbe58db7-b3f3-461a-9c47-5b803586cc53
Content-Disposition: form-data; name="MessageType"
Inbound
Content-Disposition: form-data; name="AttachmentName"
bbe58db7-b3f3-461a-9c47-5b803586cc53.tif
Content-Disposition: form-data; name="FaxSenderCSID"
Test User (123-555-1212)
Content-Disposition: form-data; name="CallerNumber"
1-123-555-1212
Content-Disposition: form-data; name="CalledNumber"
1-123-555-6789
Content-Disposition: form-data; name="FaxReceivedDateTime"
10/07/2018 10:45:21 AM
Content-Disposition: form-data; name="FaxRecipientTimeZone"
(GMT-06:00) Central Time (US & Canada)
Content-Disposition: form-data; name="FaxPages"
1
Content-Disposition: form-data; name="FaxResolution"
1
Content-Disposition: form-data; name="FaxSpeed"
2
Content-Disposition: form-data; name="AttachmentCount"
1
Content-Disposition: form-data; name="Account"
12345
Content-Disposition: form-data; name="AuthUser"
12345-6789-1212-12345-0001
Content-Disposition: form-data; name="File"; filename="bbe58db7-b3f3-461a-9c47-5b803586cc53.tif"
Content-Type: file
<binary file data>
------WebKitFormBoundary7MA4YWxkTrZu0gW--
Any help would be greatly appreciated, ty.
I need to upload an audio file with python, but i'm having difficulties.
If I ngrep the upload i see something like
T 172.31.41.159:15561 -> 172.31.39.127:80 [AP] #324
POST /v1/wizard-campaign/18cb83c1-b6a3-11ec-a986-02ca52d15783/media-audio HTTP/1.1.
Host: whatever.
Content-Length: 70535.
authorization: Bearer XXXXXXX.
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Safari/537.36.
content-type: multipart/form-data; boundary=----WebKitFormBoundaryWVhfRY6Fs7rKntcI.
origin: https://whatever.
referer: https://whatever/.
accept-encoding: gzip, deflate, br.
accept-language: en-US,en;q=0.9.
.
T 172.31.41.159:15561 -> 172.31.39.127:80 [A] #326
------WebKitFormBoundaryWVhfRY6Fs7rKntcI.
Content-Disposition: form-data; name="mediaFile"; filename="mensaje-de-saludo.mp3".
Content-Type: audio/mpeg.
.
ID3......!TXXX......
I tried with
audioDataJSON = {
"originalName": filename,
"time": math.ceil(audio.info.length)
}
file = { open(filename,'rb') }
audioData = json.dumps(audioDataJSON)
resp = requests.post(base_url + "/v1/wizard-campaign/" + id + "/media-audio", headers=headers, data=audioData , files=file)
But it's failing with "too many values to unpack"
Any help is greatly appreciated!
David
when I am trying to send an attachment with an email I got this error can anyone tell me what am I doing wrong here is the code which i have tried
var payload = `From: 'Amarjeet Singh' <${sender}>
To: ${recipient}
Subject: AWS SES Attachment
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887
--NextPart
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
This is the <b>body</b> of the email.
--7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887
Content-Type: application/json; charset=UTF-8;
Content-Disposition: attachment; filename="colors.json"
Content-Transfer-Encoding: base64
${file}
--NextPart--`;
var params = {
RawMessage: {
Data: payload
},
Source: "xyz#gmail.com"
};
If you are using AWS SES your payload has two issues, first your boundary is different, and second just keep in mind the line breaks, your string template should be like this:
var payload = `From: 'Amarjeet Singh' <${sender}>
To: ${recipient}
Subject: AWS SES Attachment
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887
--7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
This is the <b>body</b> of the email.
--7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887
Content-Type: application/json; charset=UTF-8;
Content-Disposition: attachment; filename="colors.json"
Content-Transfer-Encoding: base64
${file}
--7f1a313ee430f85a8b054f085ae67abd6ee9c52aa8d056e7f7e19c6e2887--`;
Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here
I'm new to this and probably doing something very stupid, but if I carry on like this, I'll be bald!
As a test, I am just trying to send a TXT document for signing to DocuSign via a HTTP Post. I have followed the examples they give (I believe) but what ever I try, I get the same error.
Error:
"INVALID_MULTI_PART_REQUEST",
"message": "An error was found while parsing the multipart request. Boundary terminator '--AAAAA--' was not found in the request.
Ultimately I want to send base64 encoded PDFs but if I cannot even get TXTs to work...
I am using an XML scripting language specific to our in-house application to make the HTTP request, and the process has a diagnostic mode which can dump the request elements out to file to help sort issues: These TXT files are next and I have removed any sensitive data:
REQUESTHEADERS.TXT:
Content-Type: multipart/form-data; boundary=AAAAA
Content-Length: 565
X-DocuSign-Authentication: <DocuSignCredentials> <Username>myemail#myemail.com</Username><Password>mypassword</Password> <IntegratorKey>mykey</IntegratorKey></DocuSignCredentials>
Host: demo.docusign.net
Accept: application/json; charset=UTF-8
Accept-Encoding: identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)
REQUESTDATA.TXT
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: form-data
{
"status":"created",
"emailSubject":"Test",
"emailBlurb":"This is a test",
"documents":[
{
"name":"test1.txt",
"documentId":"1",
"order":"1"
}
],
"recipients":{
"signers":[
{
"email":"myemail#myemail.com",
"name":"Fred Blogs",
"recipientId":"1"
}
]
}
}
--AAAAA
Content-Type: text/plain; charset=UTF-8
Content-Disposition: file; filename="test1.txt";documentid=1
Please sign this test document
--AAAAA--
Please, if any one can tell me what is wrong I would be very grateful indeed!
It's the formatting of the call, please take note on where I have line breaks and where I do not.
Request
--AAAAA
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: form-data
{
"status":"created",
"emailSubject":"Test",
"emailBlurb":"This is a test",
"documents":[
{
"name":"test1.txt",
"documentId":"1",
"order":"1"
}
],
"recipients":{
"signers":[
{
"email":"myemail#myemail.com",
"name":"Fred Blogs",
"recipientId":"1"
}
]
}
}
--AAAAA
Content-Type: text/plain; charset=UTF-8
Content-Disposition: file; filename="test1.txt";documentid=1
Please sign this test document
--AAAAA--
Response
{
"envelopeId": "{envelopeId}",
"uri": "/envelopes/{envelopeId}",
"statusDateTime": "2015-02-27T18:53:39.5700000Z",
"status": "created"
}