I need to upload an audio file with python, but i'm having difficulties.
If I ngrep the upload i see something like
T 172.31.41.159:15561 -> 172.31.39.127:80 [AP] #324
POST /v1/wizard-campaign/18cb83c1-b6a3-11ec-a986-02ca52d15783/media-audio HTTP/1.1.
Host: whatever.
Content-Length: 70535.
authorization: Bearer XXXXXXX.
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Safari/537.36.
content-type: multipart/form-data; boundary=----WebKitFormBoundaryWVhfRY6Fs7rKntcI.
origin: https://whatever.
referer: https://whatever/.
accept-encoding: gzip, deflate, br.
accept-language: en-US,en;q=0.9.
.
T 172.31.41.159:15561 -> 172.31.39.127:80 [A] #326
------WebKitFormBoundaryWVhfRY6Fs7rKntcI.
Content-Disposition: form-data; name="mediaFile"; filename="mensaje-de-saludo.mp3".
Content-Type: audio/mpeg.
.
ID3......!TXXX......
I tried with
audioDataJSON = {
"originalName": filename,
"time": math.ceil(audio.info.length)
}
file = { open(filename,'rb') }
audioData = json.dumps(audioDataJSON)
resp = requests.post(base_url + "/v1/wizard-campaign/" + id + "/media-audio", headers=headers, data=audioData , files=file)
But it's failing with "too many values to unpack"
Any help is greatly appreciated!
David
Related
From lighthouse chrome test page:
URL Transfer Size Potential Savings
/three.module.min.js 630.3 KiB 477.3 KiB
I minify but performance still need compression.
...
function shouldCompress (req, res) {
if (req.headers['x-no-compression']) {
// don't compress responses with this request header
return false
}
// fallback to standard filter function
return compression.filter(req, res)
}
hostingHTTP.use(compression({ filter: shouldCompress }))
...
My request :
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,ru;q=0.8
Cache-Control: no-cache
Connection: keep-alive
Host: maximumroulette.com
Pragma: no-cache
Referer: https://maximumroulette.com/apps/magic/public/module.html
sec-ch-ua: "Not_A Brand";v="99", "Google Chrome";v="109", "Chromium";v="109"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: script
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36
My response:
Accept-Ranges: bytes
Cache-Control: public, max-age=0
Connection: keep-alive
Content-Length: 394590
Content-Type: application/javascript; charset=UTF-8
Date: Mon, 13 Feb 2023 20:39:00 GMT
ETag: W/"6055e-1864c7208a7"
Keep-Alive: timeout=5
Last-Modified: Mon, 13 Feb 2023 20:23:07 GMT
X-Powered-By: Express
Did i need some action onclietn part for decopression ?
If i setup on server res.set('Content-Encoding', 'deflate'); // gzip, deflate, br a get error on chrome:
net::ERR_CONTENT_DECODING_FAILED 200 (OK)
Any suggestion ?
im opening a socket server that hosts the reddit confirmation s that you can login to the application without compromising the bot and logging in with one sting called an access token that is in the url of the socket server and what i want to do it sake that access token from the url and display it into the socket server page but the server cant detect any variables inside of the url. Here is my code if you need it. it works when implicit is not part of the url generation, but as soon as i add implicit=True (which i need to, i plan on this being an app) it no longer works.
what i want: (about, this is for the non implicit)GET /?state=state&code=code HTTP/1.1.
what i get: GET / HTTP/1.1.
that is what happens when i do data = client.recv(1024).decode("utf-8").
it will return proper data when the url is generated with url = reddit.auth.url(scopes, state).
but when i generate the url with url = reddit.auth.url(scopes, state, implicit=True).
the data will return GET / HTTP/1.1.
I need the access token to be recognized after the /.
I am using praw 7.1.0
GET / HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.15.2 Chrome/83.0.4103.122 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Language: en-US,en;q=0.9
DNT: 1
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
/
/
Traceback (most recent call last):
File "path/prawTest.py", line 81, in <module>
sys.exit(main())
File "path/prawTest.py", line 61, in main
params = {
File "path/prawTest.py", line 62, in <dictcomp>
key: value for (key, value) in [token.split("=") for token in param_tokens]
ValueError: not enough values to unpack (expected 2, got 1)
It expects to see variables after the /.
Without implicit=Tru i get the following:
GET /?state=50470&code=code HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.15.2 Chrome/83.0.4103.122 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Language: en-US,en;q=0.9
DNT: 1
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
The implicit=True url looks like this:
http://localhost:8080/#access_token=token&token_type=bearer&state=39771&expires_in=3600&scope=%2A
I'd like to pass a raw HTTP request like:
GET /foo/bar HTTP/1.1
Host: example.org
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Accept: */*
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
X-Requested-With: XMLHttpRequest
Referer: http://example.org/test
Cookie: foo=bar; lorem=ipsum;
And generate the python request such as:
import requests
burp0_url = "http://example.org:80/foo/bar"
burp0_cookies = {"foo": "bar", "lorem": "ipsum"}
burp0_headers = {"User-Agent": "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8", "Accept": "*/*", "Accept-Language": "fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3", "Accept-Encoding": "gzip,deflate", "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Keep-Alive": "115", "Connection": "keep-alive", "Content-Type": "application/x-www-form-urlencoded", "X-Requested-With": "XMLHttpRequest", "Referer": "http://example.org/test"}
requests.get(burp0_url, headers=burp0_headers, cookies=burp0_cookies)
Is there a library for that?
I could not find an existing library that does this conversion, but there is a Python library to convert curl commands to python requests code.
https://github.com/spulec/uncurl
e.g.
import uncurl
print(uncurl.parse('curl --header "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7" --compressed --header "Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3" --header "Connection: keep-alive" --header "Content-Type: application/x-www-form-urlencoded" --cookie "foo=bar; lorem=ipsum;" --header "Keep-Alive: 115" --header "Referer: http://example.org/test" --user-agent "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8" --header "X-Requested-With: XMLHttpRequest" https://example.org/foo/bar '))
I haven't found a Python library to transform raw HTTP into such a curl command. However, this Perl program does it.
Like this:
$ cat basic
GET /index.html HTTP/2
Host: example.com
Authorization: Basic aGVsbG86eW91Zm9vbA==
Accept: */*
$ ./h2c < basic
curl --http2 --header User-Agent: --user "hello:youfool" https://example.com/index.html
You could either call it from your python script, use a Python-Perl bridge or try to port it.
Postman also allows you to convert raw HTTP requests directly to python requests code, using its Code snippet generator. Although, it seems this can only be done via the GUI. It's also not Open-Source, so you can't access the code that does this transformation.
I needed something that can generate a request and couldn't find it so ended up writing it in gist:
class RequestParser(object):
def __parse_request_line(self, request_line):
request_parts = request_line.split(' ')
self.method = request_parts[0]
self.url = request_parts[1]
self.protocol = request_parts[2] if len(request_parts) > 2 else DEFAULT_HTTP_VERSION
def __init__(self, req_text):
req_lines = req_text.split(CRLF)
self.__parse_request_line(req_lines[0])
ind = 1
self.headers = dict()
while ind < len(req_lines) and len(req_lines[ind]) > 0:
colon_ind = req_lines[ind].find(':')
header_key = req_lines[ind][:colon_ind]
header_value = req_lines[ind][colon_ind + 1:]
self.headers[header_key] = header_value
ind += 1
ind += 1
self.data = req_lines[ind:] if ind < len(req_lines) else None
self.body = CRLF.join(self.data)
def __str__(self):
headers = CRLF.join(f'{key}: {self.headers[key]}' for key in self.headers)
return f'{self.method} {self.url} {self.protocol}{CRLF}' \
f'{headers}{CRLF}{CRLF}{self.body}'
def to_request(self):
req = requests.Request(method=self.method,
url=self.url,
headers=self.headers,
data=self.data, )
return req
Basically i tried sending http2.0 headers with hyper for python
https://hyper.readthedocs.io/en/latest/
https://github.com/python-hyper/hyper
Mounting HTTP20Adapter in my request.session but didnt worked as expected.
First i explain thath "from tls_version import MyAdapter" thath is used later in the Main code its these tls_version.py file
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
import ssl
class MyAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1_2)
Just to force to use tls1.2 nothing more.
The Main code is here but basically im trying to send a get call with http2.0 pseudo headers mounting hyper adapter in request.session and having control over headers order with collections.OrderectDict
import requests
from tls_version import MyAdapter
import json
import collections
from userdata import UserData
from hyper.contrib import HTTP20Adapter
headers2 = [('Upgrade-Insecure-Requests', '1'),
('User-Agent', 'Mozilla/5.0 (Linux; Android 5.1.1; google Pixel 2 Build/LMY47I; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/74.0.3729.136 Mobile Safari/537.36'),
('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3'),
('Accept-Encoding', 'gzip, deflate'),
('Accept-Language', 'es-ES,es;q=0.9,en-US;q=0.8,en;q=0.7'),
]
class post_calls():
def start(self,headers_pass,body_pass,params,url,method):
proxies = {
'http': ip,
'https': ip
}
body = str(body_pass)
#send the POST request
session = requests.session()
session.mount('https://', MyAdapter())
session.headers = collections.OrderedDict(headers_pass)
if method == 'get':
q = 'https://' + server + '.' + host
q = q.replace('.www.', '.')
session.mount('https://', HTTP20Adapter())
print('q='+q)
response = session.get(url, proxies=proxies, params=params, verify=charlesproxy)
def login_world2(sid):
a = post_calls()
q ='https://'+server+'.'+ host+'/login.php?mobile&sid='+sid+'&2'
q = q.replace('.www.','.')
params = {}
url = q
body = '0'
login = a.start(headers2,body,params,url,'get')
return login
if __name__ == "__main__":
login_get = login_world(sid)
print(login_get)
these is headers sends these file:
:method: GET
:scheme: https
:authority: server.url.com
:path: /login.php?mobile&sid=577f0967545d6acec94716d265dd4867fa4db4a446326ecde7486a97feede14702f4911438f4a4cd097677f0dd962786ef14b3f16f1184ee63a506155d522f53&2
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Linux; Android 5.1.1; google Pixel 2 Build/LMY47I; wv) AppleWebKit/537.36 (KHTML
user-agent: like Gecko) Version/4.0 Chrome/74.0.3729.136 Mobile Safari/537.36
accept: text/html
accept: application/xhtml+xml
accept: application/xml;q=0.9
accept: image/webp
accept: image/apng
accept: */*;q=0.8
accept: application/signed-exchange;v=b3
accept-encoding: gzip
accept-encoding: deflate
accept-language: es-ES
accept-language: es;q=0.9
accept-language: en-US;q=0.8
accept-language: en;q=0.7
and these is what i need to send cause if i send them like i put above, like these script does, the server rejects my get requests.
:method: GET
:authority: server.url.com
:scheme: https
:path: /login.php?mobile&sid=2ea530a62cb63af6c14be116b7df86ad85cd77c9a11aa3c881b3a460e6c14fbd1fd8b79bd66c9782073705cdff25e890e65b5aeb852fde24c2d54a6e4ee49890&2
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Linux; Android 5.1.1; google Pixel 2 Build/LMY47I; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/74.0.3729.136 Mobile Safari/537.36
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
accept-encoding: gzip, deflate
accept-language: es-ES,es;q=0.9,en-US;q=0.8,en;q=0.7
Seems thath for each "," i put in headers dict, hyper creates a new header instead of sending them in same value, witouth hyper it works normal but i need to send those headers thath are http 2.0 and witouth hyper or any other alternative i cannot, requests dosnt have support for it
:method: GET
:scheme: https
:authority: server.url.com
:path: /login.php?mobile&sid=577f0967545d6acec94716d265dd4867fa4db4a446326ecde7486a97feede14702f4911438f4a4cd097677f0dd962786ef14b3f16f1184ee63a506155d522f53&2
Set the headers in the HTTP20Adapter instead of in session and it should work.
adapter = HTTP20Adapter(headers=headers)
session.mount(prefix='https://', adapter=adapter)
Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here