I want to make a get request and then use the data I got back to make a post request but with this code all i get is the following error:
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
This code works as long as the headers are not included, but it does not hit the endpoint.
import requests
import asyncio
import json
url = 'https://test.com/add/login'
g = requests.get(url)
myHeaders = g.headers
print(myHeaders)
myCokies = g.cookies.get('Session')
print(myCokies)
myData = {'username':'test','password':'test'}
myCokies = dict(MoodleSession=myCokies)
r = requests.post(url, data=json.dumps(myData), cookies=myCokies)
print(r.text)
Related
I have a requirement to make a PUT request from Python and I have been getting a HTTP 405 response code consistently. Any pointers to the code below would be great.
filepath = './sdfdd/sdfdsst/xxxxxxxxxrrrarara.json'
with open(filepath) as fh:
mydata = fh.read()
response = requests.put('https://asdfs.sdf.sdfds.com',
data=mydata,
auth=('Authorization', 'Api-Token dsdfdsfsdfsdf'),
headers={'content-type':'application/json'},
params={'file': filepath},
allow_redirects=True
)
print(response)
It was due to incorrect API endpoint which was causing this issue.
I'm beginner in Python and I would like to build simple port sniffer.
For this purpose I'm using code from this site, as example: Simple packege snffer using python
And I would like to unpack bites from socket to exctract http header, using function struct.unpack()
What format of string should I use in unpack HTTP header, (e.g '!BBH', "!BBHHHBBH4s4s",'!HHLLBBHHH')
the HTTP header is not fixed-length, so you'll need to parse it other way, for example:
import logging
log = logging.getLogger(__name__)
def parse_http(data):
lines = data.split(b'\r\n')
if len(lines) < 1:
log.error('Invalid http header: %s', lines)
return
request = lines[0]
header = {}
rest = []
in_header = True
for line in lines[1:]:
if line == b'':
in_header = False
continue
if in_header:
try:
key, val = line.split(b': ')
except ValueError:
log.error('Invalid header line: %s', line)
continue
header[key] = val
else:
rest.append(line)
return request, header, b'\r\n'.join(rest)
In order to detect a HTTP packet, you could check if the payload starts with POST, GET, HTTP ... etc
I am facing a unique problem.
Following is my code.
url = 'ABCD.com'
cookies={'cookies':'xyz'}
r = requests.post(url,cookies=cookies)
print(r.status_code)
json_data = json.loads(r.text)
print("Printing = ",json_data)
When I use the url and cookie in the POSTMAN tool and use POST request I get JSON response . But when I use the above code with POST request method in python I get
404
Printing = {'title': 'init', 'description': "Error: couldn't find a device with id: xxxxxxxxx in ABCD: d1"}
But when I use the following code i .e with GET request method
url = 'ABCD.com'
cookies={'cookies':'xyz'}
r = requests.post(url,cookies=cookies)
print(r.status_code)
json_data = json.loads(r.text)
print("Printing = ",json_data)
I get
200
Printing = {'apiVersion': '0.4.0'}
I am not sure why POST method works with JSON repsone in POSTMAN tool and when I try using python it is not work. I use latest python 3.6.4
I finally found what was wrong following is correct way
url = 'ABCD.com'
cookies={'cookies':'xyz'}
r = requests.post(url,headers={'Cookie'=cookies)
print(r.status_code)
json_data = json.loads(r.text)
print("Printing = ",json_data)
web page was expecting headers as cookie and i got the response correctly
I'm starting to play around with Python sockets and I found the following example online for sending and receiving data to a web URL:
import socket
import sys
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print("Failed to create socket")
sys.exit()
print("Socket created")
host = input("Enter URL here: ")
port = 80
try:
remoteIP = socket.gethostbyname(host)
except socket.gaierror:
print("Host name could not be resolved")
sys.exit()
s.connect((remoteIP, port))
print("Socket connected to " + host + " on IP " + remoteIP)
message = "GET / HTTP/1.1\r\n\r\n"
messagebytes = str.encode(message)
try:
s.sendall(messagebytes)
except socket.error:
print("Send failed")
sys.exit()
print("Message sent successfully")
reply = s.recv(4096)
print(reply)
Now the issue is whenever I run this code, I get the following output:
b'HTTP/1.1 400 Bad Request\r\nCache-Control: no-cache\r\nPragma: no-cache\r\nContent-Type: text/html; charset=utf-8\r\nProxy-Connection: close\r\nConnection: close\r\nContent-Length: 1229\r\n\r\n\r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nMy Internet Access\r\n\r\n--\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nYour request could not be processed. Request could not be handled\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nThis could be caused by a misconfiguration, or possibly a malformed request.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nFor assistance, please log a call with the helpdesk.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n'
My only guess for this outcome is that the message being sent is sent as b" therefore GET / HTTP/1.1\r\n\r\n" and that b c, therefore, making it an invalid request. I tried decoding it as well and it didn't help as I get the exact same outcome as when I do not decode it.
If that is the case how can I fix this?
I'v tried decode the response and I made it.
>>> a=b'HTTP/1.1 400 Bad Request\r\nCache-Control: no-cache\r\nPragma: no-cache\r\nContent-Type: text/html; charset=utf-
8\r\nProxy-Connection: close\r\nConnection: close\r\nContent-Length: 1229\r\n\r\n\r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r
\n\r\n\r\n\r\nMy Internet Access\r\n\r\n--\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r
\n\r\n\r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nYour request could no
t be processed. Request could not be handled\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nThis could be caused by a mi
sconfiguration, or possibly a malformed request.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nFor assistance,
please log a call with the helpdesk.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n'
>>> a.decode('UTF-8')
'HTTP/1.1 400 Bad Request\r\nCache-Control: no-cache\r\nPragma: no-cache\r\nContent-Type: text/html; charset=utf-8\r\nPr
oxy-Connection: close\r\nConnection: close\r\nContent-Length: 1229\r\n\r\n\r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r\n\r\n\
r\n\r\nMy Internet Access\r\n\r\n--\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\
r\n\r\nRequest Error\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nYour request could not be pr
ocessed. Request could not be handled\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nThis could be caused by a misconfig
uration, or possibly a malformed request.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nFor assistance, please
log a call with the helpdesk.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n'
But problem with your code is in request headers. If you want to get some content of page, you can use urllib.request package.
First you need some setup:
import urllib.request as req
class Fetch:
"""
Class responsible for getting code of page at specified url
"""
def __init__(self, url):
self.url = url
self.data = None
self.obj = None
self.headers = None
self.encoding = None
def fetch_file(self):
"""
:return: str contents of a website
Returns string data form an url
"""
self.obj = req.urlopen(self.url)
self.encoding = self.obj.headers['content-type'].split('charset=')[-1]
self.data = self.obj.read().decode(self.encoding)
return self.data
Then you import Fetch class and pass desired url to init object. At the end you call fetch_file() it returns encoded response if in response headers is charset specified.
Try this when sending:
message = "GET / HTTP/1.1\r\n\r\n"
s.sendall(messagebytes.encode())
And while receiving:
reply = s.recv(4096)
print(reply.decode())
Here is my problem with urllib in python 3.
I wrote a piece of code which works well in Python 2.7 and is using urllib2. It goes to the page on Internet (which requires authorization) and grabs me the info from that page.
The real problem for me is that I can't make my code working in python 3.4 because there is no urllib2, and urllib works differently; even after few hours of googling and reading I got nothing. So if somebody can help me to solve this, I'd really appreciate that help.
Here is my code:
request = urllib2.Request('http://mysite/admin/index.cgi?index=127')
base64string = base64.encodestring('%s:%s' % ('login', 'password')).replace('\n', '')
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
resulttext = result.read()
Thankfully to you guys I finally figured out the way it works.
Here is my code:
request = urllib.request.Request('http://mysite/admin/index.cgi?index=127')
base64string = base64.b64encode(bytes('%s:%s' % ('login', 'password'),'ascii'))
request.add_header("Authorization", "Basic %s" % base64string.decode('utf-8'))
result = urllib.request.urlopen(request)
resulttext = result.read()
After all, there is one more difference with urllib: the resulttext variable in my case had the type of <bytes> instead of <str>, so to do something with text inside it I had to decode it:
text = resulttext.decode(encoding='utf-8',errors='ignore')
What about urllib.request ? It seems it has everything you need.
import base64
import urllib.request
request = urllib.request.Request('http://mysite/admin/index.cgi?index=127')
base64string = bytes('%s:%s' % ('login', 'password'), 'ascii')
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib.request.urlopen(request)
resulttext = result.read()
An alternative using OpenerDirector that installs the auth headers for all future urllib requests
login_pass = base64.b64encode(f'{login}:{password}'.encode()).decode()
opener = urllib.request.build_opener()
opener.addheaders = [('Authorization', f'Basic {login_pass}')]
urllib.request.install_opener(opener)
response = urllib.request.urlopen(API_URL)
print(response.read().decode())
A further example using HTTPBasicAuthHandler although a bit more work required if need to send credentials unconditionally:
password_mgr = urllib.request.HTTPPasswordMgrWithPriorAuth()
password_mgr.add_password(None, API_URL, login, password, is_authenticated=True)
auth_handler = request.HTTPBasicAuthHandler(password_mgr)
opener = request.build_opener(auth_handler)
request.install_opener(opener)
response = urllib.request.urlopen(API_URL)
print(response.read().decode())