Logging in to this website using requests module in python3 - python-3.x

Question

Here's what I have done so far. I had to extract the CSFRToken manually (I don't know regex, so that part is messy). Is CSFR part of cookies? Because my cookies only shows two other ID type params, so I dropped the cookie part and did it this way.
import requests
URL = r'http://login.cheezburger.com/'
client = requests.session()
login_page = client.get(URL)
index = login_page.text.find("CSRFToken")
token = login_page.text[index:index+90].split('"')[-2] # This works, I guarantee :)
#print(token) I checked it manually
login_data = {'rlm': 'Shopper',
'for': r'http://login.cheezburger.com/',
'username': 'myusername',
'password': 'mypassword',
'CSRFToken': token}
req = client.post(URL, data=login_data)
Now, there is no error per say, but I am not logging in to this site either. The text of this request shows that I am still stuck in the login page!
The parameters send are (as shown in the dev tools of firefox):
rlm: 'Shopper'
for: 'http://login.cheezburger.com/'
username: 'myusername',
password: 'mypassword',
CSRFToken: '8uhhbf67-1233-fff3-123g1-123123fsdfs22'
The websites source is as follows (the part that contains the form data):
<div class="contents-msl">
<h2>Client Login</h2>
<p>Enter username and password</p>
<div class="form-all-msl">
<form action="/login.action" id="loginForm" method="post"
enctype="application/x-www-form-urlencoded"><input type=hidden name=rlm
value="Shopper"><input
type=hidden
name=for
value="http%3a%2f%2flogin%cheezburger%2ecom%2f">
<ul class="form-section-msl">
<label class="form-label-left-msl" for="loginUserName">
Username<span class="form-required">*</span>
</label>
<div class="form-input-msl">
<input type="text" class="form-textbox-msl" id="loginUserName" name="username"
size="20">
</div>
<label class="form-label-left-msl" for="loginPwd">
Password<span class="form-required-msl">*</span>
</label>
<div class="form-input-msl">
<input type="password" class="form-textbox-msl" id="loginPwd" name="password"
size="20">
</div>
<div class="form-input-msl">
<div class="form-single-column-msl">
</div>
</div>
</ul>
</div>
<button class="members-btn-msl" type="submit">Login</button>
<input type="hidden" name="CSRFToken" class="CSRFToken" value="8uhhbf67-1233-fff3-123g1-123123fsdfs22" /> </form>
</div>
</div>

Answer 1

You maybe want to add some headers
headers = {
'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding':
'gzip, deflate, sdch',
'Accept-Language':
'en-US,en;q=0.8,vi;q=0.6',
'Cache-Control':
'max-age=0',
'Connection':
'keep-alive',
'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36',
}
Then add headers to your code:
req = client.post(URL, data=login_data)
Good luck !