How to make a request to website and download search data - python-3.x

Question

I'm trying to download some car VIN data for a list of individuals. This involves making a request to a website "http://vin.place/search.php" via requests.get in python3.7.
My code looks like this:
import requests
import pandas as pd
from bs4 import BeautifulSoup
payload = {'first name':'JOHN','last name':'DOE'}
webpage_response = requests.get("http://vin.place/search.php",data = payload)
webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")
results = soup.select(".search-content")
for x in results:
print(x.get_text())
Unfortunately, I am finding that results = [] instead. I suspect there is something wrong with the way that I am coding up the requests.get command. I am not sure how what the right keys should be for the website. Can anyone please help? Thanks.

Answer 1

The payload you have is wrong, it should be first, last. And the names in lowercase:
import requests
from bs4 import BeautifulSoup
payload = {'first':'john','last':'doe'}
webpage_response = requests.post("http://vin.place/search.php", data=payload)
webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")
results = soup.select(".search-content")
for x in results:
print(x.text)
Prints:
JOHN DOE Purchase of a 2007 HONDA ACCORD
JOHN DOE Purchase of a 2007 CHRYSLER TOWN AND COUNTRY
JOHN DOE Purchase of a 2007 DODGE RAM PICKUP 2500
JOHN DOE Purchase of a 2007 DODGE RAM PICKUP 1500
JOHN DOE Purchase of a 2007 VOLKSWAGEN PASSAT
JOHN DOE Purchase of a 2007 VOLKSWAGEN JETTA
JOHN DOE Purchase of a 2007 CHRYSLER SEBRING
JOHN DOE Purchase of a 2007 DODGE RAM PICKUP 1500
JOHN DOE Purchase of a 2007 JEEP PATRIOT
JOHN DOE Purchase of a 2007 Chevrolet TrailBlazer
JOHN DOE Purchase of a 2007 FORD EDGE
JOHN DOE Purchase of a 2007 JEEP WRANGLER UNLIMITED
JOHN DOE Purchase of a 2007 DODGE RAM PICKUP 2500
JOHN DOE Purchase of a 2007 FORD MUSTANG
JOHN DOE Purchase of a 2007 JEEP WRANGLER
JOHN DOE Purchase of a 2007 CHRYSLER 300
JOHN DOE Purchase of a 2007 JEEP WRANGLER UNLIMITED
JOHN DOE Purchase of a 2007 DODGE MAGNUM
JOHN DOE Purchase of a 2007 CADILLAC ESCALADE ESV
JOHN DOE Purchase of a 2007 FORD F-150
JOHN DOE Purchase of a 2007 HONDA ACCORD
JOHN DOE Purchase of a 2007 JEEP LIBERTY
JOHN DOE Purchase of a 2007 CADILLAC ESCALADE
JOHN DOE Purchase of a 2007 DODGE RAM PICKUP 1500
JOHN DOE Purchase of a 2007 JEEP WRANGLER

Answer 2

looking at the form data as sent by chrome when performing the search manually, the data is: first=JOHN&last=DOE, also, the site uses POST
please try this:
import requests
import pandas as pd
from bs4 import BeautifulSoup
payload = {'first':'JOHN','last':'DOE'}
webpage_response = requests.post("http://vin.place/search.php",data = payload)
webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")
results = soup.select(".search-content")
for x in results:
print(x.get_text())