I am new to this, this is my first question here.
Please guide me to improve.
Learning python automation.
Came across scenario to automate "replay server response from .har file".
Could capture, HAR file
from browsermobproxy import Server
server = Server("~/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))
driver = webdriver.Chrome(executable_path = "~/chromedriver", chrome_options=co)
proxy.new_har
driver.get(url)
proxy.har
How can I replay server response from captured HAR file (offline) ?
Is it possible with python or any other ?
And what's the significance of replay server response ?
Could you please help me ?
Check out this https://github.com/louisabraham/har2requests. It should convert the .har file into a python script.
Related
I was just wondering, how can i hide the fact that i'm using selenium to access epic games store site? It heavily kicks me even if i'm just browsing the site myself using webdriver. I saw similar posts about this topic on stack, but they didn't help me. Here's the settings I have now. What's wrong with them?
from fake_useragent import UserAgent
from selenium import webdriver
ua = UserAgent()
us_ag = ua.random
options = webdriver.ChromeOptions()
options.add_argument(f'user-agent={us_ag}')
options.add_argument('--ignore-ssl-errors')
options.add_argument('window-size=1280,800')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(options=options)
The site keeps telling me that i've done captcha wrong. Even if I try for another 10 times - he still tells me the same. There are no problems when I'm browsing with my regular browser.
I'd like to open a Chrome instance on Python though selenium using a proxy to hide my real IP and stop getting blocked when I scrape certain websites.
I have been going thought several similar posts like this and this but with no success.
I am using the following code:
from selenium import webdriver
PROXY = "165.22.62.179:8118" # IP:PORT or HOST:PORT
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(executable_path = path_to_chromedriver, options=chrome_options)
chrome.get(target_website)
The instance open correctly but when I run the command chrome.get(target_website) I get no answer:
If I open the instance without proxy it works fine. I am getting the proxies from this website. I want to create a function that takes as input a proxy IP and returns a running Chrome instance that uses that IP.
Can you please me help to fix my code? Thanks for your help!
I had tried to get the response data from a website https://www.mscdirect.com/product/details/49784960 using python requests.
But am not getting the response and i had tried selenium too. But still the error persists
Using Python requests
import requests
session=requests.session()
response=session.get('https://www.mscdirect.com/product/details/49784960')
print(response.text)
Using Selenium
from selenium import webdriver
options = Options()
#options.add_argument("--headless")
#options.add_argument('--disable-gpu')
driver = webdriver.Chrome(executable_path="chromedriver", chrome_options=options)
driver.delete_all_cookies()
driver.get("https://www.mscdirect.com/product/details/49784960")
print(driver.page_source)
Is there any thing am missing? For python requests, i has passed headers also.
When you input http://www.python.or/ (intentionally use the wrong url) in firefox or other browsers, browser show something such as below:
The connection was reset
The connection to the server was reset while the page was loading.
The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the
Now let's do the same task with selenium.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(chrome_options=chrome_options)
browser.get("http://www.python.or")
When you execute the above code in console,no error info,how to catch the exception such as firefox do with selenium?
I suggest you to try HTML requests
First, get the requests from the URL:
import requests
r = requests.get(your_url)
Now you need to get the status_code from your request:
print(r.status_code)
r.status_code contains the http's error handler.
Here is a list of all the HTTP status codes
Try this and you'll know when something wrong happened (ex Error 404)
Is there any whatsapp or webdriver API to access WhatsApp web without scanning QR code everytime while accessing it using selenium and chrome webdriver in python?
This is What you need. This code Read QR and store it in cookies
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
jokes = ["You don't need a parachute to go skydiving. You need a parachute to go skydiving twice.",
"This is Test Message."]
options = Options()
options.add_argument("--user-data-dir=chrome-data")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=options)
driver.maximize_window()
driver.get('https://web.whatsapp.com') # Already authenticated
time.sleep(20)
##################### Provide Recepient Name Here ###############################
driver.find_element_by_xpath("//*[#title='MyJakartaNumber']").click()
for joke in jokes:
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div[2]/div/div[2]').send_keys(joke)
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div[3]/button/span').click()
time.sleep(10)
time.sleep(30)
driver.close()
Your "WhatsApp" and "QR Code" don't tell anything to me, however if you're testing an application which requires an extra action to sign in I don't think you will be able to perform it using Selenium as it's browser automation framework.
Web applications identify users via Cookies - special HTTP Headers containing client-side information. When you start a web browser via Selenium bindings - it kicks off a clean browser session which is not authenticated in "WhatsApp" (whatever it is)
The solutions would be in:
Authenticate in WhatsApp manually, store your browser profile somewhere and start Selenium by pointing it to the earlier storied profile folder
Authenticate in WhatsApp manually, store your browser cookies and use WebDriver.add_cookie() function to read the stored cookies into the current clean session
You can use "pywhatkit". pywhatkit is used to send messages using whatssapp web. Run:
pip install pywhatkit
and you are good to go.