I'd like to open a Chrome instance on Python though selenium using a proxy to hide my real IP and stop getting blocked when I scrape certain websites.
I have been going thought several similar posts like this and this but with no success.
I am using the following code:
from selenium import webdriver
PROXY = "165.22.62.179:8118" # IP:PORT or HOST:PORT
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(executable_path = path_to_chromedriver, options=chrome_options)
chrome.get(target_website)
The instance open correctly but when I run the command chrome.get(target_website) I get no answer:
If I open the instance without proxy it works fine. I am getting the proxies from this website. I want to create a function that takes as input a proxy IP and returns a running Chrome instance that uses that IP.
Can you please me help to fix my code? Thanks for your help!
Related
I am a new developer. I am trying to execute selenium code in an already opened tor session.
I also tried to find my query on the internet but didn't get a satisfying answer.
My code looks like this:
binary = r'C:\Users\Admin\Desktop\Tor Browser\Browser\firefox.exe'
options = Options()
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.socks", "127.0.0.1")
options.set_preference("network.proxy.socks_port", 9050)
options.set_preference("network.proxy.socks_remote_dns", False)
options.set_capability( "debuggerAddress", f"127.0.0.1:1060")
# options.add_experimental_option("debuggerAddress", f"127.0.0.1:1060")
global browser
# only one instance of a browser opens, remove global for multiple instances
if not browser:
browser=webdriver.Firefox(firefox_binary=binary,options=options,firefox_profile=profile)
return browser
In simple words, this code doesn't connect to the running tor session but creates another session.
Hello can you tell me how to use the tor browser via selenium so far I have only found this:
from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver('C:\Users\Brotb\Downloads\win (1)\vendor\tor-bundle') as driver:
driver.get('https://google.com')
and this doesnt work
can somebody help me?
I am trying to close portable browser via selenium
I passed --remote-debugging-port=9222 because if I do not pass it then the program is stuck in object creation of webdriver.Chrome(). It will open the portable browser but not load the URL.
But after the URL is open I want to close the browser but driver.quit() it is not working for me. I have tried some other methods for close the browser but they do not work as well.
I want to close the specific instance of the browser which is opened by this program not other opened instances of the browser.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.binary_location = 'C:/Portable/GoogleChromePortable/GoogleChromePortable.exe'
chrome_options.add_argument("--remote-debugging-port=9222")
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--profile-directory=Person 1")
driver = webdriver.Chrome(options=chrome_options,executable_path='C:/Portabl/chromedriver_win32/chromedriver.exe')
url = "https://www.google.com/"
driver.get(url)
driver.quit()
I am using:
selenium 3.141.0, windows 10, python 3.8.0, portable chrome version 93.0.4577.63 (32-bit)
Your this statement
I passed --remote-debugging-port=9222 because if I do not pass it then
the program is stuck in object creation of webdriver.Chrome()
is not correct. --remote-debugging-port=9222 looks like a port number where your application is deployed and you have used chrome option to send them to browser object.
driver.quit()
this typically should have worked, what is the error when it did not work ?
also, for closing a single instance you could do
driver.close()
see if that helps.
When you input http://www.python.or/ (intentionally use the wrong url) in firefox or other browsers, browser show something such as below:
The connection was reset
The connection to the server was reset while the page was loading.
The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the
Now let's do the same task with selenium.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(chrome_options=chrome_options)
browser.get("http://www.python.or")
When you execute the above code in console,no error info,how to catch the exception such as firefox do with selenium?
I suggest you to try HTML requests
First, get the requests from the URL:
import requests
r = requests.get(your_url)
Now you need to get the status_code from your request:
print(r.status_code)
r.status_code contains the http's error handler.
Here is a list of all the HTTP status codes
Try this and you'll know when something wrong happened (ex Error 404)
I'm using Selenium 3.5 for Python to test some behaviors of my web app using both local pages and online ones.
Until now, I just used Chrome as main browser to test it and everything works fine. Then, I decided to use Firefox 55.0.3 using geckodriver v0.19.0.
I have the following issues:
selenium opens the local pages such as file:///PATH/index.html but it doesn't close the browser even using the function quit();
I would use some browser option such as the incognito mode, but I was not able to find a list of all available options for Firefox.
This is a snippet of my code to use selenium with Firefox.
from selenium import webdriver
url = 'http://www.yahoo.com'
#url = 'file:///PATH/index.html'
browser = webdriver.Firefox()
browser.get(url)
browser.quit()
# borser.close() -> I also tried with the close()
I also read this question for the first issue but it doesn't work with the current version of the used framework.
About the second I can only find partial solution such as this one for the incognito mode but it doesn't work as expected.
Any ideas? Is it a kind of bug in geckodriver?