I am trying to automate some stuff I do day-to-day online and get some experience in webscraping. I saw some stuff about Selenium and Python being good for beginners and thought I would give it a try. I ended up getting a simple script to work on my home computer but it doesn't find the Chrome binary at the path I am giving and I do not know what to do.
I used this to set my PATH environment variable to the directory that holds my chromedriver https://youtu.be/dz59GsdvUF8
I downloaded ChromeDriver 74.0.3729.6 (latest stable release) from https://sites.google.com/a/chromium.org/chromedriver/
I am using python3, visual studio code, and Ubuntu if any of that matters.
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
#Stack overflow said to try this but could not make it work
#opts = webdriver.ChromeOptions()
#opts.binary_location = "C:\webdrivers\chromedriver.exe"
#browser = webdriver.Chrome(chrome_options=opts)
#this is where my chrome is
#C:\Users\tucker.steven2\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Virtual Programs\Google Chrome\Chrome v62
#I have a chromedriver in the same directory as my python code
browser = webdriver.Chrome('chromedriver.exe')
browser.get(https://www.something.com/) #just a generic site
browser.quit()
My expected result is to open a chrome window to Hulu's website, the error I get is:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Windows NT 10.0.16299 x86_64)
If you are on Ubuntu, which is a distribution of Linux, then your path is all wrong. The YouTube video was for Windows. The file structure is completely different on Linux. Please update with your current operating system where you are trying to run this. Also you don't need to write a script - just to open hulu. You can just make a shortcut and put it on your desktop. If you are trying to do something more complicated that takes a long time to do, that would be a better use case for making a selenium script. I get that you are trying to learn this, which is commendable. If you are really on Ubuntu as your operating system, then you would have a /usr/bin folder where your programs are. If you are running Windows, it would be C:\Users\"Your User Name Goes Here"\Documents or where ever you were going.
Programs would be in C:\Program Files\ or C:\Program Files(x86). I would recommend watching some YouTube videos about the path on your operating system.
The answer depends on your operating system and where you have saved your programs.
Related
I'm running Pycharm and attempting to use selenium to launch and control an instance of firefox.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.binary_location = "/usr/lib/firefox/firefox"
browser = webdriver.Firefox(options=options)
This produces an error:
selenium.common.exceptions.InvalidArgumentException: Message: binary is not a Firefox executable
Am I writing the path of the firefox binary correctly? From the terminal within the virtual environment, there is no firefox directory in /usr/lib - presumeably because I'm in the virtual environment? From the system terminal, I can find it no problem at /usr/lib/firefox/firefox.
How do I write a path to the firefox executable in the system environment, from within my virtual environment in pycharm? What's happening here that's preventing me from seeing / accessing the system environment?
I've tried the instructions at each of the below, the issue is not that I need to set an option to the correct path - the binary is located in my system at /usr/lib/firefox/firefox - but setting the option as in the code above does not result in success, maybe because firefox does not exist at that location within the virtual environment.
I've tried all suggestions at each of the below:
Unable to open firefox using selenium-python-geckodriver
SessionNotCreatedException: Message: Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary'
Selenium on PyCharm cannot find firefox binary
Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided using GeckoDriver
I am trying to get PyVisa working on my mac. I have installed it and NI libraries but when I try to run the following simple python file from terminal in VS code:
import sys
import pyvisa as visa
rm = visa.ResourceManager()
rm.list_resources()
inst = rm.open_resources('GPIB::10::INSTR') #Instrument GPIB Addr = 10
print (inst.query("*IDN?"))
By running 'python3 temp.py' I get the following error message:
Error Domain=KMErrorDomain Code=71 "Insufficient permissions for action: Unable to load 'com.highpoint-tech.kext.HighPointRR' in unprivileged request." UserInfo={NSLocalizedDescription=Insufficient permissions for action: Unable to load 'com.highpoint-tech.kext.HighPointRR' in unprivileged request.}
zsh: abort python3 temp.py
Make sure com.ni.driver.NiViPciK extension is installed and loaded. For this, go to About This Mac -> System Report -> Extensions (in Software section) and find NiViPciK. It will likely say "Loaded: No". You need to manually allow National Instruments extensions.
Boot into recovery mode by holding Cmd-R while powering up.
Open Terminal from Tools menu.
Execute: spctl kext-consent add SKTKK2QZ74
Reboot
This did the trick for me:
I have been using NI IVI backend, but lately gave a spin to open source PyVISA-py backend instead and it has worked great for our automation purposes both on OSX and Linux.
I changed
rm = visa.ResourceManager() to rm = visa.ResourceManager('#py').
Some minor modifications may be needed (I had to remove instrument.clear() calls for some reason).
I'm trying to scrape a third party site using Selenium (I tried using Beautiful Soup, but their authentication system is too difficult to understand so I bailed and used Selenium instead).
I'm using a python script that uses the latest Selenium 3.12.0 drivers, and the selenium-server-standalone-3.12.0.jar. Both my Linux and macOS boxes are running jdk 1.8.0_172.
If I run the script on either Linux or macOS with the selenium server on macOS it works great. If I run the script on either Linux or macOS with the selenium server on Linux, it fails to load jQuery with the following error:
18:28:02.453 ERROR [DefaultJavaScriptErrorListener.loadScriptError] - Error loading JavaScript from [https://ww6.fltplan.com/js/jquery-1.12.3.min.js].
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
The site I'm scraping is written very primitively (looks like something out of GeoCities) so it's very likely it's doing something absurd. Oh, and this code was working a few weeks ago and I "froze" it, but then it stopped working in Linux and I had to start tinkering with it again.
Here's my connect code:
capabilities = {
'browserName': 'htmlunit',
'version': 'firefox',
'platform': 'ANY',
'javascriptEnabled': True,
'acceptInsecureCerts': True,
}
driver = webdriver.Remote(
# "http://192.168.1.46:4444/wd/hub", webdriver.DesiredCapabilities.HTMLUNITWITHJS
# "http://localhost:4444/wd/hub", webdriver.DesiredCapabilities.HTMLUNITWITHJS
"http://localhost:4444/wd/hub", capabilities
)
driver.get("https://www.fltplan.com/")
The first commented out Remote url is to Selenium running on my macOS box, it works. The second and third are attempts to make it work on Linux, so far unsuccessful. The second one is the one that worked up until a few weeks ago.
I think I have a solution. I make a copy of the java.security file in java_home/jre/lib/security and commented out all the "disabledAlgorithms" lines in the file. I start up the selenium server with nohup java -Djava.security.properties==java.security.copy -jar selenium-server-standalone-3.12.0.jar and it seems to work now.
I'd be interested in hearing an explanation why this works. I assume it means the site they're trying to download jquery from uses encryption that jdk 1.8 disables because it's considered too weak.
Before you let me know that this is a duplicate, the issue I am facing is that I am running my python code using a jupyter-notebook via an Ubuntu subsystem on Windows.
I have made sure that the path to the folder on Windows where chromedriver is located is correct. However, I am still getting the same error. Further, I am unable to see where my files from the Ubuntu subsystem are (ie, I can't find a folder on Windows with those files).
Anyway in which I can correct this? such as editing the ~/.bashrc file on Ubuntu?
browser = webdriver.Chrome("C:/Users/vr235/Downloads/chromedriver/chromedriver.exe")
Error:
WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
EDIT:
While a solution was pointed by user DebanjanB and marked as duplicate, I tried the user's suggestion but got the same error.
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'C:\Users\vr235\Downloads\chromedriver\chromedriver.exe')
browser.get("https://www.sciencedirect.com/")
Error:
WebDriverException: Message: 'C:\Users\vr235\Downloads\chromedriver\chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
In case one is using a WSL, one can identify the path where the Chromedriver is present by viewing the mnt folder on WSL.
For instance: My chromedriver path was in the Downloads folder in C:/ drive
Path : /mnt/c/vr235/Users/Downloads/chromedriver.exe
Once you add the same, the above lines of code work perfectly.
Using my terminal, the code "from PIL import Image" works perfectly and is recognized by my computer. This allows me to get images using the path address.
Here is my issue, when I open wingIDE and try the same code...this module isn't recognized.
Is anyone familiar with wingIDE that can help me?
I would assume PyCharm people might have the same issue with possibly a similar fix, any advice??
Thanks,
Adam
Most likely Wing is using a different Python than the one you installed Pillow into. Try this on the command line:
import sys; sys.executable
Then set the Python Executable in Wing's Project Properties to the full path of that executable (or in Wing 101 this is set in the Configure Python dialog from the Edit menu). You'll need to restart any debug process and/or the Python Shell from its Options menus.