Selenium using Python in 2019 on mac - python-3.x

I'm a non-programmer trying to use Automate the Boring Stuff with Python to do exactly that. I watched all the videos and thought I was ready, but as I tried to get going I'm stuck on square zero.
I'm on a Mac OSx Mojave Version 10.14.2
Successfully installed Python 3.7.2
I'm trying to follow this chapter: http://automatetheboringstuff.com/chapter11/
Successfully installed Selenium because:
import selenium
does not give an error.
but when I run this:
from selenium import webdriver
browser = webdriver.Firefox()
I get this error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'your\\path\\geckodriver.exe': 'your\\path\\geckodriver.exe'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
driver = webdriver.Firefox(executable_path=r'your\path\geckodriver.exe')
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
self.service.start()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'your\path\geckodriver.exe' executable needs to be in PATH.
>>> driver = webdriver.Firefox(executable_path=r'your\path\geckodriver.exe')
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'your\\path\\geckodriver.exe': 'your\\path\\geckodriver.exe'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
driver = webdriver.Firefox(executable_path=r'your\path\geckodriver.exe')
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
self.service.start()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'your\path\geckodriver.exe' executable needs to be in PATH.
I've tried everything in this: Selenium using Python - Geckodriver executable needs to be in PATH and I don't think it works anymore.
I've installed geckodriver and tried appending it to PATH but nothing seems to work. geckodriver is installed in /usr/local/bin/ and I've appended it to path by running export PATH=$PATH:/path/to/geckodriver and export PATH=$PATH:/usr/local/bin/geckodriver but I keep getting the same error
I've tried having both the newest Firefox and Firefox v46 installed because I read somewhere that it doesn't work after that version.
Any help is much appreciated.

I suggest to install with pip
pip install geckodriver-autoinstaller
and then add this code
import geckodriver_autoinstaller
# Check if the current version of geckodriver exists
# and if it doesn't exist, download it automatically,
# then add geckodriver to path
geckodriver_autoinstaller.install()

Set your executable_path
from selenium import webdriver
webdriver.Firefox(executable_path=r'path\to\geckdriver\geckodriver.exe')
executable_path –
Full path to override which geckodriver binary to use for Firefox 47.0.1 and greater, which defaults to picking up the binary from the system path.
from documentation
Edit: also works on lower versions firefox
Edit 2:
Perhaps you could verify the executable can be found by python itself.
import os.path
os.path.exists(file_path)

Related

Q: How to fix the missing dependancies in pyzbar

I am currently using spyder via anaconda with python 3.8.5 on windows 10 and I am trying to use the pyzbar package to make a barcode reader. When I try to run the script I get this error message when trying to run from pyzbar.pyzbar import decode and from pyzbar import pyzbar:
Traceback (most recent call last):
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 58, in load
dependencies, libzbar = load_objects(Path(''))
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 50, in load_objects
deps = [
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 51, in <listcomp>
cdll.LoadLibrary(str(directory.joinpath(dep)))
File "C:\Users\norinhan\Anaconda3\lib\ctypes\__init__.py", line 459, in LoadLibrary
return self._dlltype(name)
File "C:\Users\norinhan\Anaconda3\lib\ctypes\__init__.py", line 381, in __init__
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'libiconv.dll' (or one of its dependencies). Try using the full path with constructor syntax.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<ipython-input-1-95ae0761f4b5>", line 1, in <module>
from pyzbar.pyzbar import decode
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\pyzbar.py", line 7, in <module>
from .wrapper import (
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\wrapper.py", line 139, in <module>
zbar_version = zbar_function(
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\wrapper.py", line 136, in zbar_function
return prototype((fname, load_libzbar()))
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\wrapper.py", line 115, in load_libzbar
libzbar, dependencies = zbar_library.load()
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 60, in load
dependencies, libzbar = load_objects(Path(__file__).parent)
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 50, in load_objects
deps = [
File "C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\zbar_library.py", line 51, in <listcomp>
cdll.LoadLibrary(str(directory.joinpath(dep)))
File "C:\Users\norinhan\Anaconda3\lib\ctypes\__init__.py", line 459, in LoadLibrary
return self._dlltype(name)
File "C:\Users\norinhan\Anaconda3\lib\ctypes\__init__.py", line 381, in __init__
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\norinhan\Anaconda3\lib\site-packages\pyzbar\libiconv.dll' (or one of its dependencies). Try using the full path with constructor syntax.
When trying to run just import pyzbar it will compile, but it will not be able to access any of the function. When I try I get this message:
Traceback (most recent call last):
File "<ipython-input-3-26272af93b8a>", line 1, in <module>
pyzbar.pyzbar.decode()
AttributeError: module 'pyzbar' has no attribute 'pyzbar'
I have looked up the documentation for pyzbar on github which states "The zbar DLLs are included with the Windows Python wheels", but have not found this anywhere. The closest to "Windows Python wheels" I have found is the wheel package which does not contain any information about pyzbar.
Any advice on how to fix this?
Visit https://pypi.org/project/pyzbar/#files to download pyzbar-0.1.8-py2.py3-none-win_amd64.whl.
You can find all the DLL files inside the wheel package.
I have just encountered this problem.
If your OS & python are x64 version, it's just because of lack of vc 2013 x64 runtime.
You can find Visual C++ Redistributable Packages for Visual Studio 2013 here
download & install.
I was facing similar issue just installed Visual
C++ 64 bit version on my computer and restarted. Choose between vcredist_x86.exe or vcredist_x64.exe based on your machine.
https://www.microsoft.com/en-us/download/confirmation.aspx?id=40784. Hope this solves your issue as well.

Pyperclip error in Pythonista on iPad Pro

I am learning Python 3 using a Udemy course. One of the course modules asks us to install Pyperclip in order to try out simple code is the console.
import pyperclip
pyperclip.copy(‘Test’)
pyperclip.paste()
I believe this should output the word Test in the console.
I did the following:
Installed Stash
Installed Pyperclip by running the command ‘pip install Pyperclip’ from within Stash. Package installed fine
Went back to console and ran the import Pyperclip command. No errors
However, when I try to run the function Pyperclip.copy() or .paste() I get an error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/mobile/Containers/Shared/AppGroup/C6F5AC74-E13C-4A33-95BA-F075B9E680F9/Pythonista3/Documents/site-packages-3/pyperclip/__init__.py", line 120, in copy_osx_pbcopy
stdin=subprocess.PIPE, close_fds=True)
File "/var/containers/Bundle/Application/479FB713-67AD-4A41-9180-71A1C27C61E8/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/subprocess.py", line 708, in __init__
restore_signals, start_new_session)
File "/var/containers/Bundle/Application/479FB713-67AD-4A41-9180-71A1C27C61E8/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/subprocess.py", line 1261, in _execute_child
restore_signals, start_new_session, preexec_fn)
PermissionError: [Errno 1] Operation not permitted
I looked around for similar errors on stack overflow but the issues/ answers don’t seem to fit my situation. What am I doing wrong?
Pyperclip is not supported by Pythonista. Try their built in module, clipboard. http://omz-software.com/pythonista/docs/ios/clipboard.html

Error when trying to install library permanently on google colab

I'm trying to install library permanently on google colab. So I run this code first:
import os, sys
from google.colab import drive
drive.mount('/content/drive', force_remount = True)
my_path = '/content/notebooks'
os.symlink('/content/drive/My Drive/Colab Notebooks/my_env', my_path)
sys.path.insert(0, my_path)
there's no error running above code.
However, when I run the code below, I got the error:
!pip install --target=$my_path jdc
Collecting jdc
Downloading https://files.pythonhosted.org/packages/5a/cb/9afea749985eef20f3160e8826a531c7502e40c35a038dfe49b67726e9a0/jdc-0.0.9-py2.py3-none-any.whl
Installing collected packages: jdc
Successfully installed jdc-0.0.9
ERROR: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/shutil.py", line 550, in move
os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-target-21y8mx4d/lib/python/jdc' -> '/content/notebooks/jdc'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 153, in _main
status = self.run(options, args)
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py", line 505, in run
options.target_dir, target_temp_dir, options.upgrade
File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py", line 563, in _handle_target_dir
target_item_dir
File "/usr/lib/python3.6/shutil.py", line 561, in move
symlinks=True)
File "/usr/lib/python3.6/shutil.py", line 321, in copytree
os.makedirs(dst)
File "/usr/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/content/notebooks/jdc'
Is there a way to solve this?
First target your directory where you want to install the library like this:
%cd "/path/to/your/directory"
Then install using ```pip``:
pip install jdc

OSError: [Errno 8] Exec format error - wkhtmltopdf & pydf

I want to use pydf to generate a pdf in Python 3. I tested it out in terminal (Mac) and received the below error. Does anyone know why and what to do?
If possible, I would like to stick with pydf and not have to move to another library.
.
I downloaded the Mac wkhtmltopdf version from https://wkhtmltopdf.org/downloads.html
As well as pydf from pip install python-pdf
>>> import pydf
>>> pydf.generate_pdf('hello')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pydf/wkhtmltopdf.py", line 145, in generate_pdf
p = _execute_wk(*cmd_args, input=html.encode())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pydf/wkhtmltopdf.py", line 30, in _execute_wk
return subprocess.run(wk_args, input=input, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 453, in run
with Popen(*popenargs, **kwargs) as process:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 756, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1499, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pydf/bin/wkhtmltopdf'
>>>
This is because the binary which is packaged with python-pdf is for linux and will not execute on MacOS, this explained briefly in the README.
Luckily the solution is easy, you need to install wkhtmltopdf via homebrew or similar, then set the WKHTMLTOPDF_PATH environment variable to point to it.
In my case that's
export WKHTMLTOPDF_PATH=/usr/local/bin/wkhtmltopdf

Getting the following error when trying to open web page with Selenium WebDriver in python 3.6?

When I try to execute this code :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver= webdriver.Firefox("C:\Program Files\Mozilla Firefox")
driver.get("http://google.com")
I get the following error :
C:\Users\vipul\PycharmProjects\newproject\venv\Scripts\python.exe "C:/Users/vipul/PycharmProjects/newproject/Web Scraping.py"
Traceback (most recent call last):
File "C:\Users\vipul\PycharmProjects\newproject\venv\lib\site-packages\selenium\webdriver\common\service.py", line 74, in start
stdout=self.log_file, stderr=self.log_file)
File "C:\Users\vipul\AppData\Local\Programs\Python\Python36-32\Lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Users\vipul\AppData\Local\Programs\Python\Python36-32\Lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/vipul/PycharmProjects/newproject/Web Scraping.py", line 4, in <module>
driver= webdriver.Firefox("C:\Program Files\Mozilla Firefox")
File "C:\Users\vipul\PycharmProjects\newproject\venv\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 148, in __init__
self.service.start()
File "C:\Users\vipul\PycharmProjects\newproject\venv\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
Process finished with exit code 1
I have faced the same issue before a week ago,
I was using Python 2.7 and the latest version of web driver of Chrome,
Then I passed geckodriver file path like,
driver= webdriver.Firefox("c://your-path")
This fixed my issue, I hoped for your too.
Download geckodriver.exe to a particular folder/directory(eg. c:\drivers) then set the driver path.
Try any one of the following ways.
Pass driver path as argument.
driver= webdriver.Firefox("c:\drivers\geckodriver.exe")
driver.get("http://google.com")
Set driver path using enviornment.
import os
os.environ['webdriver.gecko.driver'] ="c:\drivers\geckodriver.exe"
driver.get("http://google.com")
Note: if you are keeping gecko driver exe file in local folder where the script is there. then no need to set or pass the driver path.

Resources