I attempted to exectue the following code:
df = pd.DataFrame()
for file in files:
if file.endswith('.xls'):
df = df.append(pd.read_excel(file), ignore_index=True)
df.head()
but received this error on the 4th command line:.
However, when I check modules installed I receive:
Why does Jupyter not recognize xlrd as being installed? Thanks for any feedback or help.
Problem
You are using pip python's default package manager to check a package in python's default install location.
But using Anaconda's virtual environment (which probably jupyter notebook is used with) to run python script which requires packages to be installed in it's own directory via conda package maanger.
Solution
Run this command in Anaconda command pompt:
conda install -c conda-forge xlrd
Related
I'm trying to use pip to install a package. I try to run pip install from the Python shell, but I get a SyntaxError. Why do I get this error? How do I use pip to install the package?
>>> pip install selenium
^
SyntaxError: invalid syntax
pip is run from the command line, not the Python interpreter. It is a program that installs modules, so you can use them from Python. Once you have installed the module, then you can open the Python shell and do import selenium.
The Python shell is not a command line, it is an interactive interpreter. You type Python code into it, not commands.
Use the command line, not the Python shell (DOS, PowerShell in Windows).
C:\Program Files\Python2.7\Scripts> pip install XYZ
If you installed Python into your PATH using the latest installers, you don't need to be in that folder to run pip
Terminal in Mac or Linux
$ pip install XYZ
As #sinoroc suggested correct way of installing a package via pip is using separate process since pip may cause closing a thread or may require a restart of interpreter to load new installed package so this is the right way of using the API: subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'SomeProject']) but since Python allows to access internal API and you know what you're using the API for you may want to use internal API anyway eg. if you're building own GUI package manager with alternative resourcess like https://www.lfd.uci.edu/~gohlke/pythonlibs/
Following soulution is OUT OF DATE, instead of downvoting suggest updates. see https://github.com/pypa/pip/issues/7498 for reference.
UPDATE: Since pip version 10.x there is no more get_installed_distributions() or main method under import pip instead use import pip._internal as pip.
UPDATE ca. v.18 get_installed_distributions() has been removed. Instead you may use generator freeze like this:
from pip._internal.operations.freeze import freeze
print([package for package in freeze()])
# eg output ['pip==19.0.3']
If you want to use pip inside the Python interpreter, try this:
import pip
package_names=['selenium', 'requests'] #packages to install
pip.main(['install'] + package_names + ['--upgrade'])
# --upgrade to install or update existing packages
If you need to update every installed package, use following:
import pip
for i in pip.get_installed_distributions():
pip.main(['install', i.key, '--upgrade'])
If you want to stop installing other packages if any installation fails, use it in one single pip.main([]) call:
import pip
package_names = [i.key for i in pip.get_installed_distributions()]
pip.main(['install'] + package_names + ['--upgrade'])
Note: When you install from list in file with -r / --requirement parameter you do NOT need open() function.
pip.main(['install', '-r', 'filename'])
Warning: Some parameters as simple --help may cause python interpreter to stop.
Curiosity: By using pip.exe you actually use python interpreter and pip module anyway. If you unpack pip.exe or pip3.exe regardless it's python 2.x or 3.x, inside is the SAME single file __main__.py:
# -*- coding: utf-8 -*-
import re
import sys
from pip import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
To run pip in Python 3.x, just follow the instructions on Python's page: Installing Python Modules.
python -m pip install SomePackage
Note that this is run from the command line and not the python shell (the reason for syntax error in the original question).
I installed python and when I run pip command it used to throw me an error like shown in pic below.
Make Sure pip path is added in environmental variables. For me, the python and pip installation path is::
Python: C:\Users\fhhz\AppData\Local\Programs\Python\Python38\
pip: C:\Users\fhhz\AppData\Local\Programs\Python\Python38\Scripts
Both these paths were added to path in environmental variables.
Now Open a new cmd window and type pip, you should be seeing a screen as below.
Now type pip install <<package-name>>. Here I'm installing package spyder so my command line statement will be as pip install spyder and here goes my running screen..
and I hope we are done with this!!
you need to type it in cmd not in the IDLE. becuse IDLE is not an command prompt if you want to install something from IDLE type this
>>>from pip.__main__ import _main as main
>>>main(#args splitted by space in list example:['install', 'requests'])
this is calling pip like pip <commands> in terminal. The commands will be seperated by spaces that you are doing there to.
If you are doing it from command line,
try -
python -m pip install selenium
or (for Python3 and above)
python3 -m pip install selenium
I installed miniconda and just created a conda environment:
conda create -n my_env python=3.5 anaconda
I am trying to:
import cvlib
But I am getting the error:
ImportError: No module named cvlib
So I have tried to install using:
pip3 install cvlib
This seemed to work successfully, but then when I try to import cvlib I am still getting the ImportError: No module named cvlib error (I have retarted my terminal after the installation).
Is this a problem with my PYTHONPATH not containing the path to the directory that now contains cvlib? If so, how do I find where cvlib is saved so that I can add the path?
Check if the library is in your python directory. Otherwise, make a repl.it account, and install cvlib, and check the functions or the lib name. Maybe try searching a more advanced installation of cvlib.
it might have occurred due to the version of python you installed or due to the directory, you installed.
try uninstalling the current version of python and try installing an older version of python and install it in the directory as shown below:
C:\Users\Rajish\AppData\Local\Programs\Python\Python39
also, select add the path to environment variables while installing
and after that install cvlib and all other required modules and packages
it worked for me.
I've found the solution:anaconda - graphviz - can't import after installation
I want to use graphviz and follow the commend in https://anaconda.org/anaconda/graphviz
run following in terminal
conda install -c anaconda graphviz
However no matter in Jupyter Notebook, python or Pycharm to import graphviz, it always shows
ModuleNotFoundError: No module named 'graphviz'
How to solve this problem? Thank you.
PS:
when run which python in terminal: it return /opt/anaconda3/bin/python, therefore I use anaconda environment by default. And I have only one environment in anaconda that is root.
when I run conda list in terminal, I can find this line :
graphviz 2.40.1 hefbbd9a_2
I found a weird thing:
my pip and conda use the same environment:
run :which pip
get : /opt/anaconda3/bin/pip
run : which conda
get : /opt/anaconda3/bin/conda
However when I run pip list, I cannot find graphviz and many other packages which shows in conda list. For these packages show in conda list but not in pip list, I also cannot import them no matter in Jupyter notebook, python, pycharm etc. Why this happens?
After using "conda install attrs", other package installations are working fine without any http connection or ModuleNotFoundError errors. Please try and let me know.
Importing pysftp into Jupyter Notebook
While importing pysftp into Jupyter Notebook, ModuleNotFoundError is shown.
Checking import of pysftp on device?
I have verified the package installation with
pip list and pip show pysftp
Had imported pysftp package(v0.2.9) and installed it in the below location
C:\users\xxxxxx\appdata\roaming\python\python37\site-packages
Check : Package installed OKAY
Check about package correct path linking from cmd prompt?
I'm using Python 3.7.0 on a WIN machine, verifyed the site package location using
import sys and sys.path
image confirms linking of PATH to correct location and the package is successfully executed when python is run through cmd prompt
Check : Path link and cmd run OKAY
Now could anyone help me solve why the package import in Jupyter Notebook is throwing an error?
Thank you
Edit 1: Check for different environment installed? added based on one of the answer
Only one environment is present in the machine
I got the same. I solved by installing directly within Jupyter using the following command:
import sys
!{sys.executable} -m pip install dice-ml
Are you running the notebook through a virtual environment?
You can try running the same commands as you did on CMD by preceding it with ! as follows:
!pip list
Ideally this should list the same contents as shown in CMD. However the results may be different if you are running Jupyter notebook in a virtual environment. If you are unable to see pysftp, you need to install it within the virtual environment. This can be done from within your notebook as:
!pip install pysftp
When I try to import psycopg2 it show below log for me:
Traceback (most recent call last):
File "D:/Desktop/learn/python/webcatch/appserver/testpgsql.py", line 2, in <module>
import psycopg2
File "D:/Desktop/learn/python/webcatch/appserver/webcatch/lib/site-packages/psycopg2-2.6.1-py3.5-win32.egg/psycopg2/__init__.py", line 50, in <module>
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: No module named 'psycopg2._psycopg'
How can I solve it?
My platform is win10 (64) and version is python 3.5
Eureka! I pulled my hair out for 2 days trying to get this to work. Enlightenment came from this SO Question. Simply stated, you probably installed psycopg2 x64 version like I did, not realizing your python version was 32-bit. Unistall your current psycopg2, then:
Download: psycopg2-2.6.1.win32-py3.4-pg9.4.4-release.exe from HERE, then run the following in a Terminal:
C:\path\to\project> easy_install /path/to/psycopg2-2.6.1.win32-py3.4-pg9.4.4-release.exe
C:\path\to\project> python manage.py makemigrations
C:\path\to\project> python manage.py migrate
You may also need to (re)create super user with:
C:\path\to\project> python manage.py createsuperuser
I had the same problem, solved it in this way:
Reinstall the package psycopg2 using pip (by default installed with python 3)
On Linux:
pip uninstall psycopg2
Confirm with (y) and then:
pip install psycopg2
On Windows I add the prefix ('python -m') to the commands above.
I think the problem occurs when you change the version of Python. (Even between minor versions such as Python 3.5 and 3.6).
I am using psycopg in an AWS Glue Job, where is harder to follow the instructions listed in the other answers.
What I did is installing psycopg2-binary into a directory and zip up the contents of that directory:
mkdir psycopg2-binary
cd psycopg2-binary
pip install psycopg2-binary -t .
# in case using python3:
# python3 -m pip install --system psycopg2-binary -t .
zip -r9 psycopg2.zip *
I then copied psycopg2.zip to an S3 bucket and add it as an extra Python library under "Python library path" in the Glue Spark job.
I then launched the job with the following script to verify if psycopg2 is present (the zip file will be downloaded by Glue into the directory in which the Job script is located)
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
import sys
import os
import zipfile
## #params: [JOB_NAME]
args = getResolvedOptions(sys.argv, ['JOB_NAME'])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)
zip_ref = zipfile.ZipFile('./psycopg2.zip', 'r')
print os.listdir('.')
zip_ref.extractall('/tmp/packages')
zip_ref.close()
sys.path.insert(0, '/tmp/packages')
import psycopg2
print(psycopg2.__version__)
job.commit()
Download the compiled version of psycopg2 from this link https://github.com/jkehler/awslambda-psycopg2. As psycopg2 is C library for python, which need to be compiled on linux to make it work. The compile instruction also given on that link. Thanks to the https://github.com/jkehler.
This also happens to me in new Ubuntu 18.04. It is caused by missing one file _psycopg.py in the /usr/local/lib/python3.7/site-packages/psycopg2.
It is fixed by:
remove the old psycopg2 from your machine pip3 uninstall psycopg2.
download new pyscopg2 manually from the official page http://initd.org/psycopg/tarballs/PSYCOPG-2-7/psycopg2-2.7.7.tar.gz
tar xvf psycopg2-2.7.7.tar.gz
python setup.py build
sudo python setup.py install
I had this happen in Linux using Python 3.7. It is caused by missing one file _psycopg.cpython-37m-x86_64-linux-gnu.so in the /usr/local/lib/python3.7/site-packages/psycopg2.
I downloaded _psycopg.cpython-37m-x86_64-linux-gnu.so from https://github.com/jkehler/awslambda-psycopg2/tree/master/psycopg2-3.7, and Copied this file into my anaconda lib.
I had this happen in Linux using Python 2 because I had accidentally had my PYTHONPATH set to Python 3 libraries, and it was trying to load the python3 version of psycopg2. Solution was to unset PYTHONPATH.
I had the same error on Windows, this worked for me:
pip install -U psycopg2
I had an older version installed, must have depreciated
For lambda functions on Python 3.7, I ended up using the psycopg2-binary library mentioned in these threads:
https://github.com/jkehler/awslambda-psycopg2/issues/51
Using psycopg2 with Lambda to Update Redshift (Python)
pip3 install psycopg2-binary==2.8.3
Snippet from these links:
I ended up using a different library: psycopg2-binary in my requirement.txt file and it working fine now.
solved it by using psycopg2-binary==2.8.3
I came to know that most times the WINDOWS packaging does not go fine with LAMBDA.
I faced same issue while running LAMBDA with WINDOWS installed 3rd party pscyopg2 packaging.
Solution:
step1>
I installed psycopg2 in Linux.
Copied both the directories psycopg2_binary-2.8.2.dist-info and psycopg2 from Linux to windows.
step2>
Along with source *.py, packaged with copied 3rd party dependencies psycopg2 in windows to *.zip file
step3>
Upload the file to LAMBDA - Here it goes, It runs successfully without any error.
Windows 10 with conda environment manager (fresh install of Django, wagtail with PostgreSQL), had the same error. Removed psycopg2
conda remove -n myenv psycopg2
it updated some packages, removed others (it also removed django, wagtail...). Then installed psycopg2 back
conda install -n myenv psycopg2
Tested it, import worked
python
>>> import psycopg2
Installed django, wagtail back. python manage.py migrate now populated PostgreSQL.
In my case, it was other site-packages that was exposed by installing pgcli, uninstalling pgcli resolved the issue for the time being.
This somehow penetrated virtualenv too.