Error: module not found in Jupyter Notebook (sklearn) - python-3.x
I have an issue importing scikit-learn in my Jupyter notebooks, and I am at a loss as to why this is not working. I do not recall having this issue before with other packages. I have seen several threads with people having similar problems with this specific module, but none of the proposed checks have solved my problem, nor revealed to me what might be wrong. I have tried to add as much info as possible below, to try and figure out why this refuses to work.
I use anaconda to manage my packages, and scikit-learn was installed as follows:
source activate python3
conda install scikit-learn
Fetching package metadata .............
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /Users/overaa/anaconda/envs/python3:
#
scikit-learn 0.19.1 py36hffbff8c_0
Now, if I start a python session directly from the command line, import sklearn works as intended and the module is imported correctly. Doing the same in Jupyter results in
ModuleNotFoundError: No module named 'sklearn'
I have checked a number of things. First, making sure that my jupyter and jupyter-notebook point to the correct environment
which jupyter
/Users/user_name/anaconda/envs/python3/bin/jupyter
which jupyter-notebook
/Users/user_name/anaconda/envs/python3/bin/jupyter-notebook
So the notebook checks out.
Then from a command-line python session (in the same environment as where I launch my notebook):
>>> import sklearn
>>> sklearn.__file__
'/Users/user_name/anaconda/envs/python3/lib/python3.6/site_packages/sklearn/__init__.py'
So scikit-learn is also installed in the right environment.
Now comparing that I am running the same version;
From command line:
>>> print(sys.version)
3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:04:09)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
From Jupyter notebook:
print(sys.version)
3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:04:09
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
Comparing the packages path.
From command line:
import site
print(site.getsitepackages())
['/Users/user_name/anaconda/envs/python3/lib/python3.6/site-packages']
From Jupyter notebook:
import site
print(site.getsitepackages())
['/Users/user_name/anaconda/envs/python3/lib/python3.6/site-packages']
Similarly for my sys.path
command line:
>>> print(sys.path)
['', '/Users/user_name/anaconda/envs/python3/lib/python36.zip',
'/Users/user_name/anaconda/envs/python3/lib/python3.6',
'/Users/user_name/anaconda/envs/python3/lib/python3.6/lib-dynload',
'/Users/user_name/anaconda/envs/python3/lib/python3.6/site-packages',
'/Users/user_name/anaconda/envs/python3/lib/python3.6/site_packages/setuptools-27.2.0-py3.6.egg']
Jupyter:
print(sys.path)
['', '/Users/user_name/anaconda/envs/python36/lib/python36.zip',
'/Users/user_name/anaconda/envs/python36/lib/python3.6',
'/Users/user_name/anaconda/envs/python36/lib/python3.6/lib-dynload',
'/Users/user_name/anaconda/envs/python36/lib/python3.6/site-packages',
'/Users/user_name/anaconda/envs/python36/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg',
'/Users/user_name/anaconda/envs/python36/lib/python3.6/site-packages/IPython/extensions',
'/Users/user_name/.ipython']
So everything is the same, except that the directory containing the setuptools differ between the command-line version and the Jupyter version. I also noticed that scikit-learn seems to be installed in the site_packages subdir, rather than in the site-packages subdirectory, which is where Jupyter appear to be looking for modules. So I tried two things. First, adding the path to my JUPYTER_PATH in my .bash_profile before running my notebook.
export JUPYTER_PATH="/Users/user_name/anaconda/envs/python36/lib/python3.6/site-packages:$JUPYTER_PATH"
I also tried to add it to my sys.path directly in the jupyter notebook, prior to trying to import scikit-learn
sys.path.append("/Users/user_name/anaconda/envs/python3/lib/python3.6/site_packages")
Both to no avail, sklearn still refuses to import in my notebook.
Any ideas as to why scikit-learn is not getting imported into my Jupyter Notebook would be highly appreciated. Are there more things I can check? And how do I get this to work?
I have tried activating/deactivating my environment, restart the kernels etc. also without success.
EDIT: I have no problems importing other packages, like numpy or scipy
Turns out that removing the ~/Library/Jupyter/kernels folder and restarting my notebook did the trick. Even though it appears that the kernel in jupyter kernelspec list was what I expected. Nevertheless it is now working. It appears that it is not a sklearn exclusive issue. I basically followed what
this person wrote.
Make sure you have both numpy and scipy installed - scikit-learn depends on them but does not install them for you.
Also, after installing them, restart your notebook server.
There is a simple solution to this, which is to use the Anaconda-Navigator GUI to manually install scikit-learn. So first of all choose an environment you intend to work in or create a new environment other than the base (root) level. Next, start Anaconda-Navigator either by starting the application or by typing "anaconda-navigator" at the command prompt given that the proper environment has been activated. Next, go to the "Environments" window on the GUI by clicking on its icon on the left-hand panel. Then, choose the that higher-level environment you have decided to use earlier. On the right-hand panel, you should now see a list of all installed "conda" software in this environment. After that, choose instead "Not installed" in the field to the immediate left of the "Channels" button at the top of the right-hand panel. Select scikit-learn from the list, and check the box to the left of its name. Finally, click the "Apply" button that shows up at the bottom right corner to install the software. Now you should be all set.
Screenshot of Anaconda-Navigator GUI
Related
Cannot import torch Error loading ..\caffe2_nvrtc.dll" or one of its dependencies
I am on a Windows 10 64 bit system. Pytorch for cuda has been working successfully for some time. Today I tried to upgrade to the latest version of Pytorch (1.13) using conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia Now I cannot import torch. I get the error: OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\alan\anaconda3\lib\site-packages\torch\lib\caffe2_nvrtc.dll" or one of its dependencies. I have tried both in a Jupyter notebook and in the Spyder IDE. I have tried completely removing Anaconda and reinstalling afresh and then reinstalling Pytorch with no success. I do not believe I have any other versions of python installed. The offending dll (caffe2_nvrtc.dll) does seem to be in the file location specified. I have found various similar problems reported but they all date back to 2020 or earlier and none of them seemed to have a satisfactory solution. Can anyone point me in the correct direction
I still do not understand why using conda did not work but I tried again using pip and that did work
I experienced the same problem as yours today. It turns out that when I use the anaconda prompt then the problem disappears. Anaconda prompt could do it Then I speculated that the only difference between these two scenarios is that when I use the anaconda prompt, I use the base anaconda environment, probably somehow the conda environment in spyder is not activated. So the solution is to open the spyder in anaconda prompt like this: Then it works. This picture will tell you the cause for the problem. problem...
PyCharm 2017.1, No module named 'kivy'
I could use (troubleshooting)help with getting Kivy imported in PyCharm. I am using: Anaconda with Python 3 on 64-bit Windows 10 Pro PyCharm 2017.1 Packages: NumPy, SciPy, BeautifulSoup, Pandas, Scrapy, Pattern, NetworkX, NLTK, scikit-learn, Selenium I have installed Kivy following the instructions at https://kivy.org/docs/installation/installation-windows.html#installation As evident from the screen shot, no errors occurred. Still, when I run "import kivy" from PyCharm I get "ModuleNotFoundError: No module named 'kivy'". I ran through the installation with someone on the #kivy channel and was assured that kivy is successfully installed. When I pass import kivy; print(kivy.file) to the interpreter (opened from the CMD command line), I get returned among others: C:\Users\Steve\Anaconda3\lib\site-packages\kivy\__init__.py Someone on #kivy suggested I check where PyCharm looks for kivy. How do I figure that out? I'd appreciate any suggestions to identify the problem / resolve my issue.
Indeed, the installation of Kivy went fine. The problem turned out to be that I had to select the correct (updated) interpreter in the PyCharm settings.
PyCharm Import Error: Claims 'matplotlib' is not a package, but works successfully in IDLE
Happy October everyone, I've successfully downloaded modules before using either the pycharm installer or pip through the command screen, but for some reason when installing matplotlib pycharm cannot recognize it. I've uninstalled and reinstalled, I've installed through both methods, I've followed past similar questions asked on this site which make sure that you have the same interpreter and that it was installed in the right folder (pycharm error while importing, even though it works in the terminal). So, here's the whole problem. Here's is the simple code, submitted into both pycharm and IDLE: import matplotlib.pyplot as plt plt.plot([1,2,3],[2,1,3]) plt.show() When submitted into IDLE, my plot appears. When submitted into pycharm, the following error appears: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:/PythonProject/matplotlib.py", line 1, in <module> import matplotlib.pyplot as plt File "C:\PythonProject\matplotlib.py", line 1, in <module> import matplotlib.pyplot as plt ImportError: No module named 'matplotlib.pyplot'; 'matplotlib' is not a package I am currently running Python 3.4, PyCharm 2016.2.3, and my matplotlib folders are indeed in my site-packages folder inside my Python34 folder. Also for further verification: PyCharm installation Please help I've become frustrated since this is the only module I've run into trouble with. I've scoured StackOverflow and related websites to help, I've made sure I have all the requirements, etc.
I guess if you named your current writing python module as matplotlib.py.That cause the python load your current writing module instead of the actual matplotlib.py, which triggers an error.
I recommend you to use virtualenv. Is not strictly necessary but is good for dividing your project environments. This is how I tested matplotlib on my Windows 10 installation, hope it helps. Be sure that you have the python 3 installation folder listed in your Windows PATH environment variable, should be already listed if you checked "Add Python 3.5 to PATH": You need also to set the Scripts folder in your PATH environment variable usually should be this path: C:\Users\<your username>\AppData\Local\Programs\Python\Python35\Scripts If you don't do that you have to prepend python -m to every command below like this: python -m <command>, so the command below would be python -m pip install virtualenv. I prefer the first solution. To test matplotlib on Pycharm I've used virtualenv, here is how; first install virtualenv: pip install virtualenv Then you create your virtual environment in a folder of your choice, in my case I used python_3_env_00: virtualenv python_3_env_00 After that you can activate you python 3 virtual environment: python_3_env_00/Scripts/activate.bat Now you should see in your command line the active virtual environment (python_3_venv_00), like this: Now you can install matplotlib: pip install matplotlib Fire up PyCharm and add your virtual environment as you project interpreter, go to File->Settings search for Project Interpreter click on the gear icon and Add Local and set the path of your virtual environment, should look like something like this: Test it:
import sys print(sys.path) run this code in where the import worked, and run it in the Pycharm project. Compare the lists. Find out which path that is not present in Pycharm sys.path. Before importing pyplot, append the missing path to sys.path. import sys sys.path.append("the path") import matplotlib.pyplot as plt Does this work?
Please follow below steps if you are still getting an error: If you are using PyCharm, it automatically create virtualenv. Ensure Scripts path is set into PATH C:\Users\<Username>\AppData\Local\Programs\Python\Python37-32 Then open PyCharm and go to File-> settings. Search for Project Interpreter. You will see window like this sample image Click on setting icon -> Existing Environment -> click on ... give below path C:\Users\Krunal\AppData\Local\Programs\Python\Python37-32\python.exe Click on Apply -> ok and you are good to go.
After installing matplotlib When I was trying to use matplotlib.pyplot it was giving error module not found. I browsed some white papers and found out that we also need to install scipy library to use the matplotlib so I used the below in my command prompt python -mpip install scipy Restarted my kernel session. It worked!!!
I was also facing issue while importing matplotlib but it got resolved and now I am able to use it from pycharm as well. Please make sure you should have visual c++ 14 installed in your system. 2.If you have more than two python version installed on your system then please install matplotlib from both the version. Eg. pip install matplotlib pip3 install matplotlib If matplotlib is working from python idle then please check whether you are using correct interpreter in pycharm or not and try to choose pythonw.exe path from your installed location. Hope this will help, Please do let me know if you are still facing issue.
I had similar issue but I solved it very easily on pycharm 2019.3.2. In case anyone looking for an easier solution: I just opened the terminal window on pycharm and typed pip install matplotlib and it was all good to go. Every project has its own virtual environment. Opening terminal window of IDE cds to project directory by default. So the installing command was enough.
Using TensorFlow through Jupyter (Python 3)
Apologies in advance, I think the issue is quite perplexing! I would like to use TensorFlow through Jupyter, with a Python3 kernel. However the command import tensorflow as tf returns the error: ImportError: No module named tensorflow when either Python2 or Python3 is specified as the Jupyter kernel. I have Python 2 and Python 3 installed on my Mac and can access both versions through Terminal. I installed TensorFlow for Python 3, however I can only access it via Python 2 on the Terminal. As such, this question is really two-fold: I want to get TensorFlow working with Python3 ...which should lead to TensorFlow working with Jupyter on the Python3 terminal.
I had the same problem and solved it using the tutorial Using a virtualenv in an IPython notebook. I'll walk you through the steps I took. I am using Anaconda, and I installed a new environment tensorflow using these instructions at tensorflow.org. After that, here is how I got tensorflow to work in a Jupyter notebook: Open Terminal Run source activate tensorflow. You should now see (tensorflow) at the beginning of the prompt. Now that we are in the tensorflow environment, we want to install ipython and jupyter in this environment: Run conda install ipython and conda install jupyter Now follow the instructions in the tutorial linked above. I'll repeat them here with a bit more information added. First run ipython kernelspec install-self --user The result for me was Installed kernelspec python3 in /Users/charliebrummitt/Library/Jupyter/kernels/python3 Run the following: mkdir -p ~/.ipython/kernels Then run the following with <kernel_name> replaced by a name of your choice (I chose tfkernel) and replace the first path (i.e., ~/.local/share/jupyter/kernels/pythonX) by the path generated in step 4: mv ~/.local/share/jupyter/kernels/pythonX ~/.ipython/kernels/<kernel_name> Now you'll see a new kernel if you open a Jupyter notebook and select Kernel -> Change kernel from the menu. But the new kernel will have the same name as your previous kernel (for me it was called Python 3). To give your new kernel a unique name, run in Terminal cd ~/.ipython/kernels/tfkernel/ and then run vim kernel.json to edit the file kernel.json so that you replace the value of "display_name" from the default (Python 3) to a new name (I chose to call it "tfkernel"). Save and exit vim by typing :wq while in command mode. Open a new Jupyter notebook and type import tensorflow as tf. If you didn't get ImportError then you are ready to go!
Anaconda3 libhdf5.so.9: cannot open shared object file [works fine on py2.7 but not on py3.4]
I just tried to use pd.HDFStore in IPython Notebook with a Python 3 kernel (Anaconda 2&3 on Ubuntu 14.04) import pandas as pd store = pd.HDFStore('/home/Jian/Downloads/test.h5') but it throws the following error ImportError: HDFStore requires PyTables, "libhdf5.so.9: cannot open shared object file: No such file or directory" problem importing I initially thought it's because pytables is somehow missing, but when I check $source activate py34 and $conda list, pytables 3.2.0 is already installed under anaconda python3 environment. Also, if I switch to Python 2, for example, $source activate py27 and start ipython notebook, it works properly and no import error is thrown. I guess that I must miss something for configuring pytables under anaconda python 3 env, but I cannot figure it out. Any help is highly appreciated. Update: I just tried on a fresh install of Anaconda3-2.3.0-Linux-x86_64 from official website and it ends up with the same error. When I try $locate libhdf5.so.9 in command line, nothing shows up.
This is a known issue that we are working on. When it is fixed, conda update --all will update the libraries and fix the issue.