I am originally a Fortran programmer, if anything. Last year I begun learning python and now I wanted to use fortranmagic to have a wrapper of f2py, but it is not working. Let me describe the steps I took and the resulting error messages, hoping for some help. I am working with Ubuntu 22.04
I begun by creating an environment conda create -n lf and activating the environment
I installed I few libraries I might need
conda install numpy scipy matplotlib jupyter nodejs
Following the instructions of the Fortran-magic developer GitHub I run pip install -U fortran-magic
Everything installs fine, but when I create a notebook and execute
%load_ext fortranmagic
I get the following error ModuleNotFoundError: No module named 'fortranmagic'
4. I finally try to install the extensions from jupyter, as it is also suggested in the developer GitHub
%install_ext https://raw.github.com/mgaitan/fortran_magic/master/fortranmagic.py
Yet only to receive the following error UsageError: Line magic function %install_ext not found.
I hope you can help
I have created an virtual env gcloudenv on my nividia nano running ubuntu. I was able to successfully install flask and required libraries and able to deploy my appengine into GCP from this virtual env. All my work is in python and I was using nano as my editor to get my code up and running. No issues so far.
my virtual env gcloudenv already has all the required packages for flask, jinga etc and I can see them when I run pip freeze.
Then I tried to work on Jupyter notebook as my code was getting little complicated and I didn't want to write full code and then run.
I already had jupyter notebook installed before creating the virtual env. I also installed jupyter within in virtual env as well.
So I followed the instruction to create a new kernel by running the following command:-
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$ pip install ipykernel
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$ ipython kernel install --user --
name=gcloudenv
Now, I ran the notebook as:
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$
/home/gcloudenv/bin/jupyter notebook
When trying to import the flask I get the following error:
ModuleNotFoundError: No module named 'flask'
Note sure what is going on as I getting blanked out.
Add
!pip install flask
in the beginning of your Jupyter notebook.
Finally I managed to solve my problem. Thanks to a wonderful post https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/.
In essence, I had 2 problems:-
1. I did not have a jupyter-notebook within my virtual env. Originally I thought I had it installed but that was incorrect. so whenever I tried to launch one, it was picking the first jupyter notebook it could find in the path.
The good way to find out which one is it pointing to , is to run which command
(gcloudenv) sunny#my-nano:~/gcloudenv$ which jupyter-notebook
For me, that was at:
/home/sunny/archiconda3/bin/jupyter-notebook
I had in fact 3 copies of jupyter-notebook on my system. One was probably installed using sudo pip and therefore went into the root folder. Probably not a good thing to do.
So I installed a fresh jupyter-notebook with the following command:-
(gcloudenv) $ pip install jupyter notebook
2.Next is to check list of Jupyter kernels available by running the following from the jupyter notebook ( or from command line):
!jupyter kernelspec list (OR (gcloudenv) $jupyter kernelspec list
My jupyter notebook was not able to import flask libraries because it was pointing to a wrong kernel config outside of my virtualenv gcloudenv.
Available kernels:
gcloudenv /home/sunny/.local/share/jupyter/kernels/gcloudenv ( correct one)
python3 /home/sunny/gcloudenv/share/jupyter/kernels/python3
You can determine which python version it is picking by doing a 'more' on the file:-
(gcloudenv) $
/more/home/sunny/.local/share/jupyter/kernels/gcloudenv/kernel.json
Once I changed my kernel to point to python3 from within the notebook, it picked the correct path and all the relevant libraries I needed.
In summary when you hit the problem as mentioned above, do the following:-
check the path of the python ( whereis python or which python)
check if you are running the 'right' notebook. This is determined by the path and if you have sourced your virtualenv.
Install jupyter notebook using pip from within your virtualenv( do not use sudo)
Check the Jupyter kernel. This may be particularly relevant if you have a common jupyter notebook and you want to work with multiple virtualenv.
After installing miniconda with python 3.7 I do the following:
conda create --name opencv_test python=3.7
conda activate opencv_test
conda install -c anaconda opencv
Then to test the environement I do:
python
>>> import cv2
>>> exit()
Everything goes according to plan
Then if I launch PyCharm or vs code, set my new environment as the interpreter and create a simple file that contains only
import cv2
On launch I get:
ImportError: DLL load failed: The specified module could not be found.
What I learned from the internets:
This, this and this, all those posts refer to a problem that seems similar, the difference being that the issue for me only happens when using IDEs
What I tried before posting here:
Recreating the environment multiple times switching channels.
As it seemed to be a path issue, I added the paths which contained opencv dll to the environment paths in PyCharm settings one at a time
What I just tried after spending a considerable amount of time creating this post:
conda update conda
This weirdly downgrades python to 3.6
conda create --name opencv_test_2 python=3.6
conda activate opencv_test_2
conda install -c anaconda opencv
This seems to fix all the issues but:
I don't know what just happened and why the issue only appeared on IDEs
I'd like to use python 3.7
So in short, why when using python 3.7 and PyCharm I seem to be unable to import opencv even if it works on command line?
Thanks in advance.
I've gotten myself into some kind of horrible virtualenv mess. Help?!
I manage environments with conda. Until recently, I only had a python2 jupyter notebook kernel, but I decided to drag myself kicking and screaming into the 21st century and installed a python3 kernel; I forget how I did it.
My main (anaconda) python defaults to 2.7.
So here I am, merrily trying to use beautiful soup from inside my shiny new python3 kernel, and I don't seem to be able to do anything to get at whatever environment it's finding packages in. Viz (all from notebook):
from bs4 import BeautifulSoup
-> ImportError: No module named 'bs4'
Ok, fine, I'll install it using shell magic. Right? Right?
! pip install bs4
--> Collecting bs4
Downloading bs4-0.0.1.tar.gz
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /Users/[MY-USER]/anaconda/lib/python2.7/site-packages (from bs4)
[...]
Successfully built bs4
Installing collected packages: bs4
Successfully installed bs4-0.0.1
from bs4 import BeautifulSoup
-> ImportError: No module named 'bs4'
Oh no. Does it think I'm in a 2.7 env even though I'm running a python3 kernel? That won't do.
! conda info --envs
--> # conda environments:
#
flaskenv /Users/[MY-USER]/anaconda/envs/flaskenv
mesa /Users/[MY-USER]/anaconda/envs/mesa
py35 /Users/[MY-USER]/anaconda/envs/py35
root * /Users/[MY-USER]/anaconda
Ok, I can fix that. One of those is a 3.5 env.
! source activate py35
--> prepending /Users/[MY-USER]/anaconda/envs/py35/bin to PATH
! conda install beautifulsoup4
--> Fetching package metadata .......
Solving package specifications: ..........
# All requested packages already installed.
# packages in environment at /Users/[MY-USER]/anaconda:
#
beautifulsoup4 4.4.1 py27_0
concerning...
! pip install bs4
--> Requirement already satisfied (use --upgrade to upgrade): bs4 in /Users/[MY-USER]/anaconda/lib/python2.7/site-packages
more concerning...
from bs4 import BeautifulSoup
-> ImportError: No module named 'bs4'
ARRGH!!! headdesk Am I going to have to kill the kernel in order to fix this (and re-run a bit of work)? Is killing the kernel even going to work? How do I get my jupyter kernel to know what environment it's supposed to be running under?
thanks!
This is a tricky part of ipython / Jupyter. The set of kernels available are independent of what your virtualenv is when you start jupyter Notebook. The trick is setting up the the ipykernel package in the environment you want to identify itself uniquely to jupyter. From docs on multiple ipykernels,
source activate ENVNAME
pip install ipykernel
python -m ipykernel install --user --name ENVNAME --display-name "Python (whatever you want to call it)"
If you only want to have a single Python 3 kernel, from the conda environment, just use python -m ipykernel install --user and it will reset the default python to the one in the virtualenv.
And yes, you will need to restart the kernel and re-run the prior steps.
See Also Using both Python 2.x and Python 3.x in IPython Notebook
#tschundler's solution works perfectly if your environment has already been created.
If you want to change the default kernel at the creation of your virtual environment and avoid any manual configuration, you just need to add jupyter at the end of the conda command:
conda create --name ENVNAME python=PYTHONVERSION jupyter
The correct kernel will then be used when you use ipython or jupyter notebook.
In my case somehow jupyter wasn't able to 'pick' the virtual environment's python. So I had to edit ~/.local/share/jupyter/kernels/{my_env_name}/kernel.json
and add path to the interpreter
in the argv key.
There is also an easy way here
workon my-virtualenv-name # activate your virtualenv, if you haven't already
pip install tornado==4.5.3
pip install ipykernel==4.8.2
You should now be able to see your kernel in the IPython notebook menu: Kernel -> Change kernel and be able to switch to it (you may need to refresh the page before it appears in the list). IPython will remember which kernel to use for that notebook from then on.
This worked for me. source
pip install --user ipykernel
python -m ipykernel install --user --name=myenv
Output
Installed kernelspec myenv in /home/user/.local/share/jupyter/kernels/myenv
and go to above directory
open kernel.json
{
"argv": [
"/home/user/anaconda3/envs/myenv/bin/python", # path to your virtual environment python
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "myenv",
"language": "python"
}
I know this is an old question, but nobody has mentioned nb_conda_kernels, which was made to solve this problem. You must have it installed in the environment from which you launched the running notebook, and any other environment which has kernel/s installed (ipykernel, r-irkernel, etc ) will be available as an option (Start Preferred Kernel):
Sharing my experience here, in case someone made the same mistake:
TL;DR
In VSCode I created the virtual environment from an Ubuntu terminal but forgot to activate the WSL extension too. Once I activated the WSL extension, I was able to change the Jupyterkernel to my virtual environment.
Full explanation
Using VSCode I work with the WSL extension (see here). I created my virtual environment from an Ubuntu terminal [Created a new terminal and selected 'Ubuntu (WSL)'] and then went to my Jupyter notebook to change my kernel, but it did not appear in the list.
I tried the solution proposed above of adding the path to kernel.json in my virtual environment, adapting to my case (I don't use conda) but I was surprised to see that the folder bin in my environment did not have a file named python.
The solution was to activate the WSL extension: lower right corner > click on the arrows (Open a remote connection) > reopen folder in WSL.
After this, I could see in the explorer two new entries in the bin folder for Python and Python3 (the arrow on the right side seem to indicate it's a symbolic link).
Virtual environment now shows the python symbolic links
Then I went to my Jupyter notebook and changed the kernel to my virtual environment.
In the end, I did not have to change the path in kernel.json.
P.S: The above is my understanding of my problem and the solution given my non-existing knowledge about Linux, so any comments to complement/improve this answer are very welcome!
I'm using Anaconda to manage both Python and Jupyter. That is:
>> which python
>> /home/.../software/anaconda3/bin/python
and
>> which jupyter
>> /home/.../software/anaconda3/bin/jupyter
But Jupyter's python kernel seems to be pointing to a system version of Python rather than my local version through Anaconda, since the sys.path is different in a Jupyter Python 3 notebook. Also, jupyter kernelspec list gives the following:
Available kernels:
ir /usr/local/share/jupyter/kernels/ir
matlab /usr/local/share/jupyter/kernels/matlab
python3 /usr/local/share/jupyter/kernels/python3
This doesn't seem altogether surprising since the docs say in section 1.5.5:
By default, kernel specs will go in a system-wide location (e.g. /usr/local/share/jupyter/kernels). If doing a --user install, the kernel specs will go in the JUPYTER_DATA_DIR location.
For personal sanity and organization, I want the version of Python that I use in the command line to be the same that is accessed in Jupyter. As a result, I think that what I should do is change my jupyter kernelspec list for python3 so that it points to my desired Anaconda python version, i.e. /home/.../software/anaconda3/bin/python. My questions are: 1) is that indeed the best solution for my stated preferences, and 2) how do I actually change my jupyter kernelspec entry for python3? Not sure if this will come up, but I don't want to be using virtual environments--I want the default to be same version of Python across both the command line and Jupyter.
I ended up reposting this to the Jupyter Github issues page, and was recommended to delete /usr/local/share/jupyter/kernels/python3. This allows Jupyter to find a default Python kernel using the same Python running Jupyter itself (i.e. Anaconda), and this worked for me.
You can find my post on Jupyter's Github page as well as an explanation for why the above solution works here.
1) Jupyter kernels in /usr/local/ are indeed a global install. But I do not see why it couldn't be linked to your anaconda python3 interpreter.
2) To explicitly link your anaconda interpreter to your jupyter install you can run :
pip install ipykernel
python -m ipykernel install --prefix=/usr/local/ --name "anaconda_kernel"
for a global install, or change /usr/local/ if you want a per user install. A doc is specially set for anaconda here
If you combine it with jupyter kernelspec remove python3 beforehand, you can then reset your anaconda kernel as default to be sure.