I'm trying to set up an R kernel to work in Jupyter Notebook and Jupyter Lab.
I have miniconda3 installed and when I activate base environment,
then type
jupyter-kernelspec list I see
python3 C:/path/to/miniconda3/share/jupyter/kernels/python3
I want the R kernel so I can use it in Jupyter Lab and Jupyter Notebook.
I already have Rstudio installed. Is there a difference if I install the IRkernel to the kernels directory above via CRAN in Rstudio or fork it from github (assuming I can find it) and then clone it to the kernels directory?
Is this what I need to do or is possible that all I need to do is alter some variable in my environment PATH?
If download/install it via CRAN in Rstudio is that kernel going to be available in my (base) environment?
If R is installed outside Conda (more common), then install through CRAN.
If R is installed in a Conda env (less common), then follow the nb_conda_kernels instructions.
Related
EDIT:
In addition to the behaviour outlined below, the Python3.10 based environment seems to be ignoring packages installed with the pip -e option (development mode).
Specifically, a package installed in development mode is not listed in pip freeze (which, it does in the Python3.9 based virtual environment) and a simple import <package_name> fails. If the package is installed normally (i.e. not in development mode) everything works as expected.
I am using virtualenv to create a virtual environment based around Python 3.10. While virtualenv finishes without errors and does seem to activate, it still fails to pick up its own Python unless the PYTHONPATH environment variable has been set manually.
I am not sure if the situation I am faced with is due to Ubuntu's way of incorporating Python or the way virtualenv is setting up the environment so that it picks up a local interpreter. Here is what I have gathered this far:
My base system is an Ubuntu 21.04. It has its own Python 3 (Python3.9.5) installation which I have not touched at all except installing the python3-virtualenv package using apt.
With Python 3.10, I installed the python3.10-dev package and proceeded to create a virtual environment in the usual way:
> virtualenv -p python3.10 the_env/
> source the_env/bin/activate
Although this looks OK so far, this environment does not have any information about its own site-packages directory. Not even the one that virtualenv is supposed to be creating which includes pip. In this installation, if you try to > pip --version you simply get an error that
the pip package does not exist (the pip "executable" location is picked up correctly, but because the interpreter does not know anything about its site-packages it fails to start pip properly).
Long story short, I created two environments, one based on Python3.9 (which works perfectly) and one based on Python3.10 (which does not work) and did a very simple test in each environment:
> python -m site
On the Python3.9 environment, sys.path includes a path that leads all the way to this particular environment's site-packages
On the Python3.10 environment, sys.path does not include that particular path but still includes the typical paths you expect to find (e.g. those pointing to the interpreter itself and the top environment directory but not the specific path that points to the site-packages location.
Following this, I defined a PYTHONPATH manually, before activating the environment which points exactly to the site-packages for that particular environment and everything worked as expected.
I suspect that this might be something to do with the fact that my system's Python is 3.9 which means that the USER_SITE variable is valid while in the case of Python3.10, it is not (because, I do not have a use for it, this is just a virtual environment I am creating). So, I suspect that this might be throwing off the way the site module determines where things are.
As I am not sure, I would like to ask the following:
Could this be something to do with the way Ubuntu handles the Python installation that might just be creating this small problem with virtual environments?
Could the problem be with virtualenv that does not explicitly specify a PYTHONPATH?
Could this behaviour be something of a corner case of the site module?
What worked for me is an installation from source. After unpacking the source code, as a summary:
$ ./configure --enable-optimizations --with-ensurepip=install --prefix=/path/to/install/to/
$ make -j
$ make test
$ make install
$ /path/to/install/to/bin/python3.10 -m venv /path/to/test
$ source /path/to/test/bin/activate
$ pip list
Package Version
---------- -------
pip 21.2.4
setuptools 58.1.0
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/path/to/test/bin/python3.10 -m pip install --upgrade pip' command.
$ python -m pip install --upgrade pip
[...]
$ pip list
Package Version
---------- -------
pip 21.3.1
setuptools 58.1.0
Edit
As mentioned below, I believe the reason why the procedure above just works is simply that when building and installing Python from source, the installation simply will be correct, which I suspect the 3.10 installation in Ubuntu 21.04 is not.
I made my Python installation from source somewhere under my home directory, in order to not risk messing things up, and I also did not permanently modify the PATH.
Setting the path to $PATH:/path/to/install/to/bin should be fine however, either permanently, or just when running mkvirtualenv.
Doing that, my new installation even seems to integrate seamlessly with my system's virtualenvwrapper.
I don't really need to be on the Python leading edge myself, but if I did, I would definitely make myself independent of Ubuntu's latest development by using the procedure described above.
By the way, if I update pip directly in the Python installation built from source, I will no longer get the message about the older pip (see above) when I install new virtual environments.
Edit 2
Bug reported to Ubuntu:
https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1955742
Edit 3
Also, since Ubuntu 21.04 has end of life in a month or so, upgrading to 21.10 could really make sense. I have just tried that, and it seems that Python 3.10 virtual environments work just fine in that release.
With the advent of Ubuntu 22.04 (which caused a few minor issues with some specific python virtualenv setups I had) and having already spent some time figureing this out, I ended up switching to using pyenv.
Pyenv made it very easy to install any python version and any number of virtual environments within it which can be further customised in complete isolation.
The only thing to be careful of is installing all necessary python prerequisites before installing a specific version to avoid missing functionality from some packages (e.g. not including the lzma library will generate an ominous warning from pandas ("your python installation is incomplete..."). This can be ignored if you are not using that functionality or otherwise, easily fixed.
I think that this is a better option overall if you have to manage different versions and specific configurations for python, even across distros.
This solution is very close to the one suggested before ("install from source") so I will be accepting that one and leave my contribution as additional information about this problem.
I am using a CentOs Distribution of Linux and have miniconda already installed along with conda environments that I created with it. I am trying to install anaconda now. Do I have to uninstall miniconda first? I have some environments that I created with miniconda that I would like to keep. Is it possible to keep those environments if I uninstall it? The reason I need anaconda is I'd like to create environments inside my singularity container. How would it be possible to keep both? Thanks.
Both Miniconda and Anaconda include Conda, so no need to install both. Given either, a new environment with all the Anaconda distribution packages can be created using
conda create --name my_anaconda_env anaconda
Note that Python versions can be specified as part of this, e.g.,
conda create --name my_anaconda37_env anaconda python=3.7
It is well known that Anaconda installation on macOS can cause trouble with other widely used package/environment managers like Homebrew, Pyenv, Virtualenv, etc.
The majority of the solutions I've found are 'Anaconda-centric', i.e. using Anaconda as the main python manager and setup conda env for homebrew etc.
However, I am looking for a solution that's kind of 'Homebrew-centric', and setup Anaconda as a compliment. Anaconda should be set up in a way that when ever conda is used, it will work with its own Python, own packages. And leave the rest of system untouched.
The motivation for such solution is because that, for example, when one's main work-flow use homebrew Python3 (python3), homebrew pip (pip3) and Pyenv (pyenv) with requirement.txt. And occasionally using Anaconda when a project is required.
Rather than using Anaconda I would suggest using Miniconda, which includes only Python and conda (and a few support packages). Miniconda does not include all of the packages in Anaconda by default, but they can all be installed (with conda install anaconda). Once you download Miniconda, you can install it into your home folder at /Users/username/miniconda3. During the installation, you will be asked if you want to add some initialization code to your .bash_profile. Either choose yes or (if you chose no), then you can run
/Users/username/miniconda3/bin/conda init
to add the conda initialization to your .bash_profile. By default, this will activate the base environment, so you can change the default setting so the environment is not activated by default:
conda config --set auto_activate_base false
You'll probably need to open a new terminal so the conda command is available. Then, when you want to use a conda environment, you can conda activate that environment, but otherwise, conda's Python should not be on your PATH.
I debated which Stackoverflow site this best fit but couldn't decide.
I'd like to contribute to an open-source project on Github, but I can't figure out how to prevent the stable version already installed on my machine and the development version I'd like to make a patch for from clashing on import.
The repository only suggests pip installing with the editable.
What I've done so far is: clone the repository locally, and then trying to import it in a Jupyter Notebook from the directory above. However, the Jupyter Notebook is referencing the stable version installed earlier with pip. I tried to append to sys.path the child directory holding the package but still the same issue. I can't seem to get relative imports working either. Do I need to uninstall the stable version?
Any tips are appreciated!
You'd use virtualenv for this. It will allow you to create an environment that is isolated from your system python and you can install the dev version of the library on it.
Basic usage (for Unix-like systems) is:
$ pip install virtualenv
$ virtualenv MY_ENV
$ cd MY_ENV
$ source bin/activate # activates the local python for this shell only
(MY_ENV)$ pip install <some-module> # installs to a local and isolated python
(MY_ENV)$ python ... # runs python in the local environment
(MY_ENV)$ deactivate # disable the isolated python
$
There is a lot of information available to run Jupyter Notebook with Anaconda but could not find any info to run Jupyter without Anaconda.
Any pointer would be much appreciated!
Basically the process is as follows:
pip3 install --upgrade pip
pip3 install jupyter
jupyter notebook # run notebook
Run a specific notebook:
jupyter notebook notebook.ipynb
Using custom IP or port:
jupyter notebook --port 9999
No browser:
jupyter notebook --no-browser
Help:
jupyter notebook --help
Answer from the following sources:
SOURCE 1
SOURCE 2
See Gordon Ball's Jupyter PPA, the most actively maintained Jupyter PPA as of this writing with support for both Ubuntu 16.04 and 18.04 (LTS).
Installation: It's a Ball
Thanks to Gordon's intrepid efforts, installation of Jupyter under Ubuntu trivially reduces to:
sudo add-apt-repository ppa:chronitis/jupyter
sudo apt-get update
sudo apt-get install jupyter
Doing so installs the jupyter metapackage, providing:
A standard collection of Jupyter packages published by this PPA, including:
Jupyter's core and client libraries.
Jupyter's console interface.
Jupyter's web-based notebook.
Tools for working with and converting notebook (ipynb) files.
The Python3 computational kernel.
The /usr/bin/jupyter executable.
As W. Dodge's pip-based solution details, the browser-based Jupyter Notebook UI may then be launched from a terminal as follows – where '/home/my_username/my_notebooks' should be replaced with the absolute path of the top-level directory containing all of your Jupyter notebook files:
jupyter notebook --notebook-dir='/home/my_username/my_notebooks'
Why Not Acanaconda or pip?
For Debian-based Linux distributions, the optimal solution is a Debian-based personal package archive (PPA). All other answers propose Debian-agnostic solutions (e.g., Anaconda, pip), which circumvent the system-wide APT package manager and hence are non-ideal.
Installing Jupyter via this or another PPA guarantees automatic updates to both Jupyter and its constellation of dependencies. Installing Jupyter via either Anaconda or pip requires manual updates to both on an ongoing basis – a constant thorn in the side that you can probably do without.
In short, PPA >>>> Anaconda >> pip.
There are two ways to install Jupyter-Notebook in Ubuntu. One is using Anaconda, the other using pip. Please go through the below added link for details.
http://jupyter.readthedocs.io/en/latest/install.html