I have three questions.
One, can I install multiple versions of Python on my machine. I have a 4 Gb RAM system.
Two, can I install multiple versions of Anaconda?
Three, what is the difference between jupyter notebook & jupyter lab?
Please help. I am a new user.
can I install multiple versions of Python on my machine?
Yes, and the conda package manager (which comes with the Anaconda distribution) will help with that. You can create a separate conda environment for each Python version you want to use by running:
conda create --name mypy36env python=3.6
conda activate mypy36env
For details on how to create and manage conda environments, take a look at the docs
can I install multiple versions of Anaconda?
You can but because of the answer above you don't need to and shouldn't. Instead of multiple Anaconda versions, just create multiple environments with the versions of packages you need.
You can create an environment with specific versions of Python and the Anaconda distribution with:
conda create -n anaconda201903 python=3.6 anaconda==2019.03
what is the difference between jupyter notebook & jupyter lab?
From the JupyterLab docs:
JupyterLab is the next-generation web-based user interface for Project Jupyter.
Basically JupyterLab is a new interface that allows you to create and run the same Jupyter notebooks you did in the past, but it also has much more functionality than the old interface.
For a bit more details, check this blog post.
Related
I am using a CentOs Distribution of Linux and have miniconda already installed along with conda environments that I created with it. I am trying to install anaconda now. Do I have to uninstall miniconda first? I have some environments that I created with miniconda that I would like to keep. Is it possible to keep those environments if I uninstall it? The reason I need anaconda is I'd like to create environments inside my singularity container. How would it be possible to keep both? Thanks.
Both Miniconda and Anaconda include Conda, so no need to install both. Given either, a new environment with all the Anaconda distribution packages can be created using
conda create --name my_anaconda_env anaconda
Note that Python versions can be specified as part of this, e.g.,
conda create --name my_anaconda37_env anaconda python=3.7
I have python3.7 installed on my windows 10 laptop
But i need python3.6 for a specific project
Can i install it in virtualenv which will override python3.7 in that environment?
I don't know whether this may be an appropriate solution for you. But this is what I generally follow. Just install Anaconda in your system and create an environment according to your needs. For your case create an environment for Python 3.6.
conda create --name py36 python=3.6
//This lines will create an environment named py36
You then install libraries according to your needs in that environment. You work in that environment without interfering with the libraries of the other environment. To use anaconda kindly follow Anaconda cheatsheet. You will get everything that you need.
I learned that there are some issues in the latest Anaconda 3 python 3.7 version for object detection and face recognition deep learning problems from various posts. The official Anaconda site is only providing the latest python 3.7 version here. I want to work on a similar deep learning project in Windows 10(64 bit) which requires Anaconda 3 with python 3.6 version.
I found several posts providing solutions on StackOverflow for a Linux environment but I could not find any solution for the Windows 10 operating system. Can anyone share with me how to download the Anaconda 3 python 3.6 version?.
Any help will be highly appreciated.
Recommendation: Learn how to use conda by reading get started with conda. The problem you encounter is a very common case could be solved by conda, as an environment manager.
Solution 1
Use an environment with Python 3.6 installed, and activate this environment each time to work with your project.
# create an environment with python 3.6
conda create -n py36 python=3.6
# activate this environment
conda activate py36
This is also the preferred way to work with different projects. One environment for one project.
Solution 2
The default environment you're using with conda is base. You can override the Python 3.7 within base with Python 3.6
conda install -n base python=3.6
We have Anaconda installed on my cloudera cluster via parcels. We have python 2.7.13 available with the version of Anaconda. We wanted to have another version of python (3.6) across all nodes.
My challenge here is, when I followed the Conda documentation to create a new environment and install python 3.6 on that using "conda create -n py36 python=3.6 anaconda". For few nodes, I am getting python3.6.6 installed for few nodes, and python3.6.7 for few nodes and 3.6.1 for few.
I would like to know if there is a way to choose the version of python while installing 3.6 on a separate environment. Or am I doing something wrong? Please help me.
Thanks
Kancharlapalli
You can specify the patch as well
conda create -n py36 python=3.6.7 anaconda
I create a dataproc cluster using the following command
gcloud dataproc clusters create datascience \
--initialization-actions \
gs://dataproc-initialization-actions/jupyter/jupyter.sh \
However when I submit my PySpark Job I got the following error
Exception: Python in worker has different version 3.4 than that in driver 3.7, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.
Any Thoughts?
This is due to a difference in the python versions between the master and the worker. By default, the jupyter image installs the latest version of miniconda, which uses the python3.7. However, the worker is still using the default python3.6.
Solution:
- specify the miniconda version when creating the master node i.e to install python3.6 in the master node
gcloud dataproc clusters create example-cluster --metadata=MINICONDA_VERSION=4.3.30
Note:
may need updating to have a more sustainable solution to managing the environment
UPDATE THE SPARK ENVIRONMENT TO USE PYTHON 3.7:
Open a new terminal and type the following command: export PYSPARK_PYTHON=python3.7 This will ensure that the worker nodes use Python 3.7 (same as the Driver) and not the default Python 3.4
DEPENDING ON VERSIONS OF PYTHON YOU HAVE, YOU MAY HAVE TO DO SOME INSTALL/UPDATE ANACONDA:
(To install see: https://www.digitalocean.com/community/tutorials/how-to-install-anaconda-on-ubuntu-18-04-quickstart)
Make sure you have anaconda 4.1.0 or higher. Open a new terminal and check your conda version by typing into a new terminal:
conda --version
checking conda version
if you are below anaconda 4.1.0, type conda update conda
Next we check to see if we have the library nb_conda_kernels by typing
conda list
Checking if we have nb_conda_kernels
If you don’t see nb_conda_kernels type
conda install nb_conda_kernels
Installing nb_conda_kernels
If you are using Python 2 and want a separate Python 3 environment please type the following
conda create -n py36 python=3.6 ipykernel
py35 is the name of the environment. You could literally name it anything you want.
Alternatively, If you are using Python 3 and want a separate Python 2 environment, you could type the following.
conda create -n py27 python=2.7 ipykernel
py27 is the name of the environment. It uses python 2.7.
Ensure the versions of python are installed successfully and close the terminal. Open a new terminal and type pyspark. You should see the new environments appearing.
We fixed it now -- thanks for the intermediate workaround #brotich. Check out the discussion in #300.
PR #306 keeps python at the same version as was already installed (3.6), and installs packages on all nodes to ensure that the master and worker python environments stay identical.
As a side effect, you can choose your python version by passing an argument to the conda init action to change the python version. E.g. --metadata 'CONDA_PACKAGES="python==3.5"'.
PR #311 pins miniconda to a particular version (currently 4.5.4), so we avoid issues like this again. You can use --metadata 'MINICONDA_VERSION=latest' to use the old behavior of always downloading the latest miniconda.