I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
Related
I have Spyder, clean Win10 installation, ver. 4.2.3.
When I run command pip, it gives message:
Note: you may need to restart the kernel to use updated packages.
C:\Users\UserName\AppData\Local\Programs\Spyder\Python\python.exe: No module named pip
I restarted Spyder, Win10, nothing helps.
How to fix it?
Thanks.
(Spyder maintainer here) We don't provide pip in our Windows app to prevent people breaking Spyder by installing any kind of packages with it.
The best way to use other packages that don't come with our app is to install Miniconda, create a conda environment after that with the packages you want to use and spyder-kernels, and finally connect Spyder to that env.
If you have python 3.4+ then pip should be installed with python.
To check that, you can run
pip help
in your windows command line.
However, if it isn't the case then download the get-pip.py file and run
python get-pip.py
and the pip installation should start. You can use pip help to verify this.
Another source of the problem might be that pip isn't listed in your PATH variables and thus isn't recognized by your command line.
I have just started using python and I have set up python3 to be installed in C:\Python37 dir. I have added python3 path to environment variables. When I run python3 -m pip install [package_name] it installs it in C:\Python37\Lib\site-packages. But when I try to install pylint with python3 -m pip install pylint it prints:
Requirement already satisfied: pylint in c:\users\radio\appdata\roaming\python\python37\site-packages (2.3.1)
It's installing it in above mentioned completely different location, and then VS Code complains how pylint is not installed. Why doesn't pip install it in C:\Python37\Lib\site-packages where it installs all the other packages?
It seems you might have multiple python installations on the computer or you are not running CMD with administrator priviledges when using pip.
I would refer you to this thread: windows pip installing libraries in wrong directory which seems to deal with a problem similar to the one you are having.
I would also try checking the environment variables to see the path set for Python if that doesn't work.
It is well known that Anaconda installation on macOS can cause trouble with other widely used package/environment managers like Homebrew, Pyenv, Virtualenv, etc.
The majority of the solutions I've found are 'Anaconda-centric', i.e. using Anaconda as the main python manager and setup conda env for homebrew etc.
However, I am looking for a solution that's kind of 'Homebrew-centric', and setup Anaconda as a compliment. Anaconda should be set up in a way that when ever conda is used, it will work with its own Python, own packages. And leave the rest of system untouched.
The motivation for such solution is because that, for example, when one's main work-flow use homebrew Python3 (python3), homebrew pip (pip3) and Pyenv (pyenv) with requirement.txt. And occasionally using Anaconda when a project is required.
Rather than using Anaconda I would suggest using Miniconda, which includes only Python and conda (and a few support packages). Miniconda does not include all of the packages in Anaconda by default, but they can all be installed (with conda install anaconda). Once you download Miniconda, you can install it into your home folder at /Users/username/miniconda3. During the installation, you will be asked if you want to add some initialization code to your .bash_profile. Either choose yes or (if you chose no), then you can run
/Users/username/miniconda3/bin/conda init
to add the conda initialization to your .bash_profile. By default, this will activate the base environment, so you can change the default setting so the environment is not activated by default:
conda config --set auto_activate_base false
You'll probably need to open a new terminal so the conda command is available. Then, when you want to use a conda environment, you can conda activate that environment, but otherwise, conda's Python should not be on your PATH.
I am a little bit confused....
I installed anaconda on my computer (I have windows 10).
Normally, when I want to install a package I simply do "pip install package_name" or "conda install package_name" and it is done.
First question: what is the difference between pip and conda?
Now I tried to install xgboost and it was really complicated I tried lot of things nothings worked until I install something called miniconda.
There it works but now, when I do "conda install package_name" it install it in miniconda3/lib/site _package and I have to copy/paste it in Anaconda3/lib/site_package if I want it to work.
Second question: how can I ask to the computer that "conda install
package_name" install it directly in anaconda3 and not miniconda3?
Finally I tried to install the package "surprise" for recommended systems. Both "pip install" or "conda install" failed.
I went in github and got the file "surprise" from https://github.com/NicolasHug/Surprise
I tried to copy it in Anaconda3/lib/site_package but it doesn't work.
When I do from surprise import Reader I did not get the error "no module name surprise" anymore but I get "cannot import name 'Reader'"
Last question: how can I make it work? I think I have to build it but
I do not now how...
Thank you in advance for anyone that can explain all this for me :-)
Similarly to you, I had issues installing the surprise package.
I tried both pip install surprise and conda install surprise unsuccessfully.
conda install -c conda-forge scikit-surprise
conda install -c conda-forge/label/gcc7 scikit-surprise
conda install -c conda-forge/label/cf201901 scikit-surprise
I found those on the anconda website and the first one worked for me.
Hopefully this would help you as well
pip vs conda
pip is a package manager that facilitates installation, upgrade,
and uninstallation of python packages. It also works with virtual python environments.
conda is a package manager for any software (installation, upgrade and uninstallation).
It also works with virtual system environments.
Conda is a packaging tool and installr that aims to do more than what pip does;
handle library dependencies outside of the Python packages as well as the Python packages themselves.
Conda also creates a virtual environment, like virtualenv does.
For more see here
Anaconda vs miniconda
The open source version of Anaconda is an easy-to-install
high performance Python and R distribution with a package manager,
environment manager and collection of 720+ open source packages.
It also comes with the options to install RStudio.
The "lite" version of Anaconda without the collection of 720 packages.
The downside is that you need to type in command line commands,
"conda install PACKAGENAME"
And Last
To install this package with conda run:
conda install -c anaconda py-xgboost=0.60
Update for surprise
The easiest way is to use pip (you'll need numpy):
$ pip install numpy
$ pip install scikit-surprise
Or you can clone the repo and build the source (you'll need Cython and numpy):
$ git clone https://github.com/NicolasHug/surprise.git
$ python setup.py install
I am trying to install TensorFlow. The installation instruction for Windows (https://www.tensorflow.org/install/install_windows) have as first step to install Python 3.5.2. And I'm doing the 'TensorFlow with CPU support only'.
Python was successfully installed in my computer as I can run it via the Start menu.
However, when I try to do the 2nd step of the installation instructions in order to install TensorFlow, this step says to:
To install TensorFlow, start a terminal. Then issue the appropriate pip3 install command in that terminal. To install the CPU-only version of TensorFlow, enter the following command:
C:\> pip3 install --upgrade tensorflow
But I'm getting an error when I perform the above statement, the error is
'pip' is not recognized as an internal or external command, oprable program or batch file.
I looked at several postings in StackOverflow and tried the commands provided in one of the postings, but I would get the same type of error.
So, how is 'pip3' installed? from what I read, it is supposed to be installed together with the installation, but obviously that did not happen.
How do I install it? I need to install TensorFlow and it seems that it needs to be done via the pip3 installation tool.
Thank you in advance for your help!
Either set the system environment path variable to include the python 3.5.x path in it, or just cd into the correct python folder to run pip3 from there.
The folder in windows 10 should be something like this:
C:\Users\YOUR_USERNAME\AppData\Local\Programs\Python\Python35\Scripts
Open the terminal, cd to that path (change YOUR_USERNAME to the correct user) and then just run the following command:
pip3 install --upgrade tensorflow
and if you want the gpu version:
pip3 install --upgrade tensorflow-gpu
Pip3 is already installed when you install Python, so there is no need to do anything else.