is there a way to specify for a given python package to install in a given conda env vs. the User's python?
I thought that if I did pip install <package> in a given conda environment, this would make the package accessible in that environment.
If I create a conda environment and install pySankey then do conda list, pySankey won't show but instead be installed in /Users/username/Python/3.7/lib/python/site-packages/
The package installs with "Defaulting to user installation because normal site-packages is not writeable". I looked at other stack posts associated with this, but I'm unclear how to modify ~/.bashrc, since my understanding is that ~/.bashrc is not unique to a conda environment. I also checked the path of the conda environment but there's no local as indicated here.
This has happened to me with a couple different packages (eg these packages do not pip install into the conda environment, other do), I'm using pySankey as an example.
thanks!
Related
I have multiple environments under my conda management, as shown below
ss-MacBook-Pro$ conda env list
# conda environments:
#
base * /miniconda2
testenv /miniconda2/envs/testenv
testenvpy3 /miniconda2/envs/testenvpy3
Can I install a package that becomes effective across multiple environments? By reading the documentation, I got the impression that it is NOT possible, because if I do
conda install package-name
it will only get installed into the base environment (the current active environment), but it does not apply to other environments. I remember I can somehow achieve install a package effective to multiple environments under virtualenv before.
Can someone share the suggestion?
conda install only installs packages for the current (activated) environment. Files will be installed in the directory for the specific environment. If you want a specific package in all environments, you'll have to conda install that package for each of your environments (base, testenv, testenvpy3).
To switch between environments you just need to activate the one you switch to. The syntax depends Anaconda version or your OS. For newer Anaconda versions, conda activate <env name> works, and for older versions, source activate <env name> for Unix systems and activate <env name> for Windows.
I'm sure you've looked at this already, but here's a helpful link.
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
I have python3.7 installed on my windows 10 laptop
But i need python3.6 for a specific project
Can i install it in virtualenv which will override python3.7 in that environment?
I don't know whether this may be an appropriate solution for you. But this is what I generally follow. Just install Anaconda in your system and create an environment according to your needs. For your case create an environment for Python 3.6.
conda create --name py36 python=3.6
//This lines will create an environment named py36
You then install libraries according to your needs in that environment. You work in that environment without interfering with the libraries of the other environment. To use anaconda kindly follow Anaconda cheatsheet. You will get everything that you need.
It is well known that Anaconda installation on macOS can cause trouble with other widely used package/environment managers like Homebrew, Pyenv, Virtualenv, etc.
The majority of the solutions I've found are 'Anaconda-centric', i.e. using Anaconda as the main python manager and setup conda env for homebrew etc.
However, I am looking for a solution that's kind of 'Homebrew-centric', and setup Anaconda as a compliment. Anaconda should be set up in a way that when ever conda is used, it will work with its own Python, own packages. And leave the rest of system untouched.
The motivation for such solution is because that, for example, when one's main work-flow use homebrew Python3 (python3), homebrew pip (pip3) and Pyenv (pyenv) with requirement.txt. And occasionally using Anaconda when a project is required.
Rather than using Anaconda I would suggest using Miniconda, which includes only Python and conda (and a few support packages). Miniconda does not include all of the packages in Anaconda by default, but they can all be installed (with conda install anaconda). Once you download Miniconda, you can install it into your home folder at /Users/username/miniconda3. During the installation, you will be asked if you want to add some initialization code to your .bash_profile. Either choose yes or (if you chose no), then you can run
/Users/username/miniconda3/bin/conda init
to add the conda initialization to your .bash_profile. By default, this will activate the base environment, so you can change the default setting so the environment is not activated by default:
conda config --set auto_activate_base false
You'll probably need to open a new terminal so the conda command is available. Then, when you want to use a conda environment, you can conda activate that environment, but otherwise, conda's Python should not be on your PATH.
So I am a python newb like first project ever newb. I jumped right in disregarding virtualenv and installed everything globally. Now I need to be able to share my project with other team members.
Can I create a virtualenv after making the mistake of installing all project packages globally?
I am using python 3. I've read these links:
pip installing in global site-packages instead of virtualenv
How to import a globally installed package to virtualenv folder
But I don't think thats what Im looking for. I want to go the requirements.txt route I think.
Any guidance or suggestions?
Yes, you can create a virtual env.
You can create a requirements.txt file for the packages you installed globally.
pip3 freeze > requirements.txt
and then you can use this requirements file for installing all the packages in the virtual env which will be isolated from your global environment.
First you need to install virtualenv:
pip3 install virtualenv
Create a new virtual env using the following command:
virtualenv -p python3 envname
You can activate the virtualenv by:
source /path/to/new/virtual/environment/bin/activate
To deactivate the environment and return to your local environment just run:
deactivate
Install the requirements from the file.
cat requirements.txt | xargs -n 1 pip3 install
This should install all your packages in the virtual environment.
To check which python you are using use which python command and to check installed packages use pip3 list
I hope this will clear your doubt.