I am trying to create a conda environment that will enable me to use JupyterLab extensions as well as matplotlib (among other things). When installing JupyterLab extensions in the past, I have been prompted that I require nodejs>=12.0. When I try installing both nodejs and matplotlib in a new environment, I receive a conflict. Any suggestions for how to proceed?
I do the following:
conda create -n test python=3.10
conda activate test
conda install jupyterlab
conda install nodejs
conda install matplotlib
Installation proceeds without a problem until the final step, at which point I am informed:
The following NEW packages will be INSTALLED:
blas anaconda/linux-64::blas-1.0-mkl None
...
zstd anaconda/linux-64::zstd-1.5.2-ha4553b6_0 None
The following packages will be DOWNGRADED:
icu 68.1-h2531618_0 --> 58.2-he6710b0_3 None
nodejs 16.13.1-hb931c9a_0 --> 10.13.0-he6710b0_0 None
Proceed ([y]/n)?
Of course, if I allow it to downgrade nodejs, then I won't be able to use Jupyter Lab extensions, which require nodejs>=12.0. Any suggestions for how to proceed?
(Other packages want to downgrade nodejs to 10.3 as well, e.g. conda install jupyter, but for clarity, I've focused on matplotlib.)
I ran into a similar issue, and I got around it by not installing node via conda. I think node just needs to be on your path so you can install with https://nodejs.org/en/download/ or if you need to support multiple node versions you can use nvm.
Related
In my environments created with anaconda, the same packages installed with conda are not compatible when I try to install with pip.
Is there a difference how pip and conda handle dependencies?
Here an example of requirements.txt
# Python version 3.9.13
django==2.2.5
djangorestframework==3.14.0
gensim==4.1.2
joblib==1.1.1
nltk==3.7
numpy==1.21.5
openpyxl==3.0.9
pandas==1.4.4
pickleshare==0.7.5
scikit-learn==1.1.3
seaborn==0.12.0
spacy==3.3.1
tensorflow==2.9.1
unidecode==1.2.0
conda allows you to create the environment, pip reports incompatibility between django and djangorestframework.
Conda checks, if all packages that will end up in the environment are compatible with each other and tries to find the optimal solution - considering all package versions.
Pip is less strict and only checks, if the new package is compatible with the existing ones. It doesn't change versions of the previously installed packeages.
Pip installs packages from from pypi.org, while conda installs from anaconda.org. The packages are not exactly the same, since the Anaconda staff authors new packages and tries to increase their compatibility with the older ones.
However, sometimes you are not interested in 100% compatibility but just want to use the latest features. Then pip is good enough because your unit tests will tell you if something goes wrong.
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
I tried conda remove nodejs and pip uninstall nodejs but it doesn't work. I get this error message PackagesNotFoundError: The following packages are missing from the target environment: - nodejs
but when I use conda search nodejs. It turns out the package still exists.
The conda search command does not give information about local installations, but instead queries Anaconda Cloud to see what packages are available to install.
To check what is installed, use the conda list command. If you wish to check every environment for a given package, see Conda: list all environments that use a certain package.
discription of the problem can be seen in the pictureI am using miniconda3 and it shows that no modules have been installed when I am accessing the module from Jupiter notebook.
I have installed the modules from miniconda prompt by pip install pandas,sklearn etc" the modules have been individually installed and it shows success full installation also, but when accessing it in jupyter notebook it shows error.
Did you install pip using conda install pip? You need to ensure that pip installs packages somewhere where miniconda looks for them. If you're installing common packages to be used inside conda, I'd recommend using the conda package manager to install them.
I am new to Python, the Anaconda environment, conda, pip, all of it, so please bear with me if these are simple questions. I've asked a couple previous questions about this install which so far have been resolved. Here was my previous question. All of my issues have to do with conda-build meta files which don't work and need some hand-editing in order to succeed.
Background:
I am trying to install the package called ibm-watson in my Python, in a separate conda environment cloned from my base environment. This is in support of a Coursera course. The courseware builds this package in its own Jupyter window with a pip install. I wanted to build the examples in my own environment, and I'm working in Anaconda at the recommendation of many people.
When I first ran into issues with conda-build which I couldn't figure out, I decided to try pip. That worked, but led to other problems (which online articles warned about). Conda (I read) doesn't know about things installed with pip, and that screws up its ability to manage packages and environments. So I decided to back out the pip install and try to make it work with conda.
First question: Why does pip install work correctly and recursively build all dependencies but conda-build does not? Am I just not using the right options for conda?
So here are the meta.yaml issues I've uncovered so far and resolved with the help of people here.
Version string that said '>=2.0,' with an extraneous comma.
package name that was shown with underscores but actually needed hyphens
Dependencies which I fixed by downloading the required packages one by one and building, frequently dealing with the same issues above in the meta.yaml
Most recently, this string, which was throwing an error till I guessed that the quote marks were the issue: typing; # [ py <'3.5' ]
Also the install command from the conda documentation conda install --use-local my-package doesn't work, and per a discussion on Github, I am instead using conda install -c ${CONDA_PREFIX}/conda-bld/my-package
Second question: Why is the conda process so buggy? Are the IBM developers just careless in their testing or is it conda that's at fault or am I using all of this wrong?
And finally, the real question
The last dependency I had to build was python-dotenv installed from PyPi. I built that with conda like the others:
conda skeleton pypi python-dotenv
conda-build python-dotenv (after making the above change to meta.yaml)
That gets all the way through building but then throws this error:
Run pip install "python-dotenv[cli]" to fix this.Tests failed for python-dotenv-0.11.0-py37_0.tar.bz2 - moving package to /Users/(myname)/opt/anaconda3/envs/coursera/conda-bld/broken
Since I'm not using pip, how do I do what it's asking me to do? I tried just doing conda-build "python-dotenv[cli]" but got "no valid recipes for python-dotenv[cli]".