I am new to Python, the Anaconda environment, conda, pip, all of it, so please bear with me if these are simple questions. I've asked a couple previous questions about this install which so far have been resolved. Here was my previous question. All of my issues have to do with conda-build meta files which don't work and need some hand-editing in order to succeed.
Background:
I am trying to install the package called ibm-watson in my Python, in a separate conda environment cloned from my base environment. This is in support of a Coursera course. The courseware builds this package in its own Jupyter window with a pip install. I wanted to build the examples in my own environment, and I'm working in Anaconda at the recommendation of many people.
When I first ran into issues with conda-build which I couldn't figure out, I decided to try pip. That worked, but led to other problems (which online articles warned about). Conda (I read) doesn't know about things installed with pip, and that screws up its ability to manage packages and environments. So I decided to back out the pip install and try to make it work with conda.
First question: Why does pip install work correctly and recursively build all dependencies but conda-build does not? Am I just not using the right options for conda?
So here are the meta.yaml issues I've uncovered so far and resolved with the help of people here.
Version string that said '>=2.0,' with an extraneous comma.
package name that was shown with underscores but actually needed hyphens
Dependencies which I fixed by downloading the required packages one by one and building, frequently dealing with the same issues above in the meta.yaml
Most recently, this string, which was throwing an error till I guessed that the quote marks were the issue: typing; # [ py <'3.5' ]
Also the install command from the conda documentation conda install --use-local my-package doesn't work, and per a discussion on Github, I am instead using conda install -c ${CONDA_PREFIX}/conda-bld/my-package
Second question: Why is the conda process so buggy? Are the IBM developers just careless in their testing or is it conda that's at fault or am I using all of this wrong?
And finally, the real question
The last dependency I had to build was python-dotenv installed from PyPi. I built that with conda like the others:
conda skeleton pypi python-dotenv
conda-build python-dotenv (after making the above change to meta.yaml)
That gets all the way through building but then throws this error:
Run pip install "python-dotenv[cli]" to fix this.Tests failed for python-dotenv-0.11.0-py37_0.tar.bz2 - moving package to /Users/(myname)/opt/anaconda3/envs/coursera/conda-bld/broken
Since I'm not using pip, how do I do what it's asking me to do? I tried just doing conda-build "python-dotenv[cli]" but got "no valid recipes for python-dotenv[cli]".
Related
I am trying to create a conda environment that will enable me to use JupyterLab extensions as well as matplotlib (among other things). When installing JupyterLab extensions in the past, I have been prompted that I require nodejs>=12.0. When I try installing both nodejs and matplotlib in a new environment, I receive a conflict. Any suggestions for how to proceed?
I do the following:
conda create -n test python=3.10
conda activate test
conda install jupyterlab
conda install nodejs
conda install matplotlib
Installation proceeds without a problem until the final step, at which point I am informed:
The following NEW packages will be INSTALLED:
blas anaconda/linux-64::blas-1.0-mkl None
...
zstd anaconda/linux-64::zstd-1.5.2-ha4553b6_0 None
The following packages will be DOWNGRADED:
icu 68.1-h2531618_0 --> 58.2-he6710b0_3 None
nodejs 16.13.1-hb931c9a_0 --> 10.13.0-he6710b0_0 None
Proceed ([y]/n)?
Of course, if I allow it to downgrade nodejs, then I won't be able to use Jupyter Lab extensions, which require nodejs>=12.0. Any suggestions for how to proceed?
(Other packages want to downgrade nodejs to 10.3 as well, e.g. conda install jupyter, but for clarity, I've focused on matplotlib.)
I ran into a similar issue, and I got around it by not installing node via conda. I think node just needs to be on your path so you can install with https://nodejs.org/en/download/ or if you need to support multiple node versions you can use nvm.
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
There is a prefix of "100" in front of all my python packages in anaconda navigator. I believe this is making it difficult for applications to use these libraries. For example, I cannot open Spyder.
Packages will be listed as "100 numpy" rather than simply "numpy"
I was in the middle of creating a virtual environment seperate from the root folder but I was doing a few install / uninstall of different packages using both Navigator and Anaconda Prompt. Suddenly just about everything with my python installation started having problems. After closing Spyder I have not been able to reopen it. I tried spyder --reset which did not work. I've tried several complete reinstalls of Anaconda and each time my root installation (as well as any other venvs I create) have a prefix of 100 in front of the package name.
I would expect the packages to read "numpy" rather than "100 numpy" etc.
I have also experienced this issue. For me, this occurred after running conda update --all in the anaconda terminal. I was able to fix this by rolling back my conda version to the version prior to my update that broke it.
To do this, go to anaconda prompt and type in conda list --revisions to find the revision number prior to whichever update broke anaconda. They you just type in conda install --revision #, replacing "#" with the revision number.
The same happened to me, however, it didn't affect Spyder. It occured after I added the conda-forge channel. After updating all packages using conda update --all, all packages were updated from the conda-forge channel.
I fixed this by removing conda-forge from my environment's channels and updating all packages using above command, which actually 'downgrades' a lot of packages that were more recent in the conda-forge channel. You can do this, by using:
conda config --remove channels conda-forge
afterwards, do
conda update --all
If this is your problem and you want to keep conda-forge because the base channel doesn't include the desired package, appending it to your channel list should work, instead of giving it highest priority, by using
conda config --append channels conda-forge
I'm trying to install graphviz on windows 10 (Python 3.6). I went through multiple documentations but nothing seems to work. I tried to run below commands which were explained in the various documentations but it didn't work.
install -c anaconda graphviz
conda install -c anaconda graphviz #I tried these commands in python console
I get below error:
Syntax Error : invalid syntax
Can anyone tell me the exact procedure of installing graphviz on windows?
pip install graphviz didn't work for me. The library could be imported, but when I tried to open real-world examples, it crashed because of missed dependencies. (I actually found this topic while trying to solve those crashes lol).
The following two commands solved all my problems:
conda install graphviz
conda install python-graphviz
I am now able to replicate all the (graphviz) plots presented in this well-known Machine Learning repository:
https://github.com/quark0/darts
PS. Perhaps executing only the second command (alone) is enough (conda install python-graphviz), I am not 100% sure. I only know that conda install graphviz doesn't work alone for Python.
You should first install graphviz. http://www.graphviz.org/download/
After installing it and adding to path installing the python bindings should be straight forward.
pip install graphviz
edit: Updated the link
I am having trouble installing and installing again the same package. I get an 'AssertionError: Multiple .dist-info directories' error. I think that it has something to do with the pip uninstall not completely getting rid of all the files of a package, but am not sure how to remedy the situation. I don't know what information would be helpful here, except maybe that this is a Django project, running on virtualenv on IIS.
EDIT: I just re-installed python and with it, a new version of pip.
Remove the related pip build directory for that particular package and try installing it again. That'd work.
If you're working in a virtual environment say venv, pip build directory will be <path-to-venv>/venv/build/<package-name>.