I recently have been trying to use Cmake to build ANTs (https://github.com/ANTsX/ANTs).
I am installing it on CentOS07, and am attempting this install in an Anaconda environment, as I do not have root permissions to install the proper dependencies in my bash.
There is a good tutorial I am following (https://www.greydongilmore.com/post/wsl_docs/ants/), but seem to run into some issues.
Does anybody have advice about the correct environment and dependencies I could use to install ANTs into my conda environment?
*FYI I made this question just so I could answer it...took me about 2 weeks to solve and I don't want someone to go through what I did.
Okay...so this install is a bit tricky...but it totally can be done with the right dependencies.
First you want to set up your Anaconda environment. I did all the installs on the conda forge channel. First Install you want to do is Cmake.
conda install -c conda-forge cmake
I was using version 3.22.1, but I think any recent version should be fine.
You can check if you are pointing to your correct cmake by typing which cmake. This should print the path to cmake in your environment bin. You can also check the version with cmake --version.
Next, you need to install the GCC compiler.
conda install -c conda-forge gcc
This should install in your environment bin as well. I was using GCC V 11.2.0, so try that one if you have issues.
Now this is where Cmake is going to yell at you...although you have GCC 11.2.0 installed it will say ERROR: GCC 5.1. or later is required. This is because YOU ALSO NEED TO INSTALL THE G++ COMPILER. DO THIS WITH THE COMMAND BELOW.
conda install -c conda-forge gxx
If that doesn't install, try...
conda install -c anaconda gxx_linux-64
Now you should be able to get through most of you make. This is where I got stuck...I kept getting an error at [100%]. Cmake errors are shit so it says Error 1, Error 2 and all this garbage...but also should provide the error undefined reference to `memcpy#GLIBC_2.14'. This is because your GCC/G++ compilers are not compatible with the current GLIB. I found the answer to this here (https://github.com/rapidsai/cuml/issues/3620)...
But basically you need to install sysroot into your conda environment with your command below.
conda install -c conda-forge sysroot_linux-64=2.17
For me this all worked, and I was able to complete the make with this environment setup.
You may need to play around with different versions of gcc,g++, and sysroot.
Feel free to PM with any questions if anyone has an issue...this took me a long time to figure out so I wanted to try and save someone a headache!!
Related
I am trying to import librosa, but I am thrown with this error:
/home/lakshya/anaconda3/envs/tff_env/lib/python3.9/site-packages/zmq/backend/cython/../../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/lakshya/anaconda3/envs/tff_env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)
I tried the following to fix it based on the other similar questions that I browsed through:
sudo apt-get install libstdc++6
It's output: libstdc++6 is already the newest version (10.2.1-6).
sudo apt-get dist-upgrade
It's output: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX
It's output: GLIBCXX version up to GLIBCXX_3.4.28
conda install libgcc in my virtual env "tff_env"
It's output: libgcc-7.2.0 installed in tff_env
Pip installed the libgcc package in the virtual environment as well. Didn't work.
What can I do?
My OS: Debian GNU/Linux 11 (bullseye)
Just been tackling a similar problem, it looks like you need to ensure you have the latest version of gcc. Running:
conda install -c conda-forge gcc=12.1.0
Fixed the error for me.
After my search on anaconda packages, I found only the package "libstdcxx" will substantially bring new "libstdc++.so.6" file to the conda environment (while "gcc", "gcc_linux-64", and "libgcc-ng" cannot).
Thus, installing the following package can fix this problem for me.
conda install -c conda-forge libstdcxx-ng=12
PS: dfcorbin's answer conda install -c conda-forge gcc=12 not works for me.
If the accepted answer didn't work, try this
I had the same conditions as the original poster and the accepted answer didn't work as conda was taking forever. I tried down-versioning scipy from 1.9.3 to 1.9.1 and it did work.
You can use the following command to do so:
conda install -c anaconda scipy==1.9.1
One solution that fixed this problem for while trying to run mujoco or mujoco-py was as follows
OSError: /home/ubuntu/anaconda3/envs/tensorflow_p36/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.20' not foundwhen starting ipython. For some reason this library isnt in the
anaconda environments libstdc++.so.6. It is in the base ubuntu
library. So link the anaconda version of this library back to the os
version:
cd /home/ubuntu/anaconda3/envs/tensorflow_p36/lib
mv libstdc++.so.6 libstdc++.so.6.old
ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 libstdc++.so.6
credits: https://bcourses.berkeley.edu/courses/1478831/pages/glibcxx-missing
So what worked for me was to manually remove Python3.10 which I had installed using make altinstall and upgrade scipy to the latest version.
None of the other answers worked for me. This did work though:
pip install scikit-build
None of the above answers worked for me, unfortunately. What I ended up doing was to manually inspect the libstdc++.so.6 file.
First, I checked the system file, and it returned nothing on my side:
strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX_3.4.30
Then, I took a look the conda file, and it was there:
strings /path-to-your-conda/envs/your-env-name/lib/libstdc++.so.6 | grep GLIBCXX_3.4.30
This indicates that the simple solution is to append the path to the conda file to the LD_LIBRARY_PATH.
export LD_LIBRARY_PATH=/path-to-your-conda/envs/your-env-name/lib:$LD_LIBRARY_PATH
In some cases, there exist multiple installations of glibcxx and runtime your app wants to connect to it but finds the library in the wrong place. It can be solved by changing the imported libraries' order. The problem has been solved by importing librosa on top of the running script.
The soln given above doesn't work for me. I got it working by downgrading my scipy from 1.9.1 to 1.6.1.
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
I am new to Python, the Anaconda environment, conda, pip, all of it, so please bear with me if these are simple questions. I've asked a couple previous questions about this install which so far have been resolved. Here was my previous question. All of my issues have to do with conda-build meta files which don't work and need some hand-editing in order to succeed.
Background:
I am trying to install the package called ibm-watson in my Python, in a separate conda environment cloned from my base environment. This is in support of a Coursera course. The courseware builds this package in its own Jupyter window with a pip install. I wanted to build the examples in my own environment, and I'm working in Anaconda at the recommendation of many people.
When I first ran into issues with conda-build which I couldn't figure out, I decided to try pip. That worked, but led to other problems (which online articles warned about). Conda (I read) doesn't know about things installed with pip, and that screws up its ability to manage packages and environments. So I decided to back out the pip install and try to make it work with conda.
First question: Why does pip install work correctly and recursively build all dependencies but conda-build does not? Am I just not using the right options for conda?
So here are the meta.yaml issues I've uncovered so far and resolved with the help of people here.
Version string that said '>=2.0,' with an extraneous comma.
package name that was shown with underscores but actually needed hyphens
Dependencies which I fixed by downloading the required packages one by one and building, frequently dealing with the same issues above in the meta.yaml
Most recently, this string, which was throwing an error till I guessed that the quote marks were the issue: typing; # [ py <'3.5' ]
Also the install command from the conda documentation conda install --use-local my-package doesn't work, and per a discussion on Github, I am instead using conda install -c ${CONDA_PREFIX}/conda-bld/my-package
Second question: Why is the conda process so buggy? Are the IBM developers just careless in their testing or is it conda that's at fault or am I using all of this wrong?
And finally, the real question
The last dependency I had to build was python-dotenv installed from PyPi. I built that with conda like the others:
conda skeleton pypi python-dotenv
conda-build python-dotenv (after making the above change to meta.yaml)
That gets all the way through building but then throws this error:
Run pip install "python-dotenv[cli]" to fix this.Tests failed for python-dotenv-0.11.0-py37_0.tar.bz2 - moving package to /Users/(myname)/opt/anaconda3/envs/coursera/conda-bld/broken
Since I'm not using pip, how do I do what it's asking me to do? I tried just doing conda-build "python-dotenv[cli]" but got "no valid recipes for python-dotenv[cli]".
I want to use Assimulo and Sundials for the solution of differential algebraic equations in Python and therefore I am trying to install it on Ubuntu.
For the installation of Sundials, I followed the installation instructions and as I understand it worked well.
% cmake -DCMAKE_INSTALL_PREFIX=/usr/local/lib/sundials-3.1.1/ ~/opt/sundials/sundials-3.1.1
% make
% make install
Then I tried to install Assimulo with the command pip3 install Assimulo, but I get an error message. I also tried to follow the instructions on Installation - Assimulo 3.0 documentation by downloading the installation files and install it with the following command. It results in the same error message.
sudo python3 setup.py install --sundials-home=/usr/local/lib/sundials-3.1.1
This is the error message I get:
target build/src.linux-x86_64-3.6/assimulo/thirdparty/hairer/dopri5module.c does not exist:
Assuming dopri5module.c was generated with "build_src --inplace" command.
error: 'assimulo/thirdparty/hairer/dopri5module.c' missing
What is wrong and how can I fix it? Any help would be appreciated!
I got the same error when installing on macos via pip install assimulo, after pip-installing numpy and cython.
For me, using a conda env did the trick:
Creating the conda env: conda create -n your_name_goes_here
conda activate your_name_goes_here
conda install python=3.6 (I noticed you can also use 3.7)
conda install -c conda-forge assimulo
I also had the same error message. As suggested in the other answer, you can get a compiled package from Conda. But if you want to compile from source yourself, it looks to me that PyPI source tarball doesn't contain all needed files. At least some *.pyf files are missing. So, I used SVN repo instead:
svn checkout https://svn.jmodelica.org/assimulo/tags/Assimulo-3.0/ assimulo
By compiling this source tree, I managed to get pass the original error you had, but I'm now having another build error that I don't know yet how to solve:
ssimulo/solvers/sundials.c: In function '__pyx_f_8assimulo_7solvers_8sundials_5CVode_initialize_cvode':
assimulo/solvers/sundials.c:33274:31: error: too many arguments to function 'CVodeCreate'
__pyx_v_self->cvode_mem = CVodeCreate(__pyx_t_3, __pyx_t_4);
I am a little bit confused....
I installed anaconda on my computer (I have windows 10).
Normally, when I want to install a package I simply do "pip install package_name" or "conda install package_name" and it is done.
First question: what is the difference between pip and conda?
Now I tried to install xgboost and it was really complicated I tried lot of things nothings worked until I install something called miniconda.
There it works but now, when I do "conda install package_name" it install it in miniconda3/lib/site _package and I have to copy/paste it in Anaconda3/lib/site_package if I want it to work.
Second question: how can I ask to the computer that "conda install
package_name" install it directly in anaconda3 and not miniconda3?
Finally I tried to install the package "surprise" for recommended systems. Both "pip install" or "conda install" failed.
I went in github and got the file "surprise" from https://github.com/NicolasHug/Surprise
I tried to copy it in Anaconda3/lib/site_package but it doesn't work.
When I do from surprise import Reader I did not get the error "no module name surprise" anymore but I get "cannot import name 'Reader'"
Last question: how can I make it work? I think I have to build it but
I do not now how...
Thank you in advance for anyone that can explain all this for me :-)
Similarly to you, I had issues installing the surprise package.
I tried both pip install surprise and conda install surprise unsuccessfully.
conda install -c conda-forge scikit-surprise
conda install -c conda-forge/label/gcc7 scikit-surprise
conda install -c conda-forge/label/cf201901 scikit-surprise
I found those on the anconda website and the first one worked for me.
Hopefully this would help you as well
pip vs conda
pip is a package manager that facilitates installation, upgrade,
and uninstallation of python packages. It also works with virtual python environments.
conda is a package manager for any software (installation, upgrade and uninstallation).
It also works with virtual system environments.
Conda is a packaging tool and installr that aims to do more than what pip does;
handle library dependencies outside of the Python packages as well as the Python packages themselves.
Conda also creates a virtual environment, like virtualenv does.
For more see here
Anaconda vs miniconda
The open source version of Anaconda is an easy-to-install
high performance Python and R distribution with a package manager,
environment manager and collection of 720+ open source packages.
It also comes with the options to install RStudio.
The "lite" version of Anaconda without the collection of 720 packages.
The downside is that you need to type in command line commands,
"conda install PACKAGENAME"
And Last
To install this package with conda run:
conda install -c anaconda py-xgboost=0.60
Update for surprise
The easiest way is to use pip (you'll need numpy):
$ pip install numpy
$ pip install scikit-surprise
Or you can clone the repo and build the source (you'll need Cython and numpy):
$ git clone https://github.com/NicolasHug/surprise.git
$ python setup.py install