Building own package for conda gcc and binutils issue - linux

This post summarize my painful but finally successful (just by chance) way to build own conda package for the
netgen meshing tool with Python interface. I found the recipe for the netgen build due to tpaviot.
After cloning the repository into 'netgen-conda' folder I ran:
conda build netgen-conda/netgen-6.2-dev
Which reports "Unsatisfiable dependencies": 'oce', 'gcc-5', 'binutils'.
So I tried to install these packages myself. Unfortunately the documentation do not emphasize the important fact that 'conda build' use its own temporary environment so it doesn't matter what you have installed (see). Nevertheless even installing 'gcc-5' together with 'binutils' manually turns out to be nearly impossible.
Hint for other newbies: Lot of my problems disappear after I learned details about channels.
First try was installing 'gcc-5' with 'binutils' from the 'salford_systems' channel suggested by anaconda:
conda install -c salford_systems binutils gcc-5
But it results in:
ERROR conda.core.link:_execute_actions(337): An error occurred while installing package 'salford_systems::gcc-5-5.3.0-0'.
LinkError: post-link script failed for package salford_systems::gcc-5-5.3.0-0
running your command again with-vwill provide additional information
location of failed script: /home/jb/miniconda3/envs/test/bin/.gcc-5-post-link.sh
Using verbose output ('-v') provides no more info. I was also confused by the fact that the script does not exist on the given path (probably automatically deleted).
With current experience I admit that the reason of problem can be dug out from the '-vv' output (reported issue). After some trying I found that only way to
install both is to first install 'gcc-5' into a clean environment and then install 'binutils'. Since 'conda build' installs everything
from scratch and there is no way to specify order of installed packages I was stuck.
Another issue that puzzled me is the 'conda build' long prefix hack. For unknown reason they use extremely long prefix for an auxiliary folder
which result in various kind of issues. I have faced to three such problems:
As is usual today, I have encrypted HOME causing a known issue.
Using a workaround '--croot /tmp' prevents creating the hard links from '/tmp' into 'HOME/miniconda3' since they are on different filesystems.
There is a fallback to use the copy. I even thought that the fallback doesn't work for a while, but it worked, just making the build running longer.
Trying to install 'gcc' (4.x) from 'default' channel complained about too short prefix. So ultimate workaroud was to set the length of the prefix manually
'--prefix-length 70'.
Finally, I found that the dependency on 'binutils' is not necessary and successfully build the package with:
conda build --prefix-length 70 -c salford_systems -c conda-forge -c dlr-sc netgen-conda/netgen-6.2-dev
Summary (of open questions):
Conda channels introduce a new kind of dependency hell already forgotten when using 'apt-get'. Is there a way to figure out what is a canonical channel for a package.
Does anyone succeed to build with combination 'gcc-5' and 'binutils'?
There is still lack of documentation about internal conda mechanisms and error messages do not provide clue to the problem.
Conda-build use a problematic prefix hack and lack ability to control order of installed packages. Does anybody know the reason for this hack?

Related

GSL not installing in Windows 10 using GIT Bash

When I run the config file for installing GSL library for Windows 10 I get the following error:
error: Something went wrong bootstrapping makefile fragments for
automatic dependency tracking. If GNU make was not used, consider
re-running the configure script with MAKE="gmake" (or whatever is
necessary). You can also try re-running configure with the
'--disable-dependency-tracking' option to at least be able to build
the package (albeit without support for automatic dependency
tracking).
If I run ./config MAKE="gmake" I still get the error. I have searched in StackOverflow and on the web and still haven't found a solution.

Idris/cabal install issues from removePathRecursive on Windows

Installing Idris on Windows 10 using instructions to:
Install Haskell following this
Replaced , with ; in multiline paths of config files as reported by #gergelybat in this
c:\Users\me\cabal update
c:\Users\me\cabal install idris
Several dependency errors arise, I follow Idris' instructions to apply with blind faith the following further flags
----reinstall
--avoid-reinstalls
--force-reinstalls
--upgrade-dependencies
Progress is made things begin to be built, we get all the way to building a temp directory with something called idris-1.3.1\…\setup.exe
Then we crash with following error:
C:\Users\me\AppData\Local\Temp\cabal-tmp-26128\idris-1.3.1\dist\setup: removeDirectoryRecursive:
…": unsatisfied constraints (The directory is not empty.)
Googling this I find this same error happens across cabal installs and logged as issues on oodles of GitHub projects. Theories are all over the map and no solutions provided. The Haskell documentation on removeDirectoryRecursive offers a clue ending with the remark that this fails on Windows if the directory is a symbolic-link.
How does one get past this problem to finish an install?
This is mostly to record the steps that worked, thanks to Michael Sonyman for the major tip.
Steps to follow.
Install Haskell (with Stack) following this
Replace , with ; in multiline paths of config files as reported by #gergelybat in this
Check that you have a version of make by checking stack exec -- which make
if you get no make then install GnuWin32,
ensure that the install directory for (e.g. c:\Program Files (x86)\GnuWin32\bin is added to your system path (follow "set environment variables").
Restart PowerShell/Command prompt, retry step 3, you should see a make found, if not recheck your steps.
Run stack install --resolver lts-12.26 idris … this may be very slow. It ends by copying the Idris compiler and other items to C:\Users\you\AppData\Roaming\local\bin:
Test by typing idris at prompt, it should load the Idris REPL.
Enjoy dependent typing for the rest of your programming life.
You could try using cabal new-build or cabal new-install, but I am not sure if that will work.

Install an external library instance of libv8-3.14 to folder

I need libv8-3.14 to run some R packages on linux, but I don't have root access/sudo access on the linux computer I'm using so I'd like to install an external folder instance of libv8-3.14. I've seen R packages reference this as external as CDFLAG="folder/v8-3.14" so I know it is possible.
I'm new(ish) to linux but I've installed external libraries before with tar.gz files which then have a configure file in them, which I set the external folder with ./configure --prefix==/folder/loc, but the only downloads I can find of libv8 are .git (which I can't get to work either).
How can I install an libv8-3.14 to a folder and install so I can set:
export PATH=$PATH:/path/to/install/
and
export `LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/install/`
I had the exact same problem. In case somebody in the future comes across this post, I will leave my suggestions and how it worked out in the end. Also, all credits go to an experienced colleague of mine.
The most sure thing to do is to consult IT, or someone who has already had the same problem, there is usually a workaround these issues.
A way you can do it yourself:
Create an anaconda environment, you can name it 'V8' or something (make sure the environment is based on the latest python version, or recent enough for r-v8).
activate it
install the conda version of the V8 R interface with conda install -c conda-forge r-v8
That's it. Whenever you need V8, fire up your environment beforehand, and it should be A-OK.
Further advice: If you run into errors when installing r-v8, it may be a good idea to update your conda and all the packages. However, depending on your conda version conda update conda and conda upgrade --all MAY BREAK your conda installation, so be careful. (For further information on this problem, see the endless complaints of people in this issue: https://github.com/conda/conda/issues/8920).
V8 doesn't use autotools, so it has no ./configure. In fact, it provides no installation facilities at all, because it is meant for embedding, not installing.
What I would try is to download the Ubuntu package (guessing from your other question, you are on Ubuntu, right?) for the right architecture from https://packages.ubuntu.com/trusty/libv8-3.14.5, and extracting it manually. .deb files are just ZIP archives.
As a side note, there's no point in setting PATH, because libv8, being a library, provides no executables. LD_LIBRARY_PATH is all you need.

Installing graph-tool library on conda installed python 3.5, Mac OS

Precisely the same as here, which hasn't been resolved.
Followed the sequential directions here; all channels added.
Tried:
Adding to .bash_profile export PKG_CONFIG_PATH=$PKG_CONFIG_PATH://anaconda/pkgs accordingly the directory where there is cairomm installed..
./configure -with--CAIROMM_CFLAGS -with--CAIROMM_LIBS
Can someone kindly make sure at least I have implemented alternate solutions correctly?
And of course I've tried the simplest conda install graph-tool after adding channels from ostrokach-forge and the like.
Instead of success, I get the following:
PackagesNotFoundError: The following packages are not available from current channels:
Whoa, so unpopular post!
For myself and other novices' use in the future ->
(after all, conda install-ed python is easier)
Slightly different but same in that certain library is not reached, acc. to this query:
conda needs to be able to find all the dependencies at once.
The -c flag only adds that channel for that one command.
conda install -c vgauthier rwest graph-tool. But an easier way is to add those channels to your configuration
conda config --add channels vgauthier --add channels rwest
And then perform
conda install graph-tool
But when I used conda install -c http://conda.anaconda.org/vgauthier graph-tool command, it worked instantly.
Before that, nothing worked. (when I only used the user name vguathier or ostrokach-forge or else)
If I type vi .condarc I see
channels:
- ostrokach-forge
- conda-forge
- defaults
And because I remain quite ignorant in using brew-installed packages with conda-installed python, I started out by following the directions to install all the necessary dependencies using brew. (including pixman)
Wonder how the command found everything though.. plus python upgraded to 3.6 from 3.5.
God this is how I am left with solving os issues.. i'm not 100 % clear how I made the computer connect the dots.
And still, nonetheless, I am still left with how to figure the install with ./configure. I want to understand the error msg that was returned and how to address it.

unable to execute 'x86_64-conda_cos6-linux-gnu-gcc': No such file or directory (pysam installation)

I am trying to install pysam.
After excecuting:
python path/to/pysam-master/setup.py build
This error is produced:
unable to execute 'x86_64-conda_cos6-linux-gnu-gcc': No such file or directory
error: command 'x86_64-conda_cos6-linux-gnu-gcc' failed with exit status 1
There are similar threads, but they all seem to address the problem assumig administriator rights, which I do not have. Is there a way around to install the needed files?
DISCLAIMER: This question derived from a previous post of mine.
manually installing pysam error: "ImportError: No module named version"
But since it might require a different approach, I made it a question of its own.
You can also receive the same error while installing some R packages if R was installed using conda (as I had).
Then just install the package by executing: conda install gxx_linux-64 to have that command available.
Source:
https://github.com/RcppCore/Rcpp/issues/770#issuecomment-346716808
It looks like Anaconda had a new release (4.3.27) that sets the C compiler path to a non-existing executable (quite an embarrassing bug; I'm sure they'll fix it soon). I had a similar issue with pip installing using the latest Miniconda, which I fixed by using the 4.3.21 version and ensuring I was not doing something like conda update conda.
See https://repo.continuum.io/miniconda/ which has release dates and versions.
It should now be safe to update conda. This is fixed in the following python packages for linux-64:
python-3.6.2-h0b30769_14.tar.bz2
python-2.7.14-h931c8b0_15.tar.bz2
python-2.7.13-hac47a24_15.tar.bz2
python-3.5.4-hc053d89_14.tar.bz2
The issue was as Jon Riehl described - we (Anaconda, formerly Continuum) build all of our packages with a new GCC package that we created using crosstool-ng. This package does not have gcc, it has a prefixed gcc - the missing command you're seeing, x86_64-conda_cos6-linux-gnu-gcc. This gets baked into python, and any extension built with that python goes looking for that compiler. We have fixed the issue using the _PYTHON_SYSCONFIGDATA_NAME variable that was added to python 3.6. We have backported that to python 2.7 and 3.5. You'll now only ever see python using default compilers (gcc), and you must set the _PYTHON_SYSCONFIGDATA_NAME to the appropriate filename to have the new compilers used. Setting this variable is something that we'll put into the activate scripts for the compiler package, so you'll never need to worry about it. It may take us a day or two to get new compiler packages out, though, so post issues on the conda-build issue tracker if you'd like to use the new compilers and need help getting started.
Relevant code changes are at:
py27: https://github.com/anacondarecipes/python-feedstock/tree/master-2.7.14
py35: https://github.com/anacondarecipes/python-feedstock/tree/master-3.5
py36: https://github.com/anacondarecipes/python-feedstock
The solution that worked for me was to use the conda to install the r packages:
conda install -c r r-tidyverse
or r-gggplot2, r-readr
Also ensure that the installation is not failing because of admin privileges.
It will save you a great deal of pain
After upgrading Golang to 1.19.1, I started to get:
# runtime/cgo
cgo: C compiler "x86_64-conda-linux-gnu-cc" not found: exec: "x86_64-conda-linux-gnu-cc": executable file not found in $PATH
Installing gcc_linux-64 from the same channel, has resolved it:
conda install -c anaconda gcc_linux-64
Somewhere in your $PATH (e.g., ~/bin), do
ln -sf $(which gcc) x86_64-conda_cos6-linux-gnu-gcc
Don't put this in a system directory or conda's bin directory, and remember to remove the link when the problem is resolved upstream. gcc --version should be version 6.
EDIT: I understand the sentiment in the comments against manipulating system paths, but maybe we can use a little critical thinking for the actual case in hand before reciting doctrine. What actually have we done with the command above? Nothing more than putting an executable (symlink) called x86_64-conda_cos6-linux-gnu-gcc in one's personal ~/bin directory.
If putting something in one's personal ~/bin directory broke future conda (after it fixes the C compiler path to point to gcc it embeds), then that would be a bug with conda. Would the existence of this verbosely named compiler mess with anything else? Unlikely either. Even if something did pick it up, it's just your system gcc after all...

Resources