I am trying to compile a haskell package without using cabal
Given a correct .conf file, this seems to work
cd src; ghc --make -dynamic -shared -fPIC -package-name adventlib System/IO/Advent.hs System/IO/Test.hs -osuf dyn_o -hisuf dyn_hi -o libHSadventlib-ghc8.6.5.so
cd src; ghc -c --make -package-name adventlib System/IO/Advent.hs System/IO/Test.hs
ar cqs src/libHSadventlib.a src/System/IO/*.o
ghc --make src/MainTest.hs
ghc --make -dynamic src/MainTest.hs -o src/MainTest_dyn
The last two lines test that I can compile an executable binary and link the library both statically and dynamically.
Reading the docs it seems that it should be possible to use -dynamic-too to combine the first two lines in a single ghc run. However, I haven't managed to make that work.
The next line produces the static and dynamic object files, but doesn't create the so file:
cd src; ghc -c --make -dynamic-too -fPIC -package-name adventlib System/IO/Advent.hs System/IO/Test.hs
I can link the so file afterwards with ghc, but then I need to add all the package dependencies manually, missing much of the benefit of using --make
If I remove the -c flag, to make ghc run the link stage like this
cd src; ghc --make -shared -dynamic-too -fPIC -package-name adventlib System/IO/Advent.hs System/IO/Test.hs -o libHSadventlib.a -dyno libHSadventlib-ghc8.6.5.so
Then it seems to fail linking the static library (I guess it is trying to link it as dynamic but using static objects):
cd src; ghc --make -shared -dynamic-too -fPIC -package-name adventlib System/IO/Advent.hs System/IO/Test.hs -o libHSadventlib.a -dyno libHSadventlib-ghc8.6.5.so
[1 of 3] Compiling System.IO.Advent ( System/IO/Advent.hs, System/IO/Advent.o )
[2 of 3] Compiling System.IO.TestInternal ( System/IO/TestInternal.hs, System/IO/TestInternal.o )
[3 of 3] Compiling System.IO.Test ( System/IO/Test.hs, System/IO/Test.o )
Linking libHSadventlib.a ...
/nix/store/0hr45a0pzlh51hhcgynmfjpzff9d3ddv-binutils-2.31.1/bin/ld: /nix/store/gdpi6mrz1wcgmvpnfm9i9la9lpsb8lag-unliftio-0.2.12/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5/unliftio-0.2.12-Au2Yw1nUjiS94bY0JG3imp/libHSunliftio-0.2.12-Au2Yw1nUjiS94bY0JG3imp.a(Environment.o): relocation R_X86_64_32S against undefined symbol `stg_ap_p_info' can not be used when making a shared object; recompile with -fPIC
/nix/store/0hr45a0pzlh51hhcgynmfjpzff9d3ddv-binutils-2.31.1/bin/ld: /nix/store/7dx9j6hiscwr1a2nq9bjj91p33s9nqgg-unliftio-core-0.1.2.0/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5/unliftio-core-0.1.2.0-DmlZdkLzX278vkyONsp8WQ/libHSunliftio-core-0.1.2.0-DmlZdkLzX278vkyONsp8WQ.a(Unlift.o): relocation R_X86_64_32S against `.text.unliftiozmcorezm0zi1zi2zi0zmDmlZZdkLzzX278vkyONsp8WQ_ControlziMonadziIOziUnlift_zdp1MonadUnliftIO_info' can not be used when making a shared object; recompile with -fPIC
... etc ...
Am I missing something or doesn't dynamic-too work to create shared libraries?
I have the full working experiment here , for reference.
Related
Is it possible to build a shared object file on Linux without using libc? I tried building the shared object using -nostdlib, and it complains that there is a conflicting type for built-in function 'memset'(I have my own version of the function defined within the shared object I am trying to build).
I am not using any libc functions from within the shared object file. I am building the shared object as follows :-
CC = gcc
CFLAGS = -Wall -Wextra -Werror -nostdlib
OUTPUTDIR = ./build
test: outputdir
$(CC) $(CFLAGS) -c -fPIC test.c -o ${OUTPUTDIR}/test.o
$(CC) $(CFLAGS) ${OUTPUTDIR}/test.o -shared -o ${OUTPUTDIR}/libtest.so
outputdir:
mkdir -p ${OUTPUTDIR}
clean:
rm -rf ${OUTPUTDIR}
If you link with -nostdlib, you should also compile with -ffreestanding and/or -fno-builtin as well.
You also have to be careful that you do not reference a libc.so.6 symbol without linking against glibc. Things may appear to work superficially, but it tends to introduce breakage in certain environments, especially once additional IFUNCs are added to glibc. (Intel did that with the ICC 16 compiler library.)
I've created a Haskell package that makes FFI calls to functions defined in CUDA code. I'd like to compile .cu file to an object (.o) file during package build and force linker to link it in.
So far, I tried to use a technique found this question. I've customized buildHook to:
run nvcc
run default buildHook
create ar library file with nvcc compiled code.
Setup.hs is available here.
This solution has a major disadvantage in restricting this package to static linking. Although cabal produces a shared library, it won't work because it has no way of resolving symbols located in the object file.
Is there a simpler way to link custom code during building?
I do a similar thing. I have a Haskell file which calls CUDA code.
Here's how I compile CUDA libraries and link with Haskell:
$(NVCC) -c -E $(NVCC_OPTS) -o build/file.i file.cu
$(NVCC) -c $(NVCC_OPTS) -o build/file.o file.cu
I then link everything into a C++ Shared Library called LibSO with Haskell options
$(CXX) -shared -Wl,-rpath=\$$$$ORIGIN $(CXX_LINK_LIBS) $(PACKAGE_RPATH) -Lbuild -rdynamic -L/usr/local/lib/ghc-7.6.3 -lHSrts-ghc7.6.3 -o build/LibSO.so build/file.o
where
CXX_LINK_LIBS = -Lbuild -lcudart -lcuda -lpthread -lcupti -lcurand -lnvidia-ml
NVCC_OPTS = --compiler-options -fPIC -maxrregcount=0 --machine 64 --DCUDA
I then take my Haskell files and compile them into o and hi files. (I compile twice because of TemplateHaskell)
ghc -v0 -Wall -rtsopts -threaded -stubdir build -ibuild/ -no-hs-main -o build/iop.o -ohi build/iop.hi -c haskell/iop.lhs
ghc -v0 -Wall -rtsopts -threaded -stubdir build -ibuild/ -no-hs-main -fPIC -dynamic -osuf dyn_o -hisuf dyn_hi -o build/iop.dyn_o -ohi build/iop.dyn_hi -c haskell/iop.lhs
So now we have haskell dynamic objects and a C++ shared library.
In the end, I link a main haskell file with everything:
ghc -optl "-Wl,-rpath=\$$ORIGIN" $(CXX_LINK_LIBS) -Lbuild -rtsopts -threaded -lstdc++ -lLibSO -o build/Main build/iop.dyn_o
Does this sort of help?
I need some help with a linker error I get during installation of an Rcpp package on a linux system where I don't have admin rights. In a nutshell, I get this error:
relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
I have a file solve.cpp that uses external library interp2d, which in turn has a GSL dependency. I specify my dependencies via [[Rcpp::depends(RcppArmadillo,RcppGSL)]] and in the DESCRIPTION. My Makevars is like the one from the RcppGSL package, with the addition of the linterp2d flag:
PKG_CPPFLAGS = -W $(GSL_CFLAGS) $(LOCAL_INCLUDE)
PKG_LIBS += $(GSL_LIBS) $(LOCAL_LIBS) -linterp2d $(RCPP_LDFLAGS)
where I define the environement variables
export LOCAL_INCLUDE="-I/data/uctpfos/local/include/"
export LOCAL_LIBS="-L/data/uctpfos/local/lib/"
on the system.
I do R CMD INSTALL bkPackage and see:
g++ -I/cm/shared/apps/R/3.0.1/lib64/R/include -DNDEBUG -W -I/cm/shared/apps/gsl/1.15/include -I/data/uctpfos/local/include -fPIC -I/usr/local/include -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppArmadillo/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppGSL/include" -fpic -O3 -fPIC -c RcppExports.cpp -o RcppExports.o
g++ -I/cm/shared/apps/R/3.0.1/lib64/R/include -DNDEBUG -W -I/cm/shared/apps/gsl/1.15/include -I/data/uctpfos/local/include -fPIC -I/usr/local/include -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppArmadillo/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppGSL/include" -fpic -O3 -fPIC -c solve.cpp -o solve.o
The problems appears after that in the linking step:
g++ -shared -L/usr/local/lib64 -o bkPackage.so RcppExports.o solve.o -L/cm/shared/apps/gsl/1.15/lib -lgsl -lgslcblas -lm -L/data/uctpfos/local/lib -linterp2d -L/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/lib -lRcpp -Wl,-rpath,/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/lib
The error follows as:
/usr/bin/ld: /data/uctpfos/local/lib/libinterp2d.a(interp2d_spline.c.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/data/uctpfos/local/lib/libinterp2d.a: could not read symbols: Bad value
As you can see I compiled this with -fPIC, so that can't be it.
It's complaining that libinterp2d.a was not compiled with -fPIC; are you sure that was also compiled with -fPIC on?
Also, it seems you both have -fpic and -fPIC in your flags; you probably just want -fPIC. I think R actually ensures that's on by default.
I'm trying to install GHC with -fPIC support in Fedora.
I've grabbed a source tarball since it seems no binary one has this.
In Build.mk i've changed the quick build type to
ifeq "$(BuildFlavour)" "quick"
SRC_HC_OPTS = -H64m -O0 -fasm -fPIC
GhcStage1HcOpts = -O -fasm -fPIC
GhcStage2HcOpts = -O0 -fasm -fPIC
GhcLibHcOpts = -O -fasm -fPIC
SplitObjs = NO
HADDOCK_DOCS = NO
BUILD_DOCBOOK_HTML = NO
BUILD_DOCBOOK_PS = NO
BUILD_DOCBOOK_PDF = NO
endif
unfortunately, when compiling i still get the ld error
ghc -fglasgow-exts --make -shared -oHs2lib.a /tmp/Hs2lib924498/Hs2lib.hs dllmain.o -static -fno-warn-deprecated-flags -O2 -package ghc -package Hs2lib -i/home/phyx/Documents/Haskell/Hs2lib -optl-Wl,-s -funfolding-use-threshold=16 -optc-O3 -optc-ffast-math
Linking a.out ...
/usr/bin/ld: /tmp/Hs2lib924498/Hs2lib.o: relocation R_X86_64_32 against `ghczmprim_GHCziUnit_Z0T_closure' can not be used when making a shared object; recompile with -fPIC
/tmp/Hs2lib924498/Hs2lib.o: could not read symbols: Bad value
So it seems that GHC-prim still isn't compiled with -FPIC
I've also told cabal to build any packages with -fPIC and shared.
Anyone have any ideas?
EDIT:
Thanks to dcouts I've been able to make some progress. But now i'm at the point where I thnk libffi isn't compiled with -fPIC. I've edited the makefile(.in) for it but so far, no luck.
The new command is:
ghc -fPIC -shared dllmain.o Hs2lib.o /usr/local/lib/ghc-7.0.3/libHSrts.a -o Hs2lib.so
where dllmain.c and Hs2lib.hs have both been compiled using -fPIC.
The error I get is:
/usr/bin/ld: /usr/local/lib/ghc-7.0.3/libHSffi.a(closures.o): relocation R_X86_64_32
against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/ghc-7.0.3/libHSffi.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
After you see this error, do the following:
cd /tmp/Hs2lib924498/
ghc -fglasgow-exts --make -shared -oHs2lib.a /tmp/Hs2lib924498/Hs2lib.hs dllmain.o -static -fno-warn-deprecated-flags -fPIC -O2 -package ghc -package Hs2lib -i/home/phyx/Documents/Haskell/Hs2lib -optl-Wl,-s -funfolding-use-threshold=16 -optc-O3 -optc-ffast-math
Note I added -fPIC to the failed ghc command.
Once the command succeeds, continue the compilation from within the tmp directory without cleaning already compiled files. It should skip them and continue where it ended.
There's an FAQ entry on this topic on the Haskell Stack page.
It basically says the problem is environment related and sometimes non-deterministic.
The issue may be related to the use of hardening flags in some cases, specifically those related to producing position independent executables (PIE).
There's also a work around suggestion for Arch Linux:
On Arch Linux, installing the ncurses5-compat-libs package from AUR resolves this issue.
This is an embarrassingly simple problem, but its solution yet eludes me. As the title indicates, I simply want to specify to GHC the location of all my source files. This should be simple; the GHC user guide:
-idirs
This flag appends a colon-separated
list of dirs to the search path.
So, I tried the following invocations:
ghc -isrc/ -v -outputdir build/ --make -Wall Main.hs
ghc -isrc/: -v -outputdir build/ --make -Wall Main.hs
ghc -i:src/: -v -outputdir build/ --make -Wall Main.hs
ghc -i"src/" -v -outputdir build/ --make -Wall Main.hs
ghc -i"src/": -v -outputdir build/ --make -Wall Main.hs
ghc -i:"src/": -v -outputdir build/ --make -Wall Main.hs
On every invocation GHC gave the error: "<no location info>: can't find file: Main.hs"
As you probably could have guessed, Main.hs is located in a subdirectory from the working directory called "src". Just in case it matters, I'm on Windows XP, and I'm using GHC 6.12.2. I'm assuming there is some small problem that I'm just missing.
-i specifies where GHC will search for other source files, other than the ones you specify on the command line. So your Main.hs there will also need a src/ prefix. E.g.
$ ghc -isrc src/Main.hs --make
[1 of 1] Compiling Main ( src/Main.hs, src/Main.o )
Linking src/Main ...
Alternatively, you could use cabal, and have cabal init generate all the build metadata for you.