I'm working on a project that produces a shared object (.so) file. The source code is in both C and Haskell. It needs to be linked with libraries for the GHC Runtime System and for the Hint Haskell module.
Currently, the command to compile the file is:
ghc -Weverything -Werror -fforce-recomp $^ -o $# -dynamic -shared -lHSrts-ghc8.10.5 -lHShint-0.9.0.4-7GR8UnOuwIbHNCqCSHKCK0-ghc8.10.5
There must be a way to automate the finding of the libraries instead of hard-coding the names. I want the Makefile to continue to work with future releases of GHC and Hint. It seems to be that ghc-pkg
should be able to provide this, but I can't find how.
Related
I am trying to build an executable from different libraries using the below statement
g++ -fPIC -O -DUSING_PCH -D_REENTRANT -I"/app1/home1/quickfast/boost_1_61_0/include/boost-1_61" -I"/app1/home1/quickfast/boost_1_61_0/." -I"../../src" -I"../src/Examples" -I"/app1/home1/quickfast/boost_1_61_0/include" -L"." -L"/app1/home1/quickfast/boost_1_61_0/stage/lib" -L/app1/home1/quickfast/quickfast-master/lib -l"QuickFAST" -l"boost_thread-gcc44-mt-1_61" -l"boost_system-gcc44-mt-1_61" -l"boost_filesystem-gcc44-mt-1_61" -l"boost_unit_test_framework-gcc44-mt-1_61" -static -ldl -lpthread -o "../../bin/testSequences"
The above compilation statement includes lot of libraries, which are availabe as dynamic libaries(.so file) as well as static libraries (.a file). For example library boost_system-gcc44-mt-1_61 is present in the system as libboost_system-gcc44-mt-1_61.so as well as libboost_system-gcc44-mt-1_61.a . What I want is that while compilation only dynamic libraries (.so files) are picked up and static libraries are ignored. Can someone please guide.
You are telling it to pick up static libraries by using the -static link option. If you remove it, it'll probably work as you expect.
http://www.vex.net/~trebla/haskell/so.xhtml describes how to compile shared library.
About compiling command:
ghc -O2 -dynamic -shared -fPIC -o libEval.so Eval.hs hsbracket.c -lHSrts-ghc7.6.3
it says:
(Could you omit -dynamic to request static libraries of other packages? Not really, they were not generated with -fPIC. In particular it is illegal on x86_64.)
Why is it so? What should one do to compile shared library without libHS* dependencies?
Since Kaiko contacted me privately for an answer, might as well post it here...
Short version
By omitting -dynamic you would be trying to take all the static .a libs and link them into one massive .so file. Those .a libs were themselves built without -fPIC. All code that ends up on a .so file must be built with -fPIC (at least on ELF x86-64). Thus the linking would fail in this case because -fPIC is required but the libs were not built with -fPIC.
Long version
There are a few things that vary between different ways of building
static and dynamic libraries:
Is it built as a .a archive or as a .so (or .dll/.dynlib) object?
Is it built with -fPIC, position independent code or not?
Are external symbols expected to be in the same DSO or an external DSO?
In principle, lots of different combinations of these things make sense
but in practice only a few are used.
On Linux (ELF), there are two standard approaches to building libraries,
fully static and fully dynamic. In the fully static approach the answer
to question 1,2,3 above are: .a archive, no -fPIC, same DSO. In the
fully dynamic approach the answers are: .so lib, -fPIC, external DSO.
Now what you want to do is different. You want all libraries to be built
as .a files, but with -fPIC and external symbols expected to be in the
same DSO. This would then let you link all those libraries together into
one huge shared library. So the crucial difference is the use of -fPIC,
since on ELF (specifically x86_64) code that ends up in a shared lib
must be built with -fPIC.
By contrast, on Windows, GHC can do exactly what you want, to link all
the Haskell libs (including the RTS etc) into one massive shared lib
(.dll). This is because on Windows (unlike ELF), position independent
code does not matter. So on Windows, one is able to take the static
libraries and link them into a big shared library.
In principle this should also be possible on Linux, if all of the
Haskell libraries were built statically but with -fPIC. This is not the
default, and that is the immediate reason why you cannot omit -dynamic
in this case on Linux.
So in principle, one could try rebuilding ghc and the core libraries
from source using the -fPIC flag and then see if it then works to omit
-dynamic and link everything into one huge shared lib.
Yes, compiling with -fPIC helps. Here is how to do that.
ghc-7.8.4/mk/build.mk:
SRC_HC_OPTS = -H64m -O
EXTRA_HC_OPTS = -fPIC
SRC_CC_OPTS = -fPIC -O
GhcStage1HcOpts = -fasm -O0
GhcStage2HcOpts = -fasm -O0
GhcLibHcOpts = -fasm -O2
GhcLibWays = v dyn
DYNAMIC_GHC_PROGRAMS = YES
DYNAMIC_BY_DEFAULT = NO
SplitObjs = NO
HADDOCK_DOCS = NO
BUILD_DOCBOOK_HTML = NO
BUILD_DOCBOOK_PS = NO
BUILD_DOCBOOK_PDF = NO
While you compile ghc:
export EXTRA_CONFIGURE_OPTS="--disable-library-profiling --enable-shared"
To build cabal packages with -fPIC use:
cabal install --enable-shared --ghc-option=-fPIC text
Test file foo.hs (Data.Text is used to see if cabal packages also work):
import Foreign.C as C
import Data.Text as T
import Data.Text.Foreign as T
foreign export ccall len :: CString -> IO CInt
len t = C.peekCString t >>= return . CInt . fromIntegral . T.length . T.pack
main = return ()
Dynamic build:
ghc -dynamic --make foo.hs
Dynamic mixed with static build (not sure if pthread is needed but it illustrates how to add dynamic linking):
ghc -fPIC -shared --make -o libfoo.so \
-optl-Wl,-Bstatic -lHSrts -lCffi \
-lHSbase-4.7.0.2 -lHSinteger-gmp-0.5.1.0 -lHSghc-prim-0.3.1.0 \
-optl-Wl,-Bdynamic -lpthread foo.hs
I am developing and testing some code on 2 different machines: my own *buntu laptop and a remote linux machine configured by someone else.
My code uses a library, let's say libfoo, that in turn depends on, say, libbase.
I would like to keep a single makefile across both the machines, but I found out things works differently when I build my project (with GCC):
On my laptop, I need to specify -lfoo -lbase for the code to link
correctly.
On the remote machine, I only need the -lfoo flag, and the linker
somehow picks up the base library automagically.
Anyone knows what is going on? Is there any flag that might have been passed when building libfoo from source that made this automatical "depencency detection" possible?
PS: I know I could just specify every library in the make file, but keeping the list of flags to the minimum looks interesting, and I would like to know what's going on under the hood.
Yes. Both -lfoo and -lbase refer to some shared libraries libfoo.so and libbase.so (perhaps with some version number).
On the remote machine, libfoo.so was built and linked to libbase.so, perhaps from some foo1.c and foo2.c like
gcc -Wall -O -fPIC foo1.c -o foo1.pic.o
gcc -Wall -O -fPIC foo2.c -o foo2.pic.o
gcc -shared -O foo1.pic.o foo2.pic.o -lbase -o libfoo.so
(of course, probably some Makefile did run above commands thru make)
On your own laptop, you (or your distribution maker) did not link -lbase inside libfoo.so
Read Drepper's how to write shared library (long) paper (and the program library howto)
It is possible to link a shared (low-level) library inside another (higher-level) shared library (and that is not possible with static libraries libfoo.a).
You could use ldd on (the absolute path of) your libraries libfoo.so and libbase.so to find out how they have been linked.
currently I'm creating a shared library "libmylib.so" using libtool with statements like this:
libtool --mode=compile g++ -Wall -fPIC -shared $(CFLAGS) $(LFLAGS) $(LSTATIC)
libtool --mode=link g++ -shared -export-symbols-regex beam_ -rpath /usr/lib -lotherlib
The library created with this depends on an other (non-static) library "libotherlib.so", that's why I'm using the statement "-lotherlib".
Now when I build an executable that links against libmylib.so something strange happens: I have to link against libotherlib.so again, means I have to add a statement "-lotherlib" for this executable too.
So it seems former linking of libotherlib.so when building libmylib.so does not to be persistent. But how can I change this? How can I build libmylib.so so that it implicitely knows about its dependency to shared library libotherlib.so?
Thanks!
The answer seems to be the same like for this question: Link a static library to a shared one during build?
Big difference: when shared objects are specified for linking instead of static ones they are not included but the generated shared library remembers its dependnecy to these so's.
I have a project where I have one static library libhelper.a and another with my actual shared object library, libtestlib.so. My goal is to link libhelper.a into libtestlib.so. Is that possible on Linux/BSD? When I tried and created a test program I got the following errors:
./prog1:/usr/local/lib/libtestlib.so.1.0: undefined symbol ''
My guess is that this is occurring because libhelper.a was not compiled with -fPIC while libtestlib.so was. What is the proper way to build programs that use shared libraries that also have dependancies on static libraries?
Thanks!
My goal is to link libhelper.a into libtestlib.so. Is that possible on Linux?
Sure. This should do:
gcc -shared -fPIC -o libtestlib.so $(OBJS) \
-Wl,--whole-archive -lhelper -Wl,--no-whole-archive
libhelper.a was not compiled with -fPIC
It's best to rebuild libhelper.a with -fPIC. If that's not possible, above command will still work on Linux/ix86, but not on e.g. Linux/x86_64.
What is the proper way to build programs that use shared libraries that also have dependancies on static libraries?
If you include libhelper.a into libtestlib.so as above, then simple:
gcc main.c -ltestlib
is all you need. If you insist on linking with libhelper.a, then you must tell the end-user that he must link with e.g.
gcc main.c -ltestlib -lhelper
There is no way to specify that libtestlib.so depends on libhelper.a.