The problem I had is solved. I'm posting this to solicit an explanation as to why the solution actually works. I've gotten great feedback here before.
I have a legacy code base that used a very simplistic build system, and my project is to migrate that to Autotools for customization and, particularly, building shared libraries. The main library is written in C, but must also be linkable from Fortran (for legacy purposes), and is distributed with some test codes in F77. The authors organized the source code into modules...
src_module1/
src_module2/
...
testc/
testf77/
Their built the library lib/libmain.a by compiling code in the src_*/ directories and archiving the objects with ranlib.
My first approach was to build a shared library from each src_*/ separately and "link" all of these into one shared library. Using Autotools, the src_module1/Makefile.am would contain
noinst_LTLIBRARIES = libmodule1.la
libmodule1_la_SOURCES = ...
and so on for the other modules, and finally the lib/Makefile.am would need only:
lib_LTLIBRARIES = libmain.la
libmain_la_SOURCES =
libmain_la_LIBADD = $(top_srcdir)/src_module1/libmodule1.la ...
This seemed to work perfectly. However, when the code in testc/ was compiled and linked against libmain.la, a "symbols not found" error was issued.
Thinking that this was an issue with libtool or shared libraries, I tried building static only, basically changing all .la to .a and all _LTLIBRARIES to _LIBRARIES. Same problem. This time, however, noticing the error "ranlib: warning for library: libmain.a the table of contents is empty (no object file members in the library define global symbols)" when trying to link libmain.a itself.
The solution that I found seems like a hack. I did not build Makefiles for any of the src_*/ directories, but instead used only for the lib/ directory and its Makefile.am had the lines:
lib_LTLIBRARIES = libmain.la
libmain_la_SOURCES = [all sources from all ../src_modules/ ]
This worked. The compiled programs in testc/ linked against libmain.la without issue. One of the "modules" is a set of Fortran bindings that wrap other C functions in the library. Even the Fortran codes in testf77/ linked against libmain.la properly.
Could someone carefully explain what happens when libtool builds a shared library? Or even when building a static library? Why is it that several static libraries can't be linked together to make one static library? Why are symbols only available when libtool/ranlib builds the library "from sources"? And what about installing a shared/static library, i.e. moving it to the /usr/local/lib --- what happens there? The Wikipedia article on static and shared libraries isn't really detailed enough for me.
I do appreciate all efforts to make sense of my longwinded question.
What you first tried ought to work. I am using this kind of setup all the time (in a C++ context). It's also documented, and part of the Automake test suite (although maybe not with Fortran).
A libtool library that is not installable, i.e., one declared with noinst_LTLIBRARIES, is called a libtool convenience library. That noinst_ makes a big difference in what is built. Even if Libtool is configured to build shared libraries, a libtool convenience library is not actually a shared library: it is just a set of object files (compiled as PIC so that they can be latter be used in a shared library) stored in an archive. You can use a libtool convenience library anywhere using this set of objects would make sense, e.g., to build a shared library.
When multiple libtool convenience libraries are LIBADDed to an installable libtool library (such as your libmain.la), Libtool has to unpack the archives containing the objects of each convenience library and link them into the final library.
There is a trap that is worth noting here: when building a shared library out of
convenience libraries, if the _SOURCES variable is empty Automake does not know which linker to use and default to the C linker. If you want to trick Automake into using the linking rule for some specific language, you can declare a nodist_EXTRA_..._SOURCES source file that do not have to exist. (See the Libtool Convenience Libraries section of the Automake manual for an example.)
Maybe that was your problem? If you have some Fortran files in the sources of some of your modules (your description suggests these are only C files), the Fortran linker will be used to build libmain.la only if a Fortran file appears in the source files declared for that libtool library. And the C linker will be used when libmain_la_SOURCES is empty.
Otherwise, I have no idea why it didn't work.
There is an small error in your example:
libmain_la_LIBADD = $(top_srcdir)/src_module1/libmodule1.la
should be
libmain_la_LIBADD = $(top_builddir)/src_module1/libmodule1.la
because the library is not created in the source directory. However I assume this is just a typo, and you won't see the difference unless you do a VPATH build or run make distcheck.
Your second try, using _LIBRARIES without Libtool is not expected to work.
_LIBRARIES can only be used to declare static archives, and in this case _LIBADD may only contain object files, not other static archives. Unpacking an archive to reuse its objects into another archive can be tricky to do portably. Automake's answer to this problem has always been: install Libtool and use _LTLIBRARIES (Libtool can be configured to build only static libraries).
Related
I would really like to create a single binary for my application however it depends on libzmq and I am building on OSX, has anyone managed to get an external library linking correctly on a mac?
I did something similar with lua. My aproach is to pass full path to the static library to ghc. In cabal file:
if os(darwin)
ghc-options: /usr/local/lib/liblua5.1.a
According to documentation ghc will pass it to linker:
Files with other suffixes (or without suffixes) are passed straight to
the linker.
However it doesn't work if the library is used in other package which you depend on. So if you are using e.g. zeromq4-haskell package, then you need to modify it locally.
I am a cmake newbie (on Debian/Sid/Linux/x86-64)
I forked libonion on https://github.com/bstarynk/onion to enable customization of malloc with Boehm's garbage collector; see this mail thread.
I added two files there onion/src/low_util.c and onion_src/low_util.h (which is #include-d successfully in several other patched files.
It is compiled but not linked.
set(SOURCES onion.c codecs.c dict.c low_util.c request.c response.c handler.c
log.c sessions.c sessions_mem.c shortcuts.c block.c mime.c url.c ${POLLER_C}
listen_point.c request_parser.c http.c ${HTTPS_C} websocket.c ${RANDOM_C} ${SQLITE3_C})
later:
SET(INCLUDES_ONION block.h codecs.h dict.h handler.h http.h https.h listen_point.h low_util.h log.h mime.h onion.h poller.h request.h response.h server.h sessions.h shortcuts.h types.h types_internal.h url.h websocket.h ${SQLITE3_H})
MESSAGE(STATUS "Found include files ${INCLUDES_ONION}")
but when I build, my file low_util.c got compiled but not linked.
Linking C executable otemplate
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_new':
dict.c:(.text+0x1bc): undefined reference to `onionlow_calloc'
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_node_data_free':
dict.c:(.text+0x2ec): undefined reference to `onionlow_free'
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_node_add':
Notice that libonion is a library (in C, providing HTTP service) and that I just want to add a low_util.c file (wrapping malloc & pthread_create etc... to make Boehm's GC happy: it is calling GC_malloc and GC_pthread_create ....) with its low_util.h header. Surprisingly, they get compiled, but do not seems to be linked. And I am not familiar with cmake and I am not familiar with how D.Moreno (the main author of libonion) has organized his cmake files.
Any clues?
Apply the following patch to make it link. The two executables which are being linked with the symbols generated from the .c file you added are missing and are added in the patch.
http://pastebin.com/mDMRiUQu
Based on what you posted, its hard to tell what could be wrong. The cake source code above says that a variable ${SOURCES} is equivalent to onion.c codecs.c dict.c low_util.c ... ${SQLITE3_C}, and a variable ${INCLUDE_ONION} is equivalent to block.h codecs.h dict.h ... ${SQLITE3_H}. You did not provide any targets or the files included in those targets.
A brief list of things that may help:
where do you define the top level library or executable? If your making a library, you will need the command add_library(). If you are making an executable, you will need the add_executable() command.
Use the command target_link_libraries() to resolve dependencies. Rather than placing all of the source files in a single library, group similar together in a single target (a target is defined by the add_* commands), and use this command to link the targets after compilation.
Use the find_package() to get any libraries which are defined on your system but not in you project. Then, link to that library using the target_link_libraries() command.
In this case, if the onion_dict_* functions are defined within the same library, your not including those files in library. When you use add_library or add_executable, ensure you add those files to the list. If the functions are within your project but not in the same library, use the target_link_libraries() command to link to the library which contains the correct files. If those commands are defined in an external library, then first find the library using find_package(), and then link to the library using target_link_libraries().
When linking against libraries using the -l option (say -lfoo), gcc will prefer a shared object to a static library if both are found (will prefer libfoo.so to libfoo.a). Is there a way to make gcc prefer the static library, if both are found?
The issue I'm trying to solve is the following: I'm creating a plugin for an application (the flight simulator called X-Plane), with the following constraints:
the plugin is to be in the form of a 32 bit shared object, even when running on a 64 bit system
the running environment does not provide a convenient way to load shared objects which are not in the 'normal' locations, say /usr/lib or /usr/lib32:
one cannot expect the user to set LD_PRELOAD or LD_LIBRARY_PATH to find shared objects shipped with my plugin
the X-Plane running environment would not add my plugins directory to ``LD_LIBRARY_PATH, before dynamically loading the plugin shared object, which would allow me to ship all my required shared objects alongside my plugin shared object
one cannot expect 64 bit users to install 32 bit shared objects that are non-trivial (say, are not included in the ia32-libs package on ubuntu)
to solve the above constraints, a possible solution is to link the generated shared object against static, 32 bit versions of all non-trivial libraries used. but, when installing such libraries, usually both static and dynamic versions are installed, and thus gcc will always link against the shared object instead of the static library.
of course, moving / removing / deleting the shared objects in question, and just leaving the static libraries in say /usr/lib32, is a work-around, but it is not a nice one
note:
yes, I did read up on how to link shared objects & libraries, and I'm not trying to creatae a 'totally statically linked shared object'
yes, I tried -Wl,-static -lfoo -Wl,-Bdynamic, but didn't bring the expected results
yes, I tried -l:libfoo.a as well, but this didn't bring the expected results either
You can specify the full path to the static libs without the -l flag to link with those.
gcc ... source.c ... /usr/lib32/libmysuperlib.a ...
Just add the .a file to the link line without -l as if it were a .o file.
It's dated, but may work: http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html
(almost end of the page)
"As noted earlier, it is also possible to link directly with individual library files by specifying the full path to the library on the command line."
What does the obj file ctr1.o does in gcc compilier ?Why does the linker link this obj file whenever an executable is generated?
I think it contains very basic stuf (crt stands for C run time) like setting up argv and argc for your main function etc ... Here is a link with some explanation
If you don't want it, because you are writing a tiny bootloader for example, without any bit of the libc, you can use the --no-stdlib options to link your program. If you go this way, youwill also need to write your own linker script.
I'm not sure to understand your question but I guess you are referring to 'crt1.o' in the GCC package.
The crt is one of the base packages of the libc which provides basic functionality to access the computer. IIRC it contains methods like 'printf' and such.
That's why it is often even included in the most basic C applications.
Object files hold your compiled code, but are not in themselves executable. It is the job of the linker to take all the object files that make up a program, and join them into a whole. This involves resolving references between object files (extern symbols), checking that there is a main() entrypoint (for C programs), and so on.
Since each source file (.c or .cpp) compiles into a separate object file, which are then read by the linker, changes to a single C file mean only that can be re-compiled, generating a new object file, which is then linked with the existing object files into a new executable. This makes development faster.
UPDATE: As stated in another answer, the "crt.o" object files holds the C runtime code, which is assumed to be needed by most C programs. You can read the gcc linker options and find the --no-stdlib option, this will tell gcc that your particular program should not be linked with the standard C runtime files.
I don't know if it's possible to do this, but I would like the /NODEFAULTLIB to be applied to a static library project.
I have many application projects (A.exe, B.dll, C.dll) that use a common static library D.lib.
This library has a lot of code and also has other .lib dependencies as well. One of them is the openssl library, which seems to have been built for win32 against the Release version of the CRT (i don't have the original project/sources).
So far, to avoid the mixing of the Release/Debug versions of CRT, I have to put the /NODEFAULTLIB:msvcrt.lib linker directive in all leaf projects (A.exe, B.dll). This works but I think it's not the ideal way of dealing with that issue.
I tried to put this property in D.lib project, but it has no effect.
Is there a way to force msvc++ to ignore the msvcrt.lib dependency from the 3rd party library?
A .lib does not have any linker settings because you don't link it, you link to it. A .lib is just an archive of .obj files, sort of like an uncompressed .zip file - that's why you have to put the setting on all projects that link to it.
If you're using VS2005+ you could use property sheets so that you only have to put the setting in one place and then use that property sheet in all projects.
However, OpenSSL is just that - Open Source, so you should be able to get the source for the version you are using and build it again (and add it to your version control system of course). I thought OpenSSL could be built as a DLL or LIB, which would solve your problem as the DLL would not interfere with the linking of your code.
Failing that, you always have the option of slitting your functionality out into a separate DLL so that you only have issues with one project.
To prevent your distributed static link library from depending on a specific MSVC runtime library you need to set this compiler option (in Visual Studio 2010 it looks like):
Configuration Properties -> C/C++ -> Advanced -> Omit Default Library Name = Yes (/ZI)
Now your users can link to your release built static lib from their debug build and not try to link to the incorrect runtime library causing problems, as well as linkers warnings.
Note that may cause link errors if your library actually depends on a specific runtime library or its behavior, and compatible components are not provided in some other way.
My understanding is that if library LIB in linked statically into a DLL, the DLL contains already all relevant code from LIB. Therefore, this coupling cannot be removed. This is just based on my understanding of statical linking, not on experiments.