I have a problem with a shared library (Linux) and a program that links against this library but does not find the symbols - although they are there. That's what I have:
A shared library "libetest.so" that is located in /usr/lib. When I do an
readelf -Ws /usr/lib/libetest.so.0
it gives (beside some others) this output:
54: 000052c0 905 FUNC LOCAL DEFAULT 11 ETEST_open_connection
Now I have an application that makes use of ETEST_open_connection(). When I build it with
gcc lib_test.cpp -DENV_LINUX -letest
it fails with a linker error
lib_test.cpp:(.text+0x32): undefined reference to `ETEST_open_connection'
As shown before the symbol is there! Any idea why this fails?
Please try:
gcc -DENV_LINUX -letest lib_test.cpp
This should be the correct argument order.
Related
I inherited a C++ code with a dependency to OpenMPI that I want to delegate to Conan and CMake, and the automated build has a strange (to me at least) behavior related to sysctl that I want to understand.
How I tried to do it
I declared the required dependencies in my root CMakeLists.txt:
Note: I added the full list of requirements because I also suspect that some of them may be in conflict? That happened before with boost, that forced me to set explicitly zlib (if i remember correctly).
# stuff ...
conan_cmake_configure(
REQUIRES
zlib/1.2.12
mp-units/0.7.0
boost/1.79.0
openmpi/4.1.0
gsl/2.7
cspice/0067
GENERATORS
cmake
# that is required for cspice
CMakeDeps
CMakeToolchain
)
# more stuff ...
and then in the application CmakeLists.txt I find, include and link the executable to the required libraries:
add_executable(spock main.cpp)
find_package(cspice REQUIRED)
find_package(openmpi REQUIRED)
target_include_directories(
spock PRIVATE $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}/include>
openmpi_INCLUDE_DIRS
cspice_INCLUDE_DIRS
)
target_link_libraries(spock
CONAN_PKG::boost
CONAN_PKG::mp-units
openmpi::openmpi
cspice::cspice
)
# We need C++ 20 activated with the concepts library
target_compile_features(spock PUBLIC cxx_std_20)
Problem: undefined reference to sysctl ... on my local system only.
Building on my local machine with CMake 3.23.2 results in the following error message:
Consolidate compiler generated dependencies of target spock
[ 25%] Building CXX object src/CMakeFiles/spock.dir/main.cpp.o
^[[A^[[A[ 50%] Linking CXX executable ../bin/spock
/usr/bin/ld: /home/becheler/.conan/data/openmpi/4.1.0/_/_/package/8f7048d1bf6fc2a7985eb087c34e69a5e64f6c86/lib/libopen-pal.a(evutil_rand.o): in function `arc4_stir.isra.0':
evutil_rand.c:(.text+0x3d2): undefined reference to `sysctl'
collect2: error: ld returned 1 exit status
gmake[2]: *** [src/CMakeFiles/spock.dir/build.make:146: bin/spock] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:135: src/CMakeFiles/spock.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
However, the same build on Github workflows on Ubuntu 20.04 works. What is weird. The only difference before apart the distribution version is that github wokflows use higher privileges than me on local (I believe?).
What I tried so far
I've been trying to read about what this sysctlreference is. And I found conflicting information:
Frrom this man page:
The sysctl() function retrieves system information and allows processes with appropriate privileges to set system information.
this similar SO question came to the conclusion that
Linux does not support this function (other OS like MacOS or FreeBSD support it)
A comment from the same post concludes that in Linux,
these details can be obtained by reading the kernel-provided
pseudofiles /proc/cpuinfo and /proc/meminfo
So here is my question: why does it compile at all on the remote server if this command is not supposed to exist on the OS used?
In my .nim code, I'm using the header pragma to include symbols from /usr/local/include/node/node_api.h (which then includes /usr/local/include/node/js_native_api.h).
proc napi_create_function(
env: napi_env,
utf8name: cstring,
length: csize_t,
cb: napi_callback,
data: pointer,
res: napi_value
): int {.header:"<node/node_api.h>".}
When I run nim c foo.nim, I get Undefined symbols for architecture x86_64 for symbols in js_native_api.h (eg: napi_create_function), but the symbols in node_api.h are found by the linker. Remember that node_api.h includes js_native_api.h (as seen here).
Undefined symbols for architecture x86_64:
"_napi_create_function", referenced from:
_createFn__NEWhgHCwqbksHULYRnxXfA in #m..#s..#s..#s.nimble#spkgs#snapibindings-0.1.0#snapibindings.nim.c.o
The root problem likely isn't related to Nim, but I don't know how to use clang to check if the problem is reproducible without Nim.
So my question is:
How do I get the linker to find the missing symbols?
Versions
nim v1.4.8
clang v12.0.0
x86_64-apple-darwin19.6.0
nodejs v14.13.1 (installed with Homebrew into /usr/local/Cellar/node/14.13.1)
nim c
/Users/alec/.nimble/bin/nim
c
--colors:on
--noNimblePath
-d:NimblePkgVersion=0.1.0
--path:/Users/alec/.nimble/pkgs/nimdbx-0.4.1
--path:/Users/alec/.nimble/pkgs/nimterop-0.6.13
--path:/Users/alec/.nimble/pkgs/regex-0.19.0
--path:/Users/alec/.nimble/pkgs/unicodedb-0.9.0
--path:/Users/alec/.nimble/pkgs/cligen-1.5.4
--path:/Users/alec/.nimble/pkgs/cbor-0.6.0
--path:/Users/alec/.nimble/pkgs/napibindings-0.1.0
--path:'/Users/alec/.nimble/pkgs/docopt-#master'
--path:/Users/alec/.nimble/pkgs/regex-0.19.0
--path:/Users/alec/.nimble/pkgs/unicodedb-0.9.0
--hints:off
-o:/Users/alec/my-project/dist/foo
/Users/alec/my-project/foo.nim
clang
clang
-o
/Users/alec/my-project/foo
/Users/alec/.cache/nim/foo_d/stdlib_assertions.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_dollars.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_formatfloat.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_io.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_system.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snapibindings-0.1.0#snapibindings#sutils.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snapibindings-0.1.0#snapibindings.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_parseutils.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_math.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_unicode.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_strutils.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_posix.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_options.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_times.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_os.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_hashes.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_tables.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimterop-0.6.13#snimterop#sglobals.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_streams.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_lexbase.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_parsejson.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_json.nim.c.o
/Users/alec/.cache/nim/foo_d/stdlib_cpuinfo.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sprivate#slibmdbx.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sprivate#svals.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sError.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sDatabase.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sData.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sCollection.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sTransaction.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#snimdbx-0.4.1#snimdbx#sCRUD.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sdata.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sdata#sfrom_json.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sdata#sto_json.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#squery.nim.c.o
/Users/alec/.cache/nim/foo_d/#m..#s..#s..#s.nimble#spkgs#scbor-0.6.0#scbor.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sdata#sfrom_cbor.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sdata#sto_cbor.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sref.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#squery#sdocument.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#sfunctions.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoopkg#seval.nim.c.o
/Users/alec/.cache/nim/foo_d/#mfoo.nim.c.o
-lm
/Users/alec/.nimble/pkgs/nimdbx-0.4.1/libmdbx-dist/libmdbx.a
-ldl
So my question is:
How do I get the linker to find the missing symbols?
make sure you are actually linking a library (statically or dynamically) which holds symbols you need. (please show how you link it.)
make sure your library actually HAS correct symbols (open file with hex editor and search for symbols)
make sure this library is of correct architecture. there are tools that let you check this. (on Windows it's dumpbin /headers file)
make sure you are importing it correctly. (i see only {.header.} pragma but others needed pragmas are absent). please show more code and command lines. so we can investigate further.
I'm porting a system of apps from AIX to linux, and all of those apps include a single shared library. I've got the shared library building on as a linux .so now - and I see at least one post here that describes how to specify what's exported from a shared library (as AIX does via a .exp file).
Just one silly question, though. On AIX, if a module in a shared library is not referenced by anything in the app that's linking to it, it is ignored by the linker. That doesn't seem to be the case on linux - but I want to make sure.
While testing my linux shared library, I left out one module with dependencies I wasn't ready to deal with yet (or more accurately, I provided a substitute module with dummy functions for all the entry points to that module, thinking that would allow it to link). So far, so good. But when I attempted to link that shared library into a trivial test app, the linker reported unresolved symbols for stuff referenced by another shared library module that is itself only referenced from within the module I replaced with dummies. I.e., I would have expeceted that module to simply be ignored...
In other words, this module is being considered by the linker as part of the final application even though nothing in the app references it. I tried the same experiment on AIX (replacing the same module with dummies and attempting to link a trivial app there). No complaints.
So, The AIX linker only attempts to resolve shared library module dependencies if those modules themselves are explicitly called in from the application. But the linux linker attempts to resolve dependencies for all shared library modules whether they're called in from the application or not.
Is this true? And if so, is there any way to override that behavior? Ultimately, when I port everything, all of the dependencies will resolve. But for now, it's hard to leave something out - even if it's not referenced...
Here's a minimal case:
main.c contains function main(), which calls function one().
one.c contains function one(), which does nothing.
two.c contains function two(), which calls function three().
There is no function three(), but libshared.so is built from
modules one.c and two.c. Program main is built from main.c and
links in libshared.so.
The linker needs to resolve function one(), which is in the shared
library. But that's all main.c requires. Still, function two() in
the library references function three(), which doesn't exist.
The linker will complain about the undefined symbol 'three', even
though program main doesn't need it.
On AIX the linker will not complain and everything will work.
main.c:
#include <stdio.h>
int one();
int main()
{
one();
}
one.c:
#include <stdio.h>
int one()
{
return 1;
}
two.c:
#include <stdio.h>
int three();
int two()
{
return three();
}
build libshared.so with modules one.c and two.c:
gcc -fPIC -shared one.c two.c -o libshared.so
Attempt to build main from main.c and libshared.so:
gcc main.c -o main -L. -lshared
./libshared.so: undefined reference to `three'
collect2: error: ld returned 1 exit status
The linker reports an undefined reference to 'three',
which is referenced from two() - but main() doesn't ever call two().
The actual answer: shared libraries are in fact shared objects: they are treated as a single object, not as a *.a library.
This shows that Linux (meaning: glibc/gcc/gold/ld) and AIX have different concepts regarding shared objects.
In Linux, when you link an executable, ld/gold checks the dependencies of the used shared objects as well -- Aix linker doesn't: it assumes that the shared objects are to be used as they are, their dependencies aren't part of the current linking. (At least this is the default behaviour.)
Here is a summary of my tests:
+----------------+--------------------+-------------------------------+
| | AIX | linux |
+----------------+--------------------+-------------------------------+
| libshared.so | only with option | yes |
| can be created | -Wl,-berok | |
+----------------+--------------------+-------------------------------+
| main | yes | only with option |
| can be created | | -Wl,--allow-shlib-undefined |
+----------------+--------------------+-------------------------------+
Note: My random thoughts regarding AIX and linking: http://lzsiga.users.sourceforge.net/aix-linking.html
By default the GNU binutils linker, ld on
Linux requires a symbol ref to be defined by some input file (i.e. object file or
shared library) in the linkage if ref is referenced by the definition of any
symbol def in any input file that the linkage needs. It doesn't matter whether def is referenced in turn.
Your program linkage needs libshared.so. libshared.so defines two, which refers to three,
so three must be defined.
You can countermand this default behaviour to tolerate undefined references in shared libraries
(but not in object files) as follows:
$ gcc main.c -o main -L. -lshared -Wl,--allow-shlib-undefined
--allow-shlib-undefined is documented in the ld manual
The notion of module in your language corresponds to translation unit at the
compilation level and object file at the linkage level. It might be helpful to
appreciate that an object file input to the linkage of a ELF program or shared library
has no distinct existence in the program or shared library. It is cut into
pieces and scattered around. So there is no sense in which it would be possible
for a linkage:
$ gcc main.c -o main -L. -lshared ...
to ignore the unreferenced module two.(c|o) within
libshared.so. There is no such thing. If that linkage did not need any
definition provided by libshared.so then it would ignore the shared library
altogether1. If it needs the shared library, then by default its references
must be resolved.
[1] That is, on Debian-clan systems where gcc is built to invoke ld with the --as-needed option
by default. On Redhat-clan systems GCC by default links shared libraries if they are input, needed or not.
I'm trying to dynamically load a camera library .so file into a Linux executable to gain access to simple camera functions.
I'm attempting to do this by:
if ( (newHandle = dlopen("./libCamera.so",RTLD_LAZY | RTLD_GLOBAL)) == NULL )
{
printf( "Could not open file : %s\n", dlerror() );
return 1;
}
However this fails and I receive the following output:
"Could not open file : libCamera.so: undefined symbol: ZTVN10_cxxabiv117__class_type_infoE"
How do I find out what symbols it is relying on?
Most likely, libCamera.so uses a symbol defined in a shared library without depending on that library.
Find a culprit. Take a real executable which links against libCamera.so (and it works). List its dependencies with ldd /path/to/executable. Among them should be a library which has a definition for ZTVN10_cxxabiv117__class_type_infoE (use grep to select likely candidates, nm -D on a library to be sure). That library won't be in the list shown by ldd ./libCamera.so.
Solve a problem. Load the library found in step 1 by dlopen first (use RTLD_GLOBAL there as well).
If there is a problem with another symbol, goto step 1.
If newly-added libraries have the same problem too, goto step 1.
Tell library authors to please fix their linking.
It could also happen that one of the prerequisites in ldd ./libCamera.so got upgraded and lost a symbol definition (maybe it was recompiled with a compiler which does name mangling differently). Then you won't find the culprit in step 1, and there is no solution but downgrading something again.
The ldd command can be used to display shared library dependencies.
ldd libCamera.so
Once you know the dependencies, you can use nm to show the symbols in each library.
nm -DC libCamera.so
I had a similar problem. It was to do with a .a library, which should have been linked to my .so and statically linked into the archive being left out.
I determined this with (OP object name used here):
nm mylibrary.so | grep ZTVN10_cxxabiv117__class_type_infoE
0000ABC0 U ZTVN10_cxxabiv117__class_type_infoE
The U here means that the symbol is "undefined". You can find the demangled name of the missing object with --demangle:
$ nm --demangle mylibrary.so | grep 0000ABC0
0000ABC0 U abi::class_type_info(params...)
(or something like that) this should help you figure out which library is missing.
In my case, even after including the library on the compiler line I still had the issue. Eventually, after some tinkering I discovered that the library-file (.a) has to come after its dependent object (.o) file like:
g++ -Wl,-E -g -m32 ... -fPIC myobjects1.o myobjects2.o missing_library.a -shared -o mylibrary.so
Now I get (no more U):
$ nm --demangle mylibrary.so | grep 0000ABC0
0000ABC0 T abi::class_type_info(params...)
and most importantly I don't get the error any more!
In your source code for libCamera.so, you have unresolved external symbol. It means that type_infoE have no definition in your source code and should be resolved.
I'm having a linking problem. I need to link against a shared library libfoo.so that depends on a function read which I would like to define myself in the file read.c.
I compile and link everything together but at runtime I get the error
/home/bar/src/libfoo.so: undefined symbol: sread.
nm reports the symbol is defined
$nm baz | grep sread
00000000000022f8 t sread
but ldd reports the symbol is undefined
$ldd -r baz | grep sread
undefined symbol: sread (/home/bar/src/libfoo.so)
What gives? Is there some isse with the fact that libfoo.so is a shared library?
First, defining a function called 'read' is a bad idea(TM), because it is a standard libc function on all UNIXen. The behavior of your program is undefined when you do this.
Second, the read function you defined in libbaz.so is marked with a 't' in nm output. This means that this function is local (not visible outside libbaz.so). Global functions are marked with 'T' by nm.
Did you use 'static int read(...)' when you defined it in read.c?
If not, did you use a linker script, or attribute((visibility(hidden))), or perhaps -fvisibility=hidden on command line when you compiled and linked libbaz.so?
The above error can also occur when C code is compiled with G++, and then linked. G++ performs name mangling, so the actual symbol may be something like "_Zsds_[function_name]_", causing the linker to choke when it searches for the un-mangled name.
I ran into the same behavior today, except my issue was resolved following the actions outlined on Wikipedia. Basically, C code compiled with a C++ compiler will have a "mangled" name in the symbol table, causing C-style symbol resolution to fail.
When you build your shared library you need to resolve all undefined symbols from either within the same library or another (shared) library. The linker will not resolve an undefined symbol from a library with a symbol from your application.