I got a core while executing an application and I saved the executable, the corefile and a shared library which the application use in /tmp to check them later. I then modified the library, rebuilt it and started the executable again.
Now when I am trying to debug the core, gdb is loading the shared library from its original path and not from the directory /tmp where I saved the original library.
For example, the original path was /opt/mydir/lib/libmylib.so.0.
gdb is loading this shared library, while I want it to load /tmp/libmylib.so.0.
The application also uses some standard libraries which are in /usr/lib and /lib directories so I don't want these paths to get changed. Just want to change /opt/mydir/lib/ -> /tmp. How can I do that?
The simplest solution is to temporarily restore /opt/mydir/lib/libmylib.so.0 to the copy that was used at crash time (i.e. the one now in /tmp), analyse the core, then restore back the new version.
If you don't want to do that, set solib-search-path and set sysroot are your friends.
Note that you must set both before loading the core. This sequence should work:
(gdb) set sysroot /no/such/file
(gdb) set solib-search-path /tmp:/usr/lib:/lib
(gdb) core /tmp/core
Related
I have built some shared libraries on Ubuntu Linux 16.0.2 from source.
They are 64-bit libs.
I manually copied them to /usr/local/lib.
I verified that the /usr/local/lib path is indeed in one of the .conf files that ld.so.conf includes.
I then ran: sudo ldconfig to update the cache.
But then when I try to run my executable which tries to dynamically load one of the .so files that I previously copied into /usr/local/lib using dlopen, it fails.
In my code, I have:
dlopen ("foobar.so", RTLD_LAZY);
Can anybody tell me what I am doing wrong?
The dynamic linker normally doesn't access the paths recursively included from /etc/ld.so.conf directly, but it uses a cache.
You can update the cache with
sudo ldconfig
See ldconfig(8) for more details.
for dlopen to work, there's no directory list of places to find the shared object. So doing dlopen("somefile", ...); probably won't work.
You don't need to use any path or to put the shared object (or to comply with naming conventions) to use a shared object via dlopen(3). That's only a requirement of the dynamic linker that loads and links all the shared libraries at launch time: linux-vdso.so.1 (in 64bit)
To test, just put the shared in your local dir and try to open it with it's basename, like you post.
For a system library, there are more requirements, like defining a soname for the library, which is used by the loader to load the library and to construct the cache database index, so if you have no idea of what I'm talking about, you will not be able to use automatic loading procedure. If you want to see if an executable file has all the libraries it needs and where the loader finds them out, just run ldd(1) with the executable as argument, and you'll se the dependencies for automatic loading and how the dynamic linker resolves the paths.
My project includes a library and example projects for how to use it. I place the library in the "bin" folder along with all executable examples. I can run the example projects on the machine where they were compiled but when I try to run them on another machine I get:
./example: error while loading shared libraries: libMyLib.so: cannot
open shared object file: No such file or directory
This makes no sense since the library is in the same folder. What is causes it to ignore the library on other machines?
Just because the library is in the same directory as the executable doesn't mean it will look there for it. By default on linux, executables will only look in a limited set of directories, set by ldconfig and the LD_LIBRARY_PATH environment variable.
One trick that is very useful is to link your program with the extra linker option
-Wl,-rpath,'$ORIGIN'
which will cause the executable to also look in the directory the executable is in for shared objects.
You can usually set this by adding to your Makefile:
LDFLAGS := -Wl,-rpath,'$$ORIGIN'
Note the double-$ here -- make will interpret this as a make variable which expands to just $
The current directory is not necessarily a place where the dynamic linker will look for dynamic libraries. The directory where the executable is much less.
You might want to check ldconfig to see where it looks for them.
I am attempting to carry out postmortem analysis of a crashed binary, "TestApp", on a linux system.
I have a copy of the binaries and and shared objects that are copied onto the device in a path:
/usr/public/target
This folder contains all the binaries in question in the directory structure used on the system under test, ie:
/usr/public/target/sbin/TestApp
/usr/public/target/lib/TestAppLib.so
/usr/public/target/usr/lib/TestAppAPILib.so
The automated build process strips the debug information from the binaries, and stores them in external, symbol files, all under:
/usr/public/target_external_symbols
So the symbol information for the above binaries would exist in files named:
/usr/public/target_external_symbols/sbin/TestApp.sym
/usr/public/target_external_symbols/lib/TestAppLib.so.sym
/usr/public/target_external_symbols/usr/lib/TestAppAPILib.so.sym
How do I get GDB to be aware of the existence of these external symbols and to load them?
I typically invoke GDB via:
gdb TestApp TestApp.core
I've referred to other articles on creating a test .gdbinit file and passing it to GDB via the -command argument, but it doesn't appear to work. Every time I attempt to get a backtrace from my core file, I get an indication from GDB that it cannot open the debug symbols. Any help in resolving this is appreciated.
(gdb) info shared
From To Syms Read Shared Object Library
0x78000000 0x780061e8 Yes (*) /usr/public/target/lib/TestAppLib.so
0x78010000 0x7806e60c Yes (*) /usr/public/target/usr/lib/TestAppAPILib.so
0x78070000 0x78091d2c Yes (*) /usr/public/target/lib/libm.so.2
(*): Shared library is missing debugging information.
Thank you.
There are commands set debug-file-directory, symbol-file and add-symbol-file to load debugging symbols from within a gdb session. The latter one might require the address where the shared library was loaded into memory.
Maybe during your build process 'gnu debuglinks' have been added to your binaries. This would mean that in the executables there's a path coded that directs gdb where to look for debug symbols. More can be found here.
I am using gdbserver on target and CodeSourcery IDE. My hardware is a gumstix with a omap3530.
I can step through code in my main application but if I attempt to step into a function in a shared library I get memory address and a debugger terminates.
This is my library that is compiled and copied to the /lib folder on the target system.(it does have debug symbols) I have attempted to use the .gbdinit file to set solib-absolute-prefix /lib
Here are the warnings from the gdb trace:
903,056 13-gdb-set sysroot-on-target /lib
903,065 13^done
903,065 (gdb)
903,065 14-target-select remote 192.168.1.101:2345
903,114 =thread-group-started,id="i1",pid="42000"
903,114 =thread-created,id="1",group-id="i1"
903,115 15-list-thread-groups --available
903,120 16-list-thread-groups
903,128 &"warning: Unable to find dynamic linker breakpoint function.\nGDB will be unable to debug shared library initializers\nand track explicitly loaded dynamic code."
903,128 &"\n"
Which leads to
903,395 &"Error while mapping shared library sections:\n"
903,397 &"/lib/libCoreLib.so: Invalid argument.\n"
903,399 =library-loaded,id="/lib/libCoreLib.so",target-name="/lib/libCoreLib.so",hostname="/lib/libCoreLib.so",low-address="0x0",high-address="0x0",symbols-loaded="0",thread-group="i1"
You can debug with the library installed on your host, provided the debugging machine is also the development machine. In that case, you use set sysroot instead of set sysroot-on-target. For example :
set sysroot /home/username/.../rootfs/
where /home/username/.../rootfs/ contains a copy of your target filesystem
I think you should also specify / instead of /lib
Target with debug symbols
This is the simplest method to get working, and it is specially useful when you are developing one particular shared library.
First copy the test executable and shared library to the target with debug information:
check with readelf ----debug-dump=decodedline libmyib.so: How can I tell if a library was compiled with -g?
I recommend using an NFS server on host, so that the compiled output gets automatically uploaded after compilation
Then on target:
gdbserver --multi :1234 ./executable_name
Host:
arm-linux-gnueabihf-gdb -q -nh \
-ex "target extended-remote target-hostname-or-ip:1234" \
-ex "file ./executable_name" \
-ex 'tb main' \
-ex 'c' \
-ex 'set solib-search-path .'
sharedlibrary libmylib.so also works.
The problem I had was that gdbserver stops at the dynamic loader, before main, and the dynamic libraries are not yet loaded at that point, and so GDB does not know where the symbols will go in memory yet.
GDB appears to have some mechanisms to automatically load shared library symbols, and if I compile for host, and run gdbserver locally, running to main is not needed. But on the ARM target, that is the most reliable thing to do.
Target gdbserver 7.12-6, host arm-linux-gnueabihf-gdb 7.6.1 from Linaro.
Target libraries without debug symbols
It is common to strip target libraries before deployment on embedded targets, since debug information makes them way larger.
For example, Buildroot does that by default, but you can disable it with BR2_STRIP_none=y.
You can identify this scenario by running:
info shared
Which shows something like:
From To Syms Read Shared Object Library
0x00007ffff7df7f90 0x00007ffff7dfcdd7 Yes (*) target:/lib/ld64-uClibc.so.0
0x00007ffff7b3a9b0 0x00007ffff7bbe05d Yes (*) target:/lib/libc.so.0
(*): Shared library is missing debugging information.
so there are asterisks (*) for both of the libraries which says that debug information is missing.
If that is the case, then you have to tell GDB to use the shared libraries on host before they were stripped.
Buildroot for example makes that easy for us, as it maintains the staging directory which contains the shared libraries before they were stripped and in the same relative paths as in the target:
set sysroot buildroot/output/staging/
When this option is set, gdb immediately searches for libraries in the host instead of target, and finds /lib/libc.so.0 at the path buildroot/output/staging/ + /lib/libc.so.0:
Reading symbols from buildroot/output/staging/lib/ld64-uClibc.so.0...done.
Reading symbols from buildroot/output/staging/lib/libc.so.0...done.
TODO: I don't think you can set more than one sysroot, so all your shared libraries must be placed in their correct relative paths as in the target image.
If you check the bad default sysroot, you will see:
show sysroot
give:
target:
which means that gdb searches for shared libraries on the target root / by default.
Similar issue was encountered while debugging. The debug was hanging up. Configuration is as follows
Host: Ubuntu 12.04LTS
IDE: Eclipse Kepler
Target: Beaglebone Black / ARM A8
OS: Angstrom
Solution
Update libraries and includes
Select properties for project in Eclipse
C/C++ General > Paths and Symbols > (include TAB) GNU C > Add > Files
systems > / > usr Change from /usr/lib/gcc/i686-linux-gnu/4/6/include
to /usr/arm-linux-gnueabi/include
C/C++ General > Paths and Symbols > (Include TAB) GNU C++> Add >
Files systems > / > usr /usr/arm-linux-gnueabi/include/c++/4.6.3/arm-linux-gnueabi
C/C++ General > Paths and Symbols > (Library Paths TAB) > Add > Files
systems > / > usr /usr/arm-linux-gnueabi/lib
Good day,
If the 'debug-file-directory' variable in GDB is set incorrectly,
then the reported error messages contains:
warning: Unable to find dynamic linker breakpoint function.
The root filesystem of the target is located on my host PC at
/opt/arm-linux-gnueabihf-rootfs
The following two commands helped me to get remote debugging working
via gdbserver using GDB (v7.11.1):
set debug-file-directory /opt/arm-linux-gnueabihf-rootfs/usr/lib/debug
set sysroot /opt/arm-linux-gnueabihf-rootfs
I've noticed that if 'sysroot' has a trailing slash in the path,
then GDB fails to use it.You will see this (incorrect output) after connecting to the remote target:
Reading /lib/ld-linux-armhf.so.3 from remote target...
or
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/lib/ld-linux-
armhf.so.3...(no debugging symbols found)...done
instead of the correct output:
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/lib/ld-linux-
armhf.so.3...
Reading symbols from /opt/arm-linux-gnueabihf-rootfs/usr/lib/debug/
lib/arm-linux-gnueabihf/ld-2.23.so...done.
Regards,
Frikkie Thirion
I have a core file generated on a remote system that I don't have direct access to. I also have local copies of the library files from the remote system, and the executable file for the crashing program.
I'd like to analyse this core dump in gdb.
For example:
gdb path/to/executable path/to/corefile
My libraries are in the current directory.
In the past I've seen debuggers implement this by supplying the option "-p ." or "-p /=."; so my question is:
How can I specify that libraries be loaded first from paths relative to my current directory when analysing a corefile in gdb?
Start gdb without specifying the executable or core file, then type the following commands:
set solib-absolute-prefix ./usr
file path/to/executable
core-file path/to/corefile
You will need to make sure to mirror your library path exactly from the target system. The above is meant for debugging targets that don't match your host, that is why it's important to replicate your root filesystem structure containing your libraries.
If you are remote debugging a server that is the same architecture and Linux/glibc version as your host, then you can do as fd suggested:
set solib-search-path <path>
If you are trying to override some of the libraries, but not all then you can copy the target library directory structure into a temporary place and use the solib-absolute-prefix solution described above.
I'm not sure this is possible at all within gdb but then I'm no expert.
However I can comment on the Linux dynamic linker. The following should print the path of all resolved shared libraries and the unresolved ones.
ldd path/to/executable
We need to know how your shared libraries were linked with your executable. To do this, use the following command:
readelf -d path/to/executable | grep RPATH
Should the command print nothing, the dynamic linker will use standard locations plus the LD_LIBRARY_PATH environment variable to find the shared libraries.
If the command prints some lines, the dynamic linker will ignore LD_LIBRARY_PATH and use the hardcoded rpaths instead.
If the listed rpaths are absolute, the only solution I know is to copy (or symlink) your libraries to the listed locations.
If the listed rpaths are relative, they will contain a $ORIGIN which will be replaced at run time by the path of the executable. Move either the executable or the libraries to match.
For further informations, you could start with:
man ld.so
I found this excerpt on developer.apple.com
set solib-search-path path
If this variable is set, path is a
colon-separated list of directories to
search for shared libraries.
solib-search-path' is used after
solib-absolute-prefix' fails to
locate the library, or if the path to
the library is relative instead of
absolute. If you want to use
solib-search-path' instead of
solib-absolute-prefix', be sure to
set `solib-absolute-prefix' to a
nonexistant directory to prevent GDB
from finding your host's libraries.
EDIT:
I don't think using the above setting prepends the directories I added, but it does seem to append them, so files missing from my current system are picked up in the paths I added. I guess setting the solib-absolute-prefix to something bogus and adding directories in the solib-search-path in the order I need might be a full solution.
You can also just set LD_PRELOAD to each of the libraries or LD_LIBRARY_PATH to the current directory when invoking gdb. This will only cause problems if gdb itself tries to use any of the libraries you're preloading.
One important note:
if you're doing a cross compiling and trying to debug with gdb, then
after you've done file ECECUTABLE_NAME if you see smth. like :
Using host libthread_db library "/lib/libthread_db.so.1"
then check whether you have libthread_db for your target system. I found a lot of similar problems on the web. Such problem cannot be solved just using "set solib-", you has to build libthread_db using your cross-compiler as well.