shared library loading - linux

My app shows that that dynamic linking is ok.
[root#centos udpfltr_fork]# ldd udpfltr
linux-gate.so.1 => (0x00dbe000)
libnfnetlink.so.0 => /usr/lib/libnfnetlink.so.0 (0x005de000)
libnetfilter_queue.so.1 => /usr/lib/libnetfilter_queue.so.1 (0x004a5000)
libnglogc.so.0 => /usr/lib/libnglogc.so.0 (0x00a51000)
libc.so.6 => /lib/libc.so.6 (0x001b9000)
/lib/ld-linux.so.2 (0x0019a000)
but when its invoked it stops with following error:
./udpfltr: error while loading shared libraries: libnglogc.so.0: cannot open shared object file: No such file or directory
I'm clueless about what's actually happening. I've tried with setting $LD_LIBRARY_PATH, reloading ldcache by ldconfig etc.

This might be the reason:-
1)check $LD_LIBRARY_PATH - this path may not have location where libglogc.so.0 resides
unset $LD_LIBRARY_PATH and then set this to .so location

Related

Extremely basic question about how shared objects are found and loaded

I have a shared object library (well, technically a MATLAB mexfile) that links against other shared object libraries. When I run ldd on it, I see this:
[spocops#draco libs]$ ldd render_prf_collection.mexa64
linux-vdso.so.1 (0x00007fffeaf9e000)
libstdc++.so.6 => /nasa/pkgsrc/sles12/2018Q3/gcc8/lib64/libstdc++.so.6 (0x00007f3d3420b000)
libtess-common.so => /nobackupp15/spocops/git/draco/spoc/code/dist/libs/libtess-common.so (0x00007f3d34006000)
libblas.so.4 => /home1/spocops/lib/libblas.so.4 (0x00007f3d33db2000)
liblapack.so.4 => /home1/spocops/lib/liblapack.so.4 (0x00007f3d33523000)
libmx.so => not found
libmex.so => not found
libmat.so => not found
libtess-mod.so => /nobackupp15/spocops/git/draco/spoc/code/dist/libs/libtess-mod.so (0x00007f3d33307000)
libm.so.6 => /lib64/libm.so.6 (0x00007f3d3300a000)
libgcc_s.so.1 => /nasa/pkgsrc/sles12/2018Q3/gcc8/lib64/libgcc_s.so.1 (0x00007f3d32df3000)
libc.so.6 => /lib64/libc.so.6 (0x00007f3d32a4e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3d34792000)
libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f3d32725000)
libquadmath.so.0 => /nasa/pkgsrc/sles12/2018Q3/gcc8/lib64/libquadmath.so.0 (0x00007f3d324e6000)
libmx.so => not found
libmex.so => not found
[spocops#draco libs]$
Consider the example of the 3rd item in the output, libtess-common.so . My understanding is that, because ldd reveals that render_prf_collection.mexa64 has a location for libtess-common.so stored, it is not necessary to supply libtess-common.so in a directory that's on the LD_LIBRARY_PATH. By contrast, libmx.so, libmex.so, and libmat.so are all not found, so in those cases some directory on LD_LIBRARY_PATH must contain the libraries or execution will fail.
Is this a correct understanding?
Is this a correct understanding?
No.
You can examine the info stored in render_prf_collection.mexa64 with readelf -d render_prf_collection.mexa64.
You will observe that neither libtess-common.so, nor libmx.so have their locations stored inside render_prf_collection.mexa64.
Q. Why is it then that one of them is found, and one isn't?
A. We can't tell. You haven't provided enough info.
There are a few likely possibilities:
you may have LD_LIBRARY_PATH set and it includes /nobackupp15/spocops/git/draco/spoc/code/dist/libs in it.
you may have modified ld.so.conf to include /nobackupp15/spocops/git/draco/spoc/code/dist/libs.
it may be that render_prf_collection.mexa64 has RUNPATH or RPATH which includes /nobackupp15/spocops/git/draco/spoc/code/dist/libs.

How to fix fglrun: symbol lookup error: /u2/acct/d84-lc/obj/4js/lib/userextension.so: undefined symbol: fglcapi_deccvasc

my 4gl program display this error. i'm using database oracle 11g. This program working fine in old server.
I run command ldd, and here are my findings:
Result in new server:
1:1046> ldd /u2/acct/d84-lc/work/EPBT/arpah.e_wb001/all/userextension.so
linux-vdso.so.1 => (0x00007ffc8b5c1000)
libc.so.6 => /lib64/libc.so.6 (0x00007faf0d364000)
/lib64/ld-linux-x86-64.so.2 (0x0000557392f4d000)
Result in OLD Server:
[live:LINUX-x86] ldd userextension.so
linux-gate.so.1 => (0xffffe000)
libc.so.6 => /lib/libc.so.6 (0xf7d5f000)
/lib/ld-linux.so.2 (0x00a75000)

QT5.7 How to deploy Qt Application on Linux

I'm trying to make a distributable linux Qt application. It will build run and install on the machine i've built it on AND on another machine (without Qt) that's the exact same version, but not others.
I'm currently building on Ubuntu 16, but on 15 and 14 it will segfault (somewhere in libc.so.6).
I use QtCreator to create a QtQuick Controls 2 application. It will automatically generate a sample with two clickable buttons. This is my test.
As i understand it, there is no linuxdeployqt (unfortunately).
So i copy all the Qt dependencies myself, including all the QML and .so files required. I make a qt.conf file to describe the paths where i've put everything.
The fact that this works on another Ubuntu 16 machine without a QT installation must mean i have everything I need correctly configured and in the right place.
This is a dynamically built application and not statically linked.
I suspect that the problem is to so (somehow) with incompatible versions of .so files. To that effect i trace the loading of the .so files on the machine that doesn't work (eg Ubuntu 15).
here's the trace, using
export LA_LIBRARY_PATH=$dirname/helloworldlib:$dirname/helloworldlib/lib
LD_TRACE_LOADED_OBJECTS=1 $dirname/helloworldlib/helloworld "$#"
(without LD_TRACE_LOADED_OBJECTS=1 is how it would normally be run, so that LD_LIBRARY_PATH is set to find the local .so files).
linux-vdso.so.1 => (0x00007ffdcebe5000)
libQt5Gui.so.5 => /usr/local/bin/./helloworldlib/lib/libQt5Gui.so.5 (0x00007f7dab418000)
libQt5Qml.so.5 => /usr/local/bin/./helloworldlib/lib/libQt5Qml.so.5 (0x00007f7daae1f000)
libQt5Core.so.5 => /usr/local/bin/./helloworldlib/lib/libQt5Core.so.5 (0x00007f7daa704000)
libstdc++.so.6 => /usr/local/bin/./helloworldlib/lib/libstdc++.so.6 (0x00007f7daa382000)
libgcc_s.so.1 => /usr/local/bin/./helloworldlib/lib/libgcc_s.so.1 (0x00007f7daa16c000)
libc.so.6 => /usr/local/bin/./helloworldlib/lib/libc.so.6 (0x00007f7da9da3000)
libpthread.so.0 => /usr/local/bin/./helloworldlib/lib/libpthread.so.0 (0x00007f7da9b86000)
libz.so.1 => /usr/local/bin/./helloworldlib/lib/libz.so.1 (0x00007f7da996c000)
libGL.so.1 => /usr/local/bin/./helloworldlib/lib/libGL.so.1 (0x00007f7da96fb000)
libm.so.6 => /usr/local/bin/./helloworldlib/lib/libm.so.6 (0x00007f7da93f2000)
libQt5Network.so.5 => /usr/local/bin/./helloworldlib/lib/libQt5Network.so.5 (0x00007f7da9094000)
librt.so.1 => /usr/local/bin/./helloworldlib/lib/librt.so.1 (0x00007f7da8e8c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7dabbc0000)
libicui18n.so.56 => /usr/local/bin/./helloworldlib/lib/libicui18n.so.56 (0x00007f7da89f2000)
libicuuc.so.56 => /usr/local/bin/./helloworldlib/lib/libicuuc.so.56 (0x00007f7da863a000)
libicudata.so.56 => /usr/local/bin/./helloworldlib/lib/libicudata.so.56 (0x00007f7da6c57000)
libdl.so.2 => /usr/local/bin/./helloworldlib/lib/libdl.so.2 (0x00007f7da6a53000)
libgthread-2.0.so.0 => /usr/local/bin/./helloworldlib/lib/libgthread-2.0.so.0 (0x00007f7da6851000)
libglib-2.0.so.0 => /usr/local/bin/./helloworldlib/lib/libglib-2.0.so.0 (0x00007f7da6540000)
libexpat.so.1 => /usr/local/bin/./helloworldlib/lib/libexpat.so.1 (0x00007f7da6317000)
libxcb-dri3.so.0 => /usr/local/bin/./helloworldlib/lib/libxcb-dri3.so.0 (0x00007f7da6114000)
libxcb-present.so.0 => /usr/local/bin/./helloworldlib/lib/libxcb-present.so.0 (0x00007f7da5f11000)
libxcb-sync.so.1 => /usr/local/bin/./helloworldlib/lib/libxcb-sync.so.1 (0x00007f7da5d0a000)
libxshmfence.so.1 => /usr/local/bin/./helloworldlib/lib/libxshmfence.so.1 (0x00007f7da5b07000)
libglapi.so.0 => /usr/local/bin/./helloworldlib/lib/libglapi.so.0 (0x00007f7da58d9000)
libXext.so.6 => /usr/local/bin/./helloworldlib/lib/libXext.so.6 (0x00007f7da56c7000)
libXdamage.so.1 => /usr/local/bin/./helloworldlib/lib/libXdamage.so.1 (0x00007f7da54c4000)
libXfixes.so.3 => /usr/local/bin/./helloworldlib/lib/libXfixes.so.3 (0x00007f7da52be000)
libX11-xcb.so.1 => /usr/local/bin/./helloworldlib/lib/libX11-xcb.so.1 (0x00007f7da50bc000)
libX11.so.6 => /usr/local/bin/./helloworldlib/lib/libX11.so.6 (0x00007f7da4d82000)
libxcb-glx.so.0 => /usr/local/bin/./helloworldlib/lib/libxcb-glx.so.0 (0x00007f7da4b69000)
libxcb-dri2.so.0 => /usr/local/bin/./helloworldlib/lib/libxcb-dri2.so.0 (0x00007f7da4964000)
libxcb.so.1 => /usr/local/bin/./helloworldlib/lib/libxcb.so.1 (0x00007f7da4742000)
libXxf86vm.so.1 => /usr/local/bin/./helloworldlib/lib/libXxf86vm.so.1 (0x00007f7da453c000)
libdrm.so.2 => /usr/local/bin/./helloworldlib/lib/libdrm.so.2 (0x00007f7da432d000)
libpcre.so.3 => /usr/local/bin/./helloworldlib/lib/libpcre.so.3 (0x00007f7da40bd000)
libXau.so.6 => /usr/local/bin/./helloworldlib/lib/libXau.so.6 (0x00007f7da3eb9000)
libXdmcp.so.6 => /usr/local/bin/./helloworldlib/lib/libXdmcp.so.6 (0x00007f7da3cb3000)
You can see that all the .so files are being loaded from within the "helloworldlib" directory.
except the loader:
/lib64/ld-linux-x86-64.so.2 (0x00007f7dabbc0000)
I understand that this is the loader itself. should i be using my own version of the .so loader? I don't know enough about the internals of this.
This is the same trace as on the working machine.
There's not much information on the segfault. even with debugging, I can only ascertain that it's somewhere in libc.so.6
Any suggestions from anyone who has got a dynamic linux qt app working, greatly apreciated.
thanks.

Running a ssh command inside Mathematica

Lately I've been trying to run a shell ssh command from a Mathematica notebook. I tried several suggested methods with no positive outcome. My search to an answer lead me to the following result:
RunProcess[$SystemShell, All, "
ssh <login>#<server>
exit
"]
but this gives a following error
<|"ExitCode" -> 127, "StandardOutput" -> "",
"StandardError" ->
"ssh: /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Libraries/\
Linux-x86-64/libcrypto.so.1.0.0: no version information available \
(required by ssh)
ssh: /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Libraries/\
Linux-x86-64/libcrypto.so.1.0.0: no version information available \
(required by ssh)
ssh: relocation error: ssh: symbol EVP_aes_128_ctr, version \
OPENSSL_1.0.1 not defined in file libcrypto.so.1.0.0 with link time \
reference
"|>
do you have an idea how to fix it?
P.S. My overall goal is to import and export data between external server and Mathematica notebook.
I found that Mathematica forces library dependencies. If I use ldd /usr/bin/ssh command in terminal I obtain following output:
linux-vdso.so.1 => (0x00007fffb31fe000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f2a309a7000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f2a305cd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2a303c8000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2a301af000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f2a2ff94000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f2a2fd4d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2a2f987000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f2a2f749000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2a30e87000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f2a2f47d000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f2a2f24e000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f2a2f04a000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f2a2ee3e000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f2a2ec3a000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2a2ea1b000)
but doing the same in Mathematica:
RunProcess[$SystemShell, All, "
ldd /usr/bin/ssh
exit
"]
gives a different dependence on:
libcrypto.so.1.0.0 => /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Libraries/Linux-x86-64/libcrypto.so.1.0.0 (0x00007f30fbe3c000)
libz.so.1 => /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Libraries/Linux-x86-64/libz.so.1 (0x00007f30fba20000)
My solution was to replace the file /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Libraries/Linux-x86-64/libcrypto.so.1.0.0 with /lib/x86_64-linux-gnu/libcrypto.so.1.0
this is probably not the best way, but works.
Łukasz Gładczuk suggestion should work, but there is a better way to accomplish this.
RunProcess provides a ProcessEnvironment option that allows you to set environment variables for your process. The default value is Inherited, which means the environment variables are inherited from Wolfram Language, which is what is causing library problems.
Run echo $PATH in a terminal to find your path, then use:
RunProcess[{"ssh", "user#server"}, ProcessEnvironment -> <|"PATH" -> yourpath|>]
where yourpath is your path, as a string.
An interesting example / possible solution. When I try to run ffmpeg from MMa (11.3, Ubuntu 17)
In[16]:= RunProcess["ffmpeg"]
Out[16]= <|ExitCode->1,StandardOutput->,StandardError->/usr/bin/ffmpeg: /usr/local/Wolfram/Mathematica/11.3/SystemFiles/Libraries/Linux-x86-64/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/x86_64-linux-gnu/libpng16.so.16)|>
it's trying to use the libz.so from the Mathematica path (as observed by #Łukasz above). Simply running with an empty ProcessEnvironment works?!
In[17]:= RunProcess["ffmpeg",ProcessEnvironment-><||>]
Out[17]= <|ExitCode -> 1, StandardOutput -> , StandardError ->
ffmpeg version 3.3.4-2 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 7 (Ubuntu 7.2.0-8ubuntu2)
which is nice/interesting.

Can I add a library refence to existing Shared Object?

I have a system "fsimage.so" that requires mkdirp, which just happens to live in libgen.so. But fsimage.so does not know this. For example:
# ldd /usr/lib/python2.4/vendor-packages/fsimage.so
libfsimage.so.1.0 => /usr/lib/libfsimage.so.1.0
libxml2.so.2 => /lib/libxml2.so.2
libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1
libpthread.so.1 => /lib/libpthread.so.1
libz.so.1 => /lib/libz.so.1
libm.so.2 => /lib/libm.so.2
libsocket.so.1 => /lib/libsocket.so.1
libnsl.so.1 => /lib/libnsl.so.1
libc.so.1 => /lib/libc.so.1
libmp.so.2 => /lib/libmp.so.2
libmd.so.1 => /lib/libmd.so.1
# ./test
Traceback (most recent call last):
File "./test", line 26, in ?
import fsimage
ImportError: ld.so.1: isapython2.4: fatal: relocation error: file /usr/lib/python2.4/vendor-packages/fsimage.so: symbol mkdirp: referenced symbol not found
# LD_PRELOAD=/usr/lib/libgen.so ./test
Usage: ./test
Naturally, if I had the sources etc, I could simply link it again, and add "-lgen", and it will add libgen.so as a dependency.
But as an exercise in hackery, say I do not have any sources, and simply wanted to add that fsimage.so needs to also load libgen.so. Using elfedit/objcopy etc, is this not possible? I don't suppose I can use "ld" to use .so as input, and write a new .so with extra library?
# elfdump /usr/lib/python2.4/vendor-packages/fsimage.so|grep NEEDED
[0] NEEDED 0x5187 libfsimage.so.1.0
[1] NEEDED 0x5152 libxml2.so.2
[2] NEEDED 0x5171 libgcc_s.so.1
First attempt at stackoverflow, go easy on me :)
Thank you "Employed Russian", you gave me the information needed to search deeper. Solaris already ships with "elfedit", so if others wants to know, these are the instructions
# elfedit libfsimage.so.1.0.0 libfsimage.so.1.0.0-new
> dyn:value DT_NEEDED
index tag value
[0] NEEDED 0x4f81 libpthread.so.1
[1] NEEDED 0x4fac libxml2.so.2
[2] NEEDED 0x4fc2 libgcc_s.so.1
> dyn:value -add -s DT_NEEDED libscf.so
index tag value
[35] NEEDED 0x500d libscf.so
> dyn:value DT_NEEDED
index tag value
[0] NEEDED 0x4f81 libpthread.so.1
[1] NEEDED 0x4fac libxml2.so.2
[2] NEEDED 0x4fc2 libgcc_s.so.1
[35] NEEDED 0x500d libscf.so
> :write
> :quit
# ldd libfsimage.so.1.0.0-new
libpthread.so.1 => /lib/libpthread.so.1
libxml2.so.2 => /lib/libxml2.so.2
libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1
libscf.so => /lib/libscf.so
libz.so.1 => /lib/libz.so.1
libm.so.2 => /lib/libm.so.2
libsocket.so.1 => /lib/libsocket.so.1
libnsl.so.1 => /lib/libnsl.so.1
libc.so.1 => /lib/libc.so.1
libuutil.so.1 => /lib/libuutil.so.1
libgen.so.1 => /lib/libgen.so.1
libnvpair.so.1 => /lib/libnvpair.so.1
libsmbios.so.1 => /usr/lib/libsmbios.so.1
libmp.so.2 => /lib/libmp.so.2
libmd.so.1 => /lib/libmd.so.1
Not easily.
Most UNIX systems (AIX is a notable exception) consider *.so a "final" link product, and it is not suitable as an input for any further link.
To add a new DT_NEEDED tag to the dynamic section of fsimage.so, you would need to rewrite its .dynamic section. Removing an entry from .dynamic is relatively easy -- you can just "slide" the other entries up, and replace the last entry with DT_NULL.
On the other hand, adding a new entry requires that you append a whole new .dynamic section to the file, and then update all pointers (offsets) within fsimage.so to point to the new section. This requires "deep" understanding of the ELF format.
There are existing tools to do this, e.g. rpath but I've had mixed success with it.
You can use PatchELF. Release 0.9 is out since 29 Feb 2016 and lets you:
Change the dynamic loader ("ELF interpreter") of executables
Change the RPATH of executables and libraries
Shrink the RPATH of executables and libraries
Remove declared dependencies on dynamic libraries (DT_NEEDED
entries)
Add a declared dependency on a dynamic library (DT_NEEDED)
Replace a declared dependency on a dynamic library with another one (DT_NEEDED)
Change SONAME of a dynamic library
In your case:
$ patchelf --add-needed /usr/lib/libgen.so /usr/lib/python2.4/vendor-packages/fsimage.so

Resources