I first cross compile my Rust project to linux target
cargo build --target x86_64-unknown-linux-gnu
Then run ldd on my local ubuntu and the linker works fine.
linux-vdso.so.1 (0x00007fffddc62000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f6d34500000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f6d340b0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6d33ea0000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6d33c90000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6d33a70000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6d33850000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6d33440000)
/lib64/ld-linux-x86-64.so.2 (0x00007f6d35a00000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6d330a0000)
But on my target os, ldd fails to find the libraries
libssl.so.1.0.0 => not found
libcrypto.so.1.0.0 => not found
/lib64/libc.so.6 => version `GLIBC_2.18' not found
Actually I have libssl.so.10 and libcrypto.so.10 installed under LD_LIBRARY_PATH. Unfortunately I am not able to install shared libraries of version 1.0.0 required by Rust.
I have read Rust Cross and the recommended solution is to install the missing shared libraries. Unfortunately that's not possible for me. So I am looking for alternative solutions to missing libraries.
libssl.so.1.0.0 and libcrypto.so.1.0.0 sound ancient. How can I tell cargo to use a later version?
How can I deal with /lib64/libc.so.6 => version GLIBC_2.18 not found?
According to this, GLIBC_2.18 can't be installed to RHEL7. I gave up dynamically linked libraries.
This post helped me out. The solution is:
[dependencies]
nats = "*"
protobuf = { version = "~2.0" }
# Add openssl-sys as a direct dependency so it can be cross compiled to
# x86_64-unknown-linux-musl using the "vendored" feature below
openssl-sys = "*"
[features]
# Force openssl-sys to staticly link in the openssl library. Necessary when
# cross compiling to x86_64-unknown-linux-musl.
vendored = ["openssl-sys/vendored"]
This way I can compile to a single executable with no library dependencies using:
cargo build --target=x86_64-unknown-linux-musl --features vendored
Related
Will two libraries together produce a third library that does not even exist?
Try it here
Make sure there are icu and g++ on your machine
Output
ldd out1:
linux-vdso.so.1 (0x00007ffd5cdaf000)
liblcf.so.0 => /home/aleck099/.local/lib/liblcf.so.0 (0x00007ff200600000)
libicuuc.so.72 => /usr/lib/libicuuc.so.72 (0x00007ff200200000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ff1ffe00000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007ff200518000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ff2008a1000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007ff1ffc19000)
libicui18n.so.71 => not found
libicuuc.so.71 => not found
libicudata.so.71 => not found
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007ff200874000)
libicudata.so.72 => /usr/lib/libicudata.so.72 (0x00007ff1fde00000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007ff2008ea000)
ldd out2:
linux-vdso.so.1 (0x00007ffdfed78000)
libicui18n.so.72 => /usr/lib/libicui18n.so.72 (0x00007fa088000000)
libicuuc.so.72 => /usr/lib/libicuuc.so.72 (0x00007fa087c00000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fa087800000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fa08834d000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fa087fe0000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fa087619000)
libicudata.so.72 => /usr/lib/libicudata.so.72 (0x00007fa085800000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fa08845e000)
You can see that three "not found"s in the ldd output of out1
They are even duplicates of existing icu libraries
How could this happen?
Other information
both clang and gcc produce the same result
liblcf.so is built from easyrpg
It looks like the link command used to create out1 picked up symbols from two different versions of the ICU library: libicuuc.so.71 and libicuuc.so.72. The 71 vs 72 is the version number. Those appear to be two different versions of the shared library that were both in the linker's library path. It's "not found" at ldd time because one copy is not in the runtime shared library search path, and even if it was mixing both is probably a bad idea.
The solution is likely to inspect your link command and ensure the -L options only include one version/copy of the ICU library.
EDIT: Linking with verbosity enabled (g++ -v option) will provide additional information about the libraries and paths being used.
Note that it's possible that some of the shared libraries pulled in by out1 are transitively pulling in the (non-existent) libicuuc.so.71. Comparing your two ldd executable outputs, I would be suspicious of /home/aleck099/.local/lib/liblcf.so.0 (and possibly /usr/lib/libexpat.so.1) that only appear in out1. The following command may help:
$ ldd /home/aleck099/.local/lib/liblcf.so.0 /usr/lib/libexpat.so.1
I'm developing a libfoo-sys crate with low level bindings for an third-party C-library and a libfoo-crate on top of that. In the build-script of libfoo-sys I check out the original source from git.
I want to prevent that users have to globally install stuff into their system.
I build the library in the directory of the git submodule, and install the library locally into ./lib-out. Inside ./lib-out After building, I get the shared object file that has the following dependencies:
linux-vdso.so.1 (0x00007ffc9eff6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f07870b0000)
libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f078702b000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0786e49000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0786c57000)
/lib64/ld-linux-x86-64.so.2 (0x00007f078734f000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0786c3c000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0786aeb000)
My build-script tells Rustc to use static linkage:
println!("cargo:rustc-link-lib=static=foo");
However, as soon as I use static linkage, Rustc can't find C++ symbols. etc. I guess I have to also link all the dependencies manually. However, I don't know a generic way how println!("cargo:rustc-link-lib=static=stdc++"); will work in the end. I can't find a libstdc++.so in my file system but only a libstdc++.so.6 which is not compatible with cargo:rustc-link-lib (or at least I don't know how). Any advices?
PS: I'm aware of that you should refrain from linking glibc statically and I guess the same applies for the c++ std as well. However, primarily I'd like to figure out if it makes sense and brings a benefit to statically link the library and use the self compiled version instead of a globally installed version
I use rust 1.34 and 1.35. Currently it links to GLIBC_2.18.
How can I limit cargo build to link GLIBC up to version 2.14?
Unfortunately, you can't. Not really, and not consistently. This is a problem with any binary that dynamically links to GLIBC. You can try setting up multiple GLIBC versions and linking to one, or you can try patching the resulting binary, but it's inconsistent and impractical.
So what are some practical options?
Compile Statically
By using MUSL instead of GLIBC we can compile statically.
To install the MUSL target with rustup (assuming x86_64 architecture):
$ rustup component add rust-std-x86_64-unknown-linux-musl
And to use it when compiling:
$ cargo build --target x86_64-unknown-linux-musl
This is the easiest method by far, but won't always work, especially when using native libraries, unless they can also be compiled statically.
Make a VM That Has an Older Version
This is a common approach. By using an OS with an outdated, GLIBC the binary will have GLIBC symbols that are compatible with it.
Use a Docker Container
This is probably the most convenient method, in my opinion. If you have Docker, you can just compile your project with a container that contains an old GLIBC. View the Rust contianer's README for compilation instructions. The command below will compile a project using Rust 1.67 and GLIBC 2.28 (which comes with buster):
$ docker run --rm --user "$(id -u)":"$(id -g)" -v "$PWD":/usr/src/myapp -w /usr/src/myapp rust:1.67-buster cargo build --release
I compiled this on Ubuntu 22.04 and tested it on Ubuntu 20.04.
To test further, I made sure the binary relied on another dynamic library (OpenSSL) and here's the result of ldd ./mybinary after compiling with the Docker container:
$ ldd ./mybinary
linux-vdso.so.1 (0x00007ffd98fdf000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fe49e248000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe49e22d000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe49e223000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe49e200000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe49e0b1000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe49e0ab000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe49deb7000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe49ea30000)
And this is what it looks like without the container:
$ ldd ./mybinary
linux-vdso.so.1 (0x00007ffd5d7b7000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fe85564c000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe85562c000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe855545000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe85531d000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe855f98000)
I used crosstools-ng to compile shadowsocks for my router, on my router, I can just ldd myExecutable to list the dependencies,
$ ssh root#my-router-ip
root#unknown:/tmp/home/root# ldd ./ss-server
libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0x2aabf000)
libm.so.0 => /lib/libm.so.0 (0x2ac23000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2ac37000)
libpthread.so.0 => /lib/libpthread.so.0 (0x2ac56000)
libc.so.0 => /lib/libc.so.0 (0x2ac78000)
libdl.so.0 => /lib/libdl.so.0 (0x2accb000)
ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
but right now I don't have my router with me, how do I check it?
tried with the tools from crosstools-ng compiled toolchain which used to compile the executable
$ mipsel-unknown-linux-uclibc-ldd /home/oglop/Downloads/ss-install/bin/ss-server
mipsel-unknown-linux-uclibc-ldd: no root given
Try `mipsel-unknown-linux-uclibc-ldd --help' for more information
$ mipsel-unknown-linux-uclibc-readelf -d /home/oglop/Downloads/ss-install/bin/ss-server
There is no dynamic section in this file.
I have already tried methods here in How to list library dependencies of a non-native binary?, none of them works.
I am having issues linking to a Haskell library we wrote. It goes wrong on Ubuntu, but not on Arch Linux. The error on Ubuntu we get is this:
/usr/bin/ld: warning: libHSdeepseq-1.3.0.0-ghc7.4.1.so, needed by /usr/lib/ghc/containers-0.4.2.1/libHScontainers-0.4.2.1-ghc7.4.1.so, not found (try using -rpath or -rpath-link)
/usr/lib/ghc/containers-0.4.2.1/libHScontainers-0.4.2.1-ghc7.4.1.so: undefined reference to 'deepseqzm1zi3zi0zi0_ControlziDeepSeq_zdfNFDataArrayzuzdcrnf1_info'
The issue seems to be caused by the fact that libHScontainers-0.4.2.1-ghc7.4.1.so is incorrectly linked as one can see by the output of ldd:
ldd /usr/lib/ghc/containers-0.4.2.1/libHScontainers-0.4.2.1-ghc7.4.1.so
linux-vdso.so.1 => (0x00007fffe95a2000)
libHSdeepseq-1.3.0.0-ghc7.4.1.so => not found
libHSbase-4.5.0.0-ghc7.4.1.so => not found
libHSghc-prim-0.2.0.0-ghc7.4.1.so => not found
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f89a5a59000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f89a569a000)
/lib64/ld-linux-x86-64.so.2 (0x00007f89a5fd8000)
Apparently the dependent libraries can not be found. They are installed. However if I do the same on Arch:
ldd /usr/lib/ghc-7.8.3/deepseq-1.3.0.2/libHSdeepseq-1.3.0.2-ghc7.8.3.so
linux-vdso.so.1 (0x00007fff09dfe000)
libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007fb8d3e96000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fb8d3b91000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007fb8d3988000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fb8d3784000)
libffi.so.6 => /usr/lib/libffi.so.6 (0x00007fb8d357b000)
libHSarray-0.5.0.0-ghc7.8.3.so => /usr/lib/ghc-7.8.3/deepseq-1.3.0.2/../array-0.5.0.0/libHSarray-0.5.0.0-ghc7.8.3.so (0x00007fb8d32e1000)
libHSbase-4.7.0.1-ghc7.8.3.so => /usr/lib/ghc-7.8.3/deepseq-1.3.0.2/../base-4.7.0.1/libHSbase-4.7.0.1-ghc7.8.3.so (0x00007fb8d2967000)
libHSinteger-gmp-0.5.1.0-ghc7.8.3.so => /usr/lib/ghc-7.8.3/deepseq-1.3.0.2/../integer-gmp-0.5.1.0/libHSinteger-gmp-0.5.1.0-ghc7.8.3.so (0x00007fb8d274c000)
libHSghc-prim-0.3.1.0-ghc7.8.3.so => /usr/lib/ghc-7.8.3/deepseq-1.3.0.2/../ghc-prim-0.3.1.0/libHSghc-prim-0.3.1.0-ghc7.8.3.so (0x00007fb8d24cf000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fb8d212c000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fb8d1f10000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007fb8d435f000)
The libraries are found.
As suggested I can solve this on Ubuntu by using -rpath in the application we try to link to the Haskell library. But this means that we have to do this for every Haskell package which seems wrong to me.
We can also fix this by adding a line to /etc/ld.so.conf.d/ghc.conf. But this also has to be done for every package and is not user-friendly.
A few questions I have:
What is the correct way to fix this?
Why are the packages in ghc-dynamic incorrectly linked?
Why is the linker able to find libHScontainers-0.4.2.1-ghc7.4.1.so but not libHSdeepseq-1.3.0.0-ghc7.4.1.so?
I strongly suspect that this is because the Haskell libraries installed by ghc have the locations of their dependencies (the RPATH field of their ELF header; you can verify using readelf -d) defined in terms of $ORIGIN. When library X depends on library Y, library X can indicate that library Y should be found in a location relative to its own location by using $ORIGIN. This is supported by the dynamic linker, but is not supported by the static linker.
(I'm speculating here:) Your library will define the location of its direct dependencies (in your case, I'm guessing, this includes containers) in terms of its own RPATH, which is not in terms of $ORIGIN. This is why the linker can find those, but not its transitive dependencies (again, I'm guessing, this includes deepseq in your case).
So why the difference between Arch Linux and Ubuntu? (Speculating further.) This is because unlike on Arch Linux, Ubunbu's linker uses --as-needed by default. You see, ghc will link your library against all its dependencies (including transitive ones), but then the linker will omit some of those dependencies because it doesn't directly depend on them. You could verify this by relinking with --no-as-needed.
Note that these errors by the static linkers really aren't errors, but warnings: it tries to resolve symbols, but it can't; but the dynamic linker will be able to anyway. So you can instruct the linker to ignore these errors (--unresolved-symbols=ignore-all) and all should be well.
I've been battling with adding explicit support in Cabal for generating Haskell libraries for use in C programs, and found the same problem. See https://github.com/haskell/cabal/pull/2540#issuecomment-95984067 for details.