Can I build only glibc malloc as a shared library? - malloc

I want to build glibc malloc as a shared library instead of it being part of libc.so
I'm not using any chroot but directly trying to build it.
When I make glibc as a normal build, it outputs the command that is being used to build malloc namely:
gcc malloc.c -c -std=gnu99 -fgnu89-inline -O2 -Wall -Winline -Wundef -Wwrite-strings -fmerge-all-constants -frounding-math -g -Wstrict-prototypes -fPIC -DMORECORE_CLEARS=2 -I../include -I/home/sharath.g/glibc-2.20/build/malloc -I/home/sharath.g/glibc-2.20/build -I../sysdeps/unix/sysv/linux/x86_64/64 -I../sysdeps/unix/sysv/linux/x86_64 -I../sysdeps/unix/sysv/linux/x86 -I../sysdeps/unix/sysv/linux/wordsize-64 -I../sysdeps/x86_64/nptl -I../sysdeps/unix/sysv/linux -I../sysdeps/nptl -I../sysdeps/pthread -I../sysdeps/gnu -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/x86_64 -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/x86_64/64 -I../sysdeps/x86_64/fpu/multiarch -I../sysdeps/x86_64/fpu -I../sysdeps/x86/fpu -I../sysdeps/x86_64/multiarch -I../sysdeps/x86_64 -I../sysdeps/x86 -I../sysdeps/ieee754/ldbl-96 -I../sysdeps/ieee754/dbl-64/wordsize-64 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/ieee754/flt-32 -I../sysdeps/wordsize-64 -I../sysdeps/ieee754 -I../sysdeps/generic -I.. -I../libio -I. -D_LIBC_REENTRANT -include ../include/libc-symbols.h -DPIC -DSHARED -o /home/sharath.g/glibc-2.20/build/malloc/malloc.o -MD -MP -MF /home/sharath.g/glibc-2.20/build/malloc/malloc.os.dt -MT /home/sharath.g/glibc-2.20/build/malloc/malloc.os
As you can see, malloc is built using -fPIC so I should be able to simply link it as a shared library.
However when I run this command
gcc -shared -Wl,-soname,libmalloc.so -shared -lpthread -lm -lrt -o /home/sharath.g/glibc-2.20/build/malloc/libmalloc.so /home/sharath.g/glibc-2.20/build/malloc/malloc.o
I get an error
relocation R_X86_64_PC32 against undefined symbol `__libc_multiple_threads' can not be used when making a shared object; recompile with -fPIC
I don't understand why this error shows up? clearly I've compiled malloc.c with -fPIC

I don't understand why this error shows up?
The symbol is referenced by malloc.o via inline assembly, like so:
# 69 "../sysdeps/unix/sysv/linux/x86_64/lowlevellock.h"
#define __lll_trylock_asm "cmpl $0, __libc_multiple_threads(%%rip)\n\t" "je 0f\n\t" "lock; cmpxchgl %2, %1\n\t" "jmp 1f\n\t" "0:\tcmpxchgl %2, %1\n\t" "1:"
As such, it generates R_X86_64_PC32 relocation (normal calls to external routines generate R_X86_64_PLT32 relocations when compiled with -fPIC). This form of assembly assumes that the symbol will be defined in the same ELF image.
In the normal build, this symbol is defined as a hidden symbol (meaning it's defined inside libc.so.6 and is not exported from it) in nptl/libc_multiple_threads.c.
Since you are not linking in libc_multiple_threads.o into your libmalloc.so, the symbol remains undefined, and the linker correctly complains: this symbol can't come from outside (wrong relocation for that) and isn't defined inside your libmalloc.so.
You might think that simply linking in libc_mutiple_threads.os would solve this, but you'll be wrong: your libmalloc.so would behave as if your process is single-threaded, regardless of whether it actually is or not.
TL;DR: what you are trying to do is unlikely to work, except by accident. It is very likely to be broken in multiple ways, some of which could be quite subtle.

Related

relocation R_X86_64_PC32 against symbol _ZTISt13runtime_error##GLIBCXX_3.4 can not be used when making a shared object; recompile with -fPIC

The project I am attempting compile is not in any way complex, and references nothing but the standard library and one self-contained library (everything compiles fine on another system). As indicated by the title, it can't even link against something in the standard library, due to things in there not having been compiled with -fPIC, supposedly. I didn't build it myself, nor do I want to, and reinstalling things with apt didn't seem to resolve the "recompile with -fPIC" issue.
I will say that I think one possible source of the issue is due to gcc-multilib or something being installed earlier, but I think that was purged. I don't know, maybe something was overwritten or a conflict arose. Might not even be related. Any ideas?
Running Ubuntu 18.04
g++ -I inc -I /usr/include/mono-2.0 -MMD -MF dep/Nonsense.d -std=c++17 -O3 -fno-stack-protector -fno-unroll-loops -fomit-frame-pointer -Wno-ignored-optimization-argument -c -o obj/Nonsense.o src/Nonsense.cpp
g++ -I inc -I /usr/include/mono-2.0 -MMD -MF dep/Socket.d -std=c++17 -O3 -fno-stack-protector -fno-unroll-loops -fomit-frame-pointer -Wno-ignored-optimization-argument -c -o obj/Socket.o src/Socket.cpp
g++ -shared -flto -o libNonsense.so obj/Nonsense.o obj/Socket.o -Llib -lenet
/usr/bin/x86_64-linux-gnu-ld: obj/Socket.o: relocation R_X86_64_PC32 against symbol `_ZTISt13runtime_error##GLIBCXX_3.4' can not be used when making a shared object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
makefile:22: recipe for target 'libNonsense.so' failed
make: *** [libNonsense.so] Error 1
Apparently, it is compulsory to compile with the -fPIC flag in 64bits platform. If you are making a 32-bits project with linkink with 32-bits library; you don't need -fPIC.
Indeed, without -fPIC, the dynamic linker recalculates adresses for global variables and functions (which are not known in advance in the step of compilation). However, in a 64-bits system, it is not possible to use this technique as it is required to guess in advance the size of adress.
PIC (Position Independent Code), a more sophisticated and complicated process is used. https://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64 for explanations.
As the error message says, you need to recompile with -fPIC. Your current compiler command does not show the -fPIC option:
g++ -I inc -I /usr/include/mono-2.0 -MMD -MF dep/Socket.d -std=c++17 -O3 -fno-stack-protector -fno-unroll-loops -fomit-frame-pointer -Wno-ignored-optimization-argument -c -o obj/Socket.o src/Socket.cpp
Same for -flto by the way—this flag also has to be specified when compiling in order to be effective.

Yocto/Qt5.6: Unable to build a sample Qt Application

I have a working Nitrogen6x board that runs on Yocto Krogoth-next build with core-image-sato. I have installed Qt Creator 3.5.1 (based on Qt 5.5.1) and I have added my kit for nitrogen6x board as per [Build & Install Qt5 toolchain] document. I have added a sample Qt Quick Application with component set chosen as Qt Quick 2.1 and tried to compile the source, I am noticing this below error.
19:04:25: Running steps for project untitled6...
19:04:25: Starting: "/opt/poky/2.1.1/sysroots/i686-pokysdk-linux/usr/bin/qt5/qmake" /home/test/untitled6/untitled6.pro -r -spec linux-g++
19:04:25: The process "/opt/poky/2.1.1/sysroots/i686-pokysdk-linux/usr/bin/qt5/qmake" exited normally.
19:04:25: Starting: "/usr/bin/make"
g++ -c -pipe -O2 -std=gnu++0x -Wall -W -D_REENTRANT -fPIC -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_GUI_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I../untitled6 -I. -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5 -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtQuick -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtGui -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtQml -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtNetwork -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtCore -I. -I/opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/qt5/mkspecs/linux-g++ -o main.o ../untitled6/main.cpp
/opt/poky/2.1.1/sysroots/i686-pokysdk-linux/usr/bin/qt5/rcc -name qml ../untitled6/qml.qrc -o qrc_qml.cpp
g++ -c -pipe -O2 -std=gnu++0x -Wall -W -D_REENTRANT -fPIC -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_GUI_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I../untitled6 -I. -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5 -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtQuick -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtGui -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtQml -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtNetwork -isystem /opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/QtCore -I. -I/opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/qt5/mkspecs/linux-g++ -o qrc_qml.o qrc_qml.cpp
g++ -Wl,-O1 -o untitled6 main.o qrc_qml.o -lQt5Quick -lQt5Gui -lQt5Qml -lQt5Network -lQt5Core -lGLESv2 -lpthread
/usr/bin/ld: cannot find -lQt5Quick
/usr/bin/ld: cannot find -lQt5Gui
/usr/bin/ld: cannot find -lQt5Qml
/usr/bin/ld: cannot find -lQt5Network
/usr/bin/ld: cannot find -lQt5Core
/usr/bin/ld: cannot find -lGLESv2
collect2: ld returned 1 exit status
make: *** [untitled6] Error 1
19:04:26: The process "/usr/bin/make" exited with code 2.
Error while building/deploying project untitled6 (kit: Nitrogen)
When executing step 'Make'
19:04:26: Elapsed time: 00:01.
All the Qt libraries are present under /sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/include/qt5/ and I have set the sysroot path correctly under my kit, but I am unable to figure out why is that my application is failing to reference those libraries. Any help on this is deeply appreciated. Thanks in advance.
The mkspec pointed to by your kit seem to be the wrong one (linux-g++), quoting from your build log
-I/opt/poky/2.1.1/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/qt5/mkspecs/linux-g++
When building for a nitrogen6x, it should likely be linux-oe-g++ instead. This is either set by default in the Qt version you selected, or can be set by modifying the Kit itself, in QtCreator's options, under "mkspec".
The error your's seeing is probably due to the fact the that the ABI & architecture of the libraries pointed are not compatible with the x86/64 code you compiled.
Also make sure to source Yocto's environment file before starting QtCreator if you encounter problems (namely compile error about "C" not found). This will setup the path to the cross-compiler. Example:
source /opt/poky/1.6.2/environment-setup-cortexa9hf-vfp-neon-poky-linux-gnueabi
~/Qt/Tools/QtCreator/bin/qtcreator

Use only local library with cross compiler

I'm trying to build one application with cross compiler on linux. Application is based on OpenEmbedded project. Compiler and all cross compiled libraries are located in build folder. But for one package build script is trying to use a library from my linux environment.
Command which is called from make script at the end:
/bin/sh ./arm-ABC-linux-gnueabi-libtool --tag=CXX --mode=link
arm-ABC-linux-gnueabi-g++ -march=armv7-a -mtune=cortex-a8
-mfpu=neon -mfloat-abi=softfp -g --std=c++0x -pthread -L/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/opt/my/lib
-Wl,-rpath-link,/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/opt/my/lib
-Wl,-O1 -Wl,--hash-style=gnu -L/opt/my/lib -o modbus_server_test modbus_server_test.o Serial.o SerialUtil.o Crc.o RtuFramer.o Pdu.o
SerialMaster.o
/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/usr/local/apr/lib/libapr-2.la
-lcrypt -luuid -lrt -lcrypt -lpthread -ldl -lexpat -lstdc++
Most important from here that is use -lexpat flag for libexpat.so.1.5.2 library. This library is cross compiled and located in the build folder, and at the same time, we have one in my linux environment.
As result I recieve the next error:
| arm-ABC-linux-gnueabi-libtool: link:
arm-ABC-linux-gnueabi-g++ -march=armv7-a -mtune=cortex-a8
-mfpu=neon -mfloat-abi=softfp -g --std=c++0x -pthread -Wl,-rpath-link -Wl,/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/opt/my/lib
-Wl,-O1 -Wl,--hash-style=gnu -o .libs/modbus_server_test modbus_server_test.o Serial.o SerialUtil.o Crc.o RtuFramer.o Pdu.o
SerialMaster.o
-L/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/opt/my/lib
-L/opt/my/lib /usr/local/apr/lib/libapr-2.so -L/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib
/usr/lib/x86_64-linux-gnu/libexpat.so
/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib/libuuid.so
-lrt -lcrypt -lpthread -ldl /home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib/libexpat.so
/home/ABC/build/build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib/libstdc++.so
-lm -pthread -Wl,-rpath -Wl,/usr/local/apr/lib -Wl,-rpath -Wl,/home/ABC/build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib
| /usr/lib/x86_64-linux-gnu/libexpat.so: file not recognized: File
format not recognized
| collect2: ld returned 1 exit status
| make[2]: *** [modbus_server_test] Error 1
From output I can see that it include two libraries: one from linux environment (/usr/lib/x86_64-linux-gnu/libexpat.so which is x86) and one from the build folder (build/sysroots/armv7a-ABC-linux-gnueabi/usr/lib/libexpat.so which is for ARM).At the end it complains about x86 type.
How can I exclude this library from searching in global environment? Why it even trying to use both of them?

Linking Rcpp to interp2d (GSL-type library)

I need some help with a linker error I get during installation of an Rcpp package on a linux system where I don't have admin rights. In a nutshell, I get this error:
relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
I have a file solve.cpp that uses external library interp2d, which in turn has a GSL dependency. I specify my dependencies via [[Rcpp::depends(RcppArmadillo,RcppGSL)]] and in the DESCRIPTION. My Makevars is like the one from the RcppGSL package, with the addition of the linterp2d flag:
PKG_CPPFLAGS = -W $(GSL_CFLAGS) $(LOCAL_INCLUDE)
PKG_LIBS += $(GSL_LIBS) $(LOCAL_LIBS) -linterp2d $(RCPP_LDFLAGS)
where I define the environement variables
export LOCAL_INCLUDE="-I/data/uctpfos/local/include/"
export LOCAL_LIBS="-L/data/uctpfos/local/lib/"
on the system.
I do R CMD INSTALL bkPackage and see:
g++ -I/cm/shared/apps/R/3.0.1/lib64/R/include -DNDEBUG -W -I/cm/shared/apps/gsl/1.15/include -I/data/uctpfos/local/include -fPIC -I/usr/local/include -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppArmadillo/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppGSL/include" -fpic -O3 -fPIC -c RcppExports.cpp -o RcppExports.o
g++ -I/cm/shared/apps/R/3.0.1/lib64/R/include -DNDEBUG -W -I/cm/shared/apps/gsl/1.15/include -I/data/uctpfos/local/include -fPIC -I/usr/local/include -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppArmadillo/include" -I"/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/RcppGSL/include" -fpic -O3 -fPIC -c solve.cpp -o solve.o
The problems appears after that in the linking step:
g++ -shared -L/usr/local/lib64 -o bkPackage.so RcppExports.o solve.o -L/cm/shared/apps/gsl/1.15/lib -lgsl -lgslcblas -lm -L/data/uctpfos/local/lib -linterp2d -L/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/lib -lRcpp -Wl,-rpath,/data/uctpfos/R/x86_64-unknown-linux-gnu-library/3.0/Rcpp/lib
The error follows as:
/usr/bin/ld: /data/uctpfos/local/lib/libinterp2d.a(interp2d_spline.c.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/data/uctpfos/local/lib/libinterp2d.a: could not read symbols: Bad value
As you can see I compiled this with -fPIC, so that can't be it.
It's complaining that libinterp2d.a was not compiled with -fPIC; are you sure that was also compiled with -fPIC on?
Also, it seems you both have -fpic and -fPIC in your flags; you probably just want -fPIC. I think R actually ensures that's on by default.

glibc error while building linux from scratch

The following error came up when i did 'make' of glibc-2.14.1 for LFS 7.1. I am using ubuntu as the host OS.
gcc ../sysdeps/unix/sysv/linux/syslog.c -c -std=gnu99 -fgnu89-inline -O2 -Wall -Winline
-Wwrite-strings -fmerge-all-constants -g -Wstrict-prototypes -mpreferred-stack-
boundary=2 -Wa,-mtune=i686 -fexceptions -I../include -I/mnt/lfs/sources/glibc-
build/misc -I/mnt/lfs/sources/glibc-build -I../sysdeps/i386/elf -
I../nptl/sysdeps/unix/sysv/linux/i386/i686 -I../sysdeps/unix/sysv/linux/i386/i686 -
I../nptl/sysdeps/unix/sysv/linux/i386 -I../sysdeps/unix/sysv/linux/i386 -
I../nptl/sysdeps/unix/sysv/linux -I../nptl/sysdeps/pthread -I../sysdeps/pthread -
I../sysdeps/unix/sysv/linux -I../sysdeps/gnu -I../sysdeps/unix/common -
I../sysdeps/unix/mman -I../sysdeps/unix/inet -I../sysdeps/unix/sysv/i386 -
I../nptl/sysdeps/unix/sysv -I../sysdeps/unix/sysv -I../sysdeps/unix/i386 -
I../nptl/sysdeps/unix -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/i386/i686/fpu -
I../sysdeps/i386/i686/multiarch -I../nptl/sysdeps/i386/i686 -I../sysdeps/i386/i686 -
I../sysdeps/i386/i486 -I../nptl/sysdeps/i386/i486 -I../sysdeps/i386/fpu -
I../nptl/sysdeps/i386 -I../sysdeps/i386 -I../sysdeps/wordsize-32 -
I../sysdeps/ieee754/ldbl-96 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/ieee754/flt-32 -
I../sysdeps/ieee754 -I../sysdeps/generic/elf -I../sysdeps/generic -I../nptl -I.. -
I../libio -I. -nostdinc -isystem /usr/lib/gcc/i686-linux-gnu/4.6/include -isystem
/usr/lib/gcc/i686-linux-gnu/4.6/include-fixed -isystem /tools/include -D_LIBC_REENTRANT
-include ../include/libc-symbols.h -o /mnt/lfs/sources/glibc-build/misc/syslog.o
-MD -MP -MF /mnt/lfs/sources/glibc-build/misc/syslog.o.dt -MT /mnt/lfs/sources/glibc-
build/misc/syslog.o
In file included from ../sysdeps/unix/sysv/linux/syslog.c:10:0:
../misc/syslog.c: In function '__vsyslog_chk':
../misc/syslog.c:144:9: warning: variable 'prioff' set but not used [-Wunused-but-set-
variable]
../misc/syslog.c:123:1: sorry, unimplemented: inlining failed in call to 'syslog':
function body not available
../misc/syslog.c:155:9: sorry, unimplemented: called from here
make[2]: *** [/mnt/lfs/sources/glibc-build/misc/syslog.o] Error 1
make[2]: Leaving directory `/mnt/lfs/sources/glibc-2.14.1/misc'
make[1]: *** [misc/subdir_lib] Error 2
make[1]: Leaving directory `/mnt/lfs/sources/glibc-2.14.1'
make: *** [all] Error 2
Can anyone tell me what to do?
For glibc 2.15, I had to do like ks1322:
$ ../glibc2.15/configure CFLAGS="-O2 -U_FORTIFY_SOURCE -fno-stack-protector"
$ make
The -O2 is required to recover the default optimisation, required to avoid a number of failures.
The -U_FORTIFY_SOURCE is required to avoid failure of gcc ../sysdeps/unix/sysv/linux/syslog.c (In function ‘__vsyslog_chk’: [...] inlining failed in call to ‘syslog’)
The -fno-stack-protector is required to avoid failure of gcc [...] elf/dl-allobjs.os libc_pic.a ((init-first.os):(.data+0x0): multiple definition of '__libc_multiple_libcs').
With these CFLAGS, I get just a warning (errlist.c count 133 inflated to GLIBC_2.12 count 134 (old errno.h?)).
P.S. It is also very important to not use the gold linker (ld.gold, from the binutils-gold package), because libc's config will reject it. This is easily done by changing the /usr/bin/ld symbolic link from ld.gold to ld.bfd.
After make failed, the following worked for me:
make clean
make -j2 CFLAGS="-U_FORTIFY_SOURCE -O2 -fno-stack-protector"
See details in this blog http://www.yonch.com/tech/78-compiling-glibc.
glibc build issues, inlining failed
Need to add -U_FORTIFY_SOURCE to the gcc invocation.
#avinash - try the following ...
1) re-run configure ... CFLAGS="-O2 -U_FORTIFY_SOURCE" ./configure
2) re-run make ... CFLAGS="-O2 -U_FORTIFY_SOURCE" make

Resources