How to link .so library properly in C++? - linux

I have project structure like this,
a.pb.h --- includes --> protobuf.h
b.grpc.pb.h --- includes --> a.pb.h & grpcpp.h
Also there are a.pb.cc and b.grpc.cc files.
A C++ wrapper with extern C which is wrapper.cc and wrapper.h which includes b.grpc.pb.h and grpcpp.h.
The function inside extern C is char* helloWorld(const char*, const char*, const char*);
Creating .o of a.pb.h and b.grpc.pb.h:
g++ -fpic -std=c++11 `pkg-config --cflags protobuf grpc` -c -o a.pb.o a.pb.cc
g++ -fpic -std=c++11 `pkg-config --cflags protobuf grpc` -c -o b.grpc.pb.o b.grpc.pb.cc
Steps to create libcombined.so:
The grpc and protobuf so are already provided under /usr/local/lib.
First created .so of a.pb.o and b.grpc.pb.o to compile wrapper file as:
g++ -shared -o libcombined.so *.o
Compiled wrapper as:
g++ -fpic wrapper.cc -l:./libcombined.so -c -o wrapper.o -std=c++11
.so of a.pb.o, b.grpc.pb.o and wrapper.o as libcombined.so:
g++ -shared -o libcombinedwrapper.so *.o
Compiled main.c as:
gcc main.c -l:./libcombinedwrapper.so -o main -ldl
I am calling helloWorld from my main.c file which is:
#include <stdio.h>
#include <dlfcn.h>
int main(){
char* (*fn)(const char*,const char*,const char*);
void *handle = dlopen("path_to/libcombined.so",RTLD_NOW);
if(handle==NULL){
fprintf(stderr, "Error: %s\n", dlerror());
}
fn = (char* (*)(const char*,const char*,const char*))dlsym(handle, "helloWorld");
if (!fn) {
/* no such symbol */
fprintf(stderr, "Error: %s\n", dlerror());
dlclose(handle);
return 0;
}
char* msg = fn("asd","asdas","asdasd");
printf("%s",msg);
return 0;
}
Error after executing: ./main
Error: path_to/libcombinedwrapper.so: undefined symbol: _ZN6google8protobuf2io20ZeroCopyOutputStream15WriteAliasedRawEPKvi
Error: ./main: undefined symbol: helloWorld
Segmentation fault (core dumped)
The first above error is from symbol from protobuf.h file.
Can someone please suggest what I am doing wrong while linking or is there something I am doing wrong in main.c file?

g++ -shared -o libcombined.so *.o
You need to also link in all dependencies of the objects (libgrpc here).
You can add -Wl,--no-allow-shlib-undefined to verify that libcombined.so is linking everything it needs.
P.S. To avoid core dump, you should exit or return once dlopen fails.
P.P.S. It is generally a very bad idea(TM) to link *.o. Use proper Makefile to avoid unnecessary compilations and explicitly list objects that you are intending to put into libcombined.so.

Related

gcc undefined reference to `pthread_atfork'

I'm trying to compile a recent version of openssl on an optware-ng install with gcc 7.2.0 on x86_64 architecture
The system has 2 libpthreads, one in /lib:/usr/lib:/lib64:/usr/lib64 (link to each other). And one in the optware-ng install in /opt/lib:/opt/lib64 (also link to each other)
Compiling goes fine, but at link time I get the following error:
gcc -pthread -m64 -Wa,--noexecstack -Wall -O3 -L. -o apps/openssl apps/asn1pars.o apps/ca.o apps/ciphers.o apps/cms.o apps/crl.o apps/crl2p7.o apps/dgst.o apps/dhparam.o apps/dsa.o apps/dsaparam.o apps/ec.o apps/ecparam.o apps/enc.o apps/engine.o apps/errstr.o apps/gendsa.o apps/genpkey.o apps/genrsa.o apps/nseq.o apps/ocsp.o apps/openssl.o apps/passwd.o apps/pkcs12.o apps/pkcs7.o apps/pkcs8.o apps/pkey.o apps/pkeyparam.o apps/pkeyutl.o apps/prime.o apps/rand.o apps/rehash.o apps/req.o apps/rsa.o apps/rsautl.o apps/s_client.o apps/s_server.o apps/s_time.o apps/sess_id.o apps/smime.o apps/speed.o apps/spkac.o apps/srp.o apps/storeutl.o apps/ts.o apps/verify.o apps/version.o apps/x509.o apps/libapps.a -lssl -lcrypto -ldl -pthread
./libcrypto.so: undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
Adding -v to the output of gcc:
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/lto-wrapper
Target: x86_64-buildroot-linux-gnu
Configured with: ../gcc-7.2.0/configure --build=x86_64-pc-linux-gnu --host=x86_64-buildroot-linux-gnu --target=x86_64-buildroot-linux-gnu --prefix=/opt --disable-nls --disable-static --with-as=/opt/bin/as --with-ld=/opt/bin/ld --enable-languages=c,c++,go --disable-multilib --disable-werror --with-gxx-include-dir=/opt/include/c++/7.2.0 --disable-__cxa_atexit --with-gnu-ld --disable-libssp --disable-multilib --enable-libquadmath --enable-tls --disable-libmudflap --enable-threads --without-isl --without-cloog --disable-decimal-float --with-arch=nocona --enable-shared --disable-libgomp --with-gmp=/home/jenkins/Optware-ng/buildroot-x86_64/staging/opt --with-mpfr=/home/jenkins/Optware-ng/buildroot-x86_64/staging/opt --with-mpc=/home/jenkins/Optware-ng/buildroot-x86_64/staging/opt --with-system-zlib
Thread model: posix
gcc version 7.2.0 (GCC)
COMPILER_PATH=/opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/:/opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/:/opt/libexec/gcc/x86_64-buildroot-linux-gnu/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/bin/
LIBRARY_PATH=/opt/lib64/../lib64/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/opt/lib64/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/lib/:/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-pthread' '-m64' '-Wall' '-O3' '-L.' '-o' 'apps/openssl' '-pthread' '-mtune=generic' '-march=nocona'
/opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/collect2 -plugin /opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/liblto_plugin.so -plugin-opt=/opt/libexec/gcc/x86_64-buildroot-linux-gnu/7.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/cc5Txvvm.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr -m elf_x86_64 -dynamic-linker /opt/lib64/ld-linux-x86-64.so.2 -o apps/openssl /opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/lib/crt1.o /opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/lib/crti.o /opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/crtbegin.o -L. -L/opt/lib64/../lib64 -L/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0 -L/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/lib64 -L/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/lib -L/opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../.. -rpath=/opt/lib64 apps/asn1pars.o apps/ca.o apps/ciphers.o apps/cms.o apps/crl.o apps/crl2p7.o apps/dgst.o apps/dhparam.o apps/dsa.o apps/dsaparam.o apps/ec.o apps/ecparam.o apps/enc.o apps/engine.o apps/errstr.o apps/gendsa.o apps/genpkey.o apps/genrsa.o apps/nseq.o apps/ocsp.o apps/openssl.o apps/passwd.o apps/pkcs12.o apps/pkcs7.o apps/pkcs8.o apps/pkey.o apps/pkeyparam.o apps/pkeyutl.o apps/prime.o apps/rand.o apps/rehash.o apps/req.o apps/rsa.o apps/rsautl.o apps/s_client.o apps/s_server.o apps/s_time.o apps/sess_id.o apps/smime.o apps/speed.o apps/spkac.o apps/srp.o apps/storeutl.o apps/ts.o apps/verify.o apps/version.o apps/x509.o apps/libapps.a -lssl -lcrypto -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/crtend.o /opt/lib/gcc/x86_64-buildroot-linux-gnu/7.2.0/../../../../x86_64-buildroot-linux-gnu/lib/crtn.o
./libcrypto.so: undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
which seem to indicate in LIBRARY_PATH that it is first looking in /opt/lib64, which should be ok.
The output of
nm -g /opt/lib64/libpthread.so | grep pthread_atfork
000000000000edd0 T pthread_atfork#GLIBC_2.2.5
indicates that the method should be there.
The following piece of C code, creates the same link issue on this system, but compiles/links fine on other systems:
// compile with: gcc -pthread mycode.c
#include <stdio.h>
#include <pthread.h>
int main() {
pthread_t f2_thread, f1_thread;
void *f2(), *f1(), prepare();
int i1,i2;
i1 = 1;
i2 = 2;
// int pthread_atfork(void (*prepare)(void), void (*parent)(void), void (*child)(void));
pthread_create(&f1_thread,NULL,f1,&i1);
pthread_atfork(&prepare,NULL,NULL);
pthread_create(&f2_thread,NULL,f2,&i2);
pthread_join(f1_thread,NULL);
pthread_join(f2_thread,NULL);
}
void prepare() {
printf("prepare");
}
void *f1(int *x) {
int i;
i = *x;
sleep(1);
printf("f1: %d",i);
pthread_exit(0);
}
void *f2(int *x){
int i;
i = *x;
sleep(1);
printf("f2: %d",i);
pthread_exit(0);
}
It seems to me that gcc is somehow still using the /lib version, but I'm stuck.
I've tried adding
-Wl,-rpath=/opt/lib64
But it doesn't help. Any suggestions?

Problem running "Hello World" linux module

I'm trying to compile and run a "Hello World" module from the book "Linux Device Drivers"
the program ~/ldd3/hello.c I'm trying to compile is:
/*
* $Id: hello.c,v 1.5 2004/10/26 03:32:21 corbet Exp $
*/
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("Dual BSD/GPL");
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
And the Makefile is:
obj-m += hello.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
When I run make, the compilation seems ok:
root#deb:/home/deb/ldd3# make
make -C /lib/modules/4.14.86/build M=/home/deb/ldd3 modules
make[1]: Entering directory '/home/deb/src/linux-4.14.86'
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory '/home/deb/src/linux-4.14.86'
But when I run :
root#deb:/home/deb/ldd3# insmod ./hello.ko
root#deb:/home/deb/ldd3#
the "Hello World" message doesnt get printed.
Nor do I get a message printed when I run
rmmod hello
Can you tell me why?
Thanks
In order to see kernel messages, you can use dmesg.
Alternatively, you can see the syslog tail var/log/syslog.

Linking QSsh Botan-2 std::thread

I compiled successfully BOTAN-2 (botan.exe BOTAN-test.exe runnable).
When I build QSsh's Project under Qt Creator I've got a linking error on std::thead (Compiler MinGW32)
rsa.cpp:-1: erreur : undefined reference to std::thread::_State::~_State()
rsa.cpp:-1: erreur : undefined reference to std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)())
A added unsuccessfully in the file qssh/src/libs/ssh.pro :
LIBS += -lbotan-2
LIBS += -lpthread
QMAKE_CXXFLAGS += -std=c++11 -pthread
QMAKE_CFLAGS += -std=c++11 -pthread
Note : I made a simple program ("hello world" style) using pthread with success.
g++ main.cpp -o build32/prog.exe -std=c++11 -pthread (MSYS2/MinGW32)
What's wrong with QSsh ?
Thanks

Building pjSip 2.5 - unable to undefined reference to 'pj_jni_jvm'

Trying to build pjSIP from sources for Android, using following guide - https://trac.pjsip.org/repos/wiki/Getting-Started/Android
Checkout sources of pjsip (2.5)
export ANDROID_NDK_ROOT=/home/dev/android/android-ndk-r9b
export TARGET_ABI=armeabi
export APP_PLATFORM=android-13
./configure-android --use-ndk-cflags --disable-video --disable-l16-codec --disable-gsm-codec --disable-g722-codec --disable-g7221-codec --disable-ssl --disable-ilbc-codec --disable-tls
configuration is completed successfully.
make dep && make clean && make
And I got following error - >
../src/pj/guid_android.c:43: error: undefined reference to 'pj_jni_jvm'
../src/pj/guid_android.c:117: error: undefined reference to 'pj_jni_jvm'
The command invoked by linked which leads to this error
/home/dev/android/android-ndk-r9b/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc -o ../bin/pjlib-test-arm-unknown-linux-androideabi \
output/pjlib-test-arm-unknown-linux-androideabi/main.o output/pjlib-test-arm-unknown-linux-androideabi/activesock.o output/pjlib-test-arm-unknown-linux-androideabi/atomic.o output/pjlib-test-arm-unknown-linux-androideabi/echo_clt.o output/pjlib-test-arm-unknown-linux-androideabi/errno.o output/pjlib-test-arm-unknown-linux-androideabi/exception.o output/pjlib-test-arm-unknown-linux-androideabi/fifobuf.o output/pjlib-test-arm-unknown-linux-androideabi/file.o output/pjlib-test-arm-unknown-linux-androideabi/hash_test.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_perf.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_udp.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_unreg.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_tcp.o output/pjlib-test-arm-unknown-linux-androideabi/list.o output/pjlib-test-arm-unknown-linux-androideabi/mutex.o output/pjlib-test-arm-unknown-linux-androideabi/os.o output/pjlib-test-arm-unknown-linux-androideabi/pool.o output/pjlib-test-arm-unknown-linux-androideabi/pool_perf.o output/pjlib-test-arm-unknown-linux-androideabi/rand.o output/pjlib-test-arm-unknown-linux-androideabi/rbtree.o output/pjlib-test-arm-unknown-linux-androideabi/select.o output/pjlib-test-arm-unknown-linux-androideabi/sleep.o output/pjlib-test-arm-unknown-linux-androideabi/sock.o output/pjlib-test-arm-unknown-linux-androideabi/sock_perf.o output/pjlib-test-arm-unknown-linux-androideabi/ssl_sock.o output/pjlib-test-arm-unknown-linux-androideabi/string.o output/pjlib-test-arm-unknown-linux-androideabi/test.o output/pjlib-test-arm-unknown-linux-androideabi/thread.o output/pjlib-test-arm-unknown-linux-androideabi/timer.o output/pjlib-test-arm-unknown-linux-androideabi/timestamp.o output/pjlib-test-arm-unknown-linux-androideabi/udp_echo_srv_sync.o output/pjlib-test-arm-unknown-linux-androideabi/udp_echo_srv_ioqueue.o output/pjlib-test-arm-unknown-linux-androideabi/util.o -lpj-arm-unknown-linux-androideabi -nostdlib -L/home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib -L/home/dev/android/android-ndk-r9b/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi/ -lm /home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib/crtbegin_so.o -lgnustl_static -lc -lgcc -ldl -lOpenSLES -llog -L/home/dev/android/pjproject-2.5/pjlib/lib -L/home/dev/android/pjproject-2.5/pjlib-util/lib -L/home/dev/android/pjproject-2.5/pjnath/lib -L/home/dev/android/pjproject-2.5/pjmedia/lib -L/home/dev/android/pjproject-2.5/pjsip/lib -L/home/dev/android/pjproject-2.5/third_party/lib -nostdlib -L/home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib -L/home/dev/android/android-ndk-r9b/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi/
Code of guid_androind.c
#include <pj/guid.h>
#include <pj/log.h>
#include <pj/string.h>
#include <jni.h>
extern JavaVM *pj_jni_jvm;
static pj_bool_t attach_jvm(JNIEnv **jni_env)
{
if ((*pj_jni_jvm)->GetEnv(pj_jni_jvm, (void **)jni_env,
JNI_VERSION_1_4) < 0)
{
if ((*pj_jni_jvm)->AttachCurrentThread(pj_jni_jvm, jni_env, NULL) < 0)
{
jni_env = NULL;
return PJ_FALSE;
}
return PJ_TRUE;
}
return PJ_FALSE;
}
#define detach_jvm(attached) \
if (attached) \
(*pj_jni_jvm)->DetachCurrentThread(pj_jni_jvm);
Struggling with this error for whole day, any clue is highly appreciated. Meanwhile I'm able to build version 2.3 without any errors, but not 2.5
I believe you missed a step.
https://trac.pjsip.org/repos/wiki/Getting-Started/Android#BuildPreparation
Set your config_site.h to the following:
/* Activate Android specific settings in the 'config_site_sample.h' */
#define PJ_CONFIG_ANDROID 1
#include <pj/config_site_sample.h>

Kernel module with multiple files - unknown symbol

Hello stackoverflowers :)
For the last several hours I've been trying to compile+load a multiple file module. The compilation emits a strange warning and the module fails to load. Here are the module, Makefile, compilation output and dmesg.
header:
// header.h
#ifndef _HEADER_H
#define _HEADER_H
void do_module_func(void);
void do_other_func(void);
#endif
'main' module file:
//mymodule.c
#include <linux/module.h>
#include <linux/kernel.h>
#include "header.h"
void do_module_func(void)
{
printk(KERN_INFO "module_func\n");
}
static int mymodule_init(void)
{
printk(KERN_INFO "Hello world\n");
do_other_func();
return 0;
}
module_init(mymodule_init);
static void mymodule_exit(void)
{
printk(KERN_INFO "Goodbye, cruel world\n");
}
module_exit(mymodule_exit);
MODULE_LICENSE("GPL")
other c file, which calls do_module_func() that sits in the 'main' module
//other_file.c
#include "header.h"
#include <linux/kernel.h>
void do_other_func(void)
{
printk(KERN_INFO "other_func\n");
do_module_func();
}
Makefile
//Makefile
obj-m := mymodule.o
mymodule-objs := other_file.o
CROSS:=arm-unknown-linux-gnueabi-
KERNEL:= ~/work/linux-davinci-2.6.38/
ARCH:=arm
PWD:=$(shell pwd)
all:
$(MAKE) CROSS_COMPILE=$(CROSS) ARCH=$(ARCH) -C $(KERNEL) M=$(PWD) modules
clean:
$(MAKE) CROSS_COMPILE=$(CROSS) ARCH=$(ARCH) -C $(KERNEL) M=$(PWD) clean
I'm cross compiling but I believe this shouldn't be a problem.
make output:
make CROSS_COMPILE....
make[1]: Entering directory .../linux-davinci-2.6.38
CC [M] .../other_file.o
LD [M] .../mymodule.o
Building modules, stage 2.
MODPOST 1 modules
WARNING: "do_module_func" [.../mymodule.o] undefined! <--- warning here
CC .../mymodule.mod.o
LD [M] .../mymodule.ko
make[1]: Leaving directory .../linux-davinci-2.6.38
insmod output:
can't insert 'mymodule.ko': unknown symbol in module, or unknown parameter
dmesg:
mymodule: Unknown symbol do_mdule_func (err 0)
Thus the module compiles with a (linkage?) warning and the module doesn't load.
Now, I see that in the make output there appears to be a linkage attempt after compiling other_file.c, but shouldn't there be a compilation of also mymodule.c before the linkage?
Thanks! :)
Turns out the problem was in the Makefile. The 'trick' is that you define in obj-m the module that will be compiled (into a .ko) and in the -objs you write all the source files.
Thus the definitions in this Makefile turn to:
obj-m := moduleko.o
moduleko-objs := other_file.o mymodule.o
and this is compiled into moduleko.ko.
This is because file_2 required file_1 symbol reference for building file_2 as LKM.
To overcome this, build file_1(LKM) and place the Module.symvers of file_1 in file_2 location. And build the file_2 again.
all:
$(MAKE) CROSS_COMPILE=$(CROSS) ARCH=$(ARCH) -C $(KERNEL) M=$(PWD) modules
instead try like this
$(MAKE) -C $(KERNEL) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS) M=$(PWD) /
this will run each file and links with object.hope this would solve your problem

Resources