When compiling this simple program: (with clang++-11 -fmodules main.cpp -o main.o -std=c++2a)
// main.cpp
import "pch1.h";
int main() {
std::cout << "Hello" << std::endl;
}
I get the following error message:
main.cpp:3:8: error: header file "pch1.h" (aka './pch1.h') cannot be imported because it is not known to be a header unit
Obviously I need to do something with the header file before using it, but what flags does clang expect?
The question is twofold:
What flags do clang want to compile/precompile the header unit
What flags do clang want to feed the header unit into main.cpp
After spending many hours digging in CLang's code and options I didn't find a way how to create and use header units.
But I found a fully compatible way of importing headers as modules. I describe it here, and here is description of STD headers case.
So according to linked answers import <iostream>; is totally equivalent to doing import std_mod;. In both cases all definitions are exported and all macros are preserved. Also in both cases you don't need any modifications at all to your (or std) headers.
Related
Shared objects (*.so) in Unix-like systems are inefficient because of the symbol interposition: Every access to a global variable inside the .so needs a GOT lookup, and every call from one function to another inside the .so needs a PLT lookup. I was therefore happy to see that gcc version 5.1 has added the option -fno-semantic-interposition. However, when I try to make a .so where one function calls another without using a PLT, I get the error message:
relocation R_X86_64_PC32 against symbol `functionname' can not be used when making a shared object; recompile with -fPIC
I expected the option -fno-semantic-interposition to eliminate this error message, but it doesn't. -mcmodel=large doesn't help either.
The reference to the function is indeed position-independent, which the error message actually confirms (R_X86_64_PC32 means PC-relative 32-bit relocation in 64-bit mode). -fPIC does not really mean position-independent, as the name would imply, it actually means use GOT and PLT.
I cannot use __attribute__((visibility ("hidden"))) because the called function and the caller are compiled in seperate files (caller is in C++, called function is in assembly).
I tried to make an assembly listing to see what the option -fno-semantic-interposition does. I found out that it makes a reference to a local alias when one function calls another in the same file, but it still uses a PLT when calling a function in another file.
(g++ version is 5.2.1 Ubuntu, 64-bit mode).
Is there a way to make the linker accept a cross-reference inside a .so without the GOT/PLT lookup?
Is there a way to make the linker accept a cross-reference inside a .so without the GOT/PLT lookup?
Yes: attribute((visibility("hidden"))) is exactly the way to do it.
I cannot use attribute((visibility ("hidden"))) because the called function and the caller are compiled in seperate files
You are confused: visibility("hidden") means that the symbol will not be exported from the shared library, when it is finally linked. But the symbol is global and visible across multiple translation units before that final link.
Proof:
$ cat t1.c
extern int foo() __attribute__((visibility("hidden")));
int main() { return foo(); }
$ cat t2.c
int foo() __attribute__((visibility("hidden")));
int foo() { return 42; }
$ gcc -c -fPIC t1.c t2.c
$ gcc -shared t1.o t2.o -o t.so
$ nm -D t.so | grep foo
$
I tried to make an assembly listing to see what the option -fno-semantic-interposition does. I found out that it makes a reference to a local alias when one function calls another in the same file, but it still uses a PLT when calling a function in another file.
If you read the discussion in gcc-patches, you'll see that the -fno-semantic-interposition is about allowing inlining of possibly interposable functions, not about the way they are actually called when not inlined.
What is the purpose of the features.h header? Why and when can it be used in my code?
Does it define source features supported by the system?
Or does it define some additional things which must be defined depending on other defines?
The features.h header file provides various macro definitions that indicate standard conformance to other header files, i.e. which features (hence the name) should be turned on or off depending on which standard the user wishes to use.
Most C/C++ compilers have command line options to handle standards conformance. Let's take GCC as an example: when you pass the -std=gnu9x option, you ask for the GNU dialect of the C99 standard. The features.h header makes sure that all other headers that include it will turn exactly those features on or off that are needed to support this particular dialect. This is achieved by #define -ing or #undef - ing some "intermediate" macros.
As a bonus, features.h also provides the glibc version information macros as well, and various other bits & bobs.
I have grepped POSIX 7 as explained at: https://unix.stackexchange.com/questions/340285/install-the-latest-posix-man-pages/483198#483198 and there are no hits for features.h, so it must be a glibc extension only.
In glibc 2.28, it is present at include/features.h.
One of the interesting things that it defines are version macros:
#include <stdio.h>
#include <features.h>
int main(void) {
printf("__GLIBC__ %u\n", __GLIBC__);
printf("__GLIBC_MINOR__ %u\n", __GLIBC_MINOR__);
return 0;
}
Ubuntu 16.04, which has glibc 2.23, this outputs:
__GLIBC__ 2
__GLIBC_MINOR__ 23
See also: Check glibc version for a particular gcc compiler
Also, this header seems to get included in most / all glibc headers, which might allow you to check if glibc is being used: How to tell if glibc is used but TODO I couldn't find a documentation for that.
From features.h File Reference
Defines on whether to include algorithm variants.
Less variants reduce executable size and compile time. This file is a GNU parallel extension to the Standard C++ Library.
So this file will include some algorithm listed in the reference page.
In general if you need to use any of the variables or functions defined in a header file, you need to include it in your program. This rule is valid for features.h also. You can see a URL for features.h for your reference below:
http://repo-genesis3.cbi.utsa.edu/crossref/heccer/usr/include/features.h.html
I have just downloaded MinGW 64 bit for Windows (I'm not quite used to Linux) on http://sourceforge.net/projects/mingw-w64/?source=dlp
When I compile the following code, I get no errors:
#include <iostream>
int main()
{
std::cout << "Code 64 bit :D !" << std::endl;
return 0;
}
Also, when I execute the program in Code::Blocks, eveything works perfectly (using cb_console_runner.exe). The problem occurs when I execute my program outside Code::Blocks. I get the error "The application couldn't start properly (0xc000007b)". By the way, I translated the error from french. Do you have any idea what is the problem or how I can debug this? Oh I forgot: I don't get any errors when code is empty (i.e. when int main(){return0;}).
Thank you!
I've seen this happen here and there when GCC's standard C++ library (libstdc++) (or libgcc as well) is not in your %PATH%. Furthermore, this may also be the case that you have a mismatched version (i.e. a 32 bit version) of the same DLL in your path. Use something like Dependency Walker to determine this.
This doesn't happen when your program is empty since you are not using any symbols from the standard C++ library and thus it is not linked with your application. However when you use std::cout that references a symbol defined in the C++ library and thus it must be linked.
I have a code A that is statically linked against one version of mpich. Now comes library B, which is used by A via dlopen(). B depends on mpich as well, but is linked dynamically against it.
The problem is that now, in order for B to take advantage of mpi distribution, needs to access the communicator currently handled by A. This communicator has been created by A static version of mpich, When B invokes MPI routines, it will use a dynamic version of MPI which is not guarateed to be compatible with the static version attached to A.
This is the overall picture. I think that the only solution is to have mpich dynamically linked for both A and B. What I am not fully understanding is however the following:
how does the linker handle shared objects dependencies when dlopening? Will I have two instances of mpich in VM also with dynamic linking, or is the linker smart enough to realize that the symbols required by the dlopened B are already in the address space and will resolve against those.
Is it possible to tell the linker: when you dlopen this library, don't go fetch the dynamic dependency, but resolve it with the static symbols that are already provided by A
In short: it depends on dlopen options. By default, if a symbol needed by the requested library already exists in the global scope, it will be reused (this is what you want). But you can bypass this behavior with RTLD_DEEPBIND, with this flag, the dependencies won't be reused from the global scope, and will be loaded a second time.
Here is some code to reproduce your situation and demo the effect of this flag.
Let's make a common library that will be used by both lib A and program B. This library will exist in two versions.
$ cat libcommon_v1.c
int common_func(int a)
{
return a+1;
}
$ cat libcommon_v2.c
int common_func(int a)
{
return a+2;
}
Now let's write lib A that uses libcommon_v2:
$ cat liba.c
int common_func(int a);
int a_func(int a)
{
return common_func(a)+1;
}
And finally program B that dynamically links to libcommon_v1 and dlopens lib A:
$ cat progb.c
#include <stdio.h>
#include <dlfcn.h>
int common_func(int a);
int a_func(int a);
int main(int argc, char *argv[])
{
void *dl_handle;
int (*a_ptr)(int);
char c;
/* just make sure common_func is registered in our global scope */
common_func(42);
printf("press 1 for global scope lookup, 2 for deep bind\n");
c = getchar();
if(c == '1')
{
dl_handle = dlopen("./liba.so", RTLD_NOW);
}
else if(c == '2')
{
dl_handle = dlopen("./liba.so", RTLD_NOW | RTLD_DEEPBIND);
}
else
{
printf("wrong choice\n");
return 1;
}
if( ! dl_handle)
{
printf("dlopen failed: %s\n", dlerror());
return 2;
}
a_ptr = dlsym(dl_handle, "a_func");
if( ! a_ptr)
{
printf("dlsym failed: %s\n", dlerror());
return 3;
}
printf("calling a_func(42): %d\n", (*a_ptr)(42));
return 0;
}
Let's build and run all the things:
$ export LD_LIBRARY_PATH=.
$ gcc -o libcommon_v1.so -fPIC -shared libcommon_v1.c
$ gcc -o libcommon_v2.so -fPIC -shared libcommon_v2.c
$ gcc -Wall -g -o progb progb.c -L. -lcommon_v1 -ldl
$ gcc -o liba.so -fPIC -shared liba.c -L. -lcommon_v2
$ ./progb
press 1 for global scope lookup, 2 for deep bind
1
calling a_func(42): 44
$ ./progb
press 1 for global scope lookup, 2 for deep bind
2
calling a_func(42): 45
We can clearly see that with default options, dlopen reuses the symbol common_func that was present in program B and that with RTLD_DEEPBIND, libcommon was loaded again and library A got its own version of common_func.
You did not say which Toolchain (GCC, LLVM, MSC, etc.) you are using, the most helpful answer will depend upon this information.
May I suggest you look at "GCC Exception Frames" http://www.airs.com/blog/archives/166 .
If that is helpful then the Gold Linker, which is available for GCC and LLVM, supports 'Link Time Optimization' and can run in "Make" with DLLTool http://sourceware.org/binutils/docs/binutils/dlltool.html .
Indeed it is possible to have both Static and Dynamic Code call each other, the Computer does not care; it will 'run' anything it is fed -- whether that ends up working exactly the way you want or HCFs depends on correct Code and correct Linker commands.
Using a Debugger will not be fun. It would be best to mangle the names prior to Linking so that when debugging you can see from which module the Code came. Once it is up and running you can undef the Mangle and have the same-named Functions Link (to ensure it still functions).
Compiler / Linker Bugs will not be your friend.
This sort of scenario (Static and Dynamic Linking) occurs more often with MinGW and Cygwin where some of the Libraries are Static yet a Library you download from the Internet is only available in Dynamic form (without Source).
If the Library is from two different Compiler Toolchains then other issues arise, see this StackOverflow Thread: " linking dilemma (undefined reference) between MinGW and MSVC. MinGW fails MSVC works ".
It would be best to simply get the newest version of the Library from the source and compile the whole thing yourself, rather than rely on trying to cobble together bits and pieces from different sources (though it is possible to do that).
You can even load the Dynamic Library and call it (statically) and then reload portions of it later.
How tight are you on Memory and how fast do you want Functions to run, if everything is in Memory your Program can transfer execution to called Functions straight away, if you are swapping a portion of your Code to VM your execution times will really take a hit.
Running a Profiler on your Code will help decide what portions of a Library to load if you want to do 'dynamic dynamic linking' (full control of your dyna-linking by loading a Dynamic Library so it can be used Statically). This is the stuff that headaches and nightmare are made of. GL.
I have a file with the void initGui() function in it. It does stuff.
I also have a .so shared library made with that file.
The problem is, when I try to launch a dlsym(..., "initGui"), dlerror() tells me that it didn't found the symbol (of course, I used dlopen to open it). So I tried to nm my shared lib. I "understood" that _Z7initGuiiii might be what I'm looking for. So I tried to dlsym it ... And it worked.
Please can someone tell me how to have clean symbols in my shared object library ?
I compile with g++ -Wall -Wextra -Werror -c -fPIC.
The usual practice when dlsym-ing inside some dlopen-ed shared library coded in C++ is to have the convention that those seeked symbols (those that you are dlsym-ing) are declared extern "C". Then their name is easily visible with dlsym. So you need to declare
extern "C" void initGui(void);
and then to do
typedef void initguiroutine_sig_t(void);
initguiroutine_sig_t* initguiptr = dlsym(dlhandle,"initGui");
if (!initguiptr) {
fprintf(stderr, "initGui not found: %s\n", dlerror());
exit (EXIT_FAILURE);
};
// later, call initguiptr like
(*initguiptr) ();
I don't recommend understanding in great details how your C++ name mangling works, it is not very well defined, and details depend upon particular version of the C++ libraries (notably the standard C++ library) and of the compiler version.