My initial task was to install mod_perl 2.0.6 + Apache 2.2.22.
The process stopped with a lot of errors related to off64_t when compiling mod_perl. So, I started to dig deeper. Firstly, I have installed two new instances of Perl 5.8.9 (because I'll have to use this version): a threaded version and a not-threaded one (they are identical, only usethreads differs). Trying to reproduce the same using the threaded Perl finished with success and no off64_t errors at all.
The conclusion is obvious: threaded Perl provides the neccessary off64_t, the non-threaded one doesn't.
Searching further, I have compared config.h (from core/<arch>/CORE) of both Perl'es, and at the line 3671 I can see this (in the non-threaded Perl):
/* HAS_OFF64_T:
* This symbol will be defined if the C compiler supports off64_t.
*/
/*#define HAS_OFF64_T / **/
and in the threads-enabled Perl:
#define HAS_OFF64_T /**/
perl -V for both Perl instances reports ccflags ='... -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 ...' as used compiler flags.
As I understand, off64_t is used for large files and isn't related to threads. I found this information about off_t and off64_t:
If the source is compiled with _FILE_OFFSET_BITS = 64 this type (i.e. off_t) is transparently replaced by off64_t.
Shortly: There are 2 identical Perl builds with a single difference: the usethreads configure parameter. Threaded Perl enables off64_t, non-threaded one doesn't.
My question is: Why does this happen and how threads are connected to this off64_t data type that should be used for large files, not for threads ?
Info: Arch Linux OS 32-bit (kernel 2.6.33), gcc 4.5.0, libc 2.11.1, standard Perl 5.8.9
Notes: off64_t is processed in Configure at line 15526, a simple try.c is generated and tried to be compiled. The question is why the not-threaded Perl cannot compile it while threaded Perl can.
I'm not sure if answering my own question is an accepted behaviour, but while I was searching for the solution and not just waiting for someone else to do my homework, I think it will be useful for other people reading this.
Shortly, the answer to my question is the -D_GNU_SOURCE gcc compiler flag and it seems threads have nothing in common with this off64_t type.
It appears that when -Dusethreads is used for Configure, hints/linux.sh is used and the following code is executed:
case "$usethreads" in
$define|true|[yY]*)
ccflags="-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS $ccflags"
then the code is compiled with _GNU_SOURCE defined, which allows a lot of things to be used (like is answered in these thread: What does “#define _GNU_SOURCE” imply?).
When Perl is built without threads support, these flags are skipped and a lot of bits from header files remain commented.
It seems the Perl itself is not affected by this. Even older versions of Apache were not, but Apache 2.2+ started to use code which is enabled by _GNU_SOURCE and building mod_perl is not as straightforward as before.
I don't know who should take a notice about this. Maybe core Perl maintainers themselves, maybe Apache maintainers, maybe no one and it's just my particlar case or compiler issues.
Conclusion: when building not-threaded Perl the _GNU_SOURCE is not used, as a result Perl .h files have a lot of commented #defines and building mod_perl against the Apache 2.2+ sources fails. An additional -Accflags='-D_GNU_SOURCE' should be added when building Perl.
Other answers are welcome too. Maybe I'm wrong or I'm just seeing the top of the iceberg.
Related
Error
Currently using this to compile my C++ program:
x86_64-w64-mingw32-g++ -std=c++11 -c main.cpp -o main.o -I../include
x86_64-w64-mingw32-g++ main.o -o mainWin.exe -L/usr/lib/x86_64-linux-gnu/ -L/usr/local/lib -lopengl32 -lglfw32 -lGLEW -lX11 -lXxf86vm -lXrandr -lpthread -lXi -DGLEW_STATIC
I am using Mingw to compile my C++ program from Linux (Ubuntu) to a Windows executable. I am relatively new to compiling via command line, but I would like to switch my work environment completely over to Linux.
When I attempt to compile the program, I get the following error:
*** Error in `/usr/bin/x86_64-w64-mingw32-ld`: free(): invalid pointer: [removed]***
ld terminated with signal 6 [Aborted], core dumped
I believe this is because of my build of GLEW. Whenever I make it, it wants to use a mingw32msvc version of mingw. I think I need it to use x86_64-w64-mingw32-gcc. I cannot figure out how to make it do this (if even possible).
Extra
It's also worth noting that I only get this error with GLEW_STATIC defined at the top of main.cpp. Without it, I get undefined references to GLEW.
It seems that you were using the -lGLEW flag when you're supposed to use -lglew32s/lglew32! Make sure to #define GLEW STATIC if you are statically linking...and get the appropriate binaries from their website.
If the loader (or any program) is crashing, then check to see whether you are using the most recent version. If not, get hold of the newest version and try again. If that doesn't resolve it, can you find an older version and use that? If you can't easily find a version that works, you need to report the bug to the relevant team — at MinGW or the bin-utils team at GNU. Is 32-bit compilation an option? If so, try that. You're in a hole; it will probably take some digging to get yourself out.
This problem seems to occur in 2016, even though the question is from 2014. It is a little surprising that the problem has not been fixed yet — assuming that the flawed free being run into in 2016 is the same as the one that occurred in 2014. If the loader now in use dates from (say) 2013-early 2015, then there's probably an update and you should investigate it. If the loader now in use dates from mid-2015 onwards, it is more likely (or, if that's too aggressive, then it is at least possible) that it is a different bug that manifests itself similarly.
The advice to "try an upgrade if there is one available; if that doesn't work, see whether you can find a working downgrade" remains valid. It would be worth trying to create an MCVE (Minimal, Complete, and Verifiable Example) and reporting the bug to the maintenance teams — as was suggested by nodakai in 2014. The smaller the code you use, and the fewer libraries you need, the easier it will be for the maintenance teams to discover the problem and fix it. If it is a cross-compiler running on Linux for MinGW, then you still need to minimize the code and report the issue.
Note that if you can find a 'known-to-work' version, that will probably be of interest to the maintainers. It localizes where they need to look a bit.
I should note that even if the library in use is the wrong library, the loader still shouldn't crash with the free error. It can report a problem and stop under control. It should not crash. It may still be worth reporting that it does crash.
In many ways, this is just generic advice on what to do when you encounter a bug in software.
You are (and I was) using the -lGLEW flag when you're supposed to use -lglew32s/lglew32! Make sure to #define GLEW STATIC if you are statically linking... get the appropriate binaries from their website .-.
I'm compiling my code on a server that has OpenMPI, but I need to know which version I'm on so I can read the proper documentation. Is there a constant in <mpi.h> that I can print to display my current version?
As explained in this tutorial, you may also check the MPI version running the command:
mpiexec --version
or
mpirun --version
in your terminal.
With OpenMPI, the easiest thing to do is to run ompi_info; the first few lines will give you the information you want. In your own code, if you don't mind something OpenMPI specific, you can look at use OMPI_MAJOR_VERSION, OMPI_MINOR_VERSION, and OMPI_RELEASE_VERSION in mpi.h. That obviously won't work with MPICH2 or other MPI implementations.
More standardly, as part of MPI-3, there is a standard MPI routine called MPI_Get_library_version which gives you detailed library information at run time. This is small enough and useful enough that newer versions of MPI implementations will have this very quickly - for instance it's in the OpenMPI 1.7 development trunk - but it doesn't really help you today.
I am not familier with OpenMPI but MPI has a function MPI_Get_Version, please check your mpi.h for similar functions.
You can also get the version of OpenMPI that the compiler wrapper (e.g. mpicxx/mpic++/mpicc/mpifort) comes from:
mpicxx --showme:version
This can be useful if (for any reason) you have different versions of MPI compiler wrapper and executor.
(Just mpicxx --showme will additionally show you where MPI is installed and which compiler flags uses, see the manual for more.)
I have a very complex cross-platform application. Recently my team and I have been running stress tests and have encountered several crashes (and core dumps accompanying them). Some of these core dumps are very precise, and show me the exact location where the crash occurred with around 10 or more stack frames. Others sometimes have just one stack frame with ?? being the only symbol!
What I'd like to know is:
Is there a way to increase the probability of core dumps pointing in the right direction?
Why isn't the number of stack frames reported consistent?
Any best practice advise for managing core dumps.
Here's how I compile the binaries (in release mode):
Compiler and platform: g++ with glibc-2.3.2-95.50 on CentOS 3.6 x86_64 -- This helps me maintain compatibility with older versions of Linux.
All files are compiled with the -g flag.
Debug symbols are stripped from the final binary and saved in a separate file.
When I have a core dump, I use GDB with the executable which created the core, and the symbols file. GDB never complains that there's a mismatch between the core/binary/symbols.
Yet I sometimes get core dumps with no symbols at all! It's understandable that I'm linking against non-debug version of libstdc++ and libgcc, but it would be nice if at least the stack trace shows me where in my code did the faulty instruction call originate (although it may ultimately end in ??).
Others sometimes have just one stack frame with "??" being the only symbol!
There can be many reasons for that, among others:
the stack frame was trashed (overwritten)
EBP/RBP (on x86/x64) is currently not holding any meaningful value — this can happen e.g. in units compiled with -fomit-frame-pointer or asm units that do so
Note that the second point may occur simply by, for example, glibc being compiled in such a way. Having the debug info for such system libraries installed could mitigate this (something like what the glibc-debug{info,source} packages are on openSUSE).
gdb has more control over the program than glibc, so glibc's backtrace call would naturally be unable to print a backtrace if gdb cannot do so either.
But shipping the source would be much easier :-)
As an alternative, on a glibc system, you could use the backtrace function call (or backtrace_symbols or backtrace_symbols_fd) and filter out the results yourself, so only the symbols belonging to your own code are displayed. It's a bit more work, but then, you can really tailor it to your needs.
Have you tried installing debugging symbols of the various libraries that you are using? For example, my distribution (Ubuntu) provides libc6-dbg, libstdc++6-4.5-dbg, libgcc1-dbg etc.
If you're building with optimisation enabled (eg. -O2), then the compiler can blur the boundary between stack frames, for example by inlining. I'm not sure that this would cause backtraces with just one stack frame, but in general the rule is to expect great debugging difficulty since the code you are looking it in the core dump has been modified and so does not necessarily correspond to your source.
Is it possible to compile a linux kernel(2.6) module that includes functionality defined by non-kernel includes?
For example:
kernelmodule.h
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h> // printk()
// ...
#include <openssl/sha.h>
// ...
Makefile
obj-m := kernelmodule.o
all:
$(MAKE) -C /lib/modules/`uname -r`/build M=`pwd` modules
clean:
$(MAKE) -C /lib/modules/`uname -r`/build M=`pwd` clean
$(RM) Module.markers modules.order
The kernel module I have written and are trying to compile contains functionality found in a number of openssl include files.
The standard makefile presented above doesn't allow includes outside of the linux headers. Is it possible to include this functionality, and if so, could you please point me in the right direction.
Thanks,
Mike
The kernel cannot use userspace code and must stand alone (i.e. be completely self contained, no libraries), therefore it does not pick up standard headers.
It is not clear what benefit trying to pick up userspace headers is. If there are things in there that it would be valid to use (constants, some macros perhaps provided they don't call any userspace functions), then it may be better to duplicate them and include only the kernel-compatible parts that you need.
It is not possible to link the kernel with libraries designed for userspace use - even if they don't make any OS calls - because the linking environment in the kernel cannot pick them up.
Instead, recompile any functions to be used in the kernel (assuming they don't make any OS or library calls - e.g. malloc - in which case they'll need to be modified anyway). Incorporate them into your own library to be used in your kernel modules.
Recent versions of linux contain cryptographic functions anyway, including various SHA hashes - perhaps you can use one of those instead.
Another idea would be to stop trying to do crypto in kernel-space and move the code to userspace. Userspace code is easier to write / debug / maintain etc.
I have taken bits of userspace code that I've written and converted it to work in kernel space (i.e. using kmalloc(), etc), it's not that difficult. However, you are confined to the kernel's understanding of C, not userspace, which differs slightly .. especially with various standard int types.
Just linking against user space DSO's is not possible — the Linux kernel is monolithic, completely self contained. It does not use userspace libc, libraries or other bits as others have noted.
9/10 times, you will find what you need somewhere in the kernel. It's very likely that someone else ran into the same need you have and wrote some static functions in some module to do what you want .. just grab those and re-use them.
In the case of crypto, as others have said, just use what's in the kernel. One thing to note, you'll need them to be enabled in kconfig which may or may not happen depending on what the user selects when building it. So, watch out for dependencies and be explicit, you may have to hack a few entries in kconfig that also select the crypto API you want when your module is selected. Doing that can be a bit of a pain when building out of tree.
So on the one hand we have "just copy and rename stuff while adding overall bloat", on the other you have "tell people they must have the full kernel source". It's one of the quirks that come with a monolithic kernel.
With a Microkernel, almost everything runs in userspace, no worries linking against a DSO for some driver ... it's a non issue. Please don't take that statement as a cue to re-start kernel design philosophy in comments, that's not in the scope of this question.
I am trying to build some big libraries, like Boost and OpenCV, from their source code via make and GCC under Ubuntu 8.10 on my laptop. Unfortunately the compilation of those big libraries seem to be big burden to my laptop (Acer Aspire 5000). Its fan makes higher and higher noises until out of a sudden my laptop shuts itself down without the OS gracefully turning off.
So I wonder how to reduce the compilation cost in case of make and GCC?
I wouldn't mind if the compilation will take much longer time or more space, as long as it can finish without my laptop shutting itself down.
Is building the debug version of libraries always less costly than building release version because there is no optimization?
Generally speaking, is it possible to just specify some part of a library to install instead of the full library? Can the rest be built and added into if later found needed?
Is it correct that if I restart my laptop, I can resume compilation from around where it was when my laptop shut itself down? For example, I noticed that it is true for OpenCV by looking at the progress percentage shown during its compilation does not restart from 0%. But I am not sure about Boost, since there is no obvious information for me to tell and the compilation seems to take much longer.
UPDATE:
Thanks, brianegge and Levy Chen! How to use the wrapper script for GCC and/or g++? Is it like defining some alias to GCC or g++? How to call a script to check sensors and wait until the CPU temperature drops before continuing?
I'd suggest creating a wrapper script for gcc and/or g++
#!/bin/bash
sleep 10
exec gcc "$#"
Save the above as "gccslow" or something, and then:
export CC="gccslow"
Alternatively, you can call the script gcc and put it at the front of your path. If you do that, be sure to include the full path in the script, otherwise, the script will call itself recursively.
A better implementation could call a script to check sensors and wait until the CPU temperature drops before continuing.
For your latter question: A well written Makefile will define dependencies as a directed a-cyclical graph (DAG), and it will try to satisfy those dependencies by compiling them in the order according to the DAG. Thus as a file is compiled, the dependency is satisfied and need not be compiled again.
It can, however, be tricky to write good Makefiles, and thus sometime the author will resort to a brute force approach, and recompile everything from scratch.
For your question, for such well known libraries, I will assume the Makefile is written properly, and that the build should resume from the last operation (with the caveat that it needs to rescan the DAG, and recalculate the compilation order, that should be relatively cheap).
Instead of compiling the whole thing, you can compile each target separately. You have to examine the Makefile for identifying them.
Tongue-in-cheek: What about putting the laptop into the fridge while compiling?