C program shows %zu after conversion to Windows - linux

I complied a linux program on windows via Mingw. However, the output of the program looks different on Windows than on Linux.
For example, on Windows the output is this (I get 'zu' instead of real numbers):
Approximated minimal memory consumption:
Sequence : zuM
Buffer : 1 X zuM = zuM
Table : 1 X zuM = zuM
Miscellaneous : zuM
Total : zuM
On Linux, the original program compiles (without Mingw) with a warning. On Windows, under Mingw, it compiles with zero warnings.
There is anything I should be aware about?
Does Mingw offer 100% compatibility or I have to modify the program to work on Win?
I don't know in which direction to head. Where should I start my attempt of fixing the program?
Do you think I have better chances with Cygwin?
Update:
Wikipedia mentions this: "the lack of support for C99 has caused porting problems, particularly where printf-style conversion specifiers are concerned".
Is this the thing in which I bumped my head?
Update:
My mingw version is:
MINGWBASEDIR=C:\MinGW
gcc version 4.8.1 (GCC)
gcc version 4.8.1 (GCC)
GNU gdb (GDB) 7.6.1
GNU ld (GNU Binutils) 2.24
GNU windres (GNU Binutils) 2.24
GNU dlltool (GNU Binutils) 2.24
GNU Make 3.82.90
#define __MINGW32_VERSION 3.20
#define __W32API_VERSION 3.17
(I used this code to get the version:
#echo off
REM version-of-mingw.bat
REM credit to Peter Ward work in ReactOS Build Environment RosBE.cmd it gave me a starting point that I edited.
::
:: Display the current version of GCC, ld, make and others.
::
REM %CD% works in Windows XP, not sure when it was added to Windows
REM set MINGWBASEDIR=C:\MinGW
set MINGWBASEDIR=%CD%
ECHO MINGWBASEDIR=%MINGWBASEDIR%
SET PATH=%MINGWBASEDIR%\bin;%SystemRoot%\system32
if exist %MINGWBASEDIR%\bin\gcc.exe (gcc -v 2>&1 | find "gcc version")
REM if exist %MINGWBASEDIR%\bin\gcc.exe gcc -print-search-dirs
if exist %MINGWBASEDIR%\bin\c++.exe (c++ -v 2>&1 | find "gcc version")
if exist %MINGWBASEDIR%\bin\gcc-sjlj.exe (gcc-sjlj.exe -v 2>&1 | find "gcc version")
if exist %MINGWBASEDIR%\bin\gcc-dw2.exe (gcc-dw2.exe -v 2>&1 | find "gcc version")
if exist %MINGWBASEDIR%\bin\gdb.exe (gdb.exe -v | find "GNU gdb")
if exist %MINGWBASEDIR%\bin\nasm.exe (nasm -v)
if exist %MINGWBASEDIR%\bin\ld.exe (ld -v)
if exist %MINGWBASEDIR%\bin\windres.exe (windres --version | find "GNU windres")
if exist %MINGWBASEDIR%\bin\dlltool.exe (dlltool --version | find "GNU dlltool")
if exist %MINGWBASEDIR%\bin\pexports.exe (pexports | find "PExports" )
if exist %MINGWBASEDIR%\bin\mingw32-make.exe (mingw32-make -v | find "GNU Make")
if exist %MINGWBASEDIR%\bin\make.exe (ECHO It is not recommended to have make.exe in mingw/bin)
REM ECHO "The minGW runtime version is the same as __MINGW32_VERSION"
if exist "%MINGWBASEDIR%\include\_mingw.h" (type "%MINGWBASEDIR%\include\_mingw.h" | find "__MINGW32_VERSION" | find "#define")
if exist "%MINGWBASEDIR%\include\w32api.h" (type "%MINGWBASEDIR%\include\w32api.h" | find "__W32API_VERSION")
:_end
PAUSE
)

As suggested by the bug report discussion linked in the comments, Microsoft's printf functions do not support C99. The mingw-w64 project provides alternative functions that may be used as if they were the normal C99 functions if the macro __USE_MINGW_ANSI_STDIO is set to 1 either before including any headers or on the command line. They support the standard %zu, %jd, etc. format specifiers that even the newest MSVCRT versions do not. You may invoke the function directly using mingw_printf, but it is usually easier to just define the aforementioned macro to 1 and call printf, etc.
It is worth noting that if you use Microsoft's snprintf, it will return -1 to indicate truncation if the buffer is not large enough, unless the buffer and buffer size parameters are NULL and 0 respectively, in which case the number of bytes that would be output is returned. C99 behavior is to always return the number of bytes that would be output if the buffer was sufficiently large, or a negative value if an encoding error occurs, and the mingw-w64 implementation seems to behave correctly according to C99.
And all you need to do to get all of this standard behavior is either #define __USE_MINGW_ANSI_STDIO 1 before any includes if you use any of the printf functions or simply add -D__USE_MINGW_ANSI_STDIO=1 to your compiler invocation.
If you are worried about the macro interfering with other platforms, no other implementation except the original (legacy?) MinGW[32] project that provided similar functionality should actually make use of this preprocessor macro, so it is safe to define it unconditionally.

Related

How to determine the configuration of gcc from a bash script?

I have a bash script that compiles a program as well on older versions of Ubuntu (14.04.x and 16.04.x) than on the last one (18.04.x) and on other distributions (Arch, CentOS, Debian, Fedora, etc.) and therefore... with different gcc settings.
Then, to obtain an executable that can be launched (among other ways) by a double click, I must compile this program without the "-no-pie" option with older versions of gcc setting (Ubuntu 14.04.x and 16.04.x) when I have to use this option "=no-pie" for the new version of the gcc 7.3 setting (on Ubuntu 18.04.x).
The problem is that on the last Ubuntu release (18.04.x) and its derivatives (Kubuntu, Xbuntu, etc. and maybe with other distributions) with the new configuration of gcc (7.3) having the option "--enable-default-pie", if I compile my program without the option "-no-pie", the result is that the file created is an executable which is of the "shared library" type which can not be launched by a double click.
My question is either:
a) Is there a command that allows me to determine from a bash script if gcc is configured with the "--enable-default-pie" setting?
b) if not, is there a command that allows me to determine from a bash script if the compiled file is of the "shared library" or "executable" type?
For this second option, a solution could be how to save the response of "gcc -v" in a .txt file and check if there is the "--enable-default-pie" string but I've absolutely no clue how to do it.
If I there is not an answer to my first option, the second option (it is true less elegant but just as effective) would allow me to compile my program first without the "-no-pie" option, then check the status of such a created executable and if the result is a "shared library", of restart this compilation this time using the option "-no-pie" for, in one case as in the other, get an executable that can be launched by a double click whatever the setting of gcc may be.
Thank you in advance for your time, ideas and suggestions.
Best regards.
The recommend way to check for PIE support is to compile C code like this
#if defined __PIC__ || defined __pic__ || defined PIC || defined pic
# error PIC is default.
#endif
with the requested compiler flags and check whether there is an error. If you need special treatment for PIE, this will recognize PIE if it has been specified through the CC or CFLAGS variables, even if is not immediately apparent there. For example, for technical reasons, Fedora hides the PIE flags behind a -specs argument.
Yes, you can check GCC build options with gcc -v or gcc -###
In order to have pretty print you can use:
gcc -### -E 2>&1 | grep "Configured with" | sed 's/--/\n--/g'
So bash oneliner to say you have pie or not may be:
if [[ -n "`gcc -v -E 2>&1 | grep 'Configured with' | sed 's/--/\n--/g' | grep enable-default-spie`" ]]; then echo "PIE DEFAULT"; else echo "PIE NOT DEFAULT"; fi
To check file type just use file command, eg.:
file /usr/bin/x86_64-linux-gnu-gcc-7
/usr/bin/x86_64-linux-gnu-gcc-7: ELF 64-bit LSB executable,
x86-64, version 1 (SYSV), dynamically linked, interpreter
/lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0,
BuildID[sha1]=02ac46ba938c15f55f6fab165133e0f527bc2197, stripped
file /usr/lib/libchm.so.1.0.0
/usr/lib/libchm.so.1.0.0: ELF 64-bit LSB shared object, x86-64,
version 1 (SYSV), dynamically linked,
BuildID[sha1]=7c603d9a0771b5bfd5b869b4446e2f78ef13802a, stripped
File type function
function whatfile { file $1 -i | grep application | sed 's/^.*application\/x-//g;s/,.*$//g'; }
Example output:
aaa#xxx:~ $ whatfile /boot/grub/grub.conf
aaa#xxx:~ $ whatfile /usr/lib/libnss3.so
sharedlib
aaa#xxx:~ $ whatfile /bin/zcat
executable
Unfortunately, comments don't allow CR+LF (to show pre-formatted text).
Below is my formatted translation of your gcc setting command:
check_gcc_setting()
{
if [ -n "`gcc -v -E 2>&1 | grep 'Configured with' | sed 's/--/\n--/g' | grep enable-default-pie`" ]
then
GCC_SETTING="1"
else
GCC_SETTING="0"
fi
read -p "The gcc setting is $GCC_SETTING " GCCRESULT
}
Below is the result:
whatfile { file $1 -i | grep application | sed 's/^.*application\/x-//g;s/,.*$//g'; }
-bash: syntax error near unexpected token `}'

Cmake check if the current platform is Fedora [duplicate]

Does anybody know any cmake variable or hook or something which can give me underlying platform name/flavour name on which it is getting executed ?
e.g. Linux-CentOs
Linux-Ubuntu
Linux-SLES
I know cmake has "CMAKE_SYSTEM" variable but that doesn't help differentiating flavours of linux for e.g. Any help is appreciated.
edit :
I just read that it can be done using lsb_release command ?
The following snippet populates the LSB_RELEASE_ID_SHORT cmake variable with information about the underlying Linux system:
find_program(LSB_RELEASE_EXEC lsb_release)
execute_process(COMMAND ${LSB_RELEASE_EXEC} -is
OUTPUT_VARIABLE LSB_RELEASE_ID_SHORT
OUTPUT_STRIP_TRAILING_WHITESPACE
)
On Ubuntu, for example, it yields Ubuntu.
Slightly less convoluted than checking files on the filesystem is to deduce the best you can from the available CMAKE_SYSTEM vars. For instance a CMakeLists.txt file containing lines like this:
message("-- CMAKE_SYSTEM_INFO_FILE: ${CMAKE_SYSTEM_INFO_FILE}")
message("-- CMAKE_SYSTEM_NAME: ${CMAKE_SYSTEM_NAME}")
message("-- CMAKE_SYSTEM_PROCESSOR: ${CMAKE_SYSTEM_PROCESSOR}")
message("-- CMAKE_SYSTEM: ${CMAKE_SYSTEM}")
string (REGEX MATCH "\\.el[1-9]" os_version_suffix ${CMAKE_SYSTEM})
message("-- os_version_suffix: ${os_version_suffix}")
outputs this when I ran cmake . :
-- CMAKE_SYSTEM_INFO_FILE: Platform/Linux
-- CMAKE_SYSTEM_NAME: Linux
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- CMAKE_SYSTEM: Linux-2.6.32-573.7.1.el6.x86_64
-- os_version_suffix: .el6
And for my situation, the .el6 was enough to differentiate.
Likely, you have to write such a test yourself. Here's one of the possible examples, just googled: https://htcondor-wiki.cs.wisc.edu/index.cgi/fileview?f=build/cmake/FindLinuxPlatform.cmake&v=4592599fecc08e5588c4244e2b0ceb7d32363a56
However depending on your actual needs the test may be quite complex. For example Ubuntu as a Debian-based OS always has /etc/debian_version and many RPM-based OSes traditionally have /etc/redhat-release. There's a file /etc/os-release in the Linux Standard Base (LSB) specification, but for example on the localhost this file is empty for an unknown reason :)
I know this is an old question, but as of now, there is still no cmake built-in function to find this information in good detail. I've implemented a small utility function that uses lsb_release on Linux to find a number of relevant system details:
function(get_linux_lsb_release_information)
find_program(LSB_RELEASE_EXEC lsb_release)
if(NOT LSB_RELEASE_EXEC)
message(FATAL_ERROR "Could not detect lsb_release executable, can not gather required information")
endif()
execute_process(COMMAND "${LSB_RELEASE_EXEC}" --short --id OUTPUT_VARIABLE LSB_RELEASE_ID_SHORT OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(COMMAND "${LSB_RELEASE_EXEC}" --short --release OUTPUT_VARIABLE LSB_RELEASE_VERSION_SHORT OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(COMMAND "${LSB_RELEASE_EXEC}" --short --codename OUTPUT_VARIABLE LSB_RELEASE_CODENAME_SHORT OUTPUT_STRIP_TRAILING_WHITESPACE)
set(LSB_RELEASE_ID_SHORT "${LSB_RELEASE_ID_SHORT}" PARENT_SCOPE)
set(LSB_RELEASE_VERSION_SHORT "${LSB_RELEASE_VERSION_SHORT}" PARENT_SCOPE)
set(LSB_RELEASE_CODENAME_SHORT "${LSB_RELEASE_CODENAME_SHORT}" PARENT_SCOPE)
endfunction()
Add it to your CMakeLists.txt and use it like this:
if(CMAKE_SYSTEM_NAME MATCHES "Linux")
get_linux_lsb_release_information()
message(STATUS "Linux ${LSB_RELEASE_ID_SHORT} ${LSB_RELEASE_VERSION_SHORT} ${LSB_RELEASE_CODENAME_SHORT}")
endif()
If you need further details, check what else lsb_release can provide with lsb_release -a.
Note that not every Linux has lsb_release installed. Most systems provide it, but its not mandatory. On newer Ubuntu, for example, its the default on desktop installs, and required by ubuntu-minimal. If it should be missing on your machine, you can install it with sudo apt install lsb-release.
on my machine
CMAKE_SYSTEM_INFO_FILE == "Platform/Linux"
CMAKE_SYSTEM_NAME == "Linux"
CMAKE_SYSTEM == "Linux-<kernel version>"
obtained with cmake --system-information, I know of people that use said macros in their own CMakeLists.txt files so they work as expected, probably CMAKE_SYSTEM_NAME is what you really want but here you go, you get this 3 and the command to inspect the properties of your machine as far as cmake is concerned .
Based on thiagowfx answer, If you want to get the codename of the distro (if it is available):
execute_process(COMMAND lsb_release -cs
OUTPUT_VARIABLE RELEASE_CODENAME
OUTPUT_STRIP_TRAILING_WHITESPACE
)
E.g. in Ubuntu 14.04 the variable RELEASE_CODENAME will hold trusty.
### find our os (wins, centos, ubuntu, etc)
set(VAR_OS "")
IF(CMAKE_SYSTEM_NAME MATCHES "Linux")
MESSAGE(DEBUG "Linux")
execute_process (
COMMAND bash -c "awk -F= '/^ID=/{print $2}' /etc/os-release |tr -d '\n' | tr -d '\"'"
OUTPUT_VARIABLE outOS
)
MESSAGE(DEBUG "Linux os: ${outOS}")
ELSEIF(CMAKE_SYSTEM_NAME MATCHES "Windows")
MESSAGE(DEBUG "Windows")
set(VAR_OS "wins")
ELSE()
MESSAGE(FATAL "No OS detected!")
ENDIF(CMAKE_SYSTEM_NAME MATCHES "Linux")

Linux makefile to check required minimum linux kernel version

I have a linux makefile that needs to compile on 2 different kernel versions. The makefile is not generated from automake/autoconf.
The C code are already conditioned using macros to generate different code for different kernel version, but certain features in the makefile needs to be also conditioned.
Is there a way in a makefile to do:
if (kernel_version > 2.6.30)
newer_kernel = 1
else
newer_kernel = 0
endif
Well, I can think of one quick way to do it with bash:
KERNEL_VERSION=`uname -r`
HIGHER_VERSION=`echo -e "$KERNEL_VERSION\n2.6.30" | sort -g -t '.' | tail --lines=1`
if [ "$HIGHER_VERSION" == "2.6.30" ]
# its an older kernel
else
# its a newer kernel
fi
Basically, you use uname to obtain the version of the current kernel, then compare it to 2.6.30 using sort (the -g flag enables numeric sorting, -t '.' means use dot as a field separator), then use tail to determine which of the two version was higher in the list. Not exactly a beautiful solution, but it will work.
You can put it into a separate script or directly into the makefile recipe

How can I get perf to find symbols in my program

When using perf report, I don't see any symbols for my program, instead I get output like this:
$ perf record /path/to/racket ints.rkt 10000
$ perf report --stdio
# Overhead Command Shared Object Symbol
# ........ ........ ................. ......
#
70.06% ints.rkt [unknown] [.] 0x5f99b8
26.28% ints.rkt [kernel.kallsyms] [k] 0xffffffff8103d0ca
3.66% ints.rkt perf-32046.map [.] 0x7f1d9be46650
Which is fairly uninformative.
The relevant program is built with debugging symbols, and the sysprof tool shows the appropriate symbols, as does Zoom, which I think is using perf under the hood.
Note that this is on x86-64, so the binary is compiled with -fomit-frame-pointer, but that's the case when running under the other tools as well.
This post is already over a year old, but since it came out at the top of my Google search results when I had the same problem, I thought I'd answer it here. After some more searching around, I found the answer given in this related StackOverflow question very helpful. On my Ubuntu Raring system, I then ended up doing the following:
Compile my C++ sources with -g (fairly obvious, you need debug symbols)
Run perf as
record -g dwarf -F 97 /path/to/my/program
This way perf is able to handle the DWARF 2 debug format, which is the standard format gcc uses on Linux. The -F 97 parameter reduces the sampling rate to 97 Hz. The default sampling rate was apparently too large for my system and resulted in messages like this:
Warning:
Processed 172390 events and lost 126 chunks!
Check IO/CPU overload!
and the perf report call afterwards would fail with a segmentation fault. With the reduced sampling rate everything worked out fine.
Once the perf.data file has been generated without any errors in the previous step, you can run perf report etc. I personally like the FlameGraph tools to generate SVG visualizations.
Other people reported that running
echo 0 > /proc/sys/kernel/kptr_restrict
as root can help as well, if kernel symbols are required.
In my case the solution was to delete the elf files which contained cached symbols from previous builds and were messing things up.
They are in ~/.debug/ folder
You can always use the '$ nm ' command.
here is some sample output:
Ethans-MacBook-Pro:~ phyrrus9$ nm a.out
0000000100000000 T __mh_execute_header
0000000100000f30 T _main
U _printf
0000000100000f00 T _sigint
U _signal
U dyld_stub_binder
I had this problem too, I couldn't see any userspace symbol, but I saw some kernel symbols. I thought this was a symbol loading issue. After tried all the possible solutions I could find, I still couldn't get it work.
Then I faintly remember that
ulimit -u unlimited
is needed. I tried and it magically worked.
I found from this wiki that this command is needed when you use too many file descriptors.
https://perf.wiki.kernel.org/index.php/Tutorial#Troubleshooting_and_Tips
my final command was
perf record -F 999 -g ./my_program
didn't need --call-graph
Make sure that you compile the program using -g option along with gcc(cc) so that debugging information is produced in the operating system's native format.
Try to do the following and check if there are debug symbols present in the symbol table.
$objdump -t your-elf
$readelf -a your-elf
$nm -a your-elf
How about your dev host machine? Is it also running the x86_64 OS?
If not, please make sure the perf is cross-compiled, because the perf depends on the objdump and other tools in toolchain.
I got the same problem with perf after overriding the name of my program via prctl(PR_SET_NAME)
As I can see your case is pretty similar:
70.06% ints.rkt [unknown]
Command you have executed (racket) is different from the one perf have seen.
you can check the value of kptr_restrict by cat /proc/kallsyms. If the addresses of the symbols in the result are all 0x000000, you can fix it by command echo 0 > sys/kernel/kptr_restrict . After this , you may get a wanted result of the perf report

Why is ARG_MAX not defined via limits.h?

On Fedora Core 7, I'm writing some code that relies on ARG_MAX. However, even if I #include <limits.h>, the constant is still not defined. My investigations show that it's present in <sys/linux/limits.h>, but this is supposed to be portable across Win32/Mac/Linux, so directly including it isn't an option. What's going on here?
The reason it's not in limits.h is that it's not a quantity giving the limits of the value range of an integral type based on bit width on the current architecture. That's the role assigned to limits.h by the ISO standard.
The value in which you're interested is not hardware-bound in practice and can vary from platform to platform and perhaps system build to system build.
The correct thing to do is to call sysconf and ask it for "ARG_MAX" or "_POSIX_ARG_MAX". I think that's the POSIX-compliant solution anyway.
Acc. to my documentation, you include one or both of unistd.h or limits.h based on what values you're requesting.
One other point: many implementations of the exec family of functions return E2BIG or a similar value if you try to call them with an oversized environment. This is one of the defined conditions under which exec can actually return.
For the edification of future people like myself who find themselves here after a web search for "arg_max posix", here is a demonstration of the POSIXly-correct method for discerning ARG_MAX on your system that Thomas Kammeyer refers to in his answer:
cc -x c <(echo '
#include <unistd.h>
#include <stdio.h>
int main() { printf("%li\n", sysconf(_SC_ARG_MAX)); }
')
This uses the process substitution feature of Bash; put the same lines in a file and run cc thefile.c if you are using some other shell.
Here's the output for macOS 10.14:
$ ./a.out
262144
Here's the output for a RHEL 7.x system configured for use in an HPC environment:
$ ./a.out
4611686018427387903
$ ./a.out | numfmt --to=iec-i # 'numfmt' from GNU coreutils
4.0Ei
For contrast, here is the method prescribed by https://porkmail.org/era/unix/arg-max.html, which uses the C preprocessor:
cpp <<HERE | tail -1
#include <limits.h>
ARG_MAX
HERE
This does not work on Linux for reasons still not entirely clear to me—I am not a systems programmer and not conversant in the POSIX or ISO specs—but probably explained above.
ARG_MAX is defined in /usr/include/linux/limits.h. My linux kernel version is 3.2.0-38.

Resources