I am trying to turn my flex/bison code application into a portable library (hopefully). Searching on the web revealed the following example:
https://begriffs.com/posts/2021-11-28-practical-parsing.html#using-a-parser-as-a-library
Which I've imported as a project here:
https://github.com/malaterre/using-a-parser-as-a-library
I did follow instruction to write the Makefile, but for some reason this is not working on my linux install with flex/bison. All I get is:
driver_lisp.c:8:10: fatal error: lisp.lex.h: No such file or directory
Could someone please double-check the following Makefile and rules associated with flex/bison:
% cat Makefile
LEX=flex
YACC=bison
lisp : lisp.tab.o lisp.lex.o driver_lisp.o
$(CC) -o $# driver_lisp.o lisp.tab.o lisp.lex.o -ly -ll
lisp.tab.h lisp.tab.c : lisp.y
$(YACC) -d -b lisp $?
lisp.lex.o : lisp.tab.h lisp.lex.c
driver_lisp.o : driver_lisp.c
$(CC) -o driver_lisp.o $?
lisp.lex.c : lisp.l
$(LEX) -t $? > $#
For reference:
% flex --version
flex 2.6.4
% bison --version
bison (GNU Bison) 3.7.5
Written by Robert Corbett and Richard Stallman.
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Related
I'm trying to build libc6 with a custom prefix by modifying the prefix=/usr line in debian/rules. However, this fails because the patch is applied multiple times. Curiously, patching another file does not result in the same error. I've distilled the failure down to this script:
#!/bin/bash
set -exuo pipefail
work_dir=$(mktemp -d)
cd "$work_dir"
apt-get source libc6
cd eglibc-2.19
cat <<'PATCH' > ../set-prefix.diff
Index: eglibc-2.19/debian/rules
===================================================================
--- eglibc-2.19.orig/debian/rules>2022-03-01 17:27:53.068299816 -0800
+++ eglibc-2.19/debian/rules>-2022-03-01 17:27:53.068299816 -0800
## -85,7 +85,7 ##
# Default setup
EGLIBC_PASSES ?= libc
-prefix=/usr
+prefix=/new/prefix
bindir=$(prefix)/bin
datadir=$(prefix)/share
localedir=$(prefix)/lib/locale
PATCH
cat <<'PATCH' > ../change-readme.diff
Index: eglibc-2.19/README
===================================================================
--- eglibc-2.19.orig/README 2013-10-18 14:33:25.000000000 -0700
+++ eglibc-2.19/README 2022-03-02 17:00:49.954759733 -0800
## -1,5 +1,7 ##
This directory contains the Embedded GNU C Library (EGLIBC).
+Add a line.
+
EGLIBC is a variant of the GNU C Library (GLIBC) that is designed to
work well on embedded systems. EGLIBC strives to be source and binary
compatible with GLIBC. EGLIBC's goals include reduced footprint,
PATCH
quilt import ../change-readme.diff -P any/change-readme.diff
quilt push
quilt import ../set-prefix.diff -P any/set-prefix.diff
quilt push
dpkg-buildpackage -us -uc -S -ai386
Here's the relevant output I'm seeing:
dpkg-source -b eglibc-2.19
dpkg-source: info: using options from eglibc-2.19/debian/source/options: --compression=xz
dpkg-source: info: using source format `3.0 (quilt)'
dpkg-source: info: building eglibc using existing ./eglibc_2.19.orig.tar.xz
patching file debian/rules
Reversed (or previously applied) patch detected! Skipping patch.
1 out of 1 hunk ignored
dpkg-source: info: fuzz is not allowed when applying patches
dpkg-source: info: if patch 'any/set-prefix.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -g0 -E -b -B .pc/any/set-prefix.diff/ --reject-file=- < eglibc-2.19.orig.yzNU0V/debian/patches/any/set-prefix.diff gave error exit status 1
dpkg-buildpackage: error: dpkg-source -b eglibc-2.19 gave error exit status 2
Commenting out the quilt import and quilt push commands for set-prefix.diff results in success but of course the prefix isn't updated like I want. It seems like patches are getting applied multiple times which is fine for most files but not debian/rules - maybe these patches are applied on a fresh source directory but the debian/ directory is left in tact?
What is the recommended way to build libc6 with a custom prefix without having dpkg-buildpackage/dpkg-source fail due to reapplying patches?
The debian/rules directory is special [citation needed] and shouldn't be patched using the usual quilt commands. You can modify them directly before building the package or use the patch command (patch -p1 in this case).
I try to get into XDP, for this I have this very small program:
// SPDX-License-Identifier: GPL-2.0
#include <linux/bpf.h>
#include "bpf/bpf_helpers.h"
#include "xdpsock.h"
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, MAX_SOCKS);
__uint(key_size, sizeof(int));
__uint(value_size, sizeof(int));
} xsks_map SEC(".maps");
SEC("xdp_sock") int xdp_sock_prog(struct xdp_md *ctx) {
return XDP_DROP;
}
But if I try to load it into a virtual interface veth-basic02, I get this error:
$ sudo ip -force link set dev veth-basic02 xdp object xdpsock_kern.o
section xdp_sock
Prog section 'xdp_sock' rejected: Operation not permitted (1)!
- Type: 6
- Instructions: 2 (0 over limit)
- License:
Verifier analysis:
Error fetching program/map!
Kernel-Version: 5.3.0-28-generic
This is the Makefile I am using:
OBJS = xdpsock_kern.o
LLC ?= llc
CLANG ?= clang
INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`
EXTRA_CFLAGS ?= -O2 -emit-llvm
# In case up-to-date headers are not installed locally in /usr/include,
# use source build.
linuxhdrs ?= /usr/src/linux-headers-5.1.0-050100
LINUXINCLUDE = -I$(linuxhdrs)/arch/x86/include/uapi \
-I$(linuxhdrs)/arch/x86/include/generated/uapi \
-I$(linuxhdrs)/include/generated/uapi \
-I$(linuxhdrs)/include/uapi \
-I$(linuxhdrs)/include \
-I/bpf
prefix ?= /usr/local
INSTALLPATH = $(prefix)/lib/bpf
install_PROGRAM = install
install_DIR = install -dv
all: $(OBJS)
.PHONY: clean
clean:
rm -f $(OBJS)
INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`
$(OBJS): %.o:%.c
$(CLANG) $(INC_FLAGS) \
-D__KERNEL__ -D__ASM_SYSREG_H \
-Wno-unused-value -Wno-pointer-sign \
-Wno-compare-distinct-pointer-types \
-Wno-gnu-variable-sized-type-not-at-end \
-Wno-address-of-packed-member -Wno-tautological-compare \
-Wno-unknown-warning-option \
-I../include $(LINUXINCLUDE) \
$(EXTRA_CFLAGS) -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $#
install: $(OBJS)
$(install_DIR) -d $(INSTALLPATH) ; \
$(install_PROGRAM) $^ -t $(INSTALLPATH)
uninstall: $(OBJS)
rm -rf $(INSTALLPATH)
Lockdown:
$ dmesg | grep Lockdown
[ 1.283355] Lockdown: swapper/0: Hibernation is restricted; see man kernel_lockdown.7
[ 11.313219] Lockdown: systemd: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
[ 11.337794] Lockdown: systemd: BPF is restricted; see man kernel_lockdown.7
[ 17.147844] Lockdown: Xorg: ioperm is restricted; see man kernel_lockdown.7
Edit:
echo 1 > /proc/sys/kernel/sysrq + echo x > /proc/sysrq-trigger + Alt+SysRq+x indeed solves the problem - I can finally load the XDP-Program! Funny easter egg though. Thank you #Qeole!
eBPF: Operation not permitted
There are several possible causes for a permission error (-EPERM returned by bpf(), which you can observe with strace -e bpf <command>) when working with eBPF. But no so many. Usually, they fall under one of the following items:
User does not have the required capabilities (CAP_SYS_ADMIN, CAP_NET_ADMIN, ... typically depending on the types of the programs being used). This is usually solved by running as root, who has all necessary capabilities. In your case you run with sudo, so you are covered.
Creating the BPF object (new map, or loading a program) would exceed the limit for the amount of memory that can be locked in the kernel by the user. This is usually solved (for root) by using ulimit -l <something_big> in the terminal, or setrlimit() in a C program. Very unlikely in your case, your program is very small and you did not mention having a lot of BPF objects loaded on your system.
There are a few more possibilies, like trying to write on maps that are “frozen” or read-only etc., or trying to use function calls for non-root users. These are usually for more advanced use cases and should not be hit with a program as simple as yours.
Lockdown, Secure Boot, EFI and (unfortunate) backports for bpf() restrictions
But the problem that you seem to be hitting could be related to something else. “Lockdown” is a security module that was merged into Linux 5.5 kernel. It aims at preventing users to modify the running Linux image. It turns out that several distributions decided to backport Lockdown to their kernels, and sometimes they picked patches that predated the final version that was merged to mainline Linux.
Ubuntu and Fedora, for example, have a bunch of custom patches to backport that feature to the kernels used in Disco/19.04 and Eoan/19.10 (kernel 5.3 for the latter, I don't remember for Disco). It includes a patch that completely disables the bpf() system call when Lockdown is activated, meaning that creating maps or loading BPF programs is not possible. Also, they enabled Lockdown by default when Secure Boot is activated, which, I think, is the default for machines booting with EFI.
See also this blog post: a good way to check if Lockdown is affecting your BPF usage is to try and load minimal programs, or to run dmesg | grep Lockdown to see if it says something like:
Lockdown: systemd: BPF is restricted; see man kernel_lockdown.7
So for Ubuntu 19.04 and 19.10, for example, you have to disable Lockdown to work with eBPF. This may be done with a physical stroke of the SysRq key + x (I have not tested), but NOT by writing to /proc/sysrq-trigger (Ubuntu disabled it for this operation). Alternatively, you can disable Secure Boot (in the BIOS or with mokutil, search for the relevant options on the Internet, and do not forget to check the security implications).
Note that Linux kernel 5.4 or newest has the mainline restrictions for bpf(), which do not deactivate the system call, so Focal/20.04 and newest will not be affected. Upgrading to a new kernel might thus be another workaround. I filed a ticket a few days ago to ask for this change to be backported (instead of deactivating bpf()) and the work is in progress, so by the time new readers look at the answer Lockdown impact on eBPF might well be mitigated (Edit: Should be fixed on Ubuntu 19.10 with kernel 5.3.0-43). Not sure how other distros handle this. And it will still have strong implications for tracing with eBPF, though.
I try to get into XDP, for this I have this very small program:
// SPDX-License-Identifier: GPL-2.0
#include <linux/bpf.h>
#include "bpf/bpf_helpers.h"
#include "xdpsock.h"
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, MAX_SOCKS);
__uint(key_size, sizeof(int));
__uint(value_size, sizeof(int));
} xsks_map SEC(".maps");
SEC("xdp_sock") int xdp_sock_prog(struct xdp_md *ctx) {
return XDP_DROP;
}
But if I try to load it into a virtual interface veth-basic02, I get this error:
$ sudo ip -force link set dev veth-basic02 xdp object xdpsock_kern.o
section xdp_sock
Prog section 'xdp_sock' rejected: Operation not permitted (1)!
- Type: 6
- Instructions: 2 (0 over limit)
- License:
Verifier analysis:
Error fetching program/map!
Kernel-Version: 5.3.0-28-generic
This is the Makefile I am using:
OBJS = xdpsock_kern.o
LLC ?= llc
CLANG ?= clang
INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`
EXTRA_CFLAGS ?= -O2 -emit-llvm
# In case up-to-date headers are not installed locally in /usr/include,
# use source build.
linuxhdrs ?= /usr/src/linux-headers-5.1.0-050100
LINUXINCLUDE = -I$(linuxhdrs)/arch/x86/include/uapi \
-I$(linuxhdrs)/arch/x86/include/generated/uapi \
-I$(linuxhdrs)/include/generated/uapi \
-I$(linuxhdrs)/include/uapi \
-I$(linuxhdrs)/include \
-I/bpf
prefix ?= /usr/local
INSTALLPATH = $(prefix)/lib/bpf
install_PROGRAM = install
install_DIR = install -dv
all: $(OBJS)
.PHONY: clean
clean:
rm -f $(OBJS)
INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`
$(OBJS): %.o:%.c
$(CLANG) $(INC_FLAGS) \
-D__KERNEL__ -D__ASM_SYSREG_H \
-Wno-unused-value -Wno-pointer-sign \
-Wno-compare-distinct-pointer-types \
-Wno-gnu-variable-sized-type-not-at-end \
-Wno-address-of-packed-member -Wno-tautological-compare \
-Wno-unknown-warning-option \
-I../include $(LINUXINCLUDE) \
$(EXTRA_CFLAGS) -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $#
install: $(OBJS)
$(install_DIR) -d $(INSTALLPATH) ; \
$(install_PROGRAM) $^ -t $(INSTALLPATH)
uninstall: $(OBJS)
rm -rf $(INSTALLPATH)
Lockdown:
$ dmesg | grep Lockdown
[ 1.283355] Lockdown: swapper/0: Hibernation is restricted; see man kernel_lockdown.7
[ 11.313219] Lockdown: systemd: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
[ 11.337794] Lockdown: systemd: BPF is restricted; see man kernel_lockdown.7
[ 17.147844] Lockdown: Xorg: ioperm is restricted; see man kernel_lockdown.7
Edit:
echo 1 > /proc/sys/kernel/sysrq + echo x > /proc/sysrq-trigger + Alt+SysRq+x indeed solves the problem - I can finally load the XDP-Program! Funny easter egg though. Thank you #Qeole!
eBPF: Operation not permitted
There are several possible causes for a permission error (-EPERM returned by bpf(), which you can observe with strace -e bpf <command>) when working with eBPF. But no so many. Usually, they fall under one of the following items:
User does not have the required capabilities (CAP_SYS_ADMIN, CAP_NET_ADMIN, ... typically depending on the types of the programs being used). This is usually solved by running as root, who has all necessary capabilities. In your case you run with sudo, so you are covered.
Creating the BPF object (new map, or loading a program) would exceed the limit for the amount of memory that can be locked in the kernel by the user. This is usually solved (for root) by using ulimit -l <something_big> in the terminal, or setrlimit() in a C program. Very unlikely in your case, your program is very small and you did not mention having a lot of BPF objects loaded on your system.
There are a few more possibilies, like trying to write on maps that are “frozen” or read-only etc., or trying to use function calls for non-root users. These are usually for more advanced use cases and should not be hit with a program as simple as yours.
Lockdown, Secure Boot, EFI and (unfortunate) backports for bpf() restrictions
But the problem that you seem to be hitting could be related to something else. “Lockdown” is a security module that was merged into Linux 5.5 kernel. It aims at preventing users to modify the running Linux image. It turns out that several distributions decided to backport Lockdown to their kernels, and sometimes they picked patches that predated the final version that was merged to mainline Linux.
Ubuntu and Fedora, for example, have a bunch of custom patches to backport that feature to the kernels used in Disco/19.04 and Eoan/19.10 (kernel 5.3 for the latter, I don't remember for Disco). It includes a patch that completely disables the bpf() system call when Lockdown is activated, meaning that creating maps or loading BPF programs is not possible. Also, they enabled Lockdown by default when Secure Boot is activated, which, I think, is the default for machines booting with EFI.
See also this blog post: a good way to check if Lockdown is affecting your BPF usage is to try and load minimal programs, or to run dmesg | grep Lockdown to see if it says something like:
Lockdown: systemd: BPF is restricted; see man kernel_lockdown.7
So for Ubuntu 19.04 and 19.10, for example, you have to disable Lockdown to work with eBPF. This may be done with a physical stroke of the SysRq key + x (I have not tested), but NOT by writing to /proc/sysrq-trigger (Ubuntu disabled it for this operation). Alternatively, you can disable Secure Boot (in the BIOS or with mokutil, search for the relevant options on the Internet, and do not forget to check the security implications).
Note that Linux kernel 5.4 or newest has the mainline restrictions for bpf(), which do not deactivate the system call, so Focal/20.04 and newest will not be affected. Upgrading to a new kernel might thus be another workaround. I filed a ticket a few days ago to ask for this change to be backported (instead of deactivating bpf()) and the work is in progress, so by the time new readers look at the answer Lockdown impact on eBPF might well be mitigated (Edit: Should be fixed on Ubuntu 19.10 with kernel 5.3.0-43). Not sure how other distros handle this. And it will still have strong implications for tracing with eBPF, though.
I want to find inside a Makefile, whether I'm running on a 32-bit or 64-bit machine.
Is there any macro or environment variables that can be simply accessed from within Makefile?
I found this related answer on-line:
LBITS := $(shell getconf LONG_BIT)
ifeq ($(LBITS),64)
# do 64 bit stuff here, like set some CFLAGS
else
# do 32 bit stuff here
endif
However, I doubt in all system getconf tool would be available or not.
tl;dr;
If all you need is to get the number of bits in long type in C in a portable way, it's ok to use getconf LONG_BIT.
Otherwise:
to detect host architecture, use config.guess shell script
to detect target architecture, use gcc -dumpmachine or gcc --help=target
Detecting host architecture
1) getconf
As mentioned by #Dummy00001, getconf is part of POSIX and is widely available, but it doesn't provide enough information.
Note that getconf LONG_BIT is just the number of bits in long type in C. E.g., it may be 32 on 16-bit or 64-bit machines. It's quite useless for determining host architecture.
2) uname
uname is part of POSIX as well, and it doesn't provide enough information too.
For example, it doesn't distinguish between hardfloat and softfloat ARM on Linux. Also, its architecture naming is not really portable.
3) /proc
You can use /proc to gather architecture info, e.g. /proc/cpuinfo on Linux. However, it's not portable and hard to parse.
4) config.guess
Finally, I recommend GNU config.guess script (source code). It is a standalone script that you can copy to your project. It is written in portable shell and should work on any UNIX.
$ sh config.guess
x86_64-pc-linux-gnu
This script is used in autotools, but you can use it without autotools too.
Detecting default target architecture of your toolchain
It usually makes sense to detect target architecture of toolchain being used. It's different from host architecture when cross-compiling.
1) gcc -E
You can parse gcc -E output to get toolchain tuple for target architecture of gcc:
$ gcc -v -E - </dev/null |& grep Target:
Target: x86_64-linux-gnu
This should work on any UNIX if you use GCC or clang.
Notes:
It prints the value of --target option passed to configure script when GCC was built. It's not affected by current compilations flags passed to GCC, like -m32 or -march.
Unlike config.guess, this toolchain tuple is distro-specific. E.g., different schemes are used in prebuilt toolchains in Debian and Gentoo.
2) gcc -dumpmachine
It seems that gcc -dumpmachine prints the same value as previous recipe:
$ gcc -dumpmachine
x86_64-linux-gnu
It works with GCC and clang.
3) gcc -print-multiarch
Another way to get toolchain tuple:
$ gcc -print-multiarch
x86_64-linux-gnu
It works with GCC, but not clang. Also, this option is known not to work in various cases:
on Debian, it's empty if multilib is disabled
on Gentoo, it's always empty
as mentioned by #Dummy00001, it may be empty for cross-compilation toolchains (I guess it depends on how toolchain was built)
Detecting target architecture depending on current flags
Some GCC options like -m32 or -march can affect target architecture.
1) gcc --help=target
This will print value of -march option deduced from default target architecture (configured when GCC was built) and current compilation options.
$ gcc -Q --help=target |& grep -e -march | awk '{print $2}'
x86-64
$ gcc -m32 -Q --help=target |& grep -e -march | awk '{print $2}'
i686
$ gcc -march=i386 -Q --help=target |& grep -e -march | awk '{print $2}'
i386
It doesn't work with clang.
The getconf is a POSIX tool, it should be available on all unix-like systems. But uname -m would be more portable and more traditional way to find the type of the system.
Otherwise, with high probability, you are looking in the wrong direction: the bit-ness of the host system doesn't necessarily match the bit-ness of the tool-chain. Tool-chain (compiler, linker) define what bit-ness the result would be. And the tool-chain could be customized by the user to accommodate for the local needs (e.g. compiling as 32-bit on a 64-bit OS), or even cross-compilation.
To find the bit-ness of the GCC (or clang) in your tool-chain, you can use that:
LBITS := $(shell $(CC) $(CFLAGS) -dM -E - </dev/null | grep -q "__LP64__" && echo 64 || echo 32)
Effectively, it queries what bit-ness the application is going to be compiled with, without doing actual compilation, but peeking at the compiler defines for the LP parameter, which denotes the "data model" of the architecture (LP64 means "long and pointer are 64-bit" (and, by omission, int is 32 bit); ILP32 means "int, long and pointer are 32-bit").
As long as you stay on the Linux, that would work. (You need to use the CFLAGS in the compiler invocation, since CFLAGS overridden by the user could use the -m32 or -m64 options to change the bit-ness of the output code.)
A similiar (less descriptive) post is here.
I'm trying to roll my own tool chain using recent releases on Ubuntu and was hoping there was enough of a Linux community here that I could get specific advice on certain problems as they come up.
With everyone's help I'd like to see this turn into a useful reference eventually.
First off "Crosstool" to which nearly everyone refers is a little (2006) out of date and "Crosstool-NG" which seems to now be carrying the torch fails on my machine. I'd rather debug the process itself than a script (which it would seem requires me to understand the process).
Below are basic steps of what I've got working so far; at present I'm stuck trying to compile the first pass GCC.
Where it's failing ...
It's failing because the cross compiled library containing "crti.o" is missing:
# ./gcc-4.4.1/configure --target=arm-linux --disable-thread --enable-langauges=c
/bin/bash ../../../gcc-4.4.1/libgcc/../mkinstalldirs .
/usr/src/gnu-4.4.1-build/./gcc/xgcc -B ........
/usr/local/arm-linux/bin/ld: crti.o No such file: No such file or directory
collect2: ld returned 1 exit status
make[2]: *** [libgcc_s.so] Error 1
make[2]: Leaving directory `/usr/src/gnu/gcc-4.4.1-build/arm-linux/libgcc'
make[1]: *** [all-target-libgcc] Error 2
make[1]: Leaving directory `/usr/src/gnu/gcc-4.4.1-build'
make: *** [all] Error 2
Build steps
On a 'freshly' configured Ubuntu 9.04 installation, here are the steps I've done so far:
#New configuration of Ubuntu 9.04
sudo updatedb
sudo apt-get install build-essential subversion
# For kernel configuration
sudo apt-get install libncurses5-dev
# For building GCC
sudo apt-get install libgmp3-dev libmpfr-dev
#Get Linux Headers for GCC/GLIBC compilations
# I use a hacked Linux from Artilla,
pushd ~ && svn co http://.../linux m501-linux && cd !$
make ARCH=arm m501_defconfig
make ARCH=arm menuconfig
sudo mkdir /usr/local/arm-linux/include
sudo cp -dR include/asm-arm /usr/local/arm-linux/include/asm
sudo cp -dR include/linux /usr/local/arm-linux/include/linux
cd /usr/local/arm-linux/
sudo ln -s include sys-include
popd
#Get sources:
cd /usr/src/
sudo su root
mkdir gnu
ftp ftp.gnu.org
# get gnu/binutils/binutils-2.19.1.tar.bz2
# get gnu/gcc/gcc-4.4.1/gcc-4.4.1.tar.bz2
# get gnu/glibc/glibc-2.10.1.tar.bz2
# get gnu/gdb/gdb-6.8.tar.bz2
#Build Binutils
bzcat binutils-2.19.1.tar.bz2 | tar -xv
mkdir binutils-2.19.1-build && cd !$
cp ../binutils-2.19.1/gas/config/tc-arm.c ./tc-arm.c
sed -r 's/(as_bad[ \t]*\()(.+\[.+\]\))/\1\"%s\",\2/' < ./tc-arm.c > ../binutils-2.19.1/gas/config/tc-arm.c
rm ./tc-arm.c
../binutils-2.19.1/configure --target=arm-linux
make && make install && cd ..
#Build GCC
bzcat gcc-4.4.1.tar.bz2 | tar -xv
mkdir gcc-4.4.1-build && cd !$
../gcc-4.4.1/configure --target=arm-linux --disable-thread --enable-langauges=c -with-headers=/usr/local/arm-linux/include
make
Welcome, you're not alone.
The story
I don't know why ARM cross-compiling is such a nightmare. It's not my opinion, let's see, what others say...
Building a gcc / glibc cross-toolchain
for use in embedded systems
development used to be a scary
prospect, requiring iron will, days if
not weeks of effort, lots of Unix and
Gnu lore, and sometimes willingness to
take dodgy shortcuts. ( http://www.kegel.com/crosstool/ )
My ARM computer (GuruPlug) is running on Debian, so I just need a standard G++ compiler, without any tweaks.
I'm using 32-bit Ubuntu on my notebook. There are deb packages for AVR cross-compiler, or even for Z80, but none for ARM - why? OK, we have to compile one. Let's go. The compilation process of the toolchain is a bit confusing for me. 14k lines long Makefile, thank you.
After some days (and nights) I've failed.
The solution
Finally, I've found an out-of-the box soluion. I've just downloaded the lite edition of this stuff: http://www.codesourcery.com/sgpp/lite_edition.html and now I'm happy. It has an ugly installer, but it works. It says:
arm-none-linux-gnueabi-g++ (Sourcery G++ Lite 2010q1-202) 4.4.1, which is an up-to-date G++ version.
(My friend has a Mac, and he has also failed compiling the toolchain after fighting with it for a week. He is now using this compiler on a VM running Ubuntu.)
Here is the HelenOS arm-32 toolchain installation script, this should do what you want with very little fiddling. I'm using it on Ubuntu now (I'm one of the HelenOS developers). It was written by Martin Decky:
#!/bin/bash
# Cross-Compiler Toolchain for ${PLATFORM}
# by Martin Decky <martin#decky.cz>
#
# GPL'ed, copyleft
#
check_error() {
if [ "$1" -ne "0" ]; then
echo
echo "Script failed: $2"
exit
fi
}
if [ -z "${CROSS_PREFIX}" ] ; then
CROSS_PREFIX="/usr/local"
fi
BINUTILS_VERSION="2.19.1"
GCC_VERSION="4.3.3"
BINUTILS="binutils-${BINUTILS_VERSION}.tar.gz"
GCC_CORE="gcc-core-${GCC_VERSION}.tar.bz2"
GCC_OBJC="gcc-objc-${GCC_VERSION}.tar.bz2"
GCC_CPP="gcc-g++-${GCC_VERSION}.tar.bz2"
BINUTILS_SOURCE="ftp://ftp.gnu.org/gnu/binutils/"
GCC_SOURCE="ftp://ftp.gnu.org/gnu/gcc/gcc-${GCC_VERSION}/"
PLATFORM="arm"
WORKDIR=`pwd`
TARGET="${PLATFORM}-linux-gnu"
PREFIX="${CROSS_PREFIX}/${PLATFORM}"
BINUTILSDIR="${WORKDIR}/binutils-${BINUTILS_VERSION}"
GCCDIR="${WORKDIR}/gcc-${GCC_VERSION}"
OBJDIR="${WORKDIR}/gcc-obj"
echo ">>> Downloading tarballs"
if [ ! -f "${BINUTILS}" ]; then
wget -c "${BINUTILS_SOURCE}${BINUTILS}"
check_error $? "Error downloading binutils."
fi
if [ ! -f "${GCC_CORE}" ]; then
wget -c "${GCC_SOURCE}${GCC_CORE}"
check_error $? "Error downloading GCC Core."
fi
if [ ! -f "${GCC_OBJC}" ]; then
wget -c "${GCC_SOURCE}${GCC_OBJC}"
check_error $? "Error downloading GCC Objective C."
fi
if [ ! -f "${GCC_CPP}" ]; then
wget -c "${GCC_SOURCE}${GCC_CPP}"
check_error $? "Error downloading GCC C++."
fi
echo ">>> Creating destionation directory"
if [ ! -d "${PREFIX}" ]; then
mkdir -p "${PREFIX}"
test -d "${PREFIX}"
check_error $? "Unable to create ${PREFIX}."
fi
echo ">>> Creating GCC work directory"
if [ ! -d "${OBJDIR}" ]; then
mkdir -p "${OBJDIR}"
test -d "${OBJDIR}"
check_error $? "Unable to create ${OBJDIR}."
fi
echo ">>> Unpacking tarballs"
tar -xvzf "${BINUTILS}"
check_error $? "Error unpacking binutils."
tar -xvjf "${GCC_CORE}"
check_error $? "Error unpacking GCC Core."
tar -xvjf "${GCC_OBJC}"
check_error $? "Error unpacking GCC Objective C."
tar -xvjf "${GCC_CPP}"
check_error $? "Error unpacking GCC C++."
echo ">>> Compiling and installing binutils"
cd "${BINUTILSDIR}"
check_error $? "Change directory failed."
./configure "--target=${TARGET}" "--prefix=${PREFIX}" "--program-prefix=${TARGET}-" "--disable-nls"
check_error $? "Error configuring binutils."
make all install
check_error $? "Error compiling/installing binutils."
echo ">>> Compiling and installing GCC"
cd "${OBJDIR}"
check_error $? "Change directory failed."
"${GCCDIR}/configure" "--target=${TARGET}" "--prefix=${PREFIX}" "--program-prefix=${TARGET}-" --with-gnu-as --with-gnu-ld --disable-nls --disable-threads --enable-languages=c,objc,c++,obj-c++ --disable-multilib --disable-libgcj --without-headers --disable-shared
check_error $? "Error configuring GCC."
PATH="${PATH}:${PREFIX}/bin" make all-gcc install-gcc
check_error $? "Error compiling/installing GCC."
echo
echo ">>> Cross-compiler for ${TARGET} installed."
Sorry for any line wrapping, should be easy to fix. If you want a pastebin or http link to get it, just leave a comment.
You should really have put more effort with using crosstool-NG, since the crosstool mailing list is very reactive. Since understanding the whole process is a huge task, understanding how to use a tool that you might reuse is IMHO more interesting.
I recently built a GNU toolchain for ARM using crosstool-ng. It took a bit of trial and error to figure out which versions of the toolchain components would play nice together, but I finally got it working with the following versions:
binutils 2.19.1
gcc 4.3.2
gmp 4.2.2
linux kernel 2.6.29.1
mpfr 2.3.1
uClibc 0.9.30.1
See if these work for you too.
Also, OpenWrt as a part of it's build process creates a cross-compiling toolchain. You may want to try that, selecting one of the ARM-based boards as a target, even if you aren't interested in making a wireless router.
If you're really want to build entire toolchain for yourself:
http://frank.harvard.edu/~coldwell/toolchain/
http://ftp.snapgear.org/pub/snapgear/tools/arm-linux/build-arm-linux-3.4.4
Take in mind, though, as you should search for toolchain compatibilty matrix or you may run into weird compilation errors.
If you still have the option for crosstool-ng, this is what I'm was working on for last few days:
http://blog.stranadurakov.com/2009/08/04/how-to-arm-linux/
Here you will find my crosstool-ng configuration file, which had worked for me.
I've had good luck using buildroot to build toolchains and libraries for cross-compiling Linux. Buildroot uses the lightweight uclibc C library rather than glibc, so it might not work for your purposes. (And getting the toolchain this way might not be quite as exciting and compiling it yourself.)
This was a bit of red-herring. Apparently cross-compling a tool chain for an existing system with a working GLIBC doesn't require GLIBC to be recompiled. I honestly don't know how I ignored this fact.
So even though I still don't know how to cross compile GCC and GLIBC, as I only need GCC I'm going to mark this as closed.
If I can, I'll come back to this post and mark what I eventually did.
EDIT:
Checkout this.