Why does this makefile only work with a single job? - multithreading

Question

I'm trying to write a Makefile which compiles a cross compiler.
It downloads GCC and binutils, extract the archive, run the configure script and run make.
This is the Makefile I wrote:
# Versions
GCC_VERSION := 6.3.0
BINUTILS_VERSION := 2.27
# Build
TARGET ?= i686-elf
PREFIX := $(realpath build)
# Phony tasks
.PHONY: all clean gcc binutils
# Targets
BINUTILS_TARGETS := build/bin/${TARGET}-addr2line build/bin/${TARGET}-ar \
build/bin/${TARGET}-as build/bin/${TARGET}-c++filt \
build/bin/${TARGET}-elfedit build/bin/${TARGET}-gprof \
build/bin/${TARGET}-ld build/bin/${TARGET}-ld.bfd build/bin/${TARGET}-nm \
build/bin/${TARGET}-objcopy build/bin/${TARGET}-objdump \
build/bin/${TARGET}-ranlib build/bin/${TARGET}-readelf \
build/bin/${TARGET}-size build/bin/${TARGET}-strings \
build/bin/${TARGET}-strip
GCC_TARGETS := build/bin/${TARGET}-cpp build/bin/${TARGET}-gcc \
build/bin/${TARGET}-gcc-${GCC_VERSION} build/bin/${TARGET}-gcc-ar \
build/bin/${TARGET}-gcc-nm build/bin/${TARGET}-gcc-ranlib \
build/bin/${TARGET}-gcov build/bin/${TARGET}-gcov-tool
# Aliases
all: binutils gcc
gcc: ${GCC_TARGETS}
binutils: ${BINUTILS_TARGETS}
# Download the packages
packages/gcc-${GCC_VERSION}.tar.bz2:
curl ftp://ftp.gnu.org/gnu/gcc/gcc-${GCC_VERSION}/gcc-${GCC_VERSION}.tar.bz2 \
-o $#
packages/binutils-${BINUTILS_VERSION}.tar.bz2:
curl ftp://ftp.gnu.org/gnu/binutils/binutils-${BINUTILS_VERSION}.tar.bz2 \
-o $#
# Extract them
packages/gcc-${GCC_VERSION}/configure: packages/gcc-${GCC_VERSION}.tar.bz2
tar -xmjf $< -C packages
packages/binutils-${BINUTILS_VERSION}/configure: packages/binutils-${BINUTILS_VERSION}.tar.bz2
tar -xmjf $< -C packages
# Build them
${GCC_TARGETS}: packages/gcc-${GCC_VERSION}/configure
mkdir -p build/gcc
cd build/gcc && ../../packages/gcc-${GCC_VERSION}/configure \
--target=${TARGET} --prefix=${PREFIX} --disable-nls \
--enable-languages=c --without-headers && make all-gcc && \
make all-target-libgcc && make install-gcc && make install-target-libgcc
${BINUTILS_TARGETS}: packages/binutils-${BINUTILS_VERSION}/configure
mkdir -p build/binutils
cd build/binutils && ../../packages/binutils-${BINUTILS_VERSION}/configure \
--target=${TARGET} --prefix=${PREFIX} --with-sysroot --disable-nls \
--disable-werror && make && make install
# Clean everything
clean:
rm -rf build/*
rm -rf packages/*
It works when running make with no options, but when I use the jobs option, it fails (but not everytime): it download binutils and GCC in parallel.
When binutils is fully downloaded, it extracts it and compiles it, then the GCC download ends, but it doesn't do anything and there are no errors.
Also sometimes the compilation of GCC or binutils fails with strange errors (I will edit this post with some of them).

Answer 1

Not sure about the exact source of your specific build failures, but the Makefile is problematic: the 2 build rules have multiple targets. Make does not support this well, see here for an explanation.

Answer 2

Make with jobs fails because it starts multiple instances of the same recipe but these instances can't run together correctly.
${GCC_TARGETS}: packages/gcc-${GCC_VERSION}/configure
mkdir -p build/gcc
cd build/gcc && ../../packages/gcc-${GCC_VERSION}/configure ... make ...
This is trouble maker, I guess, because Make with jobs initiates multiple configure invocations in the same directory to run asynchronously. So you have multiple configure scripts running and adding bits to the same directory. And then multiple instances of Make are invoked in the same build directory. Perfect mess.
Something like:
${GCC_TARGETS}: packages/gcc-${GCC_VERSION}/configure
mkdir -p build/gcc/$(nodir $#) && cd build/gcc/$(nodir $#) ...
Should fix the problem by means of starting configure & make sequences in different directories. You need similar fix for ${BINUTILS_TARGETS} too.