Makefile not recognizing already generated sources - linux

I'm trying to generate an RPM with a makefile, the behavior I'm expecting from the makefile is the following:
If RPM doesn't exist and the sources are not yet prepared, go ahead and do both: generate sources and then the RPM
If RPM already exist but the sources changed, go ahead and prepare the sources and generate the RPM once again
If the sources haven't changed and the RPM already exist don't do anything
However, right now the behavior I'm getting from the makefile below is not quite the way I want it, as it recognizes whether the RPM exist but when it comes down to the sources it doesn't really recognize that they already exist.
Here's the makefile:
SHELL = /bin/bash
.SHELLFLAGS = -o pipefail -c
COLORIZE := 2>&1 | sed -re "s/^(Executing|Wrote)(.*: )/"$$'\E'"[32m\1\2"$$'\E'"[0m/g" \
-e "s/(error[s]?)/"$$'\E'"[31m\1"$$'\E'"[0m/ig" \
-e "s/(warn|warning)/"$$'\E'"[33m\1"$$'\E'"[0m/ig"
SPEC := $(shell find . -name \*spec -printf '%f' -quit)
ARCH := $(shell rpm -q --qf '%{arch}' --specfile $(SPEC))
DIST := .el
NAME := $(basename $(SPEC))
RELEASE := $(shell rpm -q --qf '%{release}' --specfile $(SPEC) | cut -d. -f1)
VERSION := $(shell rpm -q --qf '%{version}' --specfile $(SPEC))
BUILDDIR := ./rpm-build
RPM := $(BUILDDIR)/RPMS/$(ARCH)/$(NAME)-$(VERSION)-$(RELEASE)$(DIST).$(ARCH).rpm
RPMBUILD := rpmbuild --define "_topdir %(pwd)/$(BUILDDIR)" \
--define "_source_filedigest_algorithm md5" \
--define "_binary_filedigest_algorithm md5" \
--define "_source_payload w9.gzdio" \
--define "_binary_payload w9.gzdio" \
--define "_sourcedir %{_topdir}/SOURCES" \
--define "_target_os linux" \
--define "dist .el"
SOURCE0 := $(BUILDDIR)/SOURCES/$(NAME)-$(VERSION).jar
.PHONY: all clean
all: $(RPM)
$(BUILDDIR):
#mkdir -p $(BUILDDIR)/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,TEMP}
$(SOURCE0): $(BUILDDIR) $(SPEC)
spectool -g -C $(BUILDDIR)/SOURCES $(SPEC)
$(RPM): $(SPEC) $(SOURCE0)
#echo -e "Building $(RPM)"
$(RPMBUILD) -bb $< $(COLORIZE)
clean:
#- $(RM) -rf ./$(BUILDDIR)
Is there anything I'm doing wrong for managing the sources, I just don't want them to be prepared everytime I run a make command ?

You should never have a target with a directory as a prerequisite, because directory timestamps are updated at unusual times. I shouldn't say "never"; it can be very useful but it means something quite different than what you think.
You can try using an order-only prerequisite for this:
$(BUILDDIR):
#mkdir -p $(BUILDDIR)/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,TEMP}
$(SOURCE0): $(SPEC) | $(BUILDDIR)
spectool -g -C $(BUILDDIR)/SOURCES $(SPEC)
Or you can just put the mkdir inside the recipe:
$(SOURCE0): $(SPEC)
#mkdir -p $(BUILDDIR)/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,TEMP}
spectool -g -C $(BUILDDIR)/SOURCES $(SPEC)
Note, you're using bash-specific issues here ({}). If you want to be portable you need to add:
SHELL := /bin/bash
to your makefile.

Related

Linux make command is deleting a source file

I have inherited a project file that has a Makefile in it that is doing something I have never seen before--It is injecting a rm command. I cannot find any reason for the rm command, so I am missing something very obvious or very esoteric.
Thanks
The results of running make are:
bison --defines --xml --graph=calc.gv -o calc.c calc.y
Bison flags =
cc -c -o calc.o calc.c
Making BASE = calc
cc -o calc calc.o
Done making BASE
rm calc.c <======== WHERE IS THIS COMING FROM?
The Makefile is:
BASE = calc
BISON = bison
XSLTPROC = xsltproc
all: $(BASE)
%.c %.h %.xml %.gv: %.y
$(BISON) $(BISONFLAGS) --defines --xml --graph=$*.gv -o $*.c $<
#echo "Bison flags = " $(BISONFLAGS)
$(BASE): $(BASE).o
#echo "Making BASE = " $(BASE)
$(CC) $(CFLAGS) -o $# $^
#echo "Done making BASE"
run: $(BASE)
#echo "Type arithmetic expressions. Quit with ctrl-d."
./$<
html: $(BASE).html
%.html: %.xml
$(XSLTPROC) $(XSLTPROCFLAGS) -o $# $$($(BISON) --print-datadir)/xslt/xml2xhtml.xsl $<
CLEANFILES = $(BASE) *.o $(BASE).[ch] $(BASE).output $(BASE).xml $(BASE).html $(BASE).gv
clean:
#echo "Running clean" $(CLEANFILES)
rm -f $(CLEANFILES)
See https://www.gnu.org/software/make/manual/make.html#Chained-Rules:
The second difference is that if make does create b in order to update something else, it deletes b later on after it is no longer needed. Therefore, an intermediate file which did not exist before make also does not exist after make. make reports the deletion to you by printing a rm -f command showing which file it is deleting.

Compile a kernel module (error stage 2, MODPOST 0)

I'm trying to compile a sample kernel module (test.c) on my Arch linux just typing make in the directory with source and Makefile.
It creates empty folder .tmp_versions and empty files Module.symvers and modules.order, but no .ko
I also tried to compile it under virtual machine with Ubuntu, but I have absolutely the same output. Samples were from the Internet (2 ones).
What I should do else to compile it?
Makefile:
obj−m += test.o
default:
$(MAKE) -C /lib/modules/$(shell uname -r)/build SUBDIRS=$(PWD) modules
clean:
$(MAKE) -C $(KDIR) M=$(PWD) clean
But it shows me:
make -C /lib/modules/4.18.14-arch1-1-ARCH/build SUBDIRS=/home/hime/Documents/module modules
make[1]: Entering directory '/usr/lib/modules/4.18.14-arch1-1-ARCH/build'
Building modules, stage 2.
MODPOST 0 modules
make[1]: Leaving directory '/usr/lib/modules/4.18.14-arch1-1-ARCH/build'
or (verbose):
make -C "/lib/modules/4.18.14-arch1-1-ARCH/build" SUBDIRS="/home/hime/Documents/module" modules V=1
make[1]: Entering directory '/usr/lib/modules/4.18.14-arch1-1-ARCH/build'
test -e include/generated/autoconf.h -a -e include/config/auto.conf || ( \
echo >&2; \
echo >&2 " ERROR: Kernel configuration is invalid."; \
echo >&2 " include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it."; \
echo >&2 ; \
/bin/false)
mkdir -p /home/hime/Documents/module/.tmp_versions ; rm -f /home/hime/Documents/module/.tmp_versions/*
make -f ./scripts/Makefile.build obj=/home/hime/Documents/module
(cat /dev/null; ) > /home/hime/Documents/module/modules.order
make -f ./scripts/Makefile.modpost
find /home/hime/Documents/module/.tmp_versions -name '*.mod' | xargs -r grep -h '\.ko$' | sort -u | sed 's/\.ko$/.o/' | scripts/mod/modpost -m -a -i ./Module.symvers -I /home/hime/Documents/module/Module.symvers -o /home/hime/Documents/module/Module.symvers -S -w -s -T -
make[1]: Leaving directory '/usr/lib/modules/4.18.14-arch1-1-ARCH/build'

Why does "make all" seem to skip "all:" rule?

I am trying to analyze the following makefile and reproduce its "behavior" step by step.
Although I type "make all" it seems this makefile skips the "all:" line and jumps straight to "build/*.o" (hence the echo's).
The file and its corresponding output:
TOOLCHAIN ?= arm-none-eabi-
SOURCES = Demo/main.c \
Demo/startup.c \
Demo/Drivers/rpi_gpio.c \
Demo/Drivers/rpi_irq.c \
Source/tasks.c \
Source/list.c \
Source/portable/GCC/RaspberryPi/port.c \
Source/portable/GCC/RaspberryPi/portisr.c \
Source/portable/MemMang/heap_4.c
OBJECTS = $(patsubst %.c,build/%.o,$(SOURCES))
INCDIRS = Source/include Source/portable/GCC/RaspberryPi \
Demo/Drivers Demo/
CFLAGS = -Wall $(addprefix -I ,$(INCDIRS))
CFLAGS += -D RPI2
CFLAGS += -march=armv7-a -mtune=cortex-a7 -mfloat-abi=hard -mfpu=neon-vfpv4
ASFLAGS += -march=armv7-a -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard
LDFLAGS =
.PHONY: all clean
all: $(MOD_NAME)
echo "in all"
$(MOD_NAME): $(OBJECTS)
echo "in mod name"
ld -shared $(LDFLAGS) $< -o $#
build/%.o: %.c
echo -e "\nin build/*.o:*.c\n"
mkdir -p $(dir $#)
$(TOOLCHAIN)gcc -c $(CFLAGS) $< -o $#
build/%.o: %.s
echo -e "in build/*.o:*.s\n"
mkdir -p $(dir $#)
$(TOOLCHAIN)as $(ASFLAGS) $< -o $#
all: kernel7.list kernel7.img kernel7.syms kernel7.hex
echo -e"in kernel all\n"
$(TOOLCHAIN)size kernel7.elf
kernel7.img: kernel7.elf
$(TOOLCHAIN)objcopy kernel7.elf -O binary $#
echo -e "in kernel7.img\n"
kernel7.list: kernel7.elf
echo -e "kernel7.list\n"
$(TOOLCHAIN)objdump -D -S kernel7.elf > $#
kernel7.syms: kernel7.elf
echo -e "kernel7.syms\n"
$(TOOLCHAIN)objdump -t kernel7.elf > $#
kernel7.hex : kernel7.elf
echo -e "kernel7.hex\n"
$(TOOLCHAIN)objcopy kernel7.elf -O ihex $#
kernel7.elf: $(OBJECTS)
echo -e "kernel7.elf\n"
$(TOOLCHAIN)ld $^ -static -Map kernel7.map -o $# -T Demo/raspberrypi.ld
clean:
rm -f $(OBJECTS)
rm -f kernel7.list kernel7.img kernel7.syms
rm -f kernel7.elf kernel7.hex kernel7.map
rm -rf build
echo -e "cleaning \n"
I tried to replicate this behaviour myself with a tiny piece of code. But it doesn't seem to work:
SOURCES = Demo/Drivers/rpi_irq.c \
Demo/Drivers/rpi_gpio.c
OBJECTS = $(patsubst %.c,build/%.o,$(SOURCES))
.PHONY: all clean
all: $(MOD_NAME)
echo "making all"$(SOURCES)
$(MOD_NAME): $(OBJECTS)
echo "MOD_NAME"
build/%.o:%.c
mkdir -p $(dir $#)
arm-none-eabi-gcc -march=armv7-a -mcpu=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=har $< -o $#
As you can see thanks to the echo's my code just doesn't even build my source code. I'd expect it to go from all->MOD_NAME->build. (This is all the output I get)
So my questions are:
How does the makefile I am analyzing manage to go straight to build/*.o?
Why does my implementation, which I think should do the same doesn't even compile my source code?
The Makefile that you copied contains 2 rules for "all".
The first depends on $(MOD_NAME) which might be empty.
The second rule depends on multiple files "kernel7.*" which themselves depend on "kernel7.elf".
Finally "kernel7.elf" depends on $(OBJECTS).
This last rule is responsible that all your source files will be compiled.
The first rule with $(MOD_NAME) does not need to cause any compilation at all.
In your own Makefile you only have a rule for "all" depending on $(MOD_NAME).
If $(MOD_NAME) is empty in your Makefile as well, you do not have any dependency for "all" at all.
If "all" does not depend on anything, no source files will be compiled.
To solve your problem you need to provide some content for $(MOD_NAME).
all is trying to build $(MOD_NAME), which has dependencies of $(OBJECTS), which it is trying to build.
There are two all's here, which is a problem.

Header in subfolder not found

my compiler does not found a header in a subfolder included in a cpp file
#include "comm\ComPublic.h"
My master makefile :
SUB_POUET=Pouet \
PouetTech \
PouetPrivate
all: $(SUB_POUET)
PouetTech:
#(cd PouetTech && $(MAKE))
Pouet:
#(cd Pouet && $(MAKE))
PouetPrivate:
#(cd PouetPrivate && $(MAKE))
.PHONY: clean mrproper $(SUB_POUET)
clean:
#(cd $(SUB_POUET) && $(MAKE) $#)
mrproper: clean
#(cd $(SUB_POUET) && $(MAKE) $#)
Sub-makefile1 :
include ../Definitions.mif
SRC=$(wildcard *.cpp)
OBJ_FILES=$(addprefix $(OBJECTS_DIR)/,$(notdir $(SRC:.cpp=.o)))
INCLUDES=-I../PouetTech -I../../libs/MPSSE -I../../libs/FTDI/$(OS) -I../shared -I../../libs/libmodbus-3.0.6/src
CXXFLAGS+=$(INCLUDES)
all: $(OBJ_FILES)
$(OBJECTS_DIR)/%o: %cpp | objects_dir
$(CXX) -o $# -c $< $(CXXFLAGS)
objects_dir:
mkdir -p $(OBJECTS_DIR)
.PHONY: clean mrproper
clean:
rm -rf *.o
Sub-makefile 2
include ../Definitions.mif
SRC=$(wildcard *.cpp)
OBJ_FILES=$(addprefix $(OBJECTS_DIR)/,$(notdir $(SRC:.cpp=.o)))
INCLUDES=-I../PouetTech -I../Pouet -I../shared
CXXFLAGS+=$(INCLUDES)
all: $(OBJ_FILES)
$(OBJECTS_DIR)/%o: %cpp | objects_dir
$(CXX) -o $# -c $< $(CXXFLAGS)
objects_dir:
mkdir -p $(OBJECTS_DIR)
.PHONY: clean mrproper
clean:
rm -rf *.o
Sub-makefile 3 is identical to 2.
And my folder are like this:
src
|_makefile
|_Pouet
| |_makefile
|_PouetTech
| |_makefile
|_PouetPrivate
| |_makefile
|_shared
|_comm
|_ComPublic.h
The problem appears during the 3rd makefile (PouetPrivate), but the folder "shared" is included in the makefile.
If I add -I../shared/comm to the makefile and change the include to #include "ComPublic.h" it works. But I dont want to include manually all folder in the makefile.
It's probable just a simple mistake, but I dont have much experience in makefile
Furthermore, its working fine when using mingw / msys under windows or Visual Studio
Edit : and I have the same issue with another subfolder of "comm"
If the include directive in your source is #include "comm\ComPublic.h" as you've written, the problem is simply that you're using the Windows path delimeter \. Linux (and g++) want / and won't recognise the path.
Fortunately, Visual C++ will accept both delimiters so / can be used throughtout.

Build on shell condition with Make

I would like to add to a build list the packages I want to build if it is not installed yet.
The goal is to install some Python packages without pip and from local sources. I don't have access to pip...
So I wrote I Makefile that looks like:
all: natsort foo bar foobar ...
natsort: natsort-4.0.4.tar.gz
tar xvzf $<
cd $(patsubst %.tar.gz,%, $<) && python setup.py install
rm -rf $(patsubst %.tar.gz,%, $<)
python -c 'import natsort'
echo -e "Installation of $< [done]\n" >> install.log
The problem with this implementation is that all the packages will be rebuild and reinstalled each time I run the Make command. I would like to check if the module is already installed. My idea is to do something like this:
ifdef $(shell python -c 'import natsort')
all: natsort
endif
How can I rewrite this to make it works?
You can absolutely do something like that. But it "costs" a shell invocation and an invocation of python every time you run make and that's a relatively high cost.
There are, basically, two ways to do what you want in a cheaper manner.
A stamp file and short-circuiting logic in the recipe.
The stamp file method is basically what you have except you add touch $# to the end of the recipe.
natsort: natsort-4.0.4.tar.gz
tar xvzf $<
cd $(patsubst %.tar.gz,%, $<) && python setup.py install
rm -rf $(patsubst %.tar.gz,%, $<)
python -c 'import natsort'
echo -e "Installation of $< [done]\n" >> install.log
touch $#
That way running the recipe the first time creates the stamp file and until natsort-4.0.4.tar.gz becomes newer than the stamp file or the stamp file gets deleted the recipe will never run again.
Note that second point though. Delete the stamp file and you install again.
That's the thing that the short-circuit logic solution solves.
Instead of your original rule of multiple commands you wrap it all in one command (this is optional but saves repeated checking costs) and check for the module to be installed before doing any work.
natsort: natsort-4.0.4.tar.gz
if ! python -c 'import natsort'; then \
tar xvzf $< || exit 1; \
cd $(patsubst %.tar.gz,%, $<) && python setup.py install || exit 1; \
rm -rf $(patsubst %.tar.gz,%, $<); \
python -c 'import natsort' || exit 1; \
echo -e "Installation of $< [done]\n" >> install.log; \
fi
Note the need to add || exit 1 since we no longer have make handling that for us. Also note that this now always runs the rule (and we should mark natsort and .PHONY) but in the most common case it will stop after the if test.
That all said you can combine these methods to get the best of both worlds.
natsort: natsort-4.0.4.tar.gz
if ! python -c 'import natsort'; then \
tar xvzf $< || exit 1; \
cd $(patsubst %.tar.gz,%, $<) && python setup.py install || exit 1; \
rm -rf $(patsubst %.tar.gz,%, $<); \
python -c 'import natsort' || exit 1; \
echo -e "Installation of $< [done]\n" >> install.log; \
fi
touch $#
and you get the benefits of both methods. The first time you run make the natsort file doesn't exist and the recipe is run. natsort isn't installed so the if test fails and the installation occurs. After that the natsort file it touched. The next time make is installed natsort is newer than natsort-4.0.4.tar.gz so make doesn't think it has anything to do. If, for some reason, you delete the natsort file then the next time make runs it checks for the module to exist, skips the installation and touches the natsort file again to get back into sync.

Resources