Make ignores the rule when run for the first time - linux

SO
I can't find out why these lines are not called for the first time I run 'make' but are called the next time:
sb_path = sb
sb_src := $(sb_path)/src
sb_build := $(sb_path)/build
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb: | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
$(sb_src)/last_git_commit_info.h: | commit_sb ;
endif
I'm just curious because there is no file generate-commit-info file and make crashes when I call it for the second time, but it compiles my program ok for the first try.
I use script on my local machine to copy sources over ssh to another machine and to run compile.sh script there:
...
scp -r $sbfolder/build $sbfolder/Makefile "$buildserver:$root/$curdate"
check_retcode
scp -r $sbfolder/sb/Makefile "$buildserver:$root/$curdate/sb/"
...
ssh $buildserver "$root/compile.sh $curdate $debug"
compile.sh:
# fix Makefile: we don't have git installed here
#DO_NOT_GENERATE_COMMIT_INFO=true
#now we can compile sb
curdir="/home/tmp/kamyshev/sb_new/$1"
cd $curdir
check_retcode
t_path=$curdir
debug=$2
config=RELEASE
if [[ debug -eq 1 ]]; then
config=DEBUG
fi
echo "building sb... CONFIG=$config"
make -j2 CONFIG=$config
check_retcode
As you see DO_NOT_GENERATE_COMMIT_INFO=true is commented out. So I just don't see a reason why the code is not run when I call a make or the script for the first time (either from the remote script or myselft from command line).
Do you have any clues?
UPDATE on Etan Reisner comment:
commit_sb target is checked, it does not exist, so it's rule is being run and it updates last_git_commit_info.h. Thus it forces to update the .h file. It also gives me a .PHONY target commit_sb so I could do it directly by calling make commit_sb.
The generate-commit-info also creates a file in a $(sb_bin) folder.
My another guess is that you are talking about a better way to organize this code.
I can update last_git_commit_info.h directly with a such rule:
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:

Thanks to the commenters on my question I've done some additional research: I've tried to make a minimal complete example. And this led me to the answer.
My code generates dependency files (look at -MMD command in SB_CXXFLAGS):
# just example - in real Makefile these are calculated on the fly
sb_deps := file1.d file2.d [...]
# rules with dependances of .o files against .h files
-include $(sb_deps)
SB_CXXFLAGS = $(CXXFLAGS) [...] -MMD
# compile and generate dependency info;
$(sb_obj)/%.o:$(sb_src)/%.cpp
$(CXX) $(SB_CXXFLAGS) $< -o $#
And when I run make for the first time there no *.d files, so no *.cpp depends on last_git_commit_info.h file and the rule is not applied.
On the subsequent runs the dependency rule appears in one of *.d files, the rule is executed and I get the error.
UPDATE: This does not concern the question directly, but this is the better way to write these rules:
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:
endif

Related

Is there a way to define custom implicit GNU Make rules?

I'm often creating png files out of dot (graphviz format) files. The command to do so is the following:
$ dot my_graph.dot -o my_graph.png -Tpng
However, I would like to be able to have a shorter command format like $ make my_graph.dot to automatically generate my png file.
For the moment, I'm using a Makefile in which I've defined the following rule, but the recipe is only available in the directory containing the Makefile
%.eps: %.dot
dot $< -o $# -Teps
Is it possible to define custom implicit GNU Make recipes ? Which would allow the above recipe to be available system-wide
If not, what solution do you use to solve those kind of problem ?
Setup:
Fedora Linux with ZSH/Bash
You could define shell functions in your shell's startup files, e.g.
dotpng()
{
echo dot ${1%.dot}.dot -o ${1%.dot}.png -Tpng;
}
This function can be called like
dotpng my_graph.dot
or
dotpng my_graph
The code ${1%.dot}.dot strips .dot from the file name if present and appends it (again) to allow both my_graph.dot and my_graph as function argument.
Is it possible to define custom implicit GNU Make recipes ?
Not without modifying the source code of GNU Make.
If not, what solution do you use to solve those kind of problem ?
I wouldn't be a fan o modyfying the system globally, but you could do:
Create a file /usr/local/lib/make/myimplicitrules.make with the content
%.eps: %.dot
dot $< -o $# -Teps
Use include /usr/local/lib/make/myimplicitrules.make in your Makefile.
I would rather use a git submodule or similar to share common configuration between projects, rather than depending on global configuration. Depending on global environment will make your program hard to test and non-portable.
I would rather go with a shell function, something along:
mymake() {
make -f <(cat <<'EOF'
%.eps: %.dot
dot $< -o $# -Teps
EOF
) "$#"
}
mymake my_graph.dot
GNU Make lets you specify extra makefiles to read using the MAKEFILES
environment variable. Quoting from info '(make)MAKEFILES Variable':
the default goal is never taken from one of these makefiles (or any
makefile included by them) and it is not an error if the files listed
in 'MAKEFILES' are not found
if you are running 'make' without a specific makefile, a makefile
in 'MAKEFILES' can do useful things to help the built-in implicit
rules work better
As an example, with no makefile in the current directory and the
following .mk files in make's include path (e.g. via
MAKEFLAGS=--include-dir="$HOME"/.local/lib/make/) you can create
subdir gen/ and convert my_graph.dot or dot/my_graph.dot by
running:
MAKEFILES=dot.mk make gen/my_graph.png
To further save some typing it's tempting to add MAKEFILES=dot.mk
to a session environment but defining MAKEFILES in startup files
can make things completely nontransparent. For that reason I prefer
seeing MAKEFILES=… on the command line.
File: dot.mk
include common.mk
genDir ?= gen/
dotDir ?= dot/
dotFlags ?= $(if $(DEBUG),-v)
Tvariant ?= :cairo:cairo
vpath %.dot $(dotDir)
$(genDir)%.png $(genDir)%.svg $(genDir)%.eps : %.dot | $(genDir).
dot $(dotFlags) $< -o $# -T'$(patsubst .%,%,$(suffix $#))$(Tvariant)'
The included common.mk is where you'd store general definitions to
manage directory creation, diagnostics etc., e.g.
.PRECIOUS: %/. ## preempt 'unlink: ...: Is a directory'
%/. : ; $(if $(wildcard $#),,mkdir -p -- $(#D))
References:
?= = := … - info '(make)Reading Makefiles'
vpath - info '(make)Selective Search'
order-only prerequisites (e.g. | $(genDir).) - info '(make)Prerequisite Types'
.PRECIOUS - info '(make)Chained Rules'

why "cmd_$#" is the previous command when build linux kernel

within linux kernel source repo, there is Makefile.build under /scripts, which is called many times when building src. there is some target : prerequisite like this:
$(obj)/%.i: $(src)/%.c FORCE
$(call if_changed_dep,cpp_i_c)
and if_changed_dep is
if_changed_dep = $(if $(newer-prereqs)$(cmd-check),$(cmd_and_fixdep),#:)
newer-prereqs is quite straightforward but cmd-check is a bit obsecure.
cmd-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(cmd_$#))), \
$(subst $(space),$(space_escape),$(strip $(cmd_$1))))
I know that $(cmd_$1) will be expanded to cmd_cpp_i_c, which is the current compiling command
and $(cmd_$#) will be expanded to $(cmd_$(obj)/%.i). for instance if it compiles i2c-core-base.c, it will be $(cmd_i2c-core-base.i) (I omit $(obj))
https://flylib.com/books/en/2.860.1.84/1/ says it is the previous command when compiling.
my question is where I am able to find the evidence since I could not find where cmd_$# is defined.
Thanks a lot for any comments.
After executing the command, the macro cmd_and_savecmd, records the command line into the file ..cmd.
In /scripts/Kbuild.include
cmd_and_savecmd = \
$(cmd); \
printf '%s\n' 'cmd_$# := $(make-cmd)' > $(dot-target).cmd
As make is invoked again during a rebuild, it will include those .*.cmd files.
In /Makefile
-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
So, cmd_$# is used to keep tracks of what command has already been done last time when building a file.

compiling a makefile that has an extenstion .x86 [duplicate]

I have a makefile in a directory of mine which builds scripts with certain environment variables set. What if I want to create another makefile in the same directory with different environment variables set? How should I name the two make files? Does makefile.1 and makefile.2 work? How do I call them?
You can give sensible names to the files like makefile.win and makefile.nix and use them:
make -f makefile.win
make -f makefile.nix
or have a Makefile that contains:
win:
make -f makefile.win
nix:
make -f makefile.nix
and use make win or make nix
You can name makefile whatever you want. I usually name it like somename.mk. To use it later you need to tell make what makefile you want. Use -f option for this:
make -f somename.mk
Actually you can have two set of environment variables in the same make file. for example
COMPILER = gcc
CCFLAGS1 = -g
CCFLAGS2 = -Wall
a: main.c
${COMPILER} ${CCFLAGS1} main.c
b: test.c
${COMPILER} ${CCFLAGS2} test.c
then you can just say make a or make b. Depending on what you want.
Also it is possible with -f flag to call which makefile you want to call.
You can do something like this rather than using multiple makefiles for the same purpose. You can pass the environment or set a flag to the same makefile. For eg:
ifeq ($(ENV),ENV1)
ENV_VAR = THIS
else
ENV_VAR = THAT
endif
default : test
.PHONY : test
test:
#echo $(ENV_VAR)
Then you can simply run the make command with arguments
make ENV=ENV1
I have two makefiles in the same directory. Many of the recipes have identical names and here are two solutions:
1. Prefix in make
proja_hello:
#echo "hello A"
projb_hello:
#echo "hello N"
2. Keep two separate files
Project A has makefile. Type make hello.
Project B has a separate make file called projb.mk. Type bmake hello.
This works since I've added alias bmake ='make -f projb.mk to my .bashrc. Note! This command can be called anywhere but only works where projb.mk exists.
Note! You lose autocompletion of make with the alias and typing make -f projb.mk hello is not better than typing make projb_hello.

Makefile isn't rebuilding dependencies?

Fair warning: I'm something of a newb at using makefiles, so this may be something obvious. What I'm trying to do is to use make to run a third-party code generation tool when and only when the source files for that generation tool (call them .abc files) change. I referenced the example at http://www.cmcrossroads.com/ask-mr-make/6795-rebuilding-when-a-files-checksum-changes which shows how to build MD5s, and I tweaked the idea a bit:
File: abc.mk
target = all
files := $(wildcard Abc/*.abc)
bltfiles := $files $(addsuffix .built,$files)
all: $bltfiles
%.built: %.abc %.abc.md5
#echo "Building $*"
# #Command that generates code from a .abc file
#touch $#
%.md5: FORCE
#echo "Checking $* for changes..."
# #Command to update the .md5 file, if the sum of the .abc file is different
FORCE:
What I'm intending to happen is for each .abc file to have two auxilary files: .abc.built & .abc.md5 . The .built file is just a dummy target & timestamp for the last time it was built, as the code produced by the generation tool cannot be readily defined as a target. The .md5 file contains a hash of the last known content of the .abc file. It should only be updated when the hash of the file changes.
However, the .built file is only created if it doesn't exist. The .md5 rule never runs at all, and the .built rule doesn't re-build even if the .abc file has a newer timestamp. Am I doing something wrong?
Update:
For posterity, here's the version I got to work:
File: abc.mk
# Call this makefile as: make all --file=abc.mk
# Default Target
target = all
COMP_ABC_FILES := $(wildcard Abc/*.abc)
COMP_BLT_FILES := $(COMP_ABC_FILES) $(addsuffix .built, $(COMP_ABC_FILES) )
# This line is needed to keep make from deleting intermediary output files:
.SECONDARY:
# Targets:
.PHONY: all
all: $(COMP_BLT_FILES)
Abc/%.abc.built: Abc/%.abc Abc/%.abc.md5
#echo "Building $*"
# #Command that generates code from a .abc file
#touch $#
%.md5: FORCE
#echo "Checking $* for changes..."
#$(if $(filter-out $(shell cat $# 2>/dev/null),$(shell md5sum $*)),md5sum $* > $#)
# Empty rule to force re-build of files:
FORCE:
clean:
#echo "Cleaning .built & .md5 files..."
#rm Abc/*.built
#rm Abc/*.md5
Fixing your makefile in three places:
target = all
files := $(wildcard Abc/*.abc)
bltfiles := $(files) $(patsubst %.abc,%.built,$(files))
all: $(bltfiles)
#Abc/%.abc.built: Abc/%.abc Abc/%.abc.md5
%.built: %.abc %.abc.md5
#echo "Building $*"
# #Command that generates code from a .abc file
#touch $#
%.md5: FORCE
#echo "Checking $* for changes..."
# #Command to update the .md5 file, if the sum of the .abc file is different
FORCE:
Note the changes:
bltfiles := $(files) $(patsubst %.abc,%.built,$(files))
Results in "Abc/a.built Abc/b.built" instead of "Abc/a.abc.built Abc/b.abc.built", which was required given how the rule for %.built was defined
all: $(bltfiles)
As above, with $(files), '$bltfiles' needed to be $(bltfiles), since otherwise make will interpret this as $(f)iles and $(b)ltfiles instead.
Tip: Having an editor with syntax highlighting for makefiles is nice here
DEMO
mkdir -pv Abc; touch Abc/{a,b,c,d,e,f,g}.abc
make -Bs -f abc.mk
output like
Checking Abc/e.abc for changes...
Building Abc/e
Checking Abc/g.abc for changes...
Building Abc/g
Checking Abc/b.abc for changes...
Building Abc/b
Checking Abc/f.abc for changes...
Building Abc/f
Checking Abc/a.abc for changes...
Building Abc/a
Checking Abc/c.abc for changes...
Building Abc/c
Checking Abc/d.abc for changes...
Building Abc/d
As sehe fixed but didn't explain: Makefile syntax isn't the same as shell syntax. By default (for reasons lost to history) make variables are only one character long. If you want a longer variable name, you have to put it in parenthesis so it parses correctly. Writing $files, for example, actually expands the string "iles" because make parses and expands only the value of the "f" variable (which is empty).
Yes, it's weird. But it's the way make works. Always put your variables in parentheses.

Recursive make is recursing too much and requires a dummy prerequisite

I have a very simple Makefile that isn't doing what I expect it would do. The ultimate goal is that it should call itself recursively, including the appropriate file each time, resulting in a build specific to what was included (I'm building several projects that all share the same code base, but utilize different combinations of the source files). I've never really dealt with recursive calls to make, so I must be missing something obvious. At the moment, I only have one .mk file in the same folder as my Makefile. It's a simple one-liner just for the purposes of this test. It will eventually contain various per-project settings.
Makefile:
SHELL = /bin/sh
ifdef MYFILE
include $(MYFILE)
PROGRAM = $(basename $(MYFILE))
endif
all: $(wildcard *.mk)
dummy:
#echo -- Entering dummy stub ... why do I need this?
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
$(PROGRAM): objs
#echo Time to link!
objs:
#echo Building objs!
test.mk
SOMEVAR = SomeValue
I have the following two problems:
Problem 1
If I remove the dummy prerequisite from my pattern rule, the pattern rule never gets called (I get the dreaded 'Nothing to be done for all' error). Is there a way I can get the recipes under the %.mk rule to run without needing that dummy prerequisite?
Problem 2
Given the two aforementioned files, I would expect make to do the following:
make[1] starts and hit the all rule
make[1] jumps down to the %.mk pattern rule
make[1] calls itself recursively (the call would look like make MYFILE=test.mk test)
make[2] starts, includes the test.mk file, and sets up the PROGRAM variable
make[2] jumps down to the $(PROGRAM) rule (since we were explicitly called with that target)
make[2] jumps to the objs rule, runs the recipes, and returns back up the chain
In actuality, make gets stuck on the %.mk pattern rule and enters an infinite loop. I don't understand why it's insisting on hitting the pattern rule, when I explicitly told it to build test in my first recursive call (which should correspond to the $(PROGRAM) target). What am I missing here?
Problem 0:
This is overdesigned. You don't need to use recursive Make here.
Problem 1:
The reason Make doesn't try to rebuild test.mk (without a dummy preq) is that test.mk is up to date. A better approach is to switch to a static pattern rule and use PHONY:
MKS = $(wildcard *.mk)
.PHONY: $(MKS)
$(MKS): %.mk:
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
An even better approach is not to use the name of a real file as a target of a rule that doesn't rebuild (or even "touch") that file.
Problem 2:
In make[2], the makefile includes test.mk. If a makefile includes another file, Make will attempt to rebuild that file before doing anything else. If there is a rule for that file (which there is) and if it succeeds (which it does) Make then reinvokes itself.
You should reconsider this design from the ground up. There are many ways to get the behavior you're looking for, depending on the specifics (how many variable will be defined in a foo.mk? do you really want to manage the build by manually moving those files around? and so on).
P.S. Here's one kludge that springs to mind. Whether it suits your case depends on the specifics:
makefile:
# includes nothing
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# -f $# $*
$(MAKE) MYFILE=$# -f $# $*
test.mk:
SOMEVAR = SomeValue
include makefile

Resources