within linux kernel source repo, there is Makefile.build under /scripts, which is called many times when building src. there is some target : prerequisite like this:
$(obj)/%.i: $(src)/%.c FORCE
$(call if_changed_dep,cpp_i_c)
and if_changed_dep is
if_changed_dep = $(if $(newer-prereqs)$(cmd-check),$(cmd_and_fixdep),#:)
newer-prereqs is quite straightforward but cmd-check is a bit obsecure.
cmd-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(cmd_$#))), \
$(subst $(space),$(space_escape),$(strip $(cmd_$1))))
I know that $(cmd_$1) will be expanded to cmd_cpp_i_c, which is the current compiling command
and $(cmd_$#) will be expanded to $(cmd_$(obj)/%.i). for instance if it compiles i2c-core-base.c, it will be $(cmd_i2c-core-base.i) (I omit $(obj))
https://flylib.com/books/en/2.860.1.84/1/ says it is the previous command when compiling.
my question is where I am able to find the evidence since I could not find where cmd_$# is defined.
Thanks a lot for any comments.
After executing the command, the macro cmd_and_savecmd, records the command line into the file ..cmd.
In /scripts/Kbuild.include
cmd_and_savecmd = \
$(cmd); \
printf '%s\n' 'cmd_$# := $(make-cmd)' > $(dot-target).cmd
As make is invoked again during a rebuild, it will include those .*.cmd files.
In /Makefile
-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
So, cmd_$# is used to keep tracks of what command has already been done last time when building a file.
Related
I'm often creating png files out of dot (graphviz format) files. The command to do so is the following:
$ dot my_graph.dot -o my_graph.png -Tpng
However, I would like to be able to have a shorter command format like $ make my_graph.dot to automatically generate my png file.
For the moment, I'm using a Makefile in which I've defined the following rule, but the recipe is only available in the directory containing the Makefile
%.eps: %.dot
dot $< -o $# -Teps
Is it possible to define custom implicit GNU Make recipes ? Which would allow the above recipe to be available system-wide
If not, what solution do you use to solve those kind of problem ?
Setup:
Fedora Linux with ZSH/Bash
You could define shell functions in your shell's startup files, e.g.
dotpng()
{
echo dot ${1%.dot}.dot -o ${1%.dot}.png -Tpng;
}
This function can be called like
dotpng my_graph.dot
or
dotpng my_graph
The code ${1%.dot}.dot strips .dot from the file name if present and appends it (again) to allow both my_graph.dot and my_graph as function argument.
Is it possible to define custom implicit GNU Make recipes ?
Not without modifying the source code of GNU Make.
If not, what solution do you use to solve those kind of problem ?
I wouldn't be a fan o modyfying the system globally, but you could do:
Create a file /usr/local/lib/make/myimplicitrules.make with the content
%.eps: %.dot
dot $< -o $# -Teps
Use include /usr/local/lib/make/myimplicitrules.make in your Makefile.
I would rather use a git submodule or similar to share common configuration between projects, rather than depending on global configuration. Depending on global environment will make your program hard to test and non-portable.
I would rather go with a shell function, something along:
mymake() {
make -f <(cat <<'EOF'
%.eps: %.dot
dot $< -o $# -Teps
EOF
) "$#"
}
mymake my_graph.dot
GNU Make lets you specify extra makefiles to read using the MAKEFILES
environment variable. Quoting from info '(make)MAKEFILES Variable':
the default goal is never taken from one of these makefiles (or any
makefile included by them) and it is not an error if the files listed
in 'MAKEFILES' are not found
if you are running 'make' without a specific makefile, a makefile
in 'MAKEFILES' can do useful things to help the built-in implicit
rules work better
As an example, with no makefile in the current directory and the
following .mk files in make's include path (e.g. via
MAKEFLAGS=--include-dir="$HOME"/.local/lib/make/) you can create
subdir gen/ and convert my_graph.dot or dot/my_graph.dot by
running:
MAKEFILES=dot.mk make gen/my_graph.png
To further save some typing it's tempting to add MAKEFILES=dot.mk
to a session environment but defining MAKEFILES in startup files
can make things completely nontransparent. For that reason I prefer
seeing MAKEFILES=… on the command line.
File: dot.mk
include common.mk
genDir ?= gen/
dotDir ?= dot/
dotFlags ?= $(if $(DEBUG),-v)
Tvariant ?= :cairo:cairo
vpath %.dot $(dotDir)
$(genDir)%.png $(genDir)%.svg $(genDir)%.eps : %.dot | $(genDir).
dot $(dotFlags) $< -o $# -T'$(patsubst .%,%,$(suffix $#))$(Tvariant)'
The included common.mk is where you'd store general definitions to
manage directory creation, diagnostics etc., e.g.
.PRECIOUS: %/. ## preempt 'unlink: ...: Is a directory'
%/. : ; $(if $(wildcard $#),,mkdir -p -- $(#D))
References:
?= = := … - info '(make)Reading Makefiles'
vpath - info '(make)Selective Search'
order-only prerequisites (e.g. | $(genDir).) - info '(make)Prerequisite Types'
.PRECIOUS - info '(make)Chained Rules'
I would like to use this Makefile both on Debian and Windows 10.
I have a set of measurements (and keep adding more) and the raw data files go into subdirectories called "measurement-12-Apr-2020" (with the date of the day I took the measurement). On those raw data files I run a series of post processing steps, each encapsulated in its own little tool, creating new output files and building on each other. Those different output files have distinct extensions: .raw for raw data files, .rot for coordinate rotations, .off for offset removal etc. I have pattern rules in my Makefile for those steps. For a few such steps I calculate special transformation matrices, that of course match just their set of data files. Those i call e.g. "transform.calibrate" or "transform.aligne"
So far the Makefile operated in just one directory and I moved the data files manually in and out of their directories.
Now I would like my top level Makefile with its pattern rules to operate on the data in the subdirectories.
How do i get it to look at the files in directories that match the "measurement-DATE" pattern? And how do i get the pattern rules to operate on one data set, using the local transformation files?
Here is my code with a recursive makefile. I would like a non-recursive makefile without the ugly
BIN := ${CURDIR}/../bin and half-good variable usage and content when the Makefile runs in the subdirectories. Furthermore i get an python error 9009 since i switched to the subdirs and the bin-directory (which is most likely related, but an other problem, really.).
ifdef OS
RM = del /Q
FixPath = $(subst /,\,$1)
COPY = copy
PYTHON = "c:\Users\Blah\AppData\Local\Programs\Python\Python37\python.exe"
else
ifeq ($(shell uname), Linux)
RM = rm -f
FixPath = $1
endif
COPY = cp
PYTHON = python
endif
.PHONY: test default
.DEFAULT_GOAL := default
# This is for running inside PyCharm
export PATH:=${CURDIR}/venv/bin:${PATH}
# file endings:
# raw
# calibrate file ending for transformation matrix
# cal calibrated data file
# ali alignement transformation
# rot rotated, aligned sensors
# off offset removed
# pha phase and
measurement_dirs := $(filter %/, $(wildcard messung-*/))
BIN := ../bin
DIRS_CMD = $(foreach subdir, $(measurement_dirs), make-rule/$(subdir))
# Makefiles = $(foreach subdir, $(measurements_dirs), $(subdir)Makefile )
default: transform.align transform.calibrate $(wildcard messung-*.pha)
all: $(DIRS_CMD)
#echo $(BIN)
transform.calibrate: calibrationsequence.raw
$(PYTHON) $(BIN)/calculateCalibration.py $<
%.cal: transform.calibrate %.raw
$(PYTHON) $(BIN)/applyCalibration.py $*.raw
transform.align: calibrationsequence.cal transform.calibrate
$(PYTHON) $(BIN)/calulateAlignement.py calibrationsequence.cal
%.rot: transform.align %.cal
$(PYTHON) $(BIN)/applyAlignment.py $*.cal
%.off: %.rot
$(PYTHON) $(BIN)/clearOffset.py $*.rot
%.pha: %.off
$(PYTHON) $(BIN)/demodulateSignals.py $*.off
test: test_%.py
$(PYTHON) pytest -v test
make-rule/%:
cp Makefile $* && cd $* && make
The heart of the problem is that two variables are involved and unknown beforehand: the names of the directories and the names of the raw files. Make can handle one variable easily, but struggles with two.
Suppose we have one directory, messung-12-Apr-2020, with three raw files alpha.raw, beta.raw and calibrationsequence.raw.
The rule is simple:
.PHONY: 12-Apr-2020
12-Apr-2020: messung-12-Apr-2020/apha.pha messung-12-Apr-2020/beta.pha messung-12-Apr-2020/calibrationsequence.pha
#echo $# done
If we do not know the names of the raw files beforehand, we can derive them:
RAW_FILES := $(wildcard messung-12-Apr-2020/*.raw)
PHA_FILES := $(patsubst %.raw,%.pha,$(RAW_FILES))
12-Apr-2020:$(PHA_FILES)
#echo $# done
If we do not know the name of the directory beforehand, we could use subst instead of patsubst and put everything into a pattern rule:
TARGETS := 12-Apr-2020
.SECONDEXPANSION:
$(TARGETS): %: $$(subst .raw,.pha,$$(wildcard messung-%/*.raw))
#echo $# done
(Note the use of $$.) This is safe as long as there is no chance of ".raw" embedded in a file name. There is a safer, more rigorous approach, but it is more complicated; if you need it, let me know in a comment and I will post it.
Now to generate the list of targets:
TARGETS := $(patsubst messung-%,%,$(wildcard messung-*))
.PHONY: all $(TARGETS)
all: $(TARGETS)
and modify the other pattern rules to be executed from the master directory:
%.pha: %.off
cd $(dir $*); $(PYTHON) $(BIN)/demodulateSignals.py $(notdir $*).off
(Clumsy, but effective.) It should be clear how to modify the pattern rules for %.off, %.rot, %.cal, %/transform.calibrate and %/transform.align the same way.
I want to run following environment check target checkenv before any of other targets,
all: build_sub_target1 build_target2
clean: clean_sub_target1 clean_target2
...
...
checkenv:
$(if $(PROJECT_ROOT), , \
$(error $(shell echo -e '\033[41;33mFATAL: Please load project settings first. \
Run "source PROJECT_ROOT_DIR/envsetup.sh"\033[0m')) \
)
I want every other target run checkenv target before they actually do their task, how can I do this? Any other way except that I add checkenv to the depends list of each targets?
Since I have many targets in this file, and I think it's not cool to add into each targets... Should there be any potential rules to do this?
Thanks a lot for your help!
You could use make conditionals for that, so that this condition is checked while the makefile is being read and before targets get evaluated:
ifndef PROJECT_ROOT
$(error "Please load project settings first. Run source PROJECT_ROOT_DIR/envsetup.sh")
endif
SO
I can't find out why these lines are not called for the first time I run 'make' but are called the next time:
sb_path = sb
sb_src := $(sb_path)/src
sb_build := $(sb_path)/build
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb: | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
$(sb_src)/last_git_commit_info.h: | commit_sb ;
endif
I'm just curious because there is no file generate-commit-info file and make crashes when I call it for the second time, but it compiles my program ok for the first try.
I use script on my local machine to copy sources over ssh to another machine and to run compile.sh script there:
...
scp -r $sbfolder/build $sbfolder/Makefile "$buildserver:$root/$curdate"
check_retcode
scp -r $sbfolder/sb/Makefile "$buildserver:$root/$curdate/sb/"
...
ssh $buildserver "$root/compile.sh $curdate $debug"
compile.sh:
# fix Makefile: we don't have git installed here
#DO_NOT_GENERATE_COMMIT_INFO=true
#now we can compile sb
curdir="/home/tmp/kamyshev/sb_new/$1"
cd $curdir
check_retcode
t_path=$curdir
debug=$2
config=RELEASE
if [[ debug -eq 1 ]]; then
config=DEBUG
fi
echo "building sb... CONFIG=$config"
make -j2 CONFIG=$config
check_retcode
As you see DO_NOT_GENERATE_COMMIT_INFO=true is commented out. So I just don't see a reason why the code is not run when I call a make or the script for the first time (either from the remote script or myselft from command line).
Do you have any clues?
UPDATE on Etan Reisner comment:
commit_sb target is checked, it does not exist, so it's rule is being run and it updates last_git_commit_info.h. Thus it forces to update the .h file. It also gives me a .PHONY target commit_sb so I could do it directly by calling make commit_sb.
The generate-commit-info also creates a file in a $(sb_bin) folder.
My another guess is that you are talking about a better way to organize this code.
I can update last_git_commit_info.h directly with a such rule:
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:
Thanks to the commenters on my question I've done some additional research: I've tried to make a minimal complete example. And this led me to the answer.
My code generates dependency files (look at -MMD command in SB_CXXFLAGS):
# just example - in real Makefile these are calculated on the fly
sb_deps := file1.d file2.d [...]
# rules with dependances of .o files against .h files
-include $(sb_deps)
SB_CXXFLAGS = $(CXXFLAGS) [...] -MMD
# compile and generate dependency info;
$(sb_obj)/%.o:$(sb_src)/%.cpp
$(CXX) $(SB_CXXFLAGS) $< -o $#
And when I run make for the first time there no *.d files, so no *.cpp depends on last_git_commit_info.h file and the rule is not applied.
On the subsequent runs the dependency rule appears in one of *.d files, the rule is executed and I get the error.
UPDATE: This does not concern the question directly, but this is the better way to write these rules:
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:
endif
I am trying to generate a make file in Linux that is fairly dynamic and will take get all the files from the /src directory of a certain type. Essentially the output of ls *.type I seem to be having difficulties in doing this. Below is what I currently have but it does not seem to work. Hopefully someone can help me out. Thanks!
JIL_B_TMPL : sh = ls *.type
JIL_LIST = $(JIL_B_TMPL)
I will also add this is not for compiling a C program.
To capture the output of a shell command in a makefile, you can do:
JIL_B_TMPL := $(shell ls *.type)
JIL_LIST := $(JIL_B_TMPL)
This is of course the same as writing:
JIL_LIST := $(shell ls *.type)
This works with GNU make, but since you mention Linux, I suppose you're using that.
Pat got the core of something that works, but in your case, you'll probably want something more like
JIL_LIST := $(wildcard *.type)
This gets rid of a call to an external program, which will be important if you decide in the future that you want to support Windows. Also, if you're using makepp, the wildcard function will also catch any .type files that can be built, regardless of whether or not they already have been.