I'm writing a static pattern rule to generate a list of dependencies for targets matching a pattern. The dependencies are generated through a shell command (the file content gives information about the dependencies). Here's an example of the explicit rule:
f1.o: $(shell gendep src/f1/f1.source)
... (some compilation command here) ...
While this works, I do not want to rewrite it for each new target since I'm maintaining the same file structure. My attempt at static pattern rule was like so:
%.o: $(shell gendep src/%/%.source)
...
I'm having trouble passing the stem (matched pattern for %) to the shell command. The shell command interprets it literally and operates on src/%/%.source, which of course doesn't exist.
I suspect there is way of passing the stem to the shell command but I don't seem to find it. Any experts here might be able to help me? Sorry if this is a newbie question (I'm indeed one).
What you're trying to do is difficult, because ordinarily Make will expand the $(shell ...) directive before running any rule, or even deciding which rules must be run. We can retard that by means of Secondary Expansion, a slightly advanced Make trick:
.SECONDEXPANSION:
%.o: $$(shell gendep src/$$*/$$*.source)
...
There are also other methods for automatic dependency generation.
Related
If you run make test on the following Makefile (with an otherwise empty directory):
test.%:
#echo $*
test: test.dummyextension
you get the following output:
dummyextension
o
cc test.o test.dummyextension -o test
clang: error: no such file or directory: 'test.o'
clang: error: no such file or directory: 'test.dummyextension'
clang: error: no input files
make: *** [test] Error 1
Why?
I suspect it has something todo with implicit rules, but I searched make -p on my machine, and can't find any implicit rules that match %: %. I would expect the output to simply be dummyextension, but it's almost like there's a phantom test.o file in my directory (despite my checking ten times that there is not).
If you put a ; after the test.dummyextension prerequisite, or add any content to the test rule, everything works as expected. This is the minimal failing example I can come up with, and I haven't a clue why you'd see this behaviour. Any ideas?
Make can chain multiple rules to create a target. In this case it has the following built-in rule:
%: %.o
$(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $#
This tells make that it can make test if it can find a way to make intermediate file test.o. So now it looks for a way to make test.o and it sees your pattern rule test.%:, which matches with stem o. So it has found a way!
You have also told make that test needs test.dummyextension, so it looks for a way to make that and again the pattern test.%: matches, this time with stem dummyextension.
So make first runs the test.% recipe twice to make the two prereqs. Then it runs the %: %.o recipe to make the final target. The $^ in the recipe is all prerequisites, so both test.o, gained from the built-in pattern rule, and test.dummyextension, gained by the explicit dependency in your Makefile, appear in the command.
You can test this by using the -r flag to disable built-in rules and then add the above pattern rule manually to your Makefile.
The key points to understand here are:
A line of the form:
test: test.dummyextension
Only adds a dependency to a target. It is not a rule to make the target. That can come from elsewhere. Make does not see this and decide test should be created with a blank recipe.
A stanza of the form:
test: test.dummyextension
;
This is a rule to make the target. Being an explicit rule it has a higher priority than a pattern rule that might also match. This does tell make it has found the rule to make test using the recipe ; and it stops looking for another rule.
Make will search for an implicit rule to make any target if it does not find an explicit one. If you don't want it to do this, you can either give it an explicit rule, like above, or declare the target as phony, with .PHONY: target. Implicit rules are not searched for phony targets.
Like
make,
scons has a large number of predefined variables and rules. (Try scons | wc on an SConstruct containing env = Environment(); print(env.Dump()) to see how extended the set is.)
But suppose we aren't after the wizardry of presets but rather want to do something a lot more primitive—simulating launching a few instructions from the (bash, etc) command line?
Also suppose we're quite happy with the default Decider('MD5'). What is the translation of the one-souce-one-target:
out/turquoise.xyz: out/chartreuse.xyz
chartreuse_to_turquoise $< $#
of the two-source-one-target:
out/purple.xyz: out/lilac.xyz out/salmon.xyz
gen_purple $< $#
and of:
run_this:
python prog.py
which we would run on-demand by typing make run_this?
What does the SConstruct for these elementary constructs look like?
All the answers you're looking for are in the users guide (and manpage)
Firstly, assuming you don't want to scan the input files to add included files specified in the input files, you can use Commmand()
(See info here: https://scons.org/doc/production/HTML/scons-user.html#chap-builders-commands)
Then you'll want an alias to specify an a non file command line target
(See here:https://scons.org/doc/production/HTML/scons-user.html#chap-alias)
Putting those two together yields
env=Environment()
# one source, one target
env.Command('out/turquoise.xyz', 'out/chartreuse.xyz', 'chartreuse_to_turquoise $SOURCE $TARGET')
# Two source, one target
env.Command('out/purple.xyz',['out/lilac.xyz','out/salmon.xyz'], 'gen_purple $SOURCES $TARGET')
# And your .phony make target which is actually not great for reproducibility and determining when it should be rerun, because you do not specify any sources or targets
env.Alias('run_this','python prog.py')
Note: SCons doesn't NOT propagate your shell environment variables. So if you depend on (for example) a non system path in your PATH, you'll need to explicitly specify that in env['ENV']['PATH'] for example. For more details take a read through the users guide, manpage and FAQ.
https://scons.org/doc/production/HTML/scons-user.html
https://scons.org/doc/production/HTML/scons-man.html
https://scons.org/faq.html
And you can reach the community directly via our discord server, IRC channel, or users mailing list
Summary
I am trying to understand a complicated chain of Makefiles, in order to get a build to succeed. I narrowed down my problem to this bit in our build script:
INF_RL=`make -f $BUILD_ROOT/Makefile BUILD_ROOT_MAKEFILE= show__BUILD_INF_RL`
$INF_RL/$BUILD_UTILS_RELDIR/BuildAll.sh
$INF_RL is being set to an empty string (or not being set). If I replace the first line with
INF_RL=/foo_rel_linx86/infrastructure_release/v8.0.14
in order to hardcode what I know $INF_RL is supposed to be, then the build goes smoothly. But I want to know how to fix this the proper way.
What I've Tried / Thought
My first thought was that make -f is failing. So I tried it in my shell:
% make -f $BUILD_ROOT/Makefile BUILD_ROOT_MAKEFILE= show__BUILD_INF_RL
% setenv | grep BUILD_ROOT
BUILD_ROOT=/userhome/andrew.cheong/TPS
Indeed, it returned an empty string. But what conclusion could I draw from this? I wasn't sure if the shell was the same thing as the environment / scope in which Make was chaining together its Makefiles. I abandoned this investigation.
Next, I looked into show__BUILD_INF_RL, which seemed to be defined in $BUILD_ROOT/Makefile:
BUILD_ROOT_MAKEFILE = 1
MAKE_DIRS = src
CASE_KITS = tpsIn tpsOut
REQUIRED_VERSIONS = "case.v$(INF_VS)"
all:
## These next 3 rules allows any variable set in this makefile (and therefore
## the included makefile.include to have it's value echoed from the command
## "make show_<variableName>"
## NOTE: the "disp" target is vital as it allows the show_% implicit rule to be
## recognised as such - implicit rules *must* have a target.
show_% := DISPLAY_MACRO = $(#:show_%=%)
show_% : disp
# echo $($(DISPLAY_MACRO))
disp:
include $(BUILD_ROOT)/makefile.include
Here, I faced more questions:
What is BUILD_ROOT_MAKEFILE for? Why is it set to 1, then seemingly something else in the make -f command?
In the make -f command, is BUILD_ROOT_MAKEFILE= its own argument? If so, what kind of target or rule is that? Otherwise, why is it being set to the macro?
In $BUILD_ROOT, there is another file, makefile.LINUX_X86.include:
BUILD_INF_RL = /foo_rel_linx86/infrastructure_release/v$(INF_VS)
$(warning $(BUILD_INF_RL))
BUILD_UTILS = $(BUILD_INF_RL)/build-utils_LINUX_X86
Though a completely ignorant guess, I think BUILD_INF_RL is being set here, and intended to be extracted into the build script's variable INF_RL when the macro show__BUILD_INF_RL is invoked. I added the middle line to see if it was indeed being set, and indeed, I get this output when running the build script:
/userhome/andrew.cheong/TPS/makefile.LINUX_X86.include:3: /foo_rel_linx86/infrastructure_release/v8.0.14
i.e. Looks like what I've hardcoded way above! But why doesn't it make it into INF_RL? There is yet another file, makefile.include, also in $BUILD_ROOT:
#
# INCLUDE THIS FILE AS THE LAST LINE IN THE LOCAL MAKEFILE
#
# makefile.include - use this file to define global build settings
# e.g. infrastructure version and location, or third-party
#
# supported macros in addition to build-utils-makefile.include
#
# BUILD_INF_RL : optional, specification of infrastructure release location
# defaults to vdev_build area
#
include $(BUILD_ROOT)/../../makefile.include.$(BUILD_ARCH).Versions
#include $(BUILD_UTILS)/makefile.archdef.include
include $(BUILD_ROOT)/makefile.$(BUILD_ARCH).include
$(warning $(BUILD_INF_RL))
_BUILD_INF_RL = $(BUILD_INF_RL)
# place the results at the root of the infdemo tree
BUILD_DEST = $(BUILD_ROOT)
INCLUDE_DIRS += $(BUILD_INF_RL)/core/$(BUILD_TARGET)/include
LINK_DIRS += $(BUILD_INF_RL)/core/$(BUILD_TARGET)/lib
# libraries required for a typical fidessa app, including OA and DB access
FIDEVMAPP_LIBS = FidApp FidInf FidCore Fidevm
include $(BUILD_UTILS)/makefile.include
That $(warning ...) is again mine, and when running the build script, I get:
/userhome/andrew.cheong/TPS/makefile.include:18: /foo_rel_linx86/infrastructure_release/v8.0.14
The Question
The fact that both $(warning ...)s show up when I run the build script that's calling the make -f ... show__BUILD_INF_RL, tells me that those Makefiles are being included. Then what is causing the macro to fail and return an empty string instead of the correct INF_RL path?
Historical Notes
These build scripts were written at a time when we were only compiling for Solaris. (The scripts were based on templates written by an infrastructure team that loosely accounted for both Solaris and Linux, but we never ran the Linux branch, as it was unnecessary.) We are now fully migrating to Linux, and hitting this issue. The reason I'm skeptical of it being a Linux versus Solaris issue is that we have at least four other products that use a similar Makefile chain and have been migrated with no issues. Not sure why this one in particular is behaving different.
Your question got very long and complex so I didn't read it all... for SO it's often better if you just ask a specific targeted question that you want to know the answer to, with a simple repro case.
I can't say why different makefiles behave differently, but this line:
show_% := DISPLAY_MACRO = $(#:show_%=%)
seems really wrong to me. This is (a) setting the variable show_%, which don't actually use anywhere, (b) to the simply expanded string DISPLAY_MACRO = because at this point in the makefile the variable $# is not set to any value.
Maybe you wanted this line to be this instead:
show_% : DISPLAY_MACRO = $(#:show_%=%)
(note : not :=) so that it's a pattern-specific variable assignment, not a simple variable assignment?
I'm trying to port linux kernel's kconfig util to my product
while compiling I got next error:
make[6]: *** No rule to make target `zconf.tab.c', needed by `zconf.tab.o'. Stop.
I found next rule in Makefile.lib for this file
$(obj)/%: $(src)/%_shipped
$(call cmd,shipped)
It looks ok for me and it just works in kernel but not in my product.
Then I added another rule right after previous one.
$(obj)/%c: $(src)/%c_shipped
$(call cmd,shipped)
And now it works just fine.
Can someone explain me what's wrong with original rule?
In my case obj=. and src=. (both = dot). Current dir contains appropriate *_shipped file.
My guess is that $(obj)/%: $(src)/%_shipped qualifies as a match-anything pattern rule. (The manual doesn't mention how targets and prerequisites with with directory components are handled, but it would make sense.)
Note the following in the manual:
A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
Since there are already built-in implicit rules for creating .c files (using parser generators for example), the match-anything rule is never considered.
The reason the error doesn't happen for the kernel makefiles is that they run make with -r, which eliminates built-in implicit rules. It's done in the top-level makefile by setting the MAKEFLAGS variable:
# Do not use make's built-in rules and variables
# (this increases performance and avoids hard-to-debug behaviour);
MAKEFLAGS += -rR
As a simple experiment, I created a file test.c_foo and the following makefile:
MAKEFLAGS += -r
%: %_foo
#echo building
make test.c without the first line gives
make: *** No rule to make target 'test.c'. Stop.
With the first line, it prints "building" instead.
I have a very simple Makefile that isn't doing what I expect it would do. The ultimate goal is that it should call itself recursively, including the appropriate file each time, resulting in a build specific to what was included (I'm building several projects that all share the same code base, but utilize different combinations of the source files). I've never really dealt with recursive calls to make, so I must be missing something obvious. At the moment, I only have one .mk file in the same folder as my Makefile. It's a simple one-liner just for the purposes of this test. It will eventually contain various per-project settings.
Makefile:
SHELL = /bin/sh
ifdef MYFILE
include $(MYFILE)
PROGRAM = $(basename $(MYFILE))
endif
all: $(wildcard *.mk)
dummy:
#echo -- Entering dummy stub ... why do I need this?
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
$(PROGRAM): objs
#echo Time to link!
objs:
#echo Building objs!
test.mk
SOMEVAR = SomeValue
I have the following two problems:
Problem 1
If I remove the dummy prerequisite from my pattern rule, the pattern rule never gets called (I get the dreaded 'Nothing to be done for all' error). Is there a way I can get the recipes under the %.mk rule to run without needing that dummy prerequisite?
Problem 2
Given the two aforementioned files, I would expect make to do the following:
make[1] starts and hit the all rule
make[1] jumps down to the %.mk pattern rule
make[1] calls itself recursively (the call would look like make MYFILE=test.mk test)
make[2] starts, includes the test.mk file, and sets up the PROGRAM variable
make[2] jumps down to the $(PROGRAM) rule (since we were explicitly called with that target)
make[2] jumps to the objs rule, runs the recipes, and returns back up the chain
In actuality, make gets stuck on the %.mk pattern rule and enters an infinite loop. I don't understand why it's insisting on hitting the pattern rule, when I explicitly told it to build test in my first recursive call (which should correspond to the $(PROGRAM) target). What am I missing here?
Problem 0:
This is overdesigned. You don't need to use recursive Make here.
Problem 1:
The reason Make doesn't try to rebuild test.mk (without a dummy preq) is that test.mk is up to date. A better approach is to switch to a static pattern rule and use PHONY:
MKS = $(wildcard *.mk)
.PHONY: $(MKS)
$(MKS): %.mk:
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
An even better approach is not to use the name of a real file as a target of a rule that doesn't rebuild (or even "touch") that file.
Problem 2:
In make[2], the makefile includes test.mk. If a makefile includes another file, Make will attempt to rebuild that file before doing anything else. If there is a rule for that file (which there is) and if it succeeds (which it does) Make then reinvokes itself.
You should reconsider this design from the ground up. There are many ways to get the behavior you're looking for, depending on the specifics (how many variable will be defined in a foo.mk? do you really want to manage the build by manually moving those files around? and so on).
P.S. Here's one kludge that springs to mind. Whether it suits your case depends on the specifics:
makefile:
# includes nothing
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# -f $# $*
$(MAKE) MYFILE=$# -f $# $*
test.mk:
SOMEVAR = SomeValue
include makefile