I am making use of a makefile to traverse a directory , store the file names in a variable , extract the file number using regex and print the file number .
Below is the code block I am using :
1. set -e ;\
2. for file in $$(MY_HOME)/mydir/python_files/ ;\
3. do \
4. string =$${file} ;\
5. [[ $$string =~ .*clause([0-9]*).py ]] ;\
6. file_num=$${BASH_REMATCH[1]} ; \
7. python $$(My_Home)/base_py_<file_num>.py ;\
8. done ;\
LINE 7 is the command in makefile I want to include in the for loop. I want the actual file_num in the above < file_num > placeholder (line 7). How can I do the same. Is there any alternative approach for the same ?
Thanks in advance
I would make a native make approach instead of looping in bash, like so:
$ cat Makefile
MY_HOME := myhome
CLAUSE_FILES_DIR := $(MY_HOME)/mydir/python_files
clause_files := $(wildcard $(CLAUSE_FILES_DIR)/*clause*.py)
clause_numbers := $(foreach clause_file,$(notdir $(clause_files:.py=)), \
$(lastword $(subst clause, ,$(clause_file))))
.PHONY: execute-clause-%
execute-clause-%: $(MY_HOME)/base_py_%.py $(CLAUSE_FILES_DIR)/*clause%.py
echo python $<
all: $(addprefix execute-clause-,$(clause_numbers))
clause_files will keep all existing files matching the pattern. clause_numbers will process the file names by stripping extension and directory, then split on clause to get only the part between clause and extension.
execute-clause-% is a generic rule to run based on existence of a specific base_py_*.py script and a matching clause file. If one or the other does not exist, the rule will not be run.
Finally all rule executes all existing clauses. And since every processing is done by a separate rule, all of them might be executed in parallel by just running make -j.
Sample output:
## Preparation stage
$ mkdir -p myhome/mydir/python_files
$ for i in `seq 1 5`; do touch myhome/base_py_$i.py; done
$ for i in `seq 1 5`; do touch myhome/mydir/python_files/${RANDOM}_clause$i.py; done
$ touch myhome/mydir/python_files/foo.py # Just to show it does not match
$ touch myhome/base_py_100.py # To demonstrate missing clause file
$ ls -R myhome/
myhome/:
base_py_1.py base_py_100.py base_py_2.py base_py_3.py base_py_4.py base_py_5.py mydir
myhome/mydir:
python_files
myhome/mydir/python_files:
14363_clause1.py 31198_clause2.py 4514_clause5.py 4767_clause4.py 7812_clause3.py foo.py
## Execution
$ make -s
python myhome/base_py_3.py
python myhome/base_py_2.py
python myhome/base_py_5.py
python myhome/base_py_4.py
python myhome/base_py_1.py
Note that neither foo.py nor base_py_100.py did not cause running the rule.
Related
I've written a snippet of Makefile that checks if file exists before executing rm upon it.
clean:
echo "Attempt to remove $(exec) file"
if test -f "${exec}" ; then \
echo "Removing file ${exec}" ; \
rm ${exec} ; \
else \
echo "No ${exec} file to remove" ; \
fi
if "$(wildcard *.o)" = "" ; then \
echo "No files found" ; \
else \
echo " Found $(wildcard *.o) " ; \
fi
First if statement works fine
Attempt to remove hello file
No hello file to remove
while the second produces this:
/bin/sh: 1: main.o factorial.o: not found
Found main.o factorial.o
My question is: How come that make recipe produces valid output ( these files truly exist ) whilst the shell does not? And why shell even tries to find them?
Thank you for your time reading this question. That's my first one here so if I did something inappropriately please let me know
if main.o is very different than if test -f main.o. The former attempts to run a command main.o, while the latter runs the command test. if "main.o factorial.o" is similar, in that it attempts to run a command named main.o factorial.o which the shell is correctly complaining that it cannot find. It is not likely that you have a file named main.o factorial.o (that's a single file with a space in its name) in your PATH.
But don't do this. There is absolutely no point in ever checking whether or not a file exists before you unlink it. There's an inherent race condition. Just attempt to remove the file, and deal with errors that may occur if the file didn't exist.
It's much easier to write rm $(wildcard *.o) and just let rm emit error messages for files that don't exist. Or rm -f $(wildcard *.o) to suppress errors. If you really insist on iterating over the files and checking, you could do something like:
for f in $(wildcard *.o); do \
if ! test -f "$$f"; then echo "$$f" does not exist; \
else rm "$$f"; fi
but it's really not worth it. Also, it seems pretty pointless since the wildcard is only going to expand to files that exist. (But note that this exhibits the same race condition: the wildcard might expand to a list of files, but new files might be created between the time the wildcard is expanded and the rm is run.) Don't use wildcard like this. Explicitly list the files that you want to work with.
i've loop which I need to add to it if else statment,
if the value ==app1 print x else print y
I've tried with the following
but when I add the else I got syntax error,
what am I doing wrong?
runners:
#for a in $(apps) ; do
if (( a == "app1"));then
echo first: $$v ;
else
echo second: $$v ;
fi
done
I prefer to use bash as my shell, because I'm used to it. Therefore I use SHELL:=/bin/bash at the top of my makefiles. But then your whole makefile fails when a system doesn't provide /bin/bash..
If you want to use the same shell for all the lines in a recipe, you can use the special target .ONESHELL This prevents you from using ;\ after every line with no whitespace behind (error prone). But if you use .ONESHELL then this is true for ALL recipes in your makefile (which I use, but you may not want).
If runners is not a file (and it seems like it, because you do not create a file in the recipe), I would recommend making it .PHONY. Your version might prevent make from executing it, when there happens to be a file named runners while phony targets are always executed.
Double parentheses ((...)) are used for arithmetic expansion. It simply cannot compare strings. Use single or double brackets. See this great post.
I also think you wanted to print a and therefore substituted $$v with $$a.
So in total, this works perfectly fine as long as you have /bin/bash:
SHELL:=/bin/bash
.ONESHELL:
.PHONY: runners
apps:=app2 app3 app1 app5
runners:
#for a in $(apps) ; do <<-- ; between commands as usual
if [[ "$$a" == "app1" ]]; then
echo first: $$a <<-- no need for ; because of .ONESHELL
else
echo second: $$a
fi
done
Output:
second: app2
second: app3
first: app1
second: app5
You use simply the wrong syntax for sh if/else. And you have to add a \ at each end of the command as all that run as a single command in the shell started by make. Attention: No whitespace after \ !
It should be something like:
runners:
#for a in $(apps) ; do \
if [ $$a = "app1" ]; then\
echo first: $$a ; \
else \
echo second: $$a ; \
fi \
done
BTW: Do you really want to print $v? I would expect $a instead ;)
Makefile:
apps=app1 app2
runners:
#for a in $(apps) ; do \
if [ $$a = "app1" ]; then \
echo "first: $$a" ; \
else \
echo "second: $$a" ; \
fi \
done
which prints
first: app1
second: app2
Take care: the solutions above are working in a recent "make" version (probably >4) only.
OSX for example includes Gnu make 3.81. So you would need to install a newer make using Homebrew
brew reinstall make --with-default-names
As part of a make install rule for a testing suite, I'd like to move all binary executables in one directory (a src directory) to a bin directory. I thought an easy way to do this would be to simply loop over each file in the src directory and then use patsubst to replace src with bin in each path. Unfortunately, I can't get it to work because I can't get make to evaluate the name of the current FILE in each loop iteration. All I have access to is the bash shell variable $$FILE, but when I use this with the make patsubst function, it doesn't actually evaluate the shell variable $$FILE... rather, the patsubst function seems to just see the string "$FILE".
So, here is what I'm trying:
install :
-- irrelevant stuff snipped --
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then mv -f $$FILE $(patsubst %/src/,%/bin/,$$FILE); fi \
done
This results in an error for each file:
mv: ‘./src/foo/bar’ and ‘./src/foo/bar’ are the same file
This error leads me to understand that the patsubst function in make is not actually evaluating shell variables, but just sees $FILE, and so the result is that it doesn't find the substitution pattern, and the final command passed to mv has the source and destination path as the same string.
So, is there a way to get patsubst to evaluate the value of a shell variable? Or is there a better way in general to accomplish what I'm trying to achieve here?
make processing has a precedence over passing commands to shell. And, once passed, they are executed by shell. So, at first make, processes the command and in:
$(patsubst %/src/,%/bin/,$$FILE)
$$FILE is substituted by $FILE and then treated literally. So, no pattern is matched and in effect patsubst returns $FILE. Please see following example:
bar:
echo $(patsubst %/src/,%/bin/,$$whatever)
It gives:
arturcz#szczaw:/tmp/m$ make bar
echo $whatever
arturcz#szczaw:/tmp/m$
As a result of your makefile rule bash is given following command to execute:
for FILE in src/a src/b src/c; do \
if [ -f $FILE ]; then mv -f $FILE $FILE; fi \
done
and that's why you got your result.
Solution
You can rely on bash to do a proper substitution, but you need to enforce it as a shell (by default it is sh, which lacks some required features):
SHELL=bash
install:
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then echo $$FILE $${FILE/\/src\//\/bin\/}; fi \
done
You can also ask make to do a loop and substitution. There are few ways you can achieve that. This one is doing all the replacement and prepares command on the fly.
install:
$(foreach d,$(BINARY_TARGETS),if [ -f $(d) ]; then mv -f $(d) $(d:./src/%=./bin/%);fi;)
You can cease checking existence of files to make too by using `$(wildcard) function:
install:
$(foreach d,$(wildcard $(BINARY_TARGETS)),mv -f $(d) $(d:./src/%=./bin/%);)
And, finally, solution which I personally prefer - do it in a make way using a proper dependencies and rules.
install: $(BINARY_TARGETS:./src/%=./bin/%)
bin/%: src/%
mv -f $< $#
If existence any of files in BINARY_TARGET is optional, you may want to use the $(wildcard) trick again:
install: $(patsubst ./src/%,./bin/%,$(wildcard $(BINARY_TARGETS)))
bin/%: src/%
mv -f $< $#
I have directory structure like this
containers/con1
containers/con2
containers/con3
Now every folder like con1, con2 has Makefile in it with targets like build, run
I run it like make run and make build
But i have to go inside that folder.
Is it possible that i have another Makefile in containers/Makefile
and i can run like
Make con1.run Make con2.run
Yes, you can do that. Something like the following should do what you want.
$ cat containers/Makefile
%.run: %
$(MAKE) -C $#
That being said as you can see the command to do what you want is trivial enough to make such a makefile not really necessary (and a simple shell script is as useful here as a makefile).
$ cat run.sh
[ -d "$1" ] || { echo 'No such directory.' >&2; exit 1; }
#make -C "$1"
# OR
#cd "$1" && make
If you wanted to be able to build all the sub-directory projects at once then a makefile could help you with that but even that is a simple enough shell one-liner.
$ for mkfile in */Makefile; do make -C "$(dirname "$mkfile"); done
$ for mkfile in */Makefile; do (cd "$(dirname "$mkfile") && make); done
As far as I understand you want this:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing anything else. If multiple -C options are specified, each is interpreted relative to the previous one: -C / -C etc is equivalent to -C /etc. This is typi‐
cally used with recursive invocations of make.
Add -C option like this: make -C con1/
Recursive makes are evil, but if you want that:
# con1.run is a phony target...
.PHONY: con1.run
con1.run:
$(MAKE) -C con1
I am new to shell script. I want to call a list of make files from Shell script in a particular order. For each makefile I want to get the result (make is success or failure). If any error I want to stop the script execution. If it is success I have to run the next makefile.
A common idiom is to create a shell script with set -e; this will cause the script to exit on the first error.
#!/bin/sh
set -e
make -f Makefile1
make -f Makefile2
:
If you need more control over the script overall, maybe remove set -e and instead explicitly exit on make failure:
make -f Makefile1 || exit
make -f Makefile2 || exit
To reduce code duplication, create a loop:
for f in Makefile1 Makefile2; do
make -f "$f" || exit
done
Just to be explicit, the || "or" and && "and" connectives are shorthand for
if make -f Makefile1; then
: "and" part
else
: "or" part
fi
Finally, the behavior you describe sounds exactly like how Make itself behaves. Perhaps a top-level Makefile would actually be a suitable solution for your scenario?
.PHONY: all
all:
$(MAKE) -f Makefile1
$(MAKE) -f Makefile2
make -f makefile1
make -f makefile2
to run make files in order
to save the output of each makefile
make -f makefile1 >> output1
make -f makefile2 >> output2
to check the result after each make file
make -f makefile1 >> output1
after this line script use
echo $? this in combination with if. if echo$? result zero then your make success so if echo$? result zero then run next file other wise exit