Makefile - make finds a file whilst shell does not - linux

I've written a snippet of Makefile that checks if file exists before executing rm upon it.
clean:
echo "Attempt to remove $(exec) file"
if test -f "${exec}" ; then \
echo "Removing file ${exec}" ; \
rm ${exec} ; \
else \
echo "No ${exec} file to remove" ; \
fi
if "$(wildcard *.o)" = "" ; then \
echo "No files found" ; \
else \
echo " Found $(wildcard *.o) " ; \
fi
First if statement works fine
Attempt to remove hello file
No hello file to remove
while the second produces this:
/bin/sh: 1: main.o factorial.o: not found
Found main.o factorial.o
My question is: How come that make recipe produces valid output ( these files truly exist ) whilst the shell does not? And why shell even tries to find them?
Thank you for your time reading this question. That's my first one here so if I did something inappropriately please let me know

if main.o is very different than if test -f main.o. The former attempts to run a command main.o, while the latter runs the command test. if "main.o factorial.o" is similar, in that it attempts to run a command named main.o factorial.o which the shell is correctly complaining that it cannot find. It is not likely that you have a file named main.o factorial.o (that's a single file with a space in its name) in your PATH.
But don't do this. There is absolutely no point in ever checking whether or not a file exists before you unlink it. There's an inherent race condition. Just attempt to remove the file, and deal with errors that may occur if the file didn't exist.
It's much easier to write rm $(wildcard *.o) and just let rm emit error messages for files that don't exist. Or rm -f $(wildcard *.o) to suppress errors. If you really insist on iterating over the files and checking, you could do something like:
for f in $(wildcard *.o); do \
if ! test -f "$$f"; then echo "$$f" does not exist; \
else rm "$$f"; fi
but it's really not worth it. Also, it seems pretty pointless since the wildcard is only going to expand to files that exist. (But note that this exhibits the same race condition: the wildcard might expand to a list of files, but new files might be created between the time the wildcard is expanded and the rm is run.) Don't use wildcard like this. Explicitly list the files that you want to work with.

Related

rm not deleting files completely

I have written an alias to create makefile (basically copying a template makefile and substituting the final exec name) in any project dir:
This is my cpmk command:
alias cpmk='f() { \
if [ "$#" -eq 0 ] ; \
then \
d="$(pwd)"; \
else \
d="$1"; \
fi; \
echo Trying to make a makefile in: $(readlink -f "$d") ; \
if [ -f $(readlink -f "$d")/makefile ] ; \
then \
echo $(readlink -f "$d")/makefile already exists, but might be a different one, dont know; \
return 1; \
fi; \
read -p "Enter exec name:" execname ;\
echo This is the name of the executable: "$execname" ; \
touch $(readlink -f "$d")/makefile;\
sed 's/hellomake/"$execname"/' ~/.makefileTemplate >$(readlink -f "$d")/makefile; \
if [ "$?" -eq 0 ] ; \
then \
echo $(readlink -f "$d")/makefile created successfully; \
unset -f f; \
return 0; \
else \
echo $(readlink -f "$d")/makefile creation failed, couldnt write to file, by the way, there is no other makefile "in" this dir by that name, something "else" erred; \
unset -f f; \
return 0; \
fi; \
}; \
f'
Then I source .bashrc (because this alias is in the bashrc).
I create a makefile using cpmk. Then I run cpmk again. This time it doesn't create a makefile because there's already one. These makefiles are a copy of a template makefile, I have kept hidden. This is basically a copy-like command with variable replacement. Back to the problem in the dir where there's already a makefile made by cpmk ran previously, I then delete this existing makefile by doing rm makefile, and rerun cpmk. This time too it asks me what to name the exec, I give it the name. It displays the name of the exec, and prints "created successfully" like message, but when I open it, I find the same deleted makefile again. How do I know? Because, It has the execname of the last one, the exec name given this time isn't found in the makefile.
The same old makefile appears again with old exec name?
how do I completely delete any file so that when next time touch is run with the same filename as the deleted one, the old deleted file doesn't reappear again?
Sometimes, the deletion of the previous makefile is successful and it doesn't reappear on running touch with the same filename as the deleted one but sed fails to substitute execname in the generated makefile
sed 's/hellomake/"$execname"/' ~/.makefileTemplate >$(readlink -f "$d")/makefile; \
This line above is not substituting $execname in the makefile. I am getting empty space in place of the substituted text hellomake. But $execname has the name of the exec file as can be seen by the messages printed by cpmk.
The problem lies in the use of quotes. The function f() is already surrounded by single quotes and inside sed' s regex, single quotes are being used again to enclose the regex 's/hellomake/"$execname"/'. That' why getting undefined behaviour. But it's surprising that because of this undefined behaviour, getting a deleted file back. Corrected it, now runs. Just substituted single quotes ' with double quotes ".
Solved!
's/hellomake/"$execname"/'
Replace the above line with this:
"s/hellomake/$execname/"

Traversing a directory using Makefile

I am making use of a makefile to traverse a directory , store the file names in a variable , extract the file number using regex and print the file number .
Below is the code block I am using :
1. set -e ;\
2. for file in $$(MY_HOME)/mydir/python_files/ ;\
3. do \
4. string =$${file} ;\
5. [[ $$string =~ .*clause([0-9]*).py ]] ;\
6. file_num=$${BASH_REMATCH[1]} ; \
7. python $$(My_Home)/base_py_<file_num>.py ;\
8. done ;\
LINE 7 is the command in makefile I want to include in the for loop. I want the actual file_num in the above < file_num > placeholder (line 7). How can I do the same. Is there any alternative approach for the same ?
Thanks in advance
I would make a native make approach instead of looping in bash, like so:
$ cat Makefile
MY_HOME := myhome
CLAUSE_FILES_DIR := $(MY_HOME)/mydir/python_files
clause_files := $(wildcard $(CLAUSE_FILES_DIR)/*clause*.py)
clause_numbers := $(foreach clause_file,$(notdir $(clause_files:.py=)), \
$(lastword $(subst clause, ,$(clause_file))))
.PHONY: execute-clause-%
execute-clause-%: $(MY_HOME)/base_py_%.py $(CLAUSE_FILES_DIR)/*clause%.py
echo python $<
all: $(addprefix execute-clause-,$(clause_numbers))
clause_files will keep all existing files matching the pattern. clause_numbers will process the file names by stripping extension and directory, then split on clause to get only the part between clause and extension.
execute-clause-% is a generic rule to run based on existence of a specific base_py_*.py script and a matching clause file. If one or the other does not exist, the rule will not be run.
Finally all rule executes all existing clauses. And since every processing is done by a separate rule, all of them might be executed in parallel by just running make -j.
Sample output:
## Preparation stage
$ mkdir -p myhome/mydir/python_files
$ for i in `seq 1 5`; do touch myhome/base_py_$i.py; done
$ for i in `seq 1 5`; do touch myhome/mydir/python_files/${RANDOM}_clause$i.py; done
$ touch myhome/mydir/python_files/foo.py # Just to show it does not match
$ touch myhome/base_py_100.py # To demonstrate missing clause file
$ ls -R myhome/
myhome/:
base_py_1.py base_py_100.py base_py_2.py base_py_3.py base_py_4.py base_py_5.py mydir
myhome/mydir:
python_files
myhome/mydir/python_files:
14363_clause1.py 31198_clause2.py 4514_clause5.py 4767_clause4.py 7812_clause3.py foo.py
## Execution
$ make -s
python myhome/base_py_3.py
python myhome/base_py_2.py
python myhome/base_py_5.py
python myhome/base_py_4.py
python myhome/base_py_1.py
Note that neither foo.py nor base_py_100.py did not cause running the rule.

Makefile patsubst using value of shell variable

As part of a make install rule for a testing suite, I'd like to move all binary executables in one directory (a src directory) to a bin directory. I thought an easy way to do this would be to simply loop over each file in the src directory and then use patsubst to replace src with bin in each path. Unfortunately, I can't get it to work because I can't get make to evaluate the name of the current FILE in each loop iteration. All I have access to is the bash shell variable $$FILE, but when I use this with the make patsubst function, it doesn't actually evaluate the shell variable $$FILE... rather, the patsubst function seems to just see the string "$FILE".
So, here is what I'm trying:
install :
-- irrelevant stuff snipped --
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then mv -f $$FILE $(patsubst %/src/,%/bin/,$$FILE); fi \
done
This results in an error for each file:
mv: ‘./src/foo/bar’ and ‘./src/foo/bar’ are the same file
This error leads me to understand that the patsubst function in make is not actually evaluating shell variables, but just sees $FILE, and so the result is that it doesn't find the substitution pattern, and the final command passed to mv has the source and destination path as the same string.
So, is there a way to get patsubst to evaluate the value of a shell variable? Or is there a better way in general to accomplish what I'm trying to achieve here?
make processing has a precedence over passing commands to shell. And, once passed, they are executed by shell. So, at first make, processes the command and in:
$(patsubst %/src/,%/bin/,$$FILE)
$$FILE is substituted by $FILE and then treated literally. So, no pattern is matched and in effect patsubst returns $FILE. Please see following example:
bar:
echo $(patsubst %/src/,%/bin/,$$whatever)
It gives:
arturcz#szczaw:/tmp/m$ make bar
echo $whatever
arturcz#szczaw:/tmp/m$
As a result of your makefile rule bash is given following command to execute:
for FILE in src/a src/b src/c; do \
if [ -f $FILE ]; then mv -f $FILE $FILE; fi \
done
and that's why you got your result.
Solution
You can rely on bash to do a proper substitution, but you need to enforce it as a shell (by default it is sh, which lacks some required features):
SHELL=bash
install:
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then echo $$FILE $${FILE/\/src\//\/bin\/}; fi \
done
You can also ask make to do a loop and substitution. There are few ways you can achieve that. This one is doing all the replacement and prepares command on the fly.
install:
$(foreach d,$(BINARY_TARGETS),if [ -f $(d) ]; then mv -f $(d) $(d:./src/%=./bin/%);fi;)
You can cease checking existence of files to make too by using `$(wildcard) function:
install:
$(foreach d,$(wildcard $(BINARY_TARGETS)),mv -f $(d) $(d:./src/%=./bin/%);)
And, finally, solution which I personally prefer - do it in a make way using a proper dependencies and rules.
install: $(BINARY_TARGETS:./src/%=./bin/%)
bin/%: src/%
mv -f $< $#
If existence any of files in BINARY_TARGET is optional, you may want to use the $(wildcard) trick again:
install: $(patsubst ./src/%,./bin/%,$(wildcard $(BINARY_TARGETS)))
bin/%: src/%
mv -f $< $#

How can i call make file from other directory

I have directory structure like this
containers/con1
containers/con2
containers/con3
Now every folder like con1, con2 has Makefile in it with targets like build, run
I run it like make run and make build
But i have to go inside that folder.
Is it possible that i have another Makefile in containers/Makefile
and i can run like
Make con1.run Make con2.run
Yes, you can do that. Something like the following should do what you want.
$ cat containers/Makefile
%.run: %
$(MAKE) -C $#
That being said as you can see the command to do what you want is trivial enough to make such a makefile not really necessary (and a simple shell script is as useful here as a makefile).
$ cat run.sh
[ -d "$1" ] || { echo 'No such directory.' >&2; exit 1; }
#make -C "$1"
# OR
#cd "$1" && make
If you wanted to be able to build all the sub-directory projects at once then a makefile could help you with that but even that is a simple enough shell one-liner.
$ for mkfile in */Makefile; do make -C "$(dirname "$mkfile"); done
$ for mkfile in */Makefile; do (cd "$(dirname "$mkfile") && make); done
As far as I understand you want this:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing anything else. If multiple -C options are specified, each is interpreted relative to the previous one: -C / -C etc is equivalent to -C /etc. This is typi‐
cally used with recursive invocations of make.
Add -C option like this: make -C con1/
Recursive makes are evil, but if you want that:
# con1.run is a phony target...
.PHONY: con1.run
con1.run:
$(MAKE) -C con1

Downloading files with Makefile rules

I was planning something like:
URLS=www.host.com/file1.tar.gz www.host2.com/file2.tar.gz
$(somefunc $URLS): #somefunc produces downloads/file1.tar.gz downloads/file2.tar.gz
mkdir -P downloads
wget whatever # I can't get the real url here because the targets don't contain the full url anymore
myproject: $(somefunc URLS)
#Files should already be in downloads/ here
The problem I have is that if I convert URLS with somefunc I lose the urls, if I don't I can't use it as a target to avoid being downloaded when it is already there.
Any ideas?
If somefunc only modifies the path, not the actual filename, and there are no duplicates, you could try searching $(URLS) for the original filename.
Maybe something like this? (untested)
$(somefunc $URLS): #somefunc produces downloads/file1.tar.gz downloads/file2.tar.gz
mkdir -p $(dir $#)
wget $(filter $(addprefix %/,$(notdir $#)),$(URLS)) -O $#
$(notdir $#) evaluates to file1.tar.gz for one of the targets.
$(addprefix %/,file1.tar.gz) evaluates to %/file1.tar.gz
$(filter %/file1.tar.gz,www.host.com/file1.tar.gz www.host2.com/file2.tar.gz) evaluates to www.host.com/file1.tar.gz.
(Presumably you want an http:// on there too?)
At this point, I don't think you can do it directly with make. But here's a solution if you are willing to use some scripting:
$(URLS):
#for url in ${URLS}; do \
if [ ! -e $$(somefunc $${url}) ]; then \
echo wget $${url}; \
wget $${url}; \
fi \
done

Resources