I have written an alias to create makefile (basically copying a template makefile and substituting the final exec name) in any project dir:
This is my cpmk command:
alias cpmk='f() { \
if [ "$#" -eq 0 ] ; \
then \
d="$(pwd)"; \
else \
d="$1"; \
fi; \
echo Trying to make a makefile in: $(readlink -f "$d") ; \
if [ -f $(readlink -f "$d")/makefile ] ; \
then \
echo $(readlink -f "$d")/makefile already exists, but might be a different one, dont know; \
return 1; \
fi; \
read -p "Enter exec name:" execname ;\
echo This is the name of the executable: "$execname" ; \
touch $(readlink -f "$d")/makefile;\
sed 's/hellomake/"$execname"/' ~/.makefileTemplate >$(readlink -f "$d")/makefile; \
if [ "$?" -eq 0 ] ; \
then \
echo $(readlink -f "$d")/makefile created successfully; \
unset -f f; \
return 0; \
else \
echo $(readlink -f "$d")/makefile creation failed, couldnt write to file, by the way, there is no other makefile "in" this dir by that name, something "else" erred; \
unset -f f; \
return 0; \
fi; \
}; \
f'
Then I source .bashrc (because this alias is in the bashrc).
I create a makefile using cpmk. Then I run cpmk again. This time it doesn't create a makefile because there's already one. These makefiles are a copy of a template makefile, I have kept hidden. This is basically a copy-like command with variable replacement. Back to the problem in the dir where there's already a makefile made by cpmk ran previously, I then delete this existing makefile by doing rm makefile, and rerun cpmk. This time too it asks me what to name the exec, I give it the name. It displays the name of the exec, and prints "created successfully" like message, but when I open it, I find the same deleted makefile again. How do I know? Because, It has the execname of the last one, the exec name given this time isn't found in the makefile.
The same old makefile appears again with old exec name?
how do I completely delete any file so that when next time touch is run with the same filename as the deleted one, the old deleted file doesn't reappear again?
Sometimes, the deletion of the previous makefile is successful and it doesn't reappear on running touch with the same filename as the deleted one but sed fails to substitute execname in the generated makefile
sed 's/hellomake/"$execname"/' ~/.makefileTemplate >$(readlink -f "$d")/makefile; \
This line above is not substituting $execname in the makefile. I am getting empty space in place of the substituted text hellomake. But $execname has the name of the exec file as can be seen by the messages printed by cpmk.
The problem lies in the use of quotes. The function f() is already surrounded by single quotes and inside sed' s regex, single quotes are being used again to enclose the regex 's/hellomake/"$execname"/'. That' why getting undefined behaviour. But it's surprising that because of this undefined behaviour, getting a deleted file back. Corrected it, now runs. Just substituted single quotes ' with double quotes ".
Solved!
's/hellomake/"$execname"/'
Replace the above line with this:
"s/hellomake/$execname/"
Related
I've written a snippet of Makefile that checks if file exists before executing rm upon it.
clean:
echo "Attempt to remove $(exec) file"
if test -f "${exec}" ; then \
echo "Removing file ${exec}" ; \
rm ${exec} ; \
else \
echo "No ${exec} file to remove" ; \
fi
if "$(wildcard *.o)" = "" ; then \
echo "No files found" ; \
else \
echo " Found $(wildcard *.o) " ; \
fi
First if statement works fine
Attempt to remove hello file
No hello file to remove
while the second produces this:
/bin/sh: 1: main.o factorial.o: not found
Found main.o factorial.o
My question is: How come that make recipe produces valid output ( these files truly exist ) whilst the shell does not? And why shell even tries to find them?
Thank you for your time reading this question. That's my first one here so if I did something inappropriately please let me know
if main.o is very different than if test -f main.o. The former attempts to run a command main.o, while the latter runs the command test. if "main.o factorial.o" is similar, in that it attempts to run a command named main.o factorial.o which the shell is correctly complaining that it cannot find. It is not likely that you have a file named main.o factorial.o (that's a single file with a space in its name) in your PATH.
But don't do this. There is absolutely no point in ever checking whether or not a file exists before you unlink it. There's an inherent race condition. Just attempt to remove the file, and deal with errors that may occur if the file didn't exist.
It's much easier to write rm $(wildcard *.o) and just let rm emit error messages for files that don't exist. Or rm -f $(wildcard *.o) to suppress errors. If you really insist on iterating over the files and checking, you could do something like:
for f in $(wildcard *.o); do \
if ! test -f "$$f"; then echo "$$f" does not exist; \
else rm "$$f"; fi
but it's really not worth it. Also, it seems pretty pointless since the wildcard is only going to expand to files that exist. (But note that this exhibits the same race condition: the wildcard might expand to a list of files, but new files might be created between the time the wildcard is expanded and the rm is run.) Don't use wildcard like this. Explicitly list the files that you want to work with.
I am making use of a makefile to traverse a directory , store the file names in a variable , extract the file number using regex and print the file number .
Below is the code block I am using :
1. set -e ;\
2. for file in $$(MY_HOME)/mydir/python_files/ ;\
3. do \
4. string =$${file} ;\
5. [[ $$string =~ .*clause([0-9]*).py ]] ;\
6. file_num=$${BASH_REMATCH[1]} ; \
7. python $$(My_Home)/base_py_<file_num>.py ;\
8. done ;\
LINE 7 is the command in makefile I want to include in the for loop. I want the actual file_num in the above < file_num > placeholder (line 7). How can I do the same. Is there any alternative approach for the same ?
Thanks in advance
I would make a native make approach instead of looping in bash, like so:
$ cat Makefile
MY_HOME := myhome
CLAUSE_FILES_DIR := $(MY_HOME)/mydir/python_files
clause_files := $(wildcard $(CLAUSE_FILES_DIR)/*clause*.py)
clause_numbers := $(foreach clause_file,$(notdir $(clause_files:.py=)), \
$(lastword $(subst clause, ,$(clause_file))))
.PHONY: execute-clause-%
execute-clause-%: $(MY_HOME)/base_py_%.py $(CLAUSE_FILES_DIR)/*clause%.py
echo python $<
all: $(addprefix execute-clause-,$(clause_numbers))
clause_files will keep all existing files matching the pattern. clause_numbers will process the file names by stripping extension and directory, then split on clause to get only the part between clause and extension.
execute-clause-% is a generic rule to run based on existence of a specific base_py_*.py script and a matching clause file. If one or the other does not exist, the rule will not be run.
Finally all rule executes all existing clauses. And since every processing is done by a separate rule, all of them might be executed in parallel by just running make -j.
Sample output:
## Preparation stage
$ mkdir -p myhome/mydir/python_files
$ for i in `seq 1 5`; do touch myhome/base_py_$i.py; done
$ for i in `seq 1 5`; do touch myhome/mydir/python_files/${RANDOM}_clause$i.py; done
$ touch myhome/mydir/python_files/foo.py # Just to show it does not match
$ touch myhome/base_py_100.py # To demonstrate missing clause file
$ ls -R myhome/
myhome/:
base_py_1.py base_py_100.py base_py_2.py base_py_3.py base_py_4.py base_py_5.py mydir
myhome/mydir:
python_files
myhome/mydir/python_files:
14363_clause1.py 31198_clause2.py 4514_clause5.py 4767_clause4.py 7812_clause3.py foo.py
## Execution
$ make -s
python myhome/base_py_3.py
python myhome/base_py_2.py
python myhome/base_py_5.py
python myhome/base_py_4.py
python myhome/base_py_1.py
Note that neither foo.py nor base_py_100.py did not cause running the rule.
As part of a make install rule for a testing suite, I'd like to move all binary executables in one directory (a src directory) to a bin directory. I thought an easy way to do this would be to simply loop over each file in the src directory and then use patsubst to replace src with bin in each path. Unfortunately, I can't get it to work because I can't get make to evaluate the name of the current FILE in each loop iteration. All I have access to is the bash shell variable $$FILE, but when I use this with the make patsubst function, it doesn't actually evaluate the shell variable $$FILE... rather, the patsubst function seems to just see the string "$FILE".
So, here is what I'm trying:
install :
-- irrelevant stuff snipped --
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then mv -f $$FILE $(patsubst %/src/,%/bin/,$$FILE); fi \
done
This results in an error for each file:
mv: ‘./src/foo/bar’ and ‘./src/foo/bar’ are the same file
This error leads me to understand that the patsubst function in make is not actually evaluating shell variables, but just sees $FILE, and so the result is that it doesn't find the substitution pattern, and the final command passed to mv has the source and destination path as the same string.
So, is there a way to get patsubst to evaluate the value of a shell variable? Or is there a better way in general to accomplish what I'm trying to achieve here?
make processing has a precedence over passing commands to shell. And, once passed, they are executed by shell. So, at first make, processes the command and in:
$(patsubst %/src/,%/bin/,$$FILE)
$$FILE is substituted by $FILE and then treated literally. So, no pattern is matched and in effect patsubst returns $FILE. Please see following example:
bar:
echo $(patsubst %/src/,%/bin/,$$whatever)
It gives:
arturcz#szczaw:/tmp/m$ make bar
echo $whatever
arturcz#szczaw:/tmp/m$
As a result of your makefile rule bash is given following command to execute:
for FILE in src/a src/b src/c; do \
if [ -f $FILE ]; then mv -f $FILE $FILE; fi \
done
and that's why you got your result.
Solution
You can rely on bash to do a proper substitution, but you need to enforce it as a shell (by default it is sh, which lacks some required features):
SHELL=bash
install:
for FILE in $(BINARY_TARGETS); do \
if [ -f $$FILE ]; then echo $$FILE $${FILE/\/src\//\/bin\/}; fi \
done
You can also ask make to do a loop and substitution. There are few ways you can achieve that. This one is doing all the replacement and prepares command on the fly.
install:
$(foreach d,$(BINARY_TARGETS),if [ -f $(d) ]; then mv -f $(d) $(d:./src/%=./bin/%);fi;)
You can cease checking existence of files to make too by using `$(wildcard) function:
install:
$(foreach d,$(wildcard $(BINARY_TARGETS)),mv -f $(d) $(d:./src/%=./bin/%);)
And, finally, solution which I personally prefer - do it in a make way using a proper dependencies and rules.
install: $(BINARY_TARGETS:./src/%=./bin/%)
bin/%: src/%
mv -f $< $#
If existence any of files in BINARY_TARGET is optional, you may want to use the $(wildcard) trick again:
install: $(patsubst ./src/%,./bin/%,$(wildcard $(BINARY_TARGETS)))
bin/%: src/%
mv -f $< $#
I would like to find a way to run sed scripts on files within a Yocto-generated OS from a .bbappend file. My OS has a read-only rootfs, which seems to stop any possibility of a post-installation script. Specifically, I need to make these changes to /etc/default/ssh (as run after booting the generated OS):
sed -i 's/var\/run/etc/' /etc/default/ssh
sed -i 's/_readonly//' /etc/default/ssh
Here's my openssh_7.1p1.bbappend which I've created in an attempt to solve these problems:
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += " \
file://ssh_host_dsa_key.pub \
file://ssh_host_rsa_key.pub \
...
"
do_install_append() {
sed -i 's/var\/run/etc/' ${D}${sysconfdir}/default/ssh
sed -i 's/_readonly//' ${D}${sysconfdir}/default/ssh
# these lines work fine
install -m 0755 ${WORKDIR}/ssh_host_dsa_key ${D}/etc/ssh
install -m 0755 ${WORKDIR}/ssh_host_dsa_key.pub ${D}/etc/ssh
...
}
FILES_${PN} += "${sysconfdir}/default/ssh"
#these lines work
FILES_${PN} += "${sysconfdir}/ssh/ssh_host_dsa_key"
FILES_${PN} += "${sysconfdir}/ssh/ssh_host_dsa_key.pub"
...
BitBake fails during the execution of do_install_append() with this error:
sed: can't read ${TMPDIR}/work/x86-poky-linux/openssh/image/etc/default/ssh: No such file or directory
(where TMPDIR is my actual tmp directory) Obviously this file doesn't exist because the proper copy is created in a separate MULTIMACH_TARGET_SYS directory (i.e. not x86-poky-linux) by image.bbclass.
Is this sort of thing possible to do from within a .bbappend file (or some other compartmentalized way)? I have found a way to do it within a .inc file with ROOTFS_POSTPROCESS_COMMAND within my core-image, but this method is leading to a poor organizational structure.
Maybe moving the operations to a post-installation function in your .bbappend is appropriate for your circumstance?
pkg_postinst_${PN} () {
#!/bin/sh
if [ x"$D" = "x" ]; then
sed -i 's/var\/run/etc/' /etc/default/ssh
sed -i 's/_readonly//' /etc/default/ssh
else
exit 1
fi
}
The function above will cause the sed operations to execute on first boot, rather than during the build.
Include the file you're trying to sed in your SRC_URI variable.
I started using "sudo rm -r" to delete files/directories. I even put it as an alias of rm.
I normally know what I am doing and I am quite experience linux user.
However, I would like that when I press the "ENTER", before the execution of rm, a list of files will show up on the screen and a prompt at the end to OK the deletion of files.
Options -i -I -v does not do what I want. I want only one prompt for all the printed files on screen.
Thank you.
##
# Double-check files to delete.
delcheck() {
printf 'Here are the %d files you said you wanted to delete:\n' "$#"
printf '"%s"\n' "$#"
read -p 'Do you want to delete them? [y/N] ' doit
case "$doit" in
[yY]) rm "$#";;
*) printf 'No files deleted\n';;
esac
}
This is a shell function that (when used properly) will do what you want. However, if you load the function in your current shell then try to use it with sudo, it won't do what you expect because sudo creates a separate shell. So you'd need to make this a shell script…
#!/bin/bash
… same code as above …
# All this script does is create the function and then execute it.
# It's lazy, but functions are nice.
delcheck "$#"
…then make sure sudo can access it. Put it in some place that is in the sudo execution PATH (Depending on sudo configuration.) Then if you really want to execute it precisely as sudo rm -r * you will still need to name the script rm, (which in my opinion is dangerous) and make sure its PATH is before /bin in your PATH. (Also dangerous). But there you go.
Here's a nice option
Alias rm to echo | xargs -p rm
The -p option means "interactive" - it will display the entire command (including any expanded file lists) and ask you to confirm
It will NOT ask about the recursively removed files. But it will expand rm * .o to:
rm -rf * .o
rm -rf program.cc program.cc~ program program.o backup?... # NO NO NO NO NO!
Which is much nicer than receiving the error
rm: .o file not found
Edit: corrected the solution based on chepner comment. My previous solutions had a bug :(
This simple script prompts for a y response before deleting the files specified.
rmc script file:
read -p "ok to delete? " ans
case $ans in
[yY]*) sudo rm "$#" ;;
*) echo "Nothing deleted";;
esac
Invoke thus
./rmc *.tmp
I created a script to do this. The solution is similar to #kojiro's.
Save the script with the filename del. Run the command sudo chmod a=r+w+x del to make the script an executable. In the directory in which you want to save the script, export the path by entering export PATH=$PATH:/path/to/the/del/executable in your '~/.bashrc' file and run source ~/.bashrc.
Here, the syntax of rm is preserved, except instead of typing rm ..., type del ... where del is the name of the bash script below.
#! /bin/bash
# Safely delete files
args=("$#") # store all arguments passed to shell
N=$# # number of arguments passed to shell
#echo $#
#echo $#
#echo ${args[#]:0}
echo "Files to delete:"
echo
n=`expr $N - 1`
for i in `seq 0 $n`
do
str=${args[i]}
if [ ${str:0:1} != "-" ]; then
echo $str
fi
done
echo
read -r -p "Delete these files? [y/n] " response
case $response in
[yY][eE][sS]|[yY])
rm ${args[#]:0}
esac