I try to use find to create a very simple way to add newline to a file. I know there are tons of other ways to do this but it bugs the hell out of me that I cannot get this way to work.
So - I'm NOT asking how to add newline to a file - I'm asking why find is weird.
find . -type f -iname 'file' -exec echo >> {} \;
results in a new file created named "{}" with the newline in while (to check that find works on my computer):
find . -type f -iname 'file' -exec echo {} \;
prints "./file".
So the >> makes find confused. The question is why and how do I solve that?
I'm asking why find is weird.
It isn't. This has nothing to do with find. In fact, when the file is created, find hasn't even started to run.
>> roughly means "redirect stdout to the end of this file, create a new file when necessary". Note how nothing of this has anything to do with whatever is left of the >>.
Redirection is a feature of the shell, find knows nothing about the redirection and the shell knows nothing about find. >> doesn't magically change its meaning just because you happened to call find. It still means the exact same thing.
If you want to use a shell feature whithin -exec, you need to use a shell within -exec:
find . -type f -iname 'file' -exec sh -c 'echo >> "{}"' \;
While the question itself has already been answered, I'd like to point out that you don't strictly need to make find do everything, rather you can use other available facilities to work together with it, for example:
find . -type f -iname 'file' | while read file; do echo >>"$file"; done
This approach also has the advantage of not executing a new process for every match, which is irrelevant in this case but potentially important if there are thousands of matches and the exec is relatively heavy.
Related
Is there any way to apply Bash variable substitution on find's output? I know I've seen someone do it on Stack Overflow but I can't seem to find that particular post anymore.
As an example, let's say I want to rename files ending in *.png to *_copy.png. I know I can do this using rename but it's just a thought experiment.
Now I'd like to be able to do something like this:
find . -name "*png" -exec mv "{}" "${{}%.*}_copy.png" \;
Which results in an invalid substitution. Of course, I could first assign the output to a variable and then apply substitution in a sub-shell, but is this really the only way?
find . -name "*.png" -exec bash -c 'var="{}"; mv "{}" "${var%.*}_copy.png"' \;
Or is there any way this can be achieved directly from {}?
Consensus
As Etan Reisner remarked, a better and safer way to handle the output of find would be to pass it as a positional argument:
find . -name "*.png" -exec bash -c 'mv "$0" "${0%.*}_copy.png"' "{}" \;
It took me a while to get the question. Basically you are asking if something like:
echo "${test.png%.png}"
could be used to get the word test.
The answer is no. The string manipulation operators starting with ${ are a subset of the parameter substitution operators. They work only on variables, not with string literals, meaning you need to store the string in a variable before. Like this:
img="test.png"
echo "${img%.png}"
Just for travellers from Google, I would use rename for this particular task. (As the OP already mentioned in his question). The command could look like this:
find -name '*png' -execdir rename 's/\.png/_copy.png/' {} +
I am figuring out a command to copy files that are modified on a Saturday.
find -type f -printf '%Ta\t%p\n'
This way the line starts with the weekday.
When I combine this with a 'egrep' command using a regular expression (starts with "za") it shows only the files which start with "za".
find -type f -printf '%Ta\t%p\n' | egrep "^(za)"
("za" is a Dutch abbreviation for "zaterdag", which means Saturday,
This works just fine.
Now I want to copy the files with this command:
find -type f -printf '%Ta\t%p\n' -exec cp 'egrep "^(za)" *' /home/richard/test/ \;
Unfortunately it doesn't work.
Any suggestions?
The immediate problem is that -printf and -exec are independent of each other. You want to process the result of -printf to decide whether or not to actually run the -exec part. Also, of course, passing an expression in single quotes simply passes a static string, and does not evaluate the expression in any way.
The immediate fix to the evaluation problem is to use a command substitution instead of single quotes, but the problem that the -printf function's result is not available to the command substitution still remains (and anyway, the command substitution would happen before find runs, not while it runs).
A common workaround would be to pass a shell script snippet to -exec, but that still doesn't expose the -printf function to the -exec part.
find whatever -printf whatever -exec sh -c '
case $something in za*) cp "$1" "$0"; esac' "$DEST_DIR" {} \;
so we have to figure out a different way to pass the $something here.
(The above uses a cheap trick to pass the value of $DEST_DIR into the subshell so we don't have to export it. The first argument to sh -c ... ends up in $0.)
Here is a somewhat roundabout way to accomplish this. We create a format string which can be passed to sh for evaluation. In order to avoid pesky file names, we print the inode numbers of matching files, then pass those to a second instance of find for performing the actual copying.
find \( -false $(find -type f \
-printf 'case %Ta in za*) printf "%%s\\n" "-o -inum %i";; esac\n' |
sh) \) -exec cp -t "$DEST_DIR" \+
Using the inode number means any file name can be processed correctly (including one containing newlines, single or double quotes, etc) but may increase running time significantly, because we need two runs of find. If you have a large directory tree, you will probably want to refactor this for your particular scenario (maybe run only in the current directory, and create a wrapper to run it in every directory you want to examine ... thinking out loud here; not sure it helps actually).
This uses features of GNU find which are not available e.g. in *BSD (including OSX). If you are not on Linux, maybe consider installing the GNU tools.
What you can do is a shell expansion. Something like
cp $(find -type f -printf '%Ta\t%p\n' | egrep "^(za)") $DEST_DIR
Assuming that the result of your find and grep is just the filenames (and full paths, at that), this will copy all the files that match your criteria to whatever you set $DEST_DIR to.
EDIT As mentioned in the comments, this won't work if your filenames contain spaces. If that's the case, you can do something like this:
find -type f -printf '%Ta\t%p\n' | egrep "^(za)" | while read file; do cp "$file" $DEST_DIR; done
This problem has been discussed extensively but I couldn't find a solution that would help me.
I'm trying to selectively copy files from a directory tree into a specific folder. After reading some Q&A, here's what I tried:
cp `find . -name "*.pdf" -type f` ../collect/
I am in the right parent directory and there indeed is a collect directory a level above. Now I'm getting the error: cp: invalid option -- 'o'
What is going wrong?
To handle difficult file names:
find . -name "*.pdf" -type f -exec cp {} ../collect/ \;
By default, find will print the file names that it finds. If one uses the -exec option, it will instead pass the file names on to a command of your choosing, in this case a cp command which is written as:
cp {} ../collect/ \;
The {} tells find where to insert the file name. The end of the command given to -exec is marked by a semicolon. Normally, the shell would eat the semicolon. So, we escape the semicolon with a backslash so that it is passed as an argument to the find command.
Because find gives the file name to cp directly without interference from the shell, this approach works for even the most difficult file names.
More efficiency
The above runs cp on every file found. If there are many files, that would be a lot of processes started. If one has GNU tools, that can be avoided as follows:
find . -name '*.pdf' -type f -exec cp -t ../collect {} +
In this variant of the command, find will supply many file names for each single invocation of cp, potentially greatly reducing the number of processes that need to be started.
I just stumbled upon Perl today while playing around with Bash scripting. When I tried to remove blank spaces in multiple file names, I found this post, which helped me a lot.
After a lot of struggling, I finally understand the rename and substitution commands and their syntax. I wanted to try to replace all "_(x)" at the end of file names with "x", due to duplicate files. But when I try to do it myself, it just does not seem to work. I have three questions with the following code:
Why is nothing executed when I run it?
I used redirection to show me the success note as an error, so I know what happened. What did I do wrong about that?
After a lot of research, I still do not entirely understand file descriptors and redirection in Bash as well as the syntax for the substitute function in Perl. Can somebody give give me a link for a good tutorial?
find -name "*_(*)." -type f | \
rename 's/)././g' && \
find -name "*_(*." -type f | \
rename 's/_(//g' 2>&1
You either need to use xargs or you need to use find's ability to execute commands:
find -name "*_(*)." -type f | xargs rename 's/)././g'
find -name "*_(*." -type f | xargs rename 's/_(//g'
Or:
find -name "*_(*)." -type f -exec rename 's/)././g' {} +
find -name "*_(*." -type f -exec rename 's/_(//g' {} +
In both cases, the file names are added to the command line of rename. As it was, rename would have to read its standard input to discover the file names — and it doesn't.
Does the first find find the files you want? Is the dot at the end of the pattern needed? Do the regexes do what you expect? OK, let's debug some of those too.
You could do it all in one command with a more complex regex:
find . -name "*_(*)" -type f -exec rename 's/_\((\d+)\)$/$1/' {} +
The find pattern is corrected to lose the requirement of a trailing .. If the _(x) is inserted before the extension, then you'd need "*_(*).*" as the pattern for find (and you'll need to revise the Perl regexes).
The Perl substitute needs dissection:
The \( matches an open parenthesis.
The ( starts a capture group.
The \d+ looks for 'one or more digits'.
The ) stops the capture group. It is the first and only, so it is given the number 1.
The \) matches a close parenthesis.
The $ matches the end of the file name.
The $1 in the replacement puts the value of capture group 1 into the replacement text.
In your code, the 2>&1 sent the error messages from the second rename command to standard output instead of standard error. That really doesn't help much here.
You need two separate tutorials; you are not going to find one tutorial that covers I/O redirection in Bash and regular expressions in Perl.
The 'official' Perl regular expression tutorial is:
perlretut, also available as perldoc perlretut on your machine.
The Bash manual covers I/O redirection, but it is somewhat terse:
I/O Redirections.
I'm trying to return a particular line from files found from this search:
find . -name "database.php"
Each of these files contains a database name, next to a php variable like $dname=
I've been trying to use -exec to execute a grep search on this file with no success
-exec "grep {\}\ dbname"
Can anyone provide me with some understanding of how to accomplish this task?
I'm running CentOS 5, and there are about 100 database.php files stored in subdirectories on my server.
Thanks
Jason
You have the arguments to grep inverted, and you need them as separate arguments:
find . -name "database.php" -exec grep '$dbname' /dev/null {} +
The presence of /dev/null ensures that the file name(s) that match are listed as well as the lines that match.
I think this will do it. Not sure if you need to make any adjustments for CentOS.
find . -name "database.php" -exec grep dbname {} \;
I worked it out using xargs
find . -name "database.php" -print | xargs grep \'database\'\=\> > list_of_databases
Feel free to post a better way if you find one (or what some rep for a good answer)
I tend to habitually avoid find because I've never learned how to use it properly, so the way I'd accomplish your task would be:
grep dbname **/database.php
Edit: This command won't be viable in all cases because it can potentially generate a very long argument list, whereas find executes its command on found files one by one like xargs. And, as I noted in my comment, it's possibly not very portable. But it's damn short ;)