I am trying to make copies of certain file and let them have a prefix.
in order to do it I thought of using find. for our use, let's call them kuku files and I want them to have a "foo" prefix:
find . -maxdepth 1 -name "kuku*" -exec cp '{}' foo_'{}' \;
but it doesn't work because the find always starts the results with ./ so i get a lot of error messages saying "cp: cannot create regular file `foo_./kuku...`: No such file or directory".
the problem is solvable by using foreach f (`ls`) and than using grep and the status var, but it is cumbersome and I want to learn a better solution (and improve my knowledge of the find command along the way...).
update foreach solution (which I don't like and want your help in finding a replacement):
foreach f (`ls`)
echo $f | grep -lq kuku
if (! $status) then
cp $f foo_$f
endif
end
but this is UGLY! (end of update)
as the header says, I'm using csh - not because I love it, just because that's what we use at work...
update
trying to use basename as a solution, because find -exec basename '{}' \; removes the ./ prefix, but i failed using the basename inside the find with backticks (`), meaning that
find -name "kuku*" -exec cp '{}' foo_`basename '{}` \;
simply doesn't work.
Here you go.. I have tested in my linux box
find . -name "kuku*" -exec sh -c 'cp {} foo_`basename {}`' \;
Related
This question already has answers here:
Recursively change file extensions in Bash
(6 answers)
Closed 6 years ago.
The user inputs a file type they are looking for; it is stored in $arg1; the file type they would like to change them is stored as $arg2. I'm able to find what I'm looking for, but I'm unsure of how to keep the filename the same but just change the type... ie., file1.txt into file1.log.
find . -type f -iname "*.$arg1" -exec mv {} \;
To enable the full power of shell parameter expansions, you can call bash -c in your exec action:
find . -type f -iname "*.$arg1" \
-exec bash -c 'echo mv "$1" "${1/%.*/$1}"' _ {} "$arg2" \;
We add {} and "$arg2" as a parameters to bash -c, so they become accessible within the command as $0 and $1. ${0%.*} removes the extension, to be replaced by whatever $arg2 expands to.
As it is, the command just prints the mv commands it would execute; to actually rename the files, the echo has to be removed.
The quoting is relevant: the argument to bash -c is in single quotes to prevent $0 and $1 from being expanded prematurely, and the two arguments to mv, and arg2 are also quoted to deal with file names with spaces in them.
Combining the find -exec bash idea with the bash loop idea, you can use the + terminator on the -exec to tell find to pass multiple filenames to a single invocation of the bash command. Pass the new type as the first argument - which shows up in $0 and so is conveniently skipped by a for loop over the rest of the command-line arguments - and you have a pretty efficient solution:
find . -type f -iname "*.$arg1" -exec bash -c \
'for arg; do mv "$arg" "${arg%.*}.$0"; done' "$arg2" {} +
Alternatively, if you have either version of the Linux rename command, you can use that. The Perl one (a.k.a. prename, installed by default on Ubuntu and other Debian-based distributions; also available for OS X from Homebrew via brew install rename) can be used like this:
find . -type f -iname "*.$arg1" -exec rename 's/\Q'"$arg1"'\E$/'"$arg2"'/' {} +
That looks a bit ugly, but it's really just the s/old/new/ substitution command familiar from many UNIX tools. The \Q and \E around $arg1 keep any weird characters inside the suffix from being interpreted as regular expression metacharacters that might match something unexpected; the $ after the \E makes sure the pattern only matches at the end of the filename.
The pattern-based version installed by default on Red Hat-based Linux distros (Fedora, CentOS, etc) is simpler:
find . -type f -iname "*.$arg1" -exec rename ".$arg1" ".$arg2" {} +
but it's also dumber: if you rename .com .exe stackoverflow.com_scanner.com, you'll get a file named stackoverflow.exe_scanner.exe.
I would do it like so:
find . -type f -iname "*.$arg1" -print0 |\
while IFS= read -r -d '' file; do
mv -- "$file" "${file%$arg1}$arg2"
done
I took your find command and fed its output to a while loop. Within that loop, I am doing the actual renaming. This way I have the name of the file as a variable that I can manipulate using bash's string manipulation operations.
If you have perl based rename command
Sample directory:
$ find
.
./a"bn.txt
./t2.abc
./abc
./abc/t1.txt
./abc/list.txt
./a bc.txt
Sample args:
$ arg1='txt'
$ arg2='log'
Dry run:
$ find -type f -iname "*.$arg1" -exec rename -n "s/$arg1$/$arg2/" {} +
rename(./a"bn.txt, ./a"bn.log)
rename(./abc/t1.txt, ./abc/t1.log)
rename(./abc/list.txt, ./abc/list.log)
rename(./a bc.txt, ./a bc.log)
Remove -n option once it is okay:
$ find -type f -iname "*.$arg1" -exec rename "s/$arg1$/$arg2/" {} +
$ find
.
./a bc.log
./t2.abc
./abc
./abc/list.log
./abc/t1.log
./a"bn.log
I would like to ensure that all files found by find with a given criteria are properly copied to the required location.
$from = '/some/path/to/the/files'
$ext = 'custom_file_extension'
$dest = '/new/destination/for/the/files/with/given/extension'
cp 'find $from -name "*.$ext"' $dest
The problem here is that, when a file found with the proper extension and it is containing space cp cannot copy it properly.
You don't do that. You can't splat filenames with spaces that way.
You either get to use something from http://mywiki.wooledge.org/BashFAQ/001 to read the output from find line-by-line or into an array or you use find -exec to do the copy work.
Something like this:
from='/some/path/to/the/files'
ext='custom_file_extension'
dest='/new/destination/for/the/files/with/given/extension'
find "$from" -name "*.$ext" -exec cp -t "$dest" {} +
Using -exec command + here means that find will only execute as many cp commands as it needs based on command length limits. Using -exec command ; here would run one cp-per-file-found (but is more portable to older systems).
See comment from gniourf_gniourf about the use of -t in that cp command to make -exec command + work correctly.
Use -exec:
find "$from" -name "*.$ext" -exec cp {} "$dest" \;
you need to copy file one by one:
for file in "$from"/*."$ext"; do
cp "$file" "$dest"
done
I just use glob here, and it's enough and complete. I think find may introduce problem if the file name contains funny character.
The solution for this sort of problem is xargs -0 and the -print0 flag for find.
-print0 instructs find to print the results with a NUL character termination, instead of a newline, while -0 for xargs tells it expect input in that format.
Finally, the -J option for xargs allows one to put the arguments in the right place for a copy.
find "$from" -name "*.$ext" -print0 | xargs -0 -J % cp % "$dest"
It's better to use -exec argument of find command to do this:
find . -type f -name "*.ext" -exec cp {} ./destination_dir \;
I've checked this case with files containing spaces and it's work for me. Also don't forger to point out '-type f' if you want to find only files, not directories.
It seems like I am unable to find a direct answer to this question.
I appreciate your help.
I'm trying to find all files with a specific name in a directory, read the last 1000 lines of the file and copy it in to a new file in the same directory. As an example:
Find all files names xyz.log in the current directory, copy the last 1000 lines to file abc.log (which doesn't exist).
I tried to use the following command with no luck:
find . -name "xyz.log" -execdir tail -1000 {} > abc.log \;
The problem I'm having is that for all the files in the current directory, they all write to abc.log in the CURRENT directory and not in the directory where xyz.log resides. Clearly the find with execdir is first executed and then the output is redirected to abc.log.
Can you guys suggest a way to fix this? I appreciate any information/help.
EDIT- I tried find . -name "xyz.log" -execdir sh -c "tail -1000 {} > abc.log" \; as suggested by some of the friends, but it gives me this error: sh: ./tail: No such file or directory error message. Do you guys have any idea what the problem is?
Luckily the solution to use -printf is working fine.
The simplest way is this:
find . -name "xyz.log" -execdir sh -c 'tail -1000 "{}" >abc.log' \;
A more flexible alternative is to first print out the commands and then execute them all with sh:
find . -name "xyz.log" -printf 'tail -1000 "%p" >"%h/abc.log"\n' | sh
You can remove the | sh from the end when you're trying it out/debugging.
There is a bug in some versions of findutils (4.2 and 4.3, though it was fixed in some 4.2.x and 4.3.x versions) that cause execdir arguments that contain {} to be prefixed with ./ (instead of the prefix being applied only to {} it is applied to the whole quoted string). To work around this you can use:
find . -name "xyz.log" -execdir sh -c 'tail -1000 "$1" >abc.log' sh {} \;
sh -c 'script' arg0 arg1 runs the sh script with arg0, arg1, etc. passed to it. By convention, arg0 is the name of the executable (here, "sh"). From the script you can access the arguments using $0 (corresponding to "sh"), $1 (corresponding to find's expansion of {}), etc.
The redirect isn't passed into execdir, so abc.log shows up in the directory you run the command in. -execdir also doesn't like embedded redirects. but you can workaround the problem by passing -execdir a shell command with a redirect embedded, like this:
find . -name "xyz.log" -execdir sh -c '/usr/bin/tail -1000 {} > abc.log' \;
Much credit to this blog post (not mine):
http://www.microhowto.info/howto/act_on_all_files_in_a_directory_tree_using_find.html
Edit
I put the full path to tail in the command (assuming it's in /usr/bin on your system), since sh may load a .profile with a PATH that differs from your current shell.
Here's another non-find (well, sorta - it still uses find but doesn't try to shoehorn find into doing the whole thing):
while read f
do
d=$(dirname "${f}")
tail -n 1000 "${f}" > "${d}/abc.log"
done < <(find . -type f -name xyz.log -print)
If I run this command:
sudo find . -name *.mp3
then I can get a listing of lots of mp3 files.
Now I want to do something with each mp3 file in a loop. For example, I could create a while loop, and inside assign the first file name to the variable file. Then I could do something with that file. Next I could assign the second file name to the variable file and do with that, etc.
How can I realize this using a linux shell command? Any help is appreciated, thanks!
For this, use the read builtin:
sudo find . -name *.mp3 |
while read filename
do
echo "$filename" # ... or any other command using $filename
done
Provided that your filenames don't use the newline (\n) character, this should work fine.
My favourites are
find . -name '*.mp3' -exec cmd {} \;
or
find . -name '*.mp3' -print0 | xargs -0 cmd
While Loop
As others have pointed out, you can frequently use a while read loop to read filenames line by line, it has the drawback of not allowing line-ends in filenames (who uses that?).
xargs vs. -exec cmd {} +
Summarizing the comments saying that -exec...+ is better, I prefer xargs because it is more versatile:
works with other commands than just find
allows 'batching' (grouping) in command lines, say xargs -n 10 (ten at a time)
allows parallellizing, say xargs -P4 (max 4 concurrent processes running at a time)
does privilige separation (such as in the OP's case, where he uses sudo find: using -exec would run all commands as the root user, whereas with xargs that isn't necessary:
sudo find -name '*.mp3' -print0 | sudo xargs -0 require_root.sh
sudo find -name '*.mp3' -print0 | xargs -0 nonroot.sh
in general, pipes are just more versatile (logging, sorting, remoting, caching, checking, parallelizing etc, you can do that)
How about using the -exec option to find?
find . -name '*.mp3' -exec mpg123 '{}' \;
That will call the command mpg123 for every file found, i.e. it will play all the files, in the order they are found.
for file in $(sudo find . -name *.mp3);
do
# do something with file
done
I want to know exactly what {} \; and {} \+ and | xargs ... do. Please clarify these with explanations.
Below 3 commands run and output same result but the first command takes a little time and the format is also little different.
find . -type f -exec file {} \;
find . -type f -exec file {} \+
find . -type f | xargs file
It's because 1st one runs the file command for every file coming from the find command. So, basically it runs as:
file file1.txt
file file2.txt
But latter 2 find with -exec commands run file command once for all files like below:
file file1.txt file2.txt
Then I run the following commands on which first one runs without problem but second one gives error message.
find . -type f -iname '*.cpp' -exec mv {} ./test/ \;
find . -type f -iname '*.cpp' -exec mv {} ./test/ \+ #gives error:find: missing argument to `-exec'
For command with {} \+, it gives me the error message
find: missing argument to `-exec'
why is that? can anyone please explain what am I doing wrong?
The manual page (or the online GNU manual) pretty much explains everything.
find -exec command {} \;
For each result, command {} is executed. All occurences of {} are replaced by the filename. ; is prefixed with a slash to prevent the shell from interpreting it.
find -exec command {} +
Each result is appended to command and executed afterwards. Taking the command length limitations into account, I guess that this command may be executed more times, with the manual page supporting me:
the total number of invocations of the command will be much less than the number of matched files.
Note this quote from the manual page:
The command line is built in much the same way that xargs builds its command lines
That's why no characters are allowed between {} and + except for whitespace. + makes find detect that the arguments should be appended to the command just like xargs.
The solution
Luckily, the GNU implementation of mv can accept the target directory as an argument, with either -t or the longer parameter --target. It's usage will be:
mv -t target file1 file2 ...
Your find command becomes:
find . -type f -iname '*.cpp' -exec mv -t ./test/ {} \+
From the manual page:
-exec command ;
Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of `;' is encountered. The string `{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a `\') or quoted to protect them from expansion by the shell. See the EXAMPLES section for examples of the use of the -exec option. The specified command is run once for each matched file. The command is executed in the starting directory. There are unavoidable security problems surrounding use of the -exec action; you should use the -execdir option instead.
-exec command {} +
This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of `{}' is allowed within the command. The command is executed in the starting directory.
I encountered the same issue on Mac OSX, using a ZSH shell: in this case there is no -t option for mv, so I had to find another solution.
However the following command succeeded:
find .* * -maxdepth 0 -not -path '.git' -not -path '.backup' -exec mv '{}' .backup \;
The secret was to quote the braces. No need for the braces to be at the end of the exec command.
I tested under Ubuntu 14.04 (with BASH and ZSH shells), it works the same.
However, when using the + sign, it seems indeed that it has to be at the end of the exec command.
The standard equivalent of find -iname ... -exec mv -t dest {} + for find implementations that don't support -iname or mv implementations that don't support -t is to use a shell to re-order the arguments:
find . -name '*.[cC][pP][pP]' -type f -exec sh -c '
exec mv "$#" /dest/dir/' sh {} +
By using -name '*.[cC][pP][pP]', we also avoid the reliance on the current locale to decide what's the uppercase version of c or p.
Note that +, contrary to ; is not special in any shell so doesn't need to be quoted (though quoting won't harm, except of course with shells like rc that don't support \ as a quoting operator).
The trailing / in /dest/dir/ is so that mv fails with an error instead of renaming foo.cpp to /dest/dir in the case where only one cpp file was found and /dest/dir didn't exist or wasn't a directory (or symlink to directory).
find . -name "*.mp3" -exec mv --target-directory=/home/d0k/Музика/ {} \+
no, the difference between + and \; should be reversed. + appends the files to the end of the exec command then runs the exec command and \; runs the command for each file.
The problem is find . -type f -iname '*.cpp' -exec mv {} ./test/ \+ should be find . -type f -iname '*.cpp' -exec mv {} ./test/ + no need to escape it or terminate the +
xargs I haven't used in a long time but I think works like +.