How do I pipe the results of a 'find' (in Linux) to be moved to a different directory? This is what I have so far.
find ./ -name '*article*' | mv ../backup
but its not yet right (I get an error missing file argument, because I didn't specify a file, because I was trying to get it from the pipe)
find ./ -name '*article*' -exec mv {} ../backup \;
OR
find ./ -name '*article*' | xargs -I '{}' mv {} ../backup
xargs is commonly used for this, and mv on Linux has a -t option to facilitate that.
find ./ -name '*article*' | xargs mv -t ../backup
If your find supports -exec ... \+ you could equivalently do
find ./ -name '*article*' -exec mv -t ../backup {} \+
The -t option is a GNU extension, so it is not portable to systems which do not have GNU coreutils (though every proper Linux I have seen has that, with the possible exception of Busybox). For complete POSIX portability, it's of course possible to roll your own replacement, maybe something like
find ./ -name '*article*' -exec sh -c 'mv "$#" "$0"' ../backup {} \+
where we shamelessly abuse the convenient fact that the first argument after sh -c 'commands' ends up as the "script name" parameter in $0 so that we don't even need to shift it.
Probably see also https://mywiki.wooledge.org/BashFAQ/020
I found this really useful having thousands of files in one folder:
ls -U | head -10000 | egrep '\.png$' | xargs -I '{}' mv {} ./png
To move all pngs in first 10000 files to subfolder png
mv $(find . -name '*article*') ../backup
Here are a few solutions.
find . -type f -newermt "2019-01-01" ! -newermt "2019-05-01" \
-exec mv {} path \;**
or
find path -type f -newermt "2019-01-01" ! -newermt "2019-05-01" \
-exec mv {} path \;
or
find /Directory/filebox/ -type f -newermt "2019-01-01" \
! -newermt "2019-05-01" -exec mv {} ../filemove/ \;
The backslash + newline is just for legibility; you can equivalently use a single long line.
xargs is your buddy here (When you have multiple actions to take)!
And using it the way I have shown will give great control to you as well.
find ./ -name '*article*' | xargs -n1 sh -c "mv {} <path/to/target/directory>"
Explanation:
-n1
Number of lines to consider for each operation ahead
sh -c
The shell command to execute giving it the lines as per previous condition
"mv {} /target/path"
The move command will take two arguments-
1) The line(s) from operation 1, i.e. {}, value substitutes automatically
2) The target path for move command, as specified
Note: the "Double Quotes" are specified to allow any number of spaces or arguments for the shell command which receives arguments from xargs
Related
Am trying to write a script that finds the files that are older than 10 hours from the sub-directories that are in the "HS_client_list". And send the Output to a file "find.log".
#!/bin/bash
while IFS= read -r line; do
echo Executing cd /moveit/$line
cd /moveit/$line
#Find files less than 600 minutes old.
find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log
done < HS_client_list
However, the script is able to cd to the folders from HS_client_list(this file contents the name of the subdirectories) but, the find command (find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log) is not working. The Output file is empty. But when I run find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log as a command it works and from the script it doesn't.
You are overwriting the file in each iteration.
You can use xargs to perform find on multiple directories; but you have to use an alternate delimiter to avoid having xargs populate the {} in the -execdir command.
sed 's%^%/moveit/%' HS_client_list |
xargs -I '<>' find '<>' -type f -iname "*.enc" -mmin +600 -execdir basename {} \; > /home/infa91punv/find.log
The xargs ls did not seem to perform any useful functionality, so I took it out. Generally, don't use ls in scripts.
With GNU find, you could avoid the call to an external utility, and use the -printf predicate to print just the part of the path name that you care about.
For added efficiency, you could invoke a shell to collect the arguments:
sed 's%^%/moveit/%' HS_client_list |
xargs sh -c 'find "$#" -type f -iname "*.enc" -mmin +600 -execdir basename {} \;' _ >/home/infa91punv/find.log
This will run as many directories as possible in a single find invocation.
If you want to keep your loop, the solution is to put the redirection after done. I would still factor out the cd, and take care to quote the variable interpolation.
while IFS= read -r line; do
find /moveit/"$line" -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';'
done < HS_client_list >/home/infa91punv/find.log
I have a directory (with subdirectories), of which I want to find all files that have a ".ipynb" extension. But I want the 'find' command to just return me these filenames without the extension.
I know the first part:
find . -type f -iname "*.ipynb" -print
But how do I then get the names without the "ipynb" extension?
Any replies greatly appreciated...
To return only filenames without the extension, try:
find . -type f -iname "*.ipynb" -execdir sh -c 'printf "%s\n" "${0%.*}"' {} ';'
or (omitting -type f from now on):
find "$PWD" -iname "*.ipynb" -execdir basename {} .ipynb ';'
or:
find . -iname "*.ipynb" -exec basename {} .ipynb ';'
or:
find . -iname "*.ipynb" | sed "s/.*\///; s/\.ipynb//"
however invoking basename on each file can be inefficient, so #CharlesDuffy suggestion is:
find . -iname '*.ipynb' -exec bash -c 'printf "%s\n" "${#%.*}"' _ {} +
or:
find . -iname '*.ipynb' -execdir basename -s '.sh' {} +
Using + means that we're passing multiple files to each bash instance, so if the whole list fits into a single command line, we call bash only once.
To print full path and filename (without extension) in the same line, try:
find . -iname "*.ipynb" -exec sh -c 'printf "%s\n" "${0%.*}"' {} ';'
or:
find "$PWD" -iname "*.ipynb" -print | grep -o "[^\.]\+"
To print full path and filename on separate lines:
find "$PWD" -iname "*.ipynb" -exec dirname "{}" ';' -exec basename "{}" .ipynb ';'
Here's a simple solution:
find . -type f -iname "*.ipynb" | sed 's/\.ipynb$//1'
I found this in a bash oneliner that simplifies the process without using find
for n in *.ipynb; do echo "${n%.ipynb}"; done
If you need to have the name with directory but without the extension :
find . -type f -iname "*.ipynb" -exec sh -c 'f=$(basename $1 .ipynb);d=$(dirname $1);echo "$d/$f"' sh {} \;
find . -type f -iname "*.ipynb" | grep -oP '.*(?=[.])'
The -o flag outputs only the matched part. The -P flag matches according to Perl regular expressions. This is necessary to make the lookahead (?=[.]) work.
Perl One Liner
what you want
find . | perl -a -F/ -lne 'print $F[-1] if /.*.ipynb/g'
Then not your code
what you do not want
find . | perl -a -F/ -lne 'print $F[-1] if !/.*.ipynb/g'
NOTE
In Perl you need to put extra .. So your pattern would be .*.ipynb
If there's no occurrence of this ".ipynb" string on any file name other than a suffix, then you can try this simpler way using tr:
find . -type f -iname "*.ipynb" -print | tr -d ".ipbyn"
If you don't know that the extension is or there are multiple you could use this:
find . -type f -exec basename {} \;|perl -pe 's/(.*)\..*$/$1/;s{^.*/}{}'
and for a list of files with no duplicates (originally differing in path or extension)
find . -type f -exec basename {} \;|perl -pe 's/(.*)\..*$/$1/;s{^.*/}{}'|sort|uniq
Another easy way which uses basename is:
find . -type f -iname '*.ipynb' -exec basename -s '.ipynb' {} +
Using + will reduce the number of invocations of the command (manpage):
-exec command {} +
This variant of the -exec action runs the specified command on
the selected files, but the command line is built by appending
each selected file name at the end; the total number of
invocations of the command will be much less than the number
of matched files. The command line is built in much the same
way that xargs builds its command lines. Only one instance of
'{}' is allowed within the command, and (when find is being
invoked from a shell) it should be quoted (for example, '{}')
to protect it from interpretation by shells. The command is
executed in the starting directory. If any invocation with
the `+' form returns a non-zero value as exit status, then
find returns a non-zero exit status. If find encounters an
error, this can sometimes cause an immediate exit, so some
pending commands may not be run at all. For this reason -exec
my-command ... {} + -quit may not result in my-command
actually being run. This variant of -exec always returns
true.
Using -s with basename runs accepts multiple filenames and removes a specified suffix (manpage):
-a, --multiple
support multiple arguments and treat each as a NAME
-s, --suffix=SUFFIX
remove a trailing SUFFIX; implies -a
I was helped out today with a command, but it doesn't seem to be working. This is the command:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 && rm {}\;
The shell returns
find: missing argument to `-exec'
What I am basically trying to do is go through a directory recursively (if it has other directories) and run the ffmpeg command on the .rm file types and convert them to .mp3 file types. Once this is done, remove the .rm file that has just been converted.
A -exec command must be terminated with a ; (so you usually need to type \; or ';' to avoid interpretion by the shell) or a +. The difference is that with ;, the command is called once per file, with +, it is called just as few times as possible (usually once, but there is a maximum length for a command line, so it might be split up) with all filenames. See this example:
$ cat /tmp/echoargs
#!/bin/sh
echo $1 - $2 - $3
$ find /tmp/foo -exec /tmp/echoargs {} \;
/tmp/foo - -
/tmp/foo/one - -
/tmp/foo/two - -
$ find /tmp/foo -exec /tmp/echoargs {} +
/tmp/foo - /tmp/foo/one - /tmp/foo/two
Your command has two errors:
First, you use {};, but the ; must be a parameter of its own.
Second, the command ends at the &&. You specified “run find, and if that was successful, remove the file named {};.“. If you want to use shell stuff in the -exec command, you need to explicitly run it in a shell, such as -exec sh -c 'ffmpeg ... && rm'.
However you should not add the {} inside the bash command, it will produce problems when there are special characters. Instead, you can pass additional parameters to the shell after -c command_string (see man sh):
$ ls
$(echo damn.)
$ find * -exec sh -c 'echo "{}"' \;
damn.
$ find * -exec sh -c 'echo "$1"' - {} \;
$(echo damn.)
You see the $ thing is evaluated by the shell in the first example. Imagine there was a file called $(rm -rf /) :-)
(Side note: The - is not needed, but the first variable after the command is assigned to the variable $0, which is a special variable normally containing the name of the program being run and setting that to a parameter is a little unclean, though it won't cause any harm here probably, so we set that to just - and start with $1.)
So your command could be something like
find -exec bash -c 'ffmpeg -i "$1" -sameq "$1".mp3 && rm "$1".mp3' - {} \;
But there is a better way. find supports and and or, so you may do stuff like find -name foo -or -name bar. But that also works with -exec, which evaluates to true if the command exits successfully, and to false if not. See this example:
$ ls
false true
$ find * -exec {} \; -and -print
true
It only runs the print if the command was successfully, which it did for true but not for false.
So you can use two exec statements chained with an -and, and it will only execute the latter if the former was run successfully.
Try putting a space before each \;
Works:
find . -name "*.log" -exec echo {} \;
Doesn't Work:
find . -name "*.log" -exec echo {}\;
I figured it out now. When you need to run two commands in exec in a find you need to actually have two separate execs. This finally worked for me.
find . -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 \; -exec rm {} \;
You have to put a space between {} and \;
So the command will be like:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 && rm {} \;
Just for your information:
I have just tried using "find -exec" command on a Cygwin system (UNIX emulated on Windows), and there it seems that the backslash before the semicolon must be removed:
find ./ -name "blabla" -exec wc -l {} ;
For anyone else having issues when using GNU find binary in a Windows command prompt. The semicolon needs to be escaped with ^
find.exe . -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 ^;
You need to do some escaping I think.
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} \-sameq {}.mp3 \&\& rm {}\;
Just in case anyone sees a similar "missing -exec args" in Amazon Opsworks Chef bash scripts, I needed to add another backslash to escape the \;
bash 'remove_wars' do
user 'ubuntu'
cwd '/'
code <<-EOH
find /home/ubuntu/wars -type f -name "*.war" -exec rm {} \\;
EOH
ignore_failure true
end
Also, if anyone else has the "find: missing argument to -exec" this might help:
In some shells you don't need to do the escaping, i.e. you don't need the "\" in front of the ";".
find <file path> -name "myFile.*" -exec rm - f {} ;
Both {} and && will cause problems due to being expanded by the command line. I would suggest trying:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i \{} -sameq \{}.mp3 \; -exec rm \{} \;
In my case I needed to execute "methods" from by bash script, which does not work when using -exec bash -c, so I add another solution I found here, as well:
UploadFile() {
curl ... -F "file=$1"
}
find . | while read file;
do
UploadFile "$file"
done
This thread pops up first when searching for solutions to execute commands for each file from find, so I hope it's okay that this solution does not use the -exec argument
I got the same error when I left a blank space after the ending ; of an -exec command.So, remove blank space after ;
If you are still getting "find: missing argument to -exec" try wrapping the execute argument in quotes.
find <file path> -type f -exec "chmod 664 {} \;"
I am trying to move about 700,000 .jpg files from one directory to another in my Ubuntu server. I tried the following:
xargs mv * -t /var/www/html/
and
echo (*.jpg|*.png|*.bmp) | xargs mv -t /var/www/html/
and
echo (*.jpg) | xargs mv -t /var/www/html/
and
find . -name "*.jpg" -print0 | xargs mv * ../
and they all give me the same error: /usr/bin/xargs: Argument list too long
what should I do? Please help me out. Thanks :)
If you use find I would recommend you to use the -exec attribute. So your result should be find . -name "*.jpg" -exec mv {} /home/new/location \;.
However I would recommend to check what the find command returns you, replacing the exec part with: -exec ls -lrt {} \;
Try:
find /path/to/old-directory -type f | xargs -i mv "{}" /path/to/new-directory
You could have tried:
for f in *.jpg do;
mv -tv $f /var/www/html/
done
for f in *.png do;
mv -tv $f /var/www/html/
done
for f in *.bmp do;
mv -tv $f /var/www/html/
done
also, you should carefully read xargs(1); I strongly suspect that
find . -name "*.jpg" -print0 | xargs -n 1000 -I '{}' mv '{}' ../
should work for you
At last, learn more about rename(1). It is probably enough for the job.
find ./dir -type f -iname "*.t[argz]*[bz2]" -print | xargs mv --target-directory=dir
seems to fail on file that has spaces in the name.
how to improve it? or alternative?
thanks for answer below: my mv doesn't support --null or -0, I'm using cygwin:
$ mv --help
Usage: mv [OPTION]... [-T] SOURCE DEST
or: mv [OPTION]... SOURCE... DIRECTORY
or: mv [OPTION]... -t DIRECTORY SOURCE...
Rename SOURCE to DEST, or move SOURCE(s) to DIRECTORY.
Mandatory arguments t
.
Use -print0 instead of -print on the find command, and the xargs -0 (or --null) option - then NULs will be used as separators rather than newlines and spaces.
find ./dir -type f -iname "*.t[argz]*[bz2]" -print0 | xargs --null mv --target-directory=dir
Have you looked at the -exec option for find?
find ./dir -type f -iname "*.t[argz][bz2]" -exec mv {} --target-directory=dir ';'
The -exec option will execute any options following it as a command until it sees the semi-colon wrapped in quotes. This way you won't have to deal with the spacing issue.
GNU find
find ./dir -type f -iname "*.t[argz]*[bz2]" -exec mv "{}" /destination +;