Currently using this command to get all my "fanart" from my TV folder, and dump it into a single folder.
find /volume1/tv/ -type f \( -name '*fanart.jpg'* -o -path '*/fanart/*.jpg' -o -path '*/extrafanart/*.jpg' \) -exec cp {} /volume1/tv/_FANART \;
Here's the issue: a lot of these files have the same name, and can't be dumped into the same folder. Example:
Folder A
fanart.jpg
Folder B
fanart.jpg
Is there a way to copy these files from their respective folders and give them a unique name in the destination folder? Name needn't be anything descriptive, random is just fine.
Thanks!
find /volume1/tv/ -type f \( -name '*fanart.jpg'* -o -path '*/fanart/*.jpg' -o -path '*/extrafanart/*.jpg' \) -exec cp --backup=numbered {} /volume1/tv/_FANART \;
..
cp --backup=numbered {}
If the file exists, this will not overwrite but make a backup with a number assigned.
The files will be hidden. Ctrl+H to view hidden files
You could copy the files while giving them names according to their locations in the original directory tree. For instance (":" is legal but
unusual in filenames), your "find" command could call a shell script (rather than "cp" directly), which might look like this:
#!/bin/sh
case "x$1" in
x/volume1/tv/_FANART/*)
;;
*)
target=`echo "$1" | sed -e 's,^/volume1/tv/,,' -e s,/,:,g`
cp "$1" "$2/$target"
;;
esac
and the corresponding "-exec" would be
-exec myscript "{}" /volume1/tv/_FANART \;
By the way, the source/destination on the original example are in the same directory tree "/volume1/tv", which is why the sample script uses a case statement - to exclude files already copied to the _FANART folder.
If you want to use the md5sum as the new name:
find /volume1/tv/ -type d -path '/volume1/tv/_FANART' -prune -o -type f \( -name '*fanart.jpg'* -o -path '*/fanart/*.jpg' -o -path '*/extrafanart/*.jpg' \) -exec sh -c 'md5=$(md5sum < "$0") && md5=${md5%% *}.jpg && echo cp "$0" "/volume1/tv/_FANART/$md5"' {} \;
Every thing happens in the sh command (all commands are separated by && but I omitted the && for clarity):
md5=$(md5sum < "$0")
md5=${md5%% *}.jpg
cp "$0" "/volume1/tv/_FANART/$md5"'
the $0 expands to the filename processed. We first compute the md5sum of the file, then only keep the md5sum (md5sum puts a hyphen next to the hash) and append .jpg to that, and finally we copy the file into the target folder, with the computed name.
Notes.
I added
-type d -path '/volume1/tv/_FANART` -prune -o
to your command to omit this folder, since you very likely don't want to process it; it would actually be weird to process it, as its content is changed throughout find's traversal.
I left an echo in the command, so that absolutely nothing is copied (as is, it's 100% safe, you can just copy and paste it in your terminal): it only shows what commands are going to be performed (and you'll also see how fast/slow it is).
The command is 100% safe regarding funny filenames with spaces, newlines, globs, etc.
I used md5sum < fileand not md5sum file, because if the filename file contains special characters (like backslashes, newlines, etc.), md5sum (at least my version) prepends the hash with a backslash. Weird. By not giving a filename, we're safe, this won't happen.
Related
I have this command:
find ~/Desktop/testrm -mindepth 1 -path ~/Desktop/testrm/.snapshot -o -mtime +2 -prune -exec rm -rf {} +
I want it to work as is, but it must avoid to remove a specific directory ($ROOT_DIR/$DATA_DIR).
it must remove the files inside the directory but not the directory itself
the flag "r" in rm is needed because it has to delete other directories
-prune is not suitable since it will discard the content and also sub directories
You can exclude individual paths using the short circuiting behavior of -o (like you already did with ~/Desktop/testrm/.snapshot).
However, for each excluded path you also have to exclude all of its parent directories. Otherwise you would delete a/b/c by deleting a/b/ or a/ with rm -rf.
In the following script, the function orParents generates a part of the find command. Example:
find $(orParents a/b/c) ... would run
find -path a/b/c -o -path a/b -o -path a -o ....
#! /usr/bin/env bash
orParents() {
p="$1"
while
printf -- '-path %q -o' "$p"
p=$(dirname "$p")
[ "$p" != . ]
do :; done
}
find ~/Desktop/testrm -mindepth 1 \
$(orParents "$ROOT_DIR/$DATA_DIR") -path ~/Desktop/testrm/.snapshot -o \
-mtime +2 -prune -exec rm -rf {} +
Warning: You have to make sure that $ROOT_DIR/$DATA_DIR does not end with a / and does not contain glob characters like *, ?, and [].
Spaces are ok as printf %q escapes them correctly. However, find -path interprets its argument as a glob pattern independently. We could do a double quoting mechanism. Maybe something like printf %q "$(sed 's/[][*?\]/\\&/' <<< "$p")", but I'm not so sure about how exactly find -path interprets its argument.
Alternatively, you could write a script isParentOf and do ...
find ... -exec isParentOf "$ROOT_DIR/$DATA_DIR" {} \; -o ...
... to exclude $ROOT_DIR/$DATA_DIR and all of its parents. This is probably safer and more portable, but slower and a hassle to set up (find -exec bash -c ... and so on) if you don't want to add a script file to your path.
I have to make a script that goes through a whole folder (/home, in my case).
I have to save all the files except the ones that start with ., and also, if I find a directory that starts with ., I don't have to care what's inside, I don't have to read it.
For the first part we use the command
for path in $(find /home \! -name ".*");do
where path is a variable that contains the path. But we don't know how to do the directory part.
I thought I'd cut the path through the / and then see if there's any .. In that case, have an if that does not save the file, but I don't know how to cut a string and save it in a variable and then go through it.
You can prune all files starting with a ..
From the man page of GNU find:
-prune True; if the file is a directory, do not descend into it. If -depth is given, false; no effect. Because -delete implies -depth, you cannot usefully use -prune and -delete together.
You should not loop over the result from find. You will get unexpected results if you have filenames with spaces or newlines.
Use xargs or -exec, e.g.
find /home -path "*/.*" -prune -o -print0 | xargs -0I{} sh -c 'echo "doing something with $1"' sh {}
or
find /home -path "*/.*" -prune -o -exec sh -c 'for i; do echo "doing something with $i"; done' sh {} +
The -prune part removes all filenames (files and directories) starting with a dot and does not descend into directories starting with a dot.
All other filenames are printed with a NUL character instead of a newline (-o -print0) and piped to xargs or a shell script is executed with your action (as few times as possible).
To save all filenames into a file:
find /home -path "*/.*" -prune -o -print > allfiles.txt
Try this
for path in $(find /home -type d -name ".*" -prune -o -type f \! -name ".*" -print);do echo $path; done
I think I would do something like that:
for path in $(find . -type f | egrep -v '/\.[^\/]+\/'); do
...
Note that you may have to take extra steps if some of your files have spaces in their names.
I have a .txt file that contains a list of directories. I want to make a script that goes through this .txt file, copies anything in the directory thats listed of a certain file type, to another directory.
I've never done this with directories, only files.
How can i edit this simple script to work for reading a directory list, looking for a .csv file, and copy it to another directory?
cat filenames.list | \
while read FILENAME
do
find . -name "$FILENAME" -exec cp '{}' new_dir\;
done
for DIRNAME in $(dirname.list); do find $DIRNAME -type f -name "*.csv" -exec cp \{} dest \; ; done;
sorry, in my first answer i didnt understand what you asking for.
The first line of code, simply, take a dirname entry in your directory list as a path and search in it for each file which end with ".csv" extension; then copy it inside the destination you want.
But you could do with less code:
for DIRNAME in $(dirname.list); do cp $DIRNAME/*.csv dest ; done
Despite the filename of the list filenames.list, let me assume the file contains the list of directory names, not filenames. Then would you please try:
while IFS= read -r dir; do
find "$dir" -type f -name "*.mp3" -exec cp -p -- {} new_dir \;
done < filenames.list
The find command searches in "$dir" for files which have an extension .mp3 then copies them to the new_dir.
The script above does not care the duplication of the filenames. If you want to keep the original directory tree and/or need a countermeasure for the duplication of the filenames, please let me know.
Using find inside a while loop works but find will run on each line of the file, another alternative is to save the list in an array, that way find can search on the directories in the list in one search.
If you have bash4+ you can use mapfile.
mapfile -t directories < filenames.list
If you're stuck at bash3.
directories=()
while IFS= read -r line; do
directories+=("$lines")
done < filenames.list
Now if you're just after one file type like files ending in *.csv.
find "${directories[#]}" -type f -name '*.csv' -exec sh -c 'cp -v -- "$#" /newdirectory' _ {} +
If you have multiple file type to match and multiple directories to copy the files.
while IFS= read -r -d '' file; do
case $file in
*.csv) cp -v -- "$file" /foodirectory;; ##: csv file copy to foodirectory
*.mp3) cp -v -- "$file" /bardirectory;; ##: mp3 file copy to bardirectory
*.avi) cp -v -- "$file" /bazdirectory;; ##: avi file copy to bazdirectory
esac
done < <(find "${directories[#]}" -type f -print0)
find's print0 will work with read's -d '' when dealing with files with white spaces and newlines. see How can I find and deal with file names containing newlines, spaces or both?
The -- is there so if you have a problematic filename that starts with a dash - cp will not interpret it as an option.
Given find ability to process multiple folder, and assuming goal is to 'flatten' all csv files into a single destination, consider the following.
Note that it assumes folder names do not have special characters (including spaces, tabs, new lines, etc).
As a side benefit, it will minimize the number of 'cp' calls, making the process efficient across large number of files/folders.
find $(<filename.list) -name '*.csv' | xargs cp -t DESTINATION/
For the more complex case, where folder names/file name can be anything (including space, '*', etc.), consider using NUL separator (-print0 and -0).
xargs -I{} -t find '{}' -name '*.csv' <dd -print0 | xargs -0 -I{} -t cp -t new/ '{}'
Which will fork multiple find and multiple cp.
I have a task, to find the pattern of the file, create a folder with the pattern name and copy the file to that folder. I am able to create the folders.
folders=`find /Location -type f -name "*.pdf" -printf "%f\n" | cut -f 1 -d '_' | sort -u`
for i in $folders
do
mkdir -p /LocationToCreateTheFolder/$i
done
Not able to go further on how to copy the files.
maybe try?
for i in $folders do mkdir -p /LocationToCreateTheFolder/$i && cp ./$i.pdf ./$i/
This will do the finding and the copying:
find Location -type f -name '*.pdf' -exec bash -c 'f=${1##*/}; d="LocationToCreateTheFolder/${f%%_*}"; mkdir -p "$d" && cp "$1" "$d"' None {} \;
This is safe for difficult file names even ones that contain spaces, tabs, or newlines in their names.
How it works
find Location -type f -name '*.pdf' -exec bash -c '...' None {} \;
This will find the pdf files under directory Location and, for each one found, the bash commands inside '...' will be executed with $1 set to the name of the file found. ($0 is set to None. We don't use $0.)
f=${1##*/}
This removes the directory names from the name of the file. This is an example of prefix removal: everything in $1 up to and including the last / is removed.
d="LocationToCreateTheFolder/${f%%_*}"
This creates the name of the directory to which we want to send the file.
${f%%_*}" is an example of suffix removal. Everything in $f from the first _ and after is removed.
mkdir -p "$d" && cp "$1" "$d"
This makes sure that the directory exists and then copies the file to it.
I want to prepend text to the name of every file of a certain type - in this case .txt files - located in the current directory or a sub-directory.
I have tried:
find -L . -type f -name "*.txt" -exec mv "{}" "PrependedTextHere{}" \;
The problem with this is dealing with the ./ part of the path that comes with the {} reference.
Any help or alternative approaches appreciated.
You can do something like this
find -L . -type f -name "*.txt" -exec bash -c 'echo "$0" "${0%/*}/PrependedTextHere${0##*/}"' {} \;
Where
bash -c '...' executes the command
$0 is the first argument passed in, in this case {} -- the full filename
${0%/*} removes everything including and after the last / in the filename
${0##*/} removes everything before and including the last / in the filename
Replace the echo with a mv once you're satisfied it's working.
Are you just trying to move the files to a new file name that has Prepend before it?
for F in *.txt; do mv "$F" Prepend"$F"; done
Or do you want it to handle subdirectories and prepend between the directory and file name:
dir1/PrependA.txt
dir2/PrependB.txt
Here's a quick shot at it. Let me know if it helps.
for file in $(find -L . -type f -name "*.txt")
do
parent=$(echo $file | sed "s=\(.*/\).*=\1=")
name=$(echo $file | sed "s=.*/\(.*\)=\1=")
mv "$file" "${parent}PrependedTextHere${name}"
done
This ought to work, as long file names does not have new line character(s). In such case make the find to use -print0 and IFS to have null.
#!/bin/sh
IFS='
'
for I in $(find -L . -name '*.txt' -print); do
echo mv "$I" "${I%/*}/prepend-${I##*/}"
done
p.s. Remove the echo to make the script effective, it's there to avoid accidental breakage for people who randomly copy paste stuff from here to their shell.