How to make a FOR loop to move files to directory based on file extension - linux

Im trying to sort thousands of files and put them in their own folder based on file extension. For example, JPG files to go into a JPG folder.
How can I create a for loop to address this?
What I have attempted to far:
# This listed out all the file extensions and the count for each extension:
find . -type f | rev | cut -d. -f1 | rev | tr '[:upper:]' '[:lower:]' | sort | uniq --count | sort -rn
#This was able to find all jpegs in the media folder
find /media -iname '*.jpg'
# This worked, however the MV command does not create the folder
find /media -iname '*.jpg' -exec mv '{}' /media/genesis/Passport/consolidated/jpg/ \; #
Im guessing that the for loop would be something like but I cant seem to figure it out:
for dir in 'find /media -iname "*.jpg"'; do
mkdir $dir;
mv $dir/*;
done

To find all the files of a particular extension and move then to a destination we can use find command as follows:
find . -name "*.jpg" -exec mv {} $destination/ \;
This being the critical logic, you can just create a for loop on top of this to iterate through all the known file extensions in your media folder
## List all the known extensions in your media folder.
declare -a arr=("jpg" "png" "svg")
## Refer question 1842254 to get all the extentsions in your folder.
## find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u
## now loop through the above array
for i in "${arr[#]}"
do
echo "$i"
## Create a directory based on file extension
destination="$i"
mkdir $destination
find /media -iname "*.$i" -exec mv {} $destination/ \;
done
Note that you can change your destination to be whatever you want. I just left it in the same directory.
Here is an executable shell script for your reference: https://onlinegdb.com/NEQSpojhM

Related

How do I retrieve all files in linux for given directory level. I want to absolute path aswell as the filename

I'm trying to retrieve all files for a given directory level. The files retrieved should also show absolute path.
git ls-files | cut -f1 | uniq | grep '.xml'
I expect the results to be a list of files in the directory level/depth 1.
Using find: using -maxdepth flag set to 1, -type f to list only files, -iname to list file of matching type. -exec to perform some action on the matched files. readlink -f to print the full path of the files.
find . -maxdepth 1 -type f -iname "*.xml" -exec readlink -f {} \;

Include folder name in renaming a file in linux

I've already used that command to rename the files in multiple directories and change JPG to jpg, so I have consistency.
find . -name '*.jpg' -exec sh -c 'mv "$0" "${0%.JPG}$.jpg"' {} \;
Do you have any idea how to change that to include the folder name in the name of the file
I am executing that in a folder that contains about 2000 folders (SKU's) or products ... and inside every SKU folder, there are 9 images. 1.jpg 2.jpg .... 9.jpg.
So the bottom-line is I have 2000 images with name 1.jpg, 2.jpg ... 9.jpg. I need those files to be unique, for example:
folder-name-1.jpg ... folder-name.2.jpg ... so on, in every folder.
Any help will be appreciated.
For example I can do as follows:
$ find . -iname '*.jpg' | while read fn; do name=$(basename "$fn") ; dir=$(dirname "$fn") ; mv "$fn" "$dir/$(basename "$dir")-$name" ;done
./lib/bukovina/version.jpg ./lib/bukovina/bukovina-version.jpg
./lib/bukovina.jpg ./lib/lib-bukovina.jpg
You can use fine one-liner:
find . -name '*.jpg' -execdir \
bash -c 'd="${PWD##*/}"; [[ "$1" != "$d-"* ]] && mv "$1" "./$d-$1"' - '{}' \;
This command uses safe approach to check whether image name is already not prefixed by the current directory name. You can run it multiple times also and image name won't be renamed after first run.
To get the folder name of a file you can do $(basename $(dirname ${FILE})), where ${FILE} is a path that may be relative but must contain at least one folder before the file name in it. This should not be a problem with find. If it is, just run it from one directory up.
find . -name '*.jpg' -exec sh -c 'mv "$0" "$(basename $(dirname $0))-${0%.JPG}$.jpg"' {} \;
Or, if you have JPEGs in your current directory:
find ../<dirname> -name '*.jpg' -exec sh -c 'mv "$0" "$(basename $(dirname $0))-${0%.JPG}$.jpg"' {} \;

Linux script Loop

I want to create a loop in Linux script that will go thru the folders in one directory and will copy photos to one folder and will overwrite photos that have the same name. Can anyone point me in the write direction?
find /path/to/source -type f -exec cp -f {} /path/to/destination \;
Would that work? Keep in mind, that will overwrite files without asking.
If you want it to confirm with you before overwriting, use the -i flag (for interactive mode) in the cp command.
find /path/to/source -type f | xargs -I {} file {} | grep <JPEG or image type> | cut -d ":" -f1 | xargs -I {} cp -rf {} /path/to/destination
With this you can find tune your copy with selecting only the image type.
Actually, you need not to loop through folders for finding photos using script, find command will do that job for you.
Try using find with xargs and cp
find source_dir -type f -iname '*.jpg' -print0 | xargs -0 -I {} cp -f {} dest_dir
Replace *.jpg with format of your photo files. (e.g. *.png etc.)
Note the use of -f option of cp, since you want to overwrite photos with the same name

Bash script to find files in a list, copy them to dest, print files not found

I would like to build on the answer I found here: Bash script to find specific files in a hierarchy of files
find $dir -name $name -exec scp {} $destination \;
I have a file with a list of file names and I need to find those files on a backup disk, then copy those files found to a destination folder, and lastly print the files that could not be found to a new file.
the last step would be helpful so that I wouldn't need to make another list of files copied and then do a compare with original list.
If the script can then make a list of the copied files, and do a compare, then print the differences, then that's exactly what's required. Unless the shell process find can print to file each time it "Can't find" a file.
Assuming that your list is separated by newlines; something like this should work
#!/bin/bash
dir=someWhere
dest=someWhereElse
toCopyList=filesomewhere
notCopied=filesomewhereElse
while read line; do
find "$dir" -name "$line" -exec cp '{}' $dest \; -printf "%f\n"
done < "$toCopyList" > cpList
#sed -i 's#'$dir'/##' cpList
# I used # instead of / in sed to not confuse sed with / in $dir
# Also, I assumed the string in $dir doesnot end with a /
cat cpList "$toCopyList" | sort | uniq -c | sed -nr '/^ +1/s/^ +1 +(.*)/\1/p' > "$notCopied"
# Will not work if you give wild cards in your "toCopyList"
Hope it helps
while read fname ; do
find /FROM/WHERE/TO/COPY/ \
-type f \
-name "$fname" \
-exec cp \{\} /DESTINATION/DIR/ \; 2>/dev/null
find /DESTINATION/DIR/ \
-type f \
-name "$fname" &>/dev/null || \
echo $fname
done < FILESTOCOPY > MISSEDFILES
Will do.

List all SVN repository URLs from a folder in recursive mode

We are looking for a script that will traverse in recursive mode all subfolders and list all SVN repository URLs and the path where it was found.
It will be used on /home folder of a user.
Recursively find directories, and for each of them try to get the SVN info. If it is successfull, then don't descend into the directory and print the directory name.
find -type d -exec bash -c "svn info {} > /dev/null 2> /dev/null" \; -prune -print
This will list the directories.
If you want the repository info, you can add it in the middle of the find exec command.
find -type d -exec bash -c "svn info {} 2> /dev/null | grep URL" \; -prune -print
Edit:
I found much better results by only testing for the presence of an .svn subdirectory. Then, svn info is called once at the end and grepped for Path and URL. (Plus using -0 to prevent from spaces in filenames.)
find -type d -exec test -d "{}/.svn" \; -prune -print0 | xargs -0 svn info | grep -e '\(Path\|URL\)'
Depending on what kind of limitations you have there could be different ways. The easiest way would be to do svn ls -R | grep -v "\." to grab all the sub-folders from the repository location you're at and feed that in to a for loop that adds the URI to the root of the ls to the front of each line. This will however not be adequate if you have files that do not contain a "." as they will be detected as folders. Unfortunately svn ls doesn't allow you to filter by file/folder, so if you need to deal with filenames without extensions then you'd have to do something different such as checking out the source and using find to get the folder names.
user_home=... # fill in the user's home dir
old_dir=/../
find $user_home -name .svn | rev | cut -f2- -d/ | rev | while read line ; do
echo -n "$line"$'\t'
wc -c <<<"$line"
done | sort -t$'\t' -k1,1 -k2,2n | while read dir size ; do
if [[ $dir != $old_dir* ]] ; then
old_dir=$dir
svn info $dir | grep URL
echo PATH: $dir
echo
fi
done
Just hope users do not store SVN under directories with spaces in names.

Resources