I just made the switch to linux and I am trying to write my first bash script. I have a folder that contains numerous folders, all with subfolders containing files. Something like:
MainFolder
Folder1
Sub1 (Contains many files)
Sub2 (Contains many files)
Folder2
Sub1 (Contains many files)
Sub2 (Contains many files)
.
.
.
I want to move all the files contained in the sub-folders to the their parent folders. My first instinct is to try and write a for-loop. I was able to do one folder at a time with the command:
mv MainFolder/Folder1/*/* MainFolder/Folder1/
But I was hoping to write a bash script to loop over all the folders in the main directory. Here is what I have so far:
#!/bin/bash
dir1="/pathto/MainFolder"
subs= ls $dir1
for i in $subs; do
mv "$dir1/$i/*/*" "$dir1/$i/"
done
This, obviously, does not work, but I do not understand where I am going wrong.
I also tried:
mv MainFolder/*/*/* MainFolder/*/
with pretty disastrous results. Once I get the file move working properly, I would also like to delete the old sub folders within the loop.
Small change. change
subs=ls $dir1
to
subs=`ls $dir1`
Notice the backquotes. Backquotes actually execute the bash command and return the result. If you issue echo $subs after the line, you'll find that it correctly lists folder1, folder2.
Second small change is to remove double quotes in the mv command. Change them to
mv $dir1/$i/*/* $dir1/$i
Double quotes take literal file names while removing quotes takes the recursive directory pattern.
After that, your initial for loop is indeed correct. It will move everything from sub1 and sub2 to folder1 etc.
Yes - this is working solution
#!/bin/bash
mainDir="$(dirname $(realpath $0))/store/media"
subs=`ls $mainDir`
for i in $subs; do
if [[ -d "$mainDir/$i" ]]; then
mv "$mainDir/$i"/* "$mainDir/"
rm -rf "$mainDir/$i"
fi
done
!/bin/bash
dir1="/pathto/MainFolder"
subs=ls $dir1
for i in $subs; do
mv $dir1/$i// $dir1/$i
done
Related
I have the following structure:
FolderA
Sub1
Sub2
filexx.csv
filexx.doc
FolderB
Sub1
Sub2
fileyy.csv
fileyy.doc
I want to write a script that will move the .csv files into the folder sub1 for each parent directory (Folder A, Folder B and so on) giving me the following structure:
FolderA
Sub1
filexx.csv
Sub2
filexx.doc
FolderB
Sub1
fileyy.csv
Sub2
fileyy.doc
This is what I have till now but I get the error mv: cannot stat *.csv: No such file or directory
for f in */*/*.csv; do
mv -v "$f" */*/Sub1;
done
for f in */*/*.doc; do
mv -v "$f" */*/Sub2;
done
I am new to bash scripting so please forgive me if I have made a very obvious mistake. I know I can do this in Python as well but it will be lengthier which is why I would like a solution using linux commands.
find . -name "*.csv" -type f -execdir mv '{}' Sub1/ \;
Using find, search for all files with the extension .csv and then when we find them, execute a move command from within the directory containing the files, moving the files to directory Sub1
find . -name "*.doc" -type f -execdir mv '{}' Sub2/ \;
Follow the same principle for files with the extension .doc but this time, move the files to Sub2.
I believe you are getting this error because no file matched your wildcard. When it happens, the for loop will give $f the value of the wildcard itself. You are basically trying to move the file *.csv which does not exist.
To prevent this behavior, you can add shopt -s nullglob at the top of your script. When using this, if no file is found, your script won't enter the loop.
My advise is, make sure you run your script from the correct location when using wildcards like this. But maybe what you meant to do by writing */*/*.csv is to recursively match all the csv files. If that's what you intended to do, this is not the right way to do it.
To recursively match all csv/doc/etc files using native bash you can add shopt -s globstar to the top of your script and use **/*.csv as wildcard
#!/bin/bash
shopt -s globstar nullglob
for f in **/*.csv; do
mv "$f" Destination/ # Note that $f is surrounded by "" to handle whitespaces in filenames
done
You could also use the find (1) utility to achieve that. But if you're planning to do more processing on the files than just moving them, a for loop might be cleaner as you won't have to inline everything in the same command.
Side note : "Linux commands" as you say are actually not Linux commands, they are part of the GNU utilities (https://www.gnu.org/gnu/linux-and-gnu.en.html)
If csv files you want to move are in the top directories (from the point of view of the current directory), but not in the subdirectories of them, then simply:
#!/bin/bash
for dir in */; do
mv -v "$dir"*.csv "${dir}Sub1/"
mv -v "$dir"*.doc "${dir}Sub2/"
done
If the files in all subdirectories are wanted to be moved similarly, then:
shopt -s globstar
for file in **/*.csv; do
mv -v "$file" "${file%/*}/Sub1/"
done
for file in **/*.doc; do
mv -v "$file" "${file%/*}/Sub2/"
done
Note that, the directories Sub1 and Sub2 are relative to the directory where csv and doc files reside.
Maybe i'm just going about this wrong and making it harder than it has to be.
This is my problem. I have 2 different scripts that download various picture files. the first downloads from email and the downloaded files go into the /attachments/ directory. The second script copies the contents of google drive, all files and folders get copied into ~/gdrive/ directory. i want to be able to move all picture files from both these folders as well as any subfolders to ~/Pictures/$today and prevent any overwriting in the case of duplicate file names. I don't mind having 2 separate scripts to handle the pictures in the 2 different directories, but I do need it to be able to get all files in subdirectories of the starting point. it also needs to be able to handle a variety of file extensions. my current solution adds a numbered extension such as .~1~ after the files normal extension .jpg, .png, .tiff, etc. I dont lose any files this way but any that wind up with a backup number after the extension are rendered useless to my project. This is what I am currently using
TODAY=$(date +"%m-%d-%Y")
mkdir -p ~/Pictures/$TODAY &&
sudo find /attachments -type f -exec mv --backup=numbered -t ~/Pictures/$TODAY {} +
My result if there are duplicate file names looks like this:
DSC07286.JPG
DSC07286.JPG.~1~
Is there a better approach than what i am doing? Is there a way to dissect the filename parts and reorganize them and do it recursively for all files in the directory? Thanks
Something like this should do it (untested; uses standard lowercase variable names and puts the index just before the extension to not mess with sorting):
for path in ~/Pictures/"$today"/*.JPG
do
index=0
for duplicate_path in "$path".~[0-9]*
do
new_path="${duplicate_path%%.*}${index}.JPG"
echo "$duplicate_path" "$new_path"
((++index))
done
done
When you're confident it's doing the right thing, simply replace echo with mv to actually move the files.
Here is my solution.
#!/bin/bash
TODAY=$(date +"%m-%d-%Y")
NOW=$(date +"%D %T")
sudo mkdir -p /home/pi/Pictures/emailpics/$TODAY &&
sudo find /attachments -type f -exec mv --backup=numbered -t /home/pi/Pictures/emailpics/$TODAY/ {} + &&
for f in /home/pi/Pictures/emailpics/$TODAY/*.~?~
do
fullfilename=$f
filepath=$(dirname "$fullfilename")
filename=$(basename "$fullfilename")
fname="${filename%.*}"
bkpnum="${filename##*.}"
file="${fname%.*}"
ext="${fname##*.}"
sudo mv $f $filepath/$file$bkpnum.$ext
done
Can't say i fully understand all the syntax for the parsing bits, but it works. maybe someone else can explain what is going on.
In linux, I have a group of folders, that all contain the same sub folder structure. E.g.
FolderA/x/y/z/file1.txt
FolderA/x/y/z/file2.txt
FolderB/x/y/z/file1.txt
FolderC/x/y/z/file1.txt
I want to run a batch process to remove one of the subfolders, but leave all files and folders beneath unchanged. E.g. if I were to remove folder "y":
FolderA/x/z/file1.txt
FolderA/x/z/file2.txt
FolderB/x/z/file1.txt
FolderC/x/z/file1.txt
I've tried putting together some combination of find and mv, but can't quite get it right
find . -name y -type d -exec sh -c '
for d; do echo mv "$d"/* "$d"/..; echo rmdir "$d"; done' _ {} +
Remove the echos if the results look like what you expect.
Considering You want to remove one of the subfolders, but leave all files and folders beneath unchanged, here is something you can do. It's NOT an exact Solution, kind of trick that might work for you. I have done it many times on my computer.
You can recursively copy the subfolder's contents to it's parent folder and then traverse to parent directory and then finally recursively remove the subfolder.
Traverse to subfolder
$ cd path/to/SubFolder
Copy all Contents Recursively to Parent Folder
$ cp -R * ..
Traverse to Parent Folder
$ cd ..
Remove recursively Subfolder
$ rm -rf SubFolder/
Say you have a folder y, then,
$ cd path/to/folder/y
$ cp -R * ..
$ cd ..
$ rm -rf y/
For running a Batch Process You can figure out the path to subfolder using find command.
Notice that this trick can consume a lot of time & resources, If you have a lot of folders in the folders. But that works !
I'm trying to learn bash on Linux, just for fun. I thought it would be pretty useful to have a .sh that would group together similar files. For example, let's say we have the directory
/home/docs/
Inside the directory we have /mathdocs/, /codingdocs/, etc.
Inside those sub-directories we have doc.txt, in all of them. Same name for all the files on the subdirectories.
Let's say I want to group them together, and I want to move all the files to /home/allthedocs/ and rename them after the directories they were in. (mathdocs.txt, codingdocs.txt, etc.)
How could I do that?
I've tried to create a script based on the ls and cp commmands, but I don't know how I can take the name of the directories to rename the files in it after I moved them. I guess it has to be some sort of iterative sentence (for X on Y directories) but I don't know how to do it.
You can move and rename your file in one shot with mv, with a loop that grabs all your files through a glob:
#!/bin/bash
dest_dir=/home/allthedocs
cd /home/docs
for file in */doc.txt; do
[[ -f "$file" ]] || continue # skip if not a regular file
dir="${file%/*}" # get the dir name from path
mv "$file" "$dest_dir/$dir.txt"
done
See this post for more info:
Copying files from multiple directories into a single destination directory
Here is a one liner solution that treats whitespaces in filenames, just as #codeforester 's solution does with the glob.
Note that white spaces are treated with the "-print0" option passed to "find", the internal field separator (IFS) in while loop and the wrapping of file3 variable with quotes.
The parameter substitution from file2 into file3 gets rid of the leading "./".
The parameter substition inside the move command turns the path into a filename (run under /home/docs/):
find . -maxdepth 2 -mindepth 2 -print0 | while IFS= read -r -d '' file; \
do file2=$(printf '%s\n' "$file"); file3=${file2#*\/*}; \
mv "$file2" ../allsil/"${file3//\//}"; done
As a rather novice Linux user, I can't seem to find how to do this.
I am trying to move unique files all in one directory into another directory.
Example:
$ ls
vehicle car.txt bicycle.txt airplane.html train.docx (more files)
I want car.txt, bicycle.txt, airplane.html, and train.docx inside vehicle.
Right now I do this by moving the files individually:
$ mv car.txt vehicle
$ mv bicycle.txt vehicle
...
How can I do this in one line?
You can do
mv car.txt bicycle.txt vehicle/
(Note that the / above is unnecessary, I include it merely to ensure that vehicle is a directory.)
You can test this as follows:
cd #Move to home directory
mkdir temp #Make a temporary directory
touch a b c d #Make test (empty) files ('touch' also updates the modification date of an existing file to the current time)
ls #Verify everything is there
mv a b c d temp/ #Move files into temp
ls #See? They are gone.
ls temp/ #Oh, there they are!
rm -rf temp/ #DESTROY (Be very, very careful with this command)
Shorthand command to move all .txt file
You can try using a wildcard. In the code below, * will match all the files which have any name ending with .txt or .docx, and move them to the vehicle folder.
mv *.txt *.docx vehicle/
If you want to move specific files to a directory
mv car.txt bicycle.txt vehicle/
Edit: As mentioned in a comment, If you are moving files by hand, I suggest using mv -i ... which will warn you in case the destination file already exists, giving you a choice of not overwriting it. Other 'file destroyer' commands like cp & rm too have a -i option
mv command in linux allow us to move more than one file into another directory. All you have to do is write the name of each file you want to move, seperated by a space.
Following command will help you:
mv car.txt bicycle.txt airplane.html train.docx vehicle
or
mv car.txt bicycle.txt airplane.html train.docx vehicle/
both of them will work.
You can move multiple files to a specific directory by using mv command.
In your scenario it can be done by,
mv car.txt bicycle.txt airplane.html train.docx vehicle/
The point you must note is that the last entry is the destination and rest everything except mv is source.
One another scenario is that the destination is not present in our directory,then we must opt for absolute path in place of vehicles/.
Note: Absolute path always starts from / ,which means we are traversing from root directory.
I have written a small bash script that will move multiple files(matched using pattern) present in multiple directories(matched using pattern) to a single location using mv and find command in bash
#!/bin/bash
for i in $(find /path/info/*/*.fna -type f) # find files and return their path
do
mv -iv $i -t ~/path/to/destination/directory # move files
done
$() is for command substitution(in other words it expand the expression inside it)
/*/ wild card for matching any directory, you can replace this with any wild card expression
*.fna is for finding any file with.fna extension
-type f is for getting the full path info of the located file
-i in mv is for prompt before overwrite( extra caution in case the wild card exp was wrong)
-v for verbose
-t for destination
NOTE: the above flags are not mandatory
Hope this helps