I have a large directory of sub-directories that has reached the ext3 limit. I need to copy some directories to an alternate path. I also need to remove some directories completely.
The directories are named numerically 1000,1001,1002,1003. I'd like to be able to copy and delete by number. i.e.
rm -rf (WHERE dirname<12000)
I also have some non-sequential directories such as 45698,59875,897526
I was able to pull these directory names from a database and I have them in an array.
What is the best way to copy and delete these?
I would copy the interesting folders first. You told that you have them in an array, maybe its better to store their names in a file? Then you cold do the following:
xargs -I {} cp -a {} /path/to/backup < list_of_interesting_folders
Then you can remove the remaining folders using brace expansion (with bash):
rm -rf {1000..12000}
Related
Lets say i have a folder named Pictures and I want to move or copy all files out of this folder.
However I also want to move and harvest all of the files who are in sub folders so:
Pictures/1.png
Pictures/yolo/2.png
Pictures/yolo/swag/sand/3.png
Pictures/extra/fire/4.png
I want to move or copy all these files to another folder like results so I get:
results/1.png
results/2.png
results/3.png
results/4.png
Only I have no idea in advance what sub folders will be in the Pictures folder.
How can I accomplish this in bash/shell scripts ?
I also appreciate making it file type neutral so any files are harvested from their directories (not only .png like in my example) and I have no idea what the file name will be (I only used 1...4 because i did not have any idea how to name them).
You can do it like this:
find /absolute/path/to/Pictures -type f -name '*.png' -exec mv -i {} /absolute/path/to/results \;
Another option is to use xargs
find /absolute/path/to/Pictures -name '*.png' | xargs -I files mv files /absolute/path/to/results
You can simply copy all files and subdirectories along with their contents using cp's recursive option:
cp -pr <source_path>/* <destination_path>/
But, moving them recursively is a bit tricky, you will need to create tar files of the subdirectories and move them and then untar the tar files in destination path. As this is a complex process, as a workaround, you can copy the files/directories recursively and then delete the files from original path.
cp -pr <source_path>/* <destination_path>/ && rm -rf <source_path>/*
I am trying to copy all the *.psd files, currently in a multi directories structure, into one single directory.
Is there an rsync parametrization to allow it?
The solution proposed at Copying files from multiple directories into a single destination directory is not a multilevel recursive directories, only single level subdirectories.
In my current case I have files in multiple recursive directories (up to 7 levels) that I would like to reconcile in a single directory.
I fear rsync can't help you here. You can use find to find all the files and copy them to the destination directory, though:
find /path/to/source/topdir -type f -name '*.psd' -exec cp {} /path/to/destination/ \;
In my opinion #choroba's answer is the right one.
For completeness (or if for any reason you needed the files to be copied with rsync) you can do something way less efficient using rsync (which is just like using cp in this case), using find, a loop and other things not really necessary.
for file in $(find ./path/to/source/topdir -name "*psd" ); do rsync $file /path/to/destination/; done
I'm attempting to create a script that will create a folder based on the current time and date. I then need the script to copy the files from a source folder to the newly created folder. I then need it to copy folders from a second source folder to the original source folder, overwriting everything that's in there.
Below is what I've tried, and it's failing in quite an epic fashion.
#!/bin/bash
d="/home/$(date +%d-%m-%y")"
mkdir "$d"
cp /home/test "$d"
cp /home/test2 /home/test
I'm aware that I don't have to define the variable, as the time between copies should be seconds and not lapse a day, but I wanted to make sure and honestly, I'm interested in learning to use variables in scripting.
There is one too many double quote here:
d="/home/$(date +%d-%m-%y")"
Actually no quoting is necessary here at all, write like this:
d=/home/$(date +%d-%m-%y)
In the rest of the script, if you want to copy directories, you will need to use cp -r instead of simply cp.
Finally, note that when you do cp -r dir1 dir2 when dir2 already exists, then dir1 will be copied inside dir2, rather than overwriting its content. That is, it will create dir2/dir1. If dir1 doesn't contain hidden files, then you can write like this to overwrite the content of dir2:
cp -r dir1/* dir2/
I have a directory that is filled with subdirectories exceeding 450 GBs. Inside of these subdirectories is an instruction file in each subdirectory. I have a script that copies the instruction file in the directory I am currently in and puts it inside every subdirectory via:
#!/bin/bash
for d in */; do cp "INSTALLATION INSTRUCTIONS.rtf" "$d"; done
I need to remove all of these files in the subdirectories and replace them with new instructions. Can I simple write another script that does this:
#!/bin/bash
for d in */; do rm "INSTALLATION INSTRUCTIONS.rtf" "$d"; done
I am very hesitant and wanted to make sue as these files are vitally important and I don't want to accidentally remove anything and making a backup of 450+ GBs is very taxing.
find . -mindepth 2 -name "INSTALLATION INSTRUCTIONS.rtf" -exec rm -f '{}' +
Since this is "vitally important" data, I would first list all files that match the file name you want to delete/overwrite, without taking any action on it (other than listing):
find /folder/ -type f -name "INSTALLATION INSTRUCTIONS.rtf" -print > /tmp/holder
That would create a list of matches on /tmp/holder. Then you could analyze this list before taking any action (either visually or programatically) to make sure that the list does not include anything you don't want to delete (when dealing with big amounts of data, strange things can happen, so be proactive on protecting the data).
If you are happy with what the list shows, then you could delete the old instructions, or if possible, overwrite them with the new file. Here's an example to overwrite the old file with the new one:
while read -r line; do cp --no-preserve=all /folder/newfile "$line"; done < /tmp/holder
The cp --no-preserve=all command (available on GNU bash) would ensure that the new file has permissions that are "adequate" to the folder where they are located. You may change that to a simple cp if you don't want that to happen.
I'm trying write a script in linux. Where I have some csv files in Two different folders(A and B) and then after some processing copy of rejected files are moving to Bad Folder.
SO I want bad files to be deleted from Table A and B which have copied to Bad Folder.
Can you help me to write this script for linux?
Best
lets say name of Bad Folder is 'badFolder' and considering 'A', 'B' and 'badFolder' are in same directory
Steps to delete files from folder A and B:
step 1: change current directory to your 'badFolder'
cd badFolder
step 2: delete identical files
find . -type f -exec rm -f ../A/{} \;
find . -type f -exec rm -f ../B/{} \;
The argument -type f tells to look for files, not directories.
The -exec ... \; argument tells that, once it finds a file in 'badFolder', it should run the command rm -f on its counterpart in the A subdirectory.
Because rm is given with the -f option, it will silently ignore files that don't exist.
Also, it will not prompt before deleting files. This is very handy when deleting a large number of files. However, be sure that you really want to delete the files before running this script.
#!/bin/bash
#Set the working folder in which you want to delete the file
Working_folder=/<Folder>/<path>
cd $Working_folder
#command to delete all files present in folders
rm <filenames seperated by space>
echo "files are deleted"
#if you want to delete all files you can use wild card character
# e.g. command rm *.*
# if you want to delete a particular file say for deleting .csv file you can use command rm *.csv command
Set variables containing the paths of your A, B and BAD directories.
Then you can do something along the lines of
for file in ls ${PATH_TO_BAD}
do
rm ${PATH_TO_A}/$file
rm ${PATH_TO_B}/$file
done
This is iterating over the BAD directory and any file it finds, it deletes from the A and B directories.