Let's say I have a bunch of *.tar.gz files located in a hierarchy of folders. What would be a good way to find those files, and then execute multiple commands on it.
I know if I just need to execute one command on the target file, I can use something like this:
$ find . -name "*.tar.gz" -exec tar xvzf {} \;
But what if I need to execute multiple commands on the target file? Must I write a bash script here, or is there any simpler way?
Samples of commands that need to be executed a A.tar.gz file:
$ tar xvzf A.tar.gz # assume it untars to folder logs
$ mv logs logs_A
$ rm A.tar.gz
Here's what works for me (thanks to Etan Reisner suggestions)
#!/bin/bash # the target folder (to search for tar.gz files) is parsed from command line
find $1 -name "*.tar.gz" -print0 | while IFS= read -r -d '' file; do # this does the magic of getting each tar.gz file and assign to shell variable `file`
echo $file # then we can do everything with the `file` variable
tar xvzf $file
# mv untar_folder $file.suffix # untar_folder is the name of folder after untar
rm $file
done
As suggested, the array way is unsafe if file name contained space(s), and also doesn't seem to work properly in this case.
Writing a shell script is probably easiest. Take a look at sh for loops. You could use the output of a find command in an array, and then loop over that array to perform a set of commands on each element.
For example,
arr=( $(find . -name "*.tar.gz" -print0) )
for i in "${arr[#]}"; do
# $i now holds each of the filenames output by find
tar xvzf $i
mv $i $i.suffix
rm $i
# etc., etc.
done
Related
I have multiple folders with multiple files. I need to rename those files with the same name like the folder where the file stored with "_partN" prefix.
As example,
I have a folder named as "new_folder_for_upload" which have 2 files. I need to convert the name of these 2 files like,
new_folder_for_upload_part1
new_folder_for_upload_part2
I have so many folders like above which have multiple files. I need to convert all the file names as I describe above.
Can anybody help me to find out for a single linux command or script to do this work automatically?
Assuming bash shell, and assuming you want the file numbering to restart for each subdirectory, and doing the moving of all files to the top directory (leaving empty subdirectories). Formatted as script for easier reading:
find . -type f -print0 | while IFS= read -r -d '' file
do
myfile=$(echo $file | sed "s#./##")
mydir=$(dirname "$myfile")
if [[ $mydir != $lastdir ]]
then
NR=1
fi
lastdir=${mydir}
mv "$myfile" "$(dirname "$myfile")_part${NR}"
((NR++))
done
Or as one-line command:
find . -type f -print0 | while IFS= read -r -d '' file; do myfile=$(echo $file | sed "s#./##"); mydir=$(dirname "$myfile"); if [[ $mydir != $lastdir ]]; then NR=1; fi; lastdir=${mydir}; mv "$myfile" "$(dirname "$myfile")_part${NR}"; ((NR++)); done
Beware. This is armed, and will do a bulk renaming / moving of every file in or below your current work directory. Use at your own risk.
To delete the empty subdirs:
find . -depth -empty -type d -delete
I am currently working on an auto grading script for a class project. It has to be able to search any number of given directories lets say
for example
usr/autograder/jdoe/
jdoe contains two files house.c and readme.txt.
I need to create a file in jdoe called jdoe.pdf
Currently i'm using this line of code below to get the path to where i need to create the file. Where $1 is user input of the path containing the projects the auto grader will grade.
find $1 -name "*.txt" -exec sh -c "dirname {}"
When I try adding /somename.pdf to the end of this statement I get readme.txt/somename.pdf
along with another -exec to get the name for the file.
\; -exec sh -c "dirname {} xargs -n 1 basename" \;
I'm having problems combining these two into one working statement.
I'm new to unix programming and would appreciate any advice or help even if it means re-writing the code using different unix tools.
The main question here is how do I create files in a path other than the directory I call my script from. Thanks in advance.
How about this?
find "$1" -name "*.txt" -exec bash -c 'd=$(dirname "$1"); touch $d"/"$(basename "$d").pdf' - {} \;
You can create files in another path using change directory command (cd).
If you start your script in usr/autograder/script and want to change to usr/autograder/jdoe you can change directory with shell command cd ../jdoe (relative) or cd usr/autograder/jdoe (absolute).
Now you are in the directory of usr/autograder/jdoe and you are able to create files in this directory, for example gedit readme.txt will open gedit and creates the file in usr/autograder/jdoe.
The simplest way is to loop over the files returned by find and then do whatever you need to do.
For example:
find "$1" -type f -name "*.txt" -print0 | while IFS= read -r -d $'\0' filename; do
dir=$(dirname "$filename")
# create pdf file
touch "$dir/${dir##*/}.pdf"
done
(Note the use of find -print0 to correctly handle filenames containing whitespace and newline characters.)
Is this what you are looking for?
function process_file {
dir=$(dirname "$1")
name=$(basename "$1")
echo name is $name and dir is $dir;
cd "$dir"
touch "${dir##*/}.pdf" # or anything else
}
# export the function, so that it is known in the child processes
export -f process_file
find . -name '*.txt' -exec bash -c "process_file '{}'" \;
There is a set of file paths that i need to copy to some other location:
find . -name '*.txt':
/a/b/c/d/e/f/filea.txt
/a/b/c/d/e/g/h/fileb.txt
/a/b/c/d/e/k/l/filec.txt
I want to copy each of the files to the path that includes e and after e.g:
cp /a/b/c/d/e/f/filea.txt e/f/
The e path will always exists.
I need first to find all files.
How can find, extract everything after e from the path and copy at the same time?
Pipe the find output through
sed 's|\(.*\)\(\*\*e\*\*.*/\)\(.*\)|cp \1\2\3 \2|'
and if the result looks ok, pipe the sed output through sh. If by **e** you just meant e, the pattern is
sed 's|\(.*\)\(e.*/\)\(.*\)|cp \1\2\3 \2|'
You can use something like this. If the generated mkdir and cp commands look ok, you can remove the echo before them.
SOURCE="/a/b/c/d"
cd $SOURCE
DESTINATION="full/path/to/whatever"
for FILE in $(find . -name '*.txt'); do
DIR=$(dirname $FILE)
# create the intermediate directories if they don't exist
echo "mkdir -p $DESTINATION/$DIR"
echo "cp $FILE $DESTINATION/$DIR"
done
I am new to Linux. I am trying to write a shell script which will move files to certain folders based on their extension, like for example in my downloads folder, I have all files of mixed file types. I have written the following script
mv *.mp3 ../Music
mv *.ogg ../Music
mv *.wav ../Music
mv *.mp4 ../Videos
mv *.flv ../Videos
How can I make it run automatically when a file is added to this folder? Now I have to manually run the script each time.
One more question, is there any way of combining these 2 statements
mv *.mp3 ../../Music
mv *.ogg ../../Music
into a single statement? I tried using || (C programming 'or' operator) and comma but they don't seem to work.
There is no trigger for when a file is added to a directory. If the file is uploaded via a webpage, you might be able to make the webpage do it.
You can put a script in crontab to do this, on unix machines (or task schedular in windows). Google crontab for a how-to.
As for combining your commands, use the following:
mv *.mp3 *.ogg ../../Music
You can include as many different "globs" (filenames with wildcards) as you like. The last thing should be the target directory.
Two ways:
find . -name '*mp3' -or -name '*ogg' -print | xargs -J% mv % ../../Music
find . -name '*mp3' -or -name '*ogg' -exec mv {} ../Music \;
The first uses a pipe and may run out of argument space; while the second may use too many forks and be slower. But, both will work.
Another way is:
mv -v {*.mp3,*.ogg,*.wav} ../Music
mv -v {*.mp4,*.flv} ../Videos
PS: option -v shows what is going on (verbose).
I like this method:
#!/bin/bash
for filename in *; do
if [[ -f "$filename" ]]; then
base=${filename%.*}
ext=${filename#$base.}
mkdir -p "${ext}"
mv "$filename" "${ext}"
fi
done
incron will watch the filesystem and perform run commands upon certain events.
You can combine multiple commands on a single line by using a command separator. The unconditional serialized command separator is ;.
command1 ; command2
You can use for loop to traverse through folders and subfolders inside the source folder.
The following code will help you move files in pair from "/source/foler/path/" to "/destination/fodler/path/". This code will move file matching their name and having different extensions.
for d in /source/folder/path/*; do
ls -tr $d |grep txt | rev | cut -f 2 -d '.' | rev | uniq | head -n 4 | xargs -I % bash -c 'mv -v '$d'/%.{txt,csv} /destination/folder/path/'
sleep 30
done
I'm trying to move media and other files which are in a specified directory to another directory and create another one if it does not exits (where the files will go), and create a directory the remaining files with different extensions will go. My first problem is that my script is not making a new directory and it is not moving the files to other directories and what code can I use to move files with different extensions to one directory?
This is what i have had so far, correct me where I'm wrong and help modify my script:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo #directory where the media files will go
WA=/home/katy/do # directory where the other files will go
if [ ! -d "$To" ]; then
mkdir -p "$To"
fi
cd $From
find path -type f -name"*.mp4" -exec mv {} $To \;
I'd solve it somewhat like this:
#!/bin/bash
From=/home/katy/doc
To=/home/katy/mo # directory where the media files will go
WA=/home/katy/do # directory where the other files will go
cd "$From"
find . -type f \
| while read file; do
dir="$(dirname "$file")"
base="$(basename "$file")"
if [[ "$file" =~ \.mp4$ ]]; then
target="$To"
else
target="$WA"
fi
mkdir -p "$target/$dir"
mv -i "$file" "$target/$dir/$base"
done
Notes:
mkdir -p will not complain if the directory already exists, so there's no need to check for that.
Put double quotes around all filenames in case they contain spaces.
By piping the output of find into a while loop, you also avoid getting bitten by spaces, because read will read until a newline.
You can modify the regex according to taste, e.g. \.(mp3|mp4|wma|ogg)$.
In case you didn't know, $(...) will run the given command and stick its output back in the place of the $(...) (called command substitution). It is almost the same as `...` but slightly better (details).
In order to test it, put echo in front of mv. (Note that quotes will disappear in the output.)
cd $From
find . -type f -name "*.mp4" -exec mv {} $To \;
^^^
or
find $From -type f -name "*.mp4" -exec mv {} $To \;
^^^^^
cd $From
mv *.mp4 $To;
mv * $WA;