I have a recursive multiple folder structures. Inside of there are multiple zip, img files. But when I extracted it, new zip,img files come out again. I want to write a loop for this.
Scan whole recursive folders, find multiple zip files and extract them. After extraction scan folders again if system found new zip files, extract it again and again continue until no left zip file in the folders.
sample.tar.gz
1.img
Thefolder
samplefolder
t.img
2.img
samplefolder2
different.gz
3.img
4.img
There are more folders also under other img files.
I tried to write for loop for this but couldn't work it. After extraction 1.img script getting error.
Error Output:
ID = 277952352
Everything is Ok
Folders: 3
Files: 1
Size: 345424152
Compressed: 671024244
+ rm -f /home/sample/1.img
+ recursiveExtract /home/sample/Thefolder
+ for file in "$path"/*
+ '[' -d /home/sample/Thefolder ']'
+ recursiveExtract /home/sample/Thefolder
Segmentation fault
The code:
#!/bin/bash
set -x
target_file=$1
path="$(realpath "$1")"
recursiveExtract () { # $1=directory
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'img' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
b=$(ls $path | grep img | wc -l)
if [[ "$b" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tar' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.tar}"
fi
c=$(ls $path | grep tar | wc -l)
if [[ "$c" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
d=$(ls $path | grep gz | wc -l)
if [[ "$d" -eq 0 ]]; then
break
fi
done
}
recursiveExtract "$1"
Latest version of my code:
#!/bin/bash
# set -x
recursiveExtract () { # $1=directory
path="$(realpath "$1")"
echo "GZ------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'gz' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.gz}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
# d=$(ls $path | grep gz | wc -l)
d=$(find $path -type f -name "*.gz" | wc -l)
echo "WC GZ-----------------:" $d
if [[ "$d" -eq 0 ]]; then
break
fi
done
echo "IMG------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'img' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.img}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
# b=$(ls $path | grep img | wc -l)
b=$(find $path -type f -name "*.img" | wc -l)
echo "WC IMG-----------------:" $b
if [[ "$b" -eq 0 ]]; then
break
fi
done
}
while true
do
d=$(find $1 -type f -name "*.gz" | wc -l)
b=$(find $1 -type f -name "*.img" | wc -l)
if [[ "$d" -eq 0 ]] && [[ "$b" -eq 0 ]]; then
break
else
recursiveExtract "$1"
fi
done
This shoud do the job
#!/bin/bash
find ./ -name "*.zip*" > current_zips.txt
while [[ `wc -l "current_zips.txt" | cut -d' ' -f1` > 0 ]]; do
find ./ -name "*.zip*" -exec unzip {} \;
while IFS= read file_; do rm $file_; done < "current_zips.txt" # for file_ in $(cat current_zips.txt);do rm $file_;done
find ./ -name "*.zip*" > current_zips.txt
done
Use the following script to generate test data
#!/bin/bash
#create some data
echo "data 0" > file0.txt
mkdir folder1 folder2
echo "data 1" > folder1/file1.txt
echo "data 2" > folder2/file1.txt
#zip data
zip file0.zip file0.txt
zip -r folder1.zip folder1
zip -r folder2.zip folder2
zip -r data.zip *.zip
#delete original data
rm -rf file* folder*
I think you want something like this:
recursiveExtract.bash
#!/bin/bash
# efi,gz,img,tar
recursiveExtract () { # $1=file/directory
local path="$(realpath "$1")" # This variable must be local.
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tgz' ]; then # file.tgz == file.tar.gz
local anotherFile="${file%.*}"'.tar'
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
#######################
# -aoa == Overwrite output files #
# -bd == Don't show progress bar #
# -bso0 == Silent #
#######################
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
local anotherFile="${file%.*}"
if [ -n "$anotherFile" -a \( "${anotherFile##*.}" = 'tar' -o "${anotherFile##*.}" = 'img' \) ]; then # file.tar.gz, file.img.gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
else # gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "{file%.*}"
fi
elif [ -f "$file" -a \( "${file##*.}" = 'img' -o "${file##*.}" = 'tar' \) ]; then # file.img or file.tar
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${file%.*}"
fi
done
}
recursiveExtract "$1"
Give the script executable permission with chmod +x recursiveExtract.bash then you can run it like:
$ ./recursiveExtract.bash <directory/file>
In your case, maybe like this:
$ ./recursiveExtract.bash 'sample.img.gz'
I am trying to create a basic recycle bin concept in a VM using bash scripting. It will need to delete files that have been entered and place them into a directory that is created and save the path(origin) to a log file to be later used in a restore function.
I will start off with my delete/recycle code which I believe works just fine but seems kind of untidy/contains redundant code:
#!/bin/sh
if [ ! -d ~/recycle ]
then mkdir ~/recycle
fi
if [ ! -d ~/recycle/recycle_log ]
then mkdir ~/recycle/recycle_log
fi
if [ ! -d ~/recycle/recycle_bin ]
then mkdir ~/recycle/recycle_bin
fi
if [ -d ~/recycle ]
then
echo "$(readlink -f "$1")" >> "$HOME/recycle/recycle_log/log_file" && mv "$1" "$HOME/recycle/recycle_bin"
echo "$(readlink -f "$2")" >> "$HOME/recycle/recycle_log/log_file" && mv "$2" "$HOME/recycle/recycle_bin"
echo "$(readlink -f "$3")" >> "$HOME/recycle/recycle_log/log_file" && mv "$3" "$HOME/recycle/recycle_bin"
echo "$(readlink -f "$4")" >> "$HOME/recycle/recycle_log/log_file" && mv "$4" "$HOME/recycle/recycle_bin"
fi
#end
Thereafter what I have for my restore script is as follows:
#!/bin/sh
cd "$HOME/recycle/recycle_bin" || exit 1
mv -i "$(grep "$1" "$HOME/recycle/recycle_log")"
I imagine this is somewhat close to what I need to return any deleted file stored in the log/recycle bin to be restored to its origin but the error I am getting is:
mv: missing destination file operand after `'
Any thoughts on where I'm going wrong?
Try this:
recycle.sh
#!/bin/sh
set -e
check_dir() {
[ ! -d $1 ] || return 0
mkdir --parents $1
}
check_dir "${HOME}/recycle/recycle_bin"
touch "${HOME}/recycle/recycle_log"
for file in "$#"; do
echo "$(readlink -f "$file")" >> "${HOME}/recycle/recycle_log"
mv "$file" "${HOME}/recycle/recycle_bin"
done
#end
restore.sh
#!/bin/sh
set -e
cd "${HOME}/recycle/recycle_bin" || exit 1
for name in "$#"; do
file=$(grep "\/${name}\$" "${HOME}/recycle/recycle_log")
mv -i $name "$file"
sed -i "/\/${name}\$/ d" "${HOME}/recycle/recycle_log"
done
Some insights:
set -e: Abort on any error, to avoid some if's
$#: The array of arguments ($1, $2...)
[ ! -d $1 ] || return 0: Since we are using set -e, do not fail if the directory exists
grep "\/${name}\$" ...: Only matches the name at the end of the path
sed -i: sed in-place editing to remove the line
I'm missing some images that should have been archived when this script runs. I think this may be to do with my indentations or my Md5 sum. I have tried everything I can think of.
here is the code with out the correct indentations:
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
rm uniquelist.txt md5.txt uniquemd5.txt list.txt
done
This loop
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
should probably be
while read line;
do md5sum "$line"
done < list.txt > md5.txt
Quote parameter expansions, and it's unclear why you needed.
I have a different HTML files in a folder. How to rename the files so that they have the names of:
1.html
2.html
3.html
...
This can make it:
i=1
for file in /your/folder/*
do
mv $file ${i}.html
i=$((i+1)) #((i++)) was giving errors (see comments)
done
It loops through all files in /your/folder and renames them according to the number $i that keeps increasing.
Here is my script
#!/bin/sh
#
# batchrename - renames files like 01.ext, 02.ext ...
#
# format : batchrename <list of files>
# or: -r <extension> <<list of files> or <dir>>
# -r - recoursively
counter=0
extrec=""
if [ "$#" -lt "1" ]; then
echo -e "\n\t\tUsage:\n\tbatchrename [opt]\nopt:"
echo -e "-r <ext> <folder> (or file list) -- renames recoursively ALL files"
echo -e "\tin folder <folder> (or by file list given) with extension .<ext>"
echo -e "<folder> -- renames ALL files in folder given"
echo -e "<file list> -- renames ALL files of given filelist.\n\n"
exit 0
fi
Name="$*"
if [ "$1" = "-r" ]; then
extrec="$2"
shift
shift
Name="$*"
[ "$Name" = "" ] && Name="./"
fi
echo -e "\n\t\t\tRENAMING"
for file in $Name
do
file=`echo "$file" | sed "s/<>/ /g"`
if [ -d "$file" ];then
echo -e "\nDiving into \033[38m $file \033[39m"
cd "$file"
if [ "$extrec" != "" ]; then
batchrename -r $extrec `ls -1 | sed "s/\ /<>/g"`
else
batchrename `ls -1 | sed "s/\ /<>/g"`
fi
cd ../
continue
fi
ext=`ext "$file"`
if [ "$ext" = "ion" ]; then
continue
fi
if [ "$extrec" = "" -o "$ext" = "$extrec" ];then
counter=`expr $counter + 1`
echo -e "Progress: $counter files\r\c"
mv "$file" "rnmd$counter.$ext"
fi
done
echo -e "\n\n\t\t\tENDING"
digits=`echo $counter|awk '{print length ($0)}'`
cnt=1
while [ $digits -gt $cnt ]
do
f=`ls -S -1|grep "rnmd[0-9]\{$cnt\}\."`
rename rnmd rnmd0 $f
cnt=`expr $cnt + 1`
done
if [ "$counter" -gt "0" ]; then
rename rnmd "" rnmd*
fi
echo -e "\n\t\t\tDone !!!\n"
After renaming all your files will looks like 001.file, 002.file, ... and so on. Amount of leading zeros depends on amount of files. So, after renaming ls will show right order of files!
It use intermediate script ext:
#!/bin/sh
#
# ext - returns file suffix (case-unsensitive)
#
File="$*"
if [ -d "$File" ]; then
echo ""
exit 0
fi
EXT=`echo $File|sed 's/.\{1,\}\.//g'`
if [ "$EXT" = "$File" ]; then
EXT=""
fi
echo $EXT| tr '[:upper:]' '[:lower:]'
Here is a similar code, just to add to rename with the same prefix and append an incremental value
declare -i x=1
for f in $(find -type f); do
mv -v $f ${f%/*}/change_me_$x ;
x=$x+1;
done
Can two files be swapped in bash?
Or, can they be swapped in a shorter way than this:
cp old tmp
cp curr old
cp tmp curr
rm tmp
$ mv old tmp && mv curr old && mv tmp curr
is slightly more efficient!
Wrapped into reusable shell function:
function swap()
{
local TMPFILE=tmp.$$
mv "$1" $TMPFILE && mv "$2" "$1" && mv $TMPFILE "$2"
}
Add this to your .bashrc:
function swap()
{
local TMPFILE=tmp.$$
mv "$1" $TMPFILE
mv "$2" "$1"
mv $TMPFILE "$2"
}
If you want to handle potential failure of intermediate mv operations, check Can Bal's answer.
Please note that neither this, nor other answers provide an atomic solution, because it's impossible to implement such using Linux syscalls and/or popular Linux filesystems. For Darwin kernel, check exchangedata syscall.
tmpfile=$(mktemp $(dirname "$file1")/XXXXXX)
mv "$file1" "$tmpfile"
mv "$file2" "$file1"
mv "$tmpfile" "$file2"
do you actually want to swap them?
i think its worth to mention that you can automatically backup overwritten file with mv:
mv new old -b
you'll get:
old and old~
if you'd like to have
old and old.old
you can use -S to change ~ to your custom suffix
mv new old -b -S .old
ls
old old.old
using this approach you can actually swap them faster, using only 2 mv:
mv new old -b && mv old~ new
Combining the best answers, I put this in my ~/.bashrc:
function swap()
{
tmpfile=$(mktemp $(dirname "$1")/XXXXXX)
mv "$1" "$tmpfile" && mv "$2" "$1" && mv "$tmpfile" "$2"
}
You could simply move them, instead of making a copy.
#!/bin/sh
# Created by Wojtek Jamrozy (www.wojtekrj.net)
mv $1 cop_$1
mv $2 $1
mv cop_$1 $2
http://www.wojtekrj.net/2008/08/bash-script-to-swap-contents-of-files/
This is what I use as a command on my system ($HOME/bin/swapfiles). I think it is relatively resilient to badness.
#!/bin/bash
if [ "$#" -ne 2 ]; then
me=`basename $0`
echo "Syntax: $me <FILE 1> <FILE 2>"
exit -1
fi
if [ ! -f $1 ]; then
echo "File '$1' does not exist!"
fi
if [ ! -f $2 ]; then
echo "File '$2' does not exist!"
fi
if [[ ! -f $1 || ! -f $2 ]]; then
exit -1
fi
tmpfile=$(mktemp $(dirname "$1")/XXXXXX)
if [ ! -f $tmpfile ]; then
echo "Could not create temporary intermediate file!"
exit -1
fi
# move files taking into account if mv fails
mv "$1" "$tmpfile" && mv "$2" "$1" && mv "$tmpfile" "$2"
A somewhat hardened version that works for both files and directories:
function swap()
{
if [ ! -z "$2" ] && [ -e "$1" ] && [ -e "$2" ] && ! [ "$1" -ef "$2" ] && (([ -f "$1" ] && [ -f "$2" ]) || ([ -d "$1" ] && [ -d "$2" ])) ; then
tmp=$(mktemp -d $(dirname "$1")/XXXXXX)
mv "$1" "$tmp" && mv "$2" "$1" && mv "$tmp"/"$1" "$2"
rmdir "$tmp"
else
echo "Usage: swap file1 file2 or swap dir1 dir2"
fi
}
This works on Linux. Not sure about OS X.
Hardy's idea was good enough for me.
So I've tried my following two files to swap "sendsms.properties", "sendsms.properties.swap".
But once I called this function as same argument "sendsms.properties" then this file deleted. Avoiding to this kind of FAIL I added some line for me :-)
function swp2file()
{ if [ $1 != $2 ] ; then
local TMPFILE=tmp.$$
mv "$1" $TMPFILE
mv "$2" "$1"
mv $TMPFILE "$2"
else
echo "swap requires 2 different filename"
fi
}
Thanks again Hardy ;-)
using mv means you have one fewer operations, no need for the final rm, also mv is only changing directory entries so you are not using extra disk space for the copy.
Temptationh then is to implementat a shell function swap() or some such. If you do be extremly careful to check error codes. Could be horribly destructive. Also need to check for pre-existing tmp file.
One problem I had when using any of the solutions provided here: your file names will get switched up.
I incorporated the use of basename and dirname to keep the file names intact*.
swap() {
if (( $# == 2 )); then
mv "$1" /tmp/
mv "$2" "`dirname $1`"
mv "/tmp/`basename $1`" "`dirname $2`"
else
echo "Usage: swap <file1> <file2>"
return 1
fi
}
I've tested this in bash and zsh.
*So to clarify how this is better:
If you start out with:
dir1/file2: this is file2
dir2/file1: this is file1
The other solutions would end up with:
dir1/file2: this is file1
dir2/file1: this is file2
The contents are swapped but the file names stayed. My solution makes it:
dir1/file1: this is file1
dir2/file2: this is file2
The contents and names are swapped.
Here is a swap script with paranoid error checking to avoid unlikely case of a failure.
if any of the operations fail it's reported.
the path of the first argument is used for the temp path (to avoid moving between file-systems).
in the unlikely case the second move fails, the first is restored.
Script:
#!/bin/sh
if [ -z "$1" ] || [ -z "$2" ]; then
echo "Expected 2 file arguments, abort!"
exit 1
fi
if [ ! -z "$3" ]; then
echo "Expected 2 file arguments but found a 3rd, abort!"
exit 1
fi
if [ ! -f "$1" ]; then
echo "File '$1' not found, abort!"
exit 1
fi
if [ ! -f "$2" ]; then
echo "File '$2' not found, abort!"
exit 1
fi
# avoid moving between drives
tmp=$(mktemp --tmpdir="$(dirname '$1')")
if [ $? -ne 0 ]; then
echo "Failed to create temp file, abort!"
exit 1
fi
# Exit on error,
mv "$1" "$tmp"
if [ $? -ne 0 ]; then
echo "Failed to to first file '$1', abort!"
rm "$tmp"
exit 1
fi
mv "$2" "$1"
if [ $? -ne 0 ]; then
echo "Failed to move first file '$2', abort!"
# restore state
mv "$tmp" "$1"
if [ $? -ne 0 ]; then
echo "Failed to move file: (unable to restore) '$1' has been left at '$tmp'!"
fi
exit 1
fi
mv "$tmp" "$2"
if [ $? -ne 0 ]; then
# this is very unlikely!
echo "Failed to move file: (unable to restore) '$1' has been left at '$tmp', '$2' as '$1'!"
exit 1
fi
Surely mv instead of cp?
mv old tmp
mv curr old
mv tmp curr
I have this in a working script I delivered. It's written as a function, but you would invoke it
d_swap lfile rfile
The GNU mv has the -b and the -T switch. You can deal with directories using the -T
switch.
The quotes are for filenames with spaces.
It's a bit verbose, but I've used it many times with both files and directories. There might be cases where you would want to rename a file with the name a directory had, but that isn't handled by this function.
This isn't very efficient if all you want to do is rename the files (leaving them where they are), that is better done with a shell variable.
d_swap() {
test $# -eq 2 || return 2
test -e "$1" || return 3
test -e "$2" || return 3
if [ -f "$1" -a -f "$2" ]
then
mv -b "$1" "$2" && mv "$2"~ "$1"
return 0
fi
if [ -d "$1" -a -d "$2" ]
then
mv -T -b "$1" "$2" && mv -T "$2"~ "$1"
return 0
fi
return 4
}
This function will rename files. It uses a temp name (it puts a dot '.' in front of the name) just in case the files/directories are in the same directory, which is usually the case.
d_swapnames() {
test $# -eq 2 || return 2
test -e "$1" || return 3
test -e "$2" || return 3
local lname="$(basename "$1")"
local rname="$(basename "$2")"
( cd "$(dirname "$1")" && mv -T "$lname" ".${rname}" ) && \
( cd "$(dirname "$2")" && mv -T "$rname" "$lname" ) && \
( cd "$(dirname "$1")" && mv -T ".${rname}" "$rname" )
}
That is a lot faster (there's no copying, just renaming). It is even uglier. And it will rename anything: files, directories, pipes, devices.