I have a recursive multiple folder structures. Inside of there are multiple zip, img files. But when I extracted it, new zip,img files come out again. I want to write a loop for this.
Scan whole recursive folders, find multiple zip files and extract them. After extraction scan folders again if system found new zip files, extract it again and again continue until no left zip file in the folders.
sample.tar.gz
1.img
Thefolder
samplefolder
t.img
2.img
samplefolder2
different.gz
3.img
4.img
There are more folders also under other img files.
I tried to write for loop for this but couldn't work it. After extraction 1.img script getting error.
Error Output:
ID = 277952352
Everything is Ok
Folders: 3
Files: 1
Size: 345424152
Compressed: 671024244
+ rm -f /home/sample/1.img
+ recursiveExtract /home/sample/Thefolder
+ for file in "$path"/*
+ '[' -d /home/sample/Thefolder ']'
+ recursiveExtract /home/sample/Thefolder
Segmentation fault
The code:
#!/bin/bash
set -x
target_file=$1
path="$(realpath "$1")"
recursiveExtract () { # $1=directory
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'img' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
b=$(ls $path | grep img | wc -l)
if [[ "$b" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tar' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.tar}"
fi
c=$(ls $path | grep tar | wc -l)
if [[ "$c" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
d=$(ls $path | grep gz | wc -l)
if [[ "$d" -eq 0 ]]; then
break
fi
done
}
recursiveExtract "$1"
Latest version of my code:
#!/bin/bash
# set -x
recursiveExtract () { # $1=directory
path="$(realpath "$1")"
echo "GZ------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'gz' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.gz}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
# d=$(ls $path | grep gz | wc -l)
d=$(find $path -type f -name "*.gz" | wc -l)
echo "WC GZ-----------------:" $d
if [[ "$d" -eq 0 ]]; then
break
fi
done
echo "IMG------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'img' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.img}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
# b=$(ls $path | grep img | wc -l)
b=$(find $path -type f -name "*.img" | wc -l)
echo "WC IMG-----------------:" $b
if [[ "$b" -eq 0 ]]; then
break
fi
done
}
while true
do
d=$(find $1 -type f -name "*.gz" | wc -l)
b=$(find $1 -type f -name "*.img" | wc -l)
if [[ "$d" -eq 0 ]] && [[ "$b" -eq 0 ]]; then
break
else
recursiveExtract "$1"
fi
done
This shoud do the job
#!/bin/bash
find ./ -name "*.zip*" > current_zips.txt
while [[ `wc -l "current_zips.txt" | cut -d' ' -f1` > 0 ]]; do
find ./ -name "*.zip*" -exec unzip {} \;
while IFS= read file_; do rm $file_; done < "current_zips.txt" # for file_ in $(cat current_zips.txt);do rm $file_;done
find ./ -name "*.zip*" > current_zips.txt
done
Use the following script to generate test data
#!/bin/bash
#create some data
echo "data 0" > file0.txt
mkdir folder1 folder2
echo "data 1" > folder1/file1.txt
echo "data 2" > folder2/file1.txt
#zip data
zip file0.zip file0.txt
zip -r folder1.zip folder1
zip -r folder2.zip folder2
zip -r data.zip *.zip
#delete original data
rm -rf file* folder*
I think you want something like this:
recursiveExtract.bash
#!/bin/bash
# efi,gz,img,tar
recursiveExtract () { # $1=file/directory
local path="$(realpath "$1")" # This variable must be local.
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tgz' ]; then # file.tgz == file.tar.gz
local anotherFile="${file%.*}"'.tar'
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
#######################
# -aoa == Overwrite output files #
# -bd == Don't show progress bar #
# -bso0 == Silent #
#######################
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
local anotherFile="${file%.*}"
if [ -n "$anotherFile" -a \( "${anotherFile##*.}" = 'tar' -o "${anotherFile##*.}" = 'img' \) ]; then # file.tar.gz, file.img.gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
else # gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "{file%.*}"
fi
elif [ -f "$file" -a \( "${file##*.}" = 'img' -o "${file##*.}" = 'tar' \) ]; then # file.img or file.tar
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${file%.*}"
fi
done
}
recursiveExtract "$1"
Give the script executable permission with chmod +x recursiveExtract.bash then you can run it like:
$ ./recursiveExtract.bash <directory/file>
In your case, maybe like this:
$ ./recursiveExtract.bash 'sample.img.gz'
I am trying to use a txt file to store folder locations to use in find command. But keep getting folder not found works with only one folder location in file
with "$addfolder"
found=$(find "$addfolder" ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
and replacing \"
addfolder="$addfolder $Folder"
folder.txt :-
Main/Public
Main/General
Not Used
Old Backup Files
#!/bin/bash
addfolder=""
filename="Settings/folders.txt"
#Read Folder.txt for locations
while read -r Folder; do
if [ ! "$Folder" == "" ];then
if [ -d "$Folder" ]; then
addfolder="$addfolder \"$Folder\""
echo "$addfolder"
fi
fi
done < "$filename"
if [ "$addfolder" == "" ]; then
exit
fi
echo -e "\e[36mEnter Filename To Find :-\e[0m"
read -p "" ffind
echo -e "\e[92mSearching:\e[0m"
found=$(find $addfolder ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
echo -e "\e[33m$found\e[0m"
echo "Press Enter To Exit"
read -s -n 1 -p ""
Regular variables should only hold single strings.
To hold lists of strings, use an array:
#!/bin/bash
addfolder=()
filename="Settings/folders.txt"
#Read Folder.txt for locations
while IFS= read -r Folder; do
if [ ! "$Folder" == "" ];then
if [ -d "$Folder" ]; then
addfolder+=( "$Folder" )
echo "${addfolder[#]}"
fi
fi
done < "$filename"
if [ "${#addfolder[#]}" == 0 ]; then
exit
fi
echo -e "\e[36mEnter Filename To Find :-\e[0m"
read -p "" ffind
echo -e "\e[92mSearching:\e[0m"
found=$(find "${addfolder[#]}" ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
echo -e "\e[33m$found\e[0m"
echo "Press Enter To Exit"
read -s -n 1 -p ""
I typed the code the same as The Linux Command Line: A Complete Introduction, page 369
but prompt the error:
line 7 `if[ -e "$FILE" ]; then`
the code is like:
#!/bin/bash
#test file exists
FILE="1"
if[ -e "$FILE" ]; then
if[ -f "$FILE" ]; then
echo :"$FILE is a regular file"
fi
if[ -d "$FILE" ]; then
echo "$FILE is a directory"
fi
else
echo "$FILE does not exit"
exit 1
fi
exit
I want to figure out what introduced the error. How can I modify the code? My system is Ubuntu.
There must be a space between if and [, like this:
#!/bin/bash
#test file exists
FILE="1"
if [ -e "$FILE" ]; then
if [ -f "$FILE" ]; then
echo :"$FILE is a regular file"
fi
...
These (and their combinations) would all be incorrect too:
if [-e "$FILE" ]; then
if [ -e"$FILE" ]; then
if [ -e "$FILE"]; then
These on the other hand are all ok:
if [ -e "$FILE" ];then # no spaces around ;
if [ -e "$FILE" ] ; then # 1 or more spaces are ok
Btw these are equivalent:
if [ -e "$FILE" ]; then
if test -e "$FILE"; then
These are also equivalent:
if [ -e "$FILE" ]; then echo exists; fi
[ -e "$FILE" ] && echo exists
test -e "$FILE" && echo exists
And, the middle part of your script would have been better with an elif like this:
if [ -f "$FILE" ]; then
echo $FILE is a regular file
elif [ -d "$FILE" ]; then
echo $FILE is a directory
fi
(I also dropped the quotes in the echo, as in this example they are unnecessary)
The solution is pretty simple.
Just give space between if and the opening square braces like given below.
if [ -f "$File" ]; then
<code>
fi
I'm missing some images that should have been archived when this script runs. I think this may be to do with my indentations or my Md5 sum. I have tried everything I can think of.
here is the code with out the correct indentations:
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
rm uniquelist.txt md5.txt uniquemd5.txt list.txt
done
This loop
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
should probably be
while read line;
do md5sum "$line"
done < list.txt > md5.txt
Quote parameter expansions, and it's unclear why you needed.
Can two files be swapped in bash?
Or, can they be swapped in a shorter way than this:
cp old tmp
cp curr old
cp tmp curr
rm tmp
$ mv old tmp && mv curr old && mv tmp curr
is slightly more efficient!
Wrapped into reusable shell function:
function swap()
{
local TMPFILE=tmp.$$
mv "$1" $TMPFILE && mv "$2" "$1" && mv $TMPFILE "$2"
}
Add this to your .bashrc:
function swap()
{
local TMPFILE=tmp.$$
mv "$1" $TMPFILE
mv "$2" "$1"
mv $TMPFILE "$2"
}
If you want to handle potential failure of intermediate mv operations, check Can Bal's answer.
Please note that neither this, nor other answers provide an atomic solution, because it's impossible to implement such using Linux syscalls and/or popular Linux filesystems. For Darwin kernel, check exchangedata syscall.
tmpfile=$(mktemp $(dirname "$file1")/XXXXXX)
mv "$file1" "$tmpfile"
mv "$file2" "$file1"
mv "$tmpfile" "$file2"
do you actually want to swap them?
i think its worth to mention that you can automatically backup overwritten file with mv:
mv new old -b
you'll get:
old and old~
if you'd like to have
old and old.old
you can use -S to change ~ to your custom suffix
mv new old -b -S .old
ls
old old.old
using this approach you can actually swap them faster, using only 2 mv:
mv new old -b && mv old~ new
Combining the best answers, I put this in my ~/.bashrc:
function swap()
{
tmpfile=$(mktemp $(dirname "$1")/XXXXXX)
mv "$1" "$tmpfile" && mv "$2" "$1" && mv "$tmpfile" "$2"
}
You could simply move them, instead of making a copy.
#!/bin/sh
# Created by Wojtek Jamrozy (www.wojtekrj.net)
mv $1 cop_$1
mv $2 $1
mv cop_$1 $2
http://www.wojtekrj.net/2008/08/bash-script-to-swap-contents-of-files/
This is what I use as a command on my system ($HOME/bin/swapfiles). I think it is relatively resilient to badness.
#!/bin/bash
if [ "$#" -ne 2 ]; then
me=`basename $0`
echo "Syntax: $me <FILE 1> <FILE 2>"
exit -1
fi
if [ ! -f $1 ]; then
echo "File '$1' does not exist!"
fi
if [ ! -f $2 ]; then
echo "File '$2' does not exist!"
fi
if [[ ! -f $1 || ! -f $2 ]]; then
exit -1
fi
tmpfile=$(mktemp $(dirname "$1")/XXXXXX)
if [ ! -f $tmpfile ]; then
echo "Could not create temporary intermediate file!"
exit -1
fi
# move files taking into account if mv fails
mv "$1" "$tmpfile" && mv "$2" "$1" && mv "$tmpfile" "$2"
A somewhat hardened version that works for both files and directories:
function swap()
{
if [ ! -z "$2" ] && [ -e "$1" ] && [ -e "$2" ] && ! [ "$1" -ef "$2" ] && (([ -f "$1" ] && [ -f "$2" ]) || ([ -d "$1" ] && [ -d "$2" ])) ; then
tmp=$(mktemp -d $(dirname "$1")/XXXXXX)
mv "$1" "$tmp" && mv "$2" "$1" && mv "$tmp"/"$1" "$2"
rmdir "$tmp"
else
echo "Usage: swap file1 file2 or swap dir1 dir2"
fi
}
This works on Linux. Not sure about OS X.
Hardy's idea was good enough for me.
So I've tried my following two files to swap "sendsms.properties", "sendsms.properties.swap".
But once I called this function as same argument "sendsms.properties" then this file deleted. Avoiding to this kind of FAIL I added some line for me :-)
function swp2file()
{ if [ $1 != $2 ] ; then
local TMPFILE=tmp.$$
mv "$1" $TMPFILE
mv "$2" "$1"
mv $TMPFILE "$2"
else
echo "swap requires 2 different filename"
fi
}
Thanks again Hardy ;-)
using mv means you have one fewer operations, no need for the final rm, also mv is only changing directory entries so you are not using extra disk space for the copy.
Temptationh then is to implementat a shell function swap() or some such. If you do be extremly careful to check error codes. Could be horribly destructive. Also need to check for pre-existing tmp file.
One problem I had when using any of the solutions provided here: your file names will get switched up.
I incorporated the use of basename and dirname to keep the file names intact*.
swap() {
if (( $# == 2 )); then
mv "$1" /tmp/
mv "$2" "`dirname $1`"
mv "/tmp/`basename $1`" "`dirname $2`"
else
echo "Usage: swap <file1> <file2>"
return 1
fi
}
I've tested this in bash and zsh.
*So to clarify how this is better:
If you start out with:
dir1/file2: this is file2
dir2/file1: this is file1
The other solutions would end up with:
dir1/file2: this is file1
dir2/file1: this is file2
The contents are swapped but the file names stayed. My solution makes it:
dir1/file1: this is file1
dir2/file2: this is file2
The contents and names are swapped.
Here is a swap script with paranoid error checking to avoid unlikely case of a failure.
if any of the operations fail it's reported.
the path of the first argument is used for the temp path (to avoid moving between file-systems).
in the unlikely case the second move fails, the first is restored.
Script:
#!/bin/sh
if [ -z "$1" ] || [ -z "$2" ]; then
echo "Expected 2 file arguments, abort!"
exit 1
fi
if [ ! -z "$3" ]; then
echo "Expected 2 file arguments but found a 3rd, abort!"
exit 1
fi
if [ ! -f "$1" ]; then
echo "File '$1' not found, abort!"
exit 1
fi
if [ ! -f "$2" ]; then
echo "File '$2' not found, abort!"
exit 1
fi
# avoid moving between drives
tmp=$(mktemp --tmpdir="$(dirname '$1')")
if [ $? -ne 0 ]; then
echo "Failed to create temp file, abort!"
exit 1
fi
# Exit on error,
mv "$1" "$tmp"
if [ $? -ne 0 ]; then
echo "Failed to to first file '$1', abort!"
rm "$tmp"
exit 1
fi
mv "$2" "$1"
if [ $? -ne 0 ]; then
echo "Failed to move first file '$2', abort!"
# restore state
mv "$tmp" "$1"
if [ $? -ne 0 ]; then
echo "Failed to move file: (unable to restore) '$1' has been left at '$tmp'!"
fi
exit 1
fi
mv "$tmp" "$2"
if [ $? -ne 0 ]; then
# this is very unlikely!
echo "Failed to move file: (unable to restore) '$1' has been left at '$tmp', '$2' as '$1'!"
exit 1
fi
Surely mv instead of cp?
mv old tmp
mv curr old
mv tmp curr
I have this in a working script I delivered. It's written as a function, but you would invoke it
d_swap lfile rfile
The GNU mv has the -b and the -T switch. You can deal with directories using the -T
switch.
The quotes are for filenames with spaces.
It's a bit verbose, but I've used it many times with both files and directories. There might be cases where you would want to rename a file with the name a directory had, but that isn't handled by this function.
This isn't very efficient if all you want to do is rename the files (leaving them where they are), that is better done with a shell variable.
d_swap() {
test $# -eq 2 || return 2
test -e "$1" || return 3
test -e "$2" || return 3
if [ -f "$1" -a -f "$2" ]
then
mv -b "$1" "$2" && mv "$2"~ "$1"
return 0
fi
if [ -d "$1" -a -d "$2" ]
then
mv -T -b "$1" "$2" && mv -T "$2"~ "$1"
return 0
fi
return 4
}
This function will rename files. It uses a temp name (it puts a dot '.' in front of the name) just in case the files/directories are in the same directory, which is usually the case.
d_swapnames() {
test $# -eq 2 || return 2
test -e "$1" || return 3
test -e "$2" || return 3
local lname="$(basename "$1")"
local rname="$(basename "$2")"
( cd "$(dirname "$1")" && mv -T "$lname" ".${rname}" ) && \
( cd "$(dirname "$2")" && mv -T "$rname" "$lname" ) && \
( cd "$(dirname "$1")" && mv -T ".${rname}" "$rname" )
}
That is a lot faster (there's no copying, just renaming). It is even uglier. And it will rename anything: files, directories, pipes, devices.