I am Attempting to use the output of ls command as an array. And I want it to be line by line [closed] - linux

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 months ago.
Improve this question
I need to find the encrypted(.enc) files from a folder and decrypt them.
I will find the .enc files
if [[ -n "$(ls -A /sodaman/tempPrabhu/temp/*.enc 2>/dev/null)" ]]; then
And I used enc_files=($( ls *.enc )) but it takes all the files as one and fails. It considers all the files for the output as one line. So I replaced it with mapfile to decrypt the files one by one but it throws an error
test.sh: line 31: syntax error near unexpected token `<' test.sh: line 31: ` mapfile -t enc_files < <(ls *.enc)'
Below is the script:
if [[ -n "$(ls -A /sodaman/tempPrabhu/temp/*.enc 2>/dev/null)" ]]; then
#create array of encrypted files
mapfile -t enc_files < <(ls *.enc)
enc_files=($( ls *.enc ))
#echo Creating array $enc_files
#decrypt all encrypted files.
echo Creating loop for encryped files
for m in "${enc_files[#]}"
do
d=$(echo "$m" | cut -f 1 -d '.')
echo $d
d=$d.dat
echo $d
/empty/extproc/hfencrypt_plus $m $d Decrypt /empty/extproc/hfsymmetrickey.dat log.log infa91punv
echo /empty/extproc/hfencrypt_plus $m $d Decrypt /empty/extproc/hfsymmetrickey.dat log.log infa91punv
if [[ -f "$d" ]]; then
mv $m /empty/sodaman/tempPrabhu/blr_temp
#echo Moving file to encrypted archive : mv "$m" /empty/sodaman/enc_archive
echo removing log file : rm log.log
rm log.log
else
echo File was not decrypted successfully
fi
done
fi

Here's a refactoring which avoids several of the http://shellcheck.net/ violations in your attempt.
for file in /sodaman/tempPrabhu/temp/*.enc; do
# avoid nullglob
test -e "$file" || continue
# prefer parameter expansion
d=${file%%.*}.dat
if /empty/extproc/hfencrypt_plus "$file" "$d" Decrypt /empty/extproc/hfsymmetrickey.dat log.log infa91punv
then
# assume hfencrypt sets exit code
mv "$file" /empty/sodaman/tempPrabhu/blr_temp
else
# print diagnostics to stderr
# mention which file failed
# mention which script emitted the warning
echo "$0: $file was not decrypted successfully" >&2
sed "s%^%log.log: $file: %" log.log >&2
fi
rm -f log.log
done
This assumes that you wanted to loop over the files in the directory you examine at the beginning of your script, not in the current directory (perhaps see also What exactly is current working directory?) and that the encryption utility sets its exit code to nonzero if encryption failed (perhaps see also Why is testing “$?” to see if a command succeeded or not, an anti-pattern? which discusses idioms around conditions involving errors). I added a sed command to include the output from log.log in the diagnostics after a failure, though perhaps you would like a different error-handling strategy (exit immediately and let the user troubleshoot? Or rename the log file to a unique name and keep it around for later?)

Don't parse ls results, but use this:
find /sodaman/tempPrabhu/temp/ -maxdepth 1 -name "*.enc"

Related

Using inotifywait to process two files in parallel

I am using:
inotifywait -m -q -e close_write --format %f . | while IFS= read -r file; do
cp -p "$file" /path/to/other/directory
done
to monitor a folder for file completion, then moving it to another folder.
Files are made in pairs but at separate times, ie File1_001.txt is made at 3pm, File1_002.txt is made at 9pm. I want to monitor for the completion of BOTH files, then launch a script.
script.sh File1_001.txt File1_002.txt
So I need to have another inotifywait command or a different utility, that can also identify that both files are present and completed, then start the script.
Does anyone know how to solve this problem?
I found a Linux box with inotifywait installed on it, so now I understand what it does and how it works. :)
Is this what you need?
#!/bin/bash
if [ "$1" = "-v" ]; then
Verbose=true
shift
else
Verbose=false
fi
file1="$1"
file2="$2"
$Verbose && printf 'Waiting for %s and %s.\n' "$file1" "$file2"
got1=false
got2=false
while read thisfile; do
$Verbose && printf ">> $thisfile"
case "$thisfile" in
$file1) got1=true; $Verbose && printf "... it's a match!" ;;
$file2) got2=true; $Verbose && printf "... it's a match!" ;;
esac
$Verbose && printf '\n'
if $got1 && $got2; then
$Verbose && printf 'Saw both files.\n'
break
fi
done < <(inotifywait -m -q -e close_write --format %f .)
This runs a single inotifywait but parses its output in a loop that exits when both files on the command line ($1 and $2) are seen to have been updated.
Note that if one file is closed and then later is reopened while the second file is closed, this script obviously will not detect the open file. But that may not be a concern in your use case.
Note that there are many ways of building a solution -- I've shown you only one.

How to extract only file name return from diff command?

I am trying to prepare a bash script for sync 2 directories. But I am not able to file name return from diff. everytime it converts to array.
Here is my code :
#!/bin/bash
DIRS1=`diff -r /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/ `
for DIR in $DIRS1
do
echo $DIR
done
And if I run this script I get out put something like this :
Only
in
/opt/lampp/htdocs/scripts/www/:
file1
diff
-r
"/opt/lampp/htdocs/scripts/dev/File
1.txt"
"/opt/lampp/htdocs/scripts/www/File
1.txt"
0a1
>
sa
das
Only
in
/opt/lampp/htdocs/scripts/www/:
File
1.txt~
Only
in
/opt/lampp/htdocs/scripts/www/:
file
2
-
second
Actually I just want to file name where I find the diffrence so I can take perticular action either copy/delete.
Thanks
I don't think diff produces output which can be parsed easily for your purposes. It's possible to solve your problem by iterating over the files in the two directories and running diff on them, using the return value from diff instead (and throwing the diff output away).
The code to do this is a bit long, but here it is:
DIR1=./one # set as required
DIR2=./two # set as required
# Process any files in $DIR1 only, or in both $DIR1 and $DIR2
find $DIR1 -type f -print0 | while read -d $'\0' -r file1; do
relative_path=${file1#${DIR1}/};
file2="$DIR2/$relative_path"
if [[ ! -f "$file2" ]]; then
echo "'$relative_path' in '$DIR1' only"
# Do more stuff here
elif diff -q "$file1" "$file2" >/dev/null; then
echo "'$relative_path' same in '$DIR1' and '$DIR2'"
# Do more stuff here
else
echo "'$relative_path' different between '$DIR1' and '$DIR2'"
# Do more stuff here
fi
done
# Process files in $DIR2 only
find $DIR2 -type f -print0 | while read -d $'\0' -r file2; do
relative_path=${file2#${DIR2}/};
file1="$DIR1/$relative_path"
if [[ ! -f "$file2" ]]; then
echo "'$relative_path' in '$DIR2 only'"
# Do more stuff here
fi
done
This code leverages some tricks to safely handle files which contain spaces, which would be very difficult to get working by parsing diff output. You can find more details on that topic here.
Of course this doesn't do anything regarding files which have the same contents but different names or are located in different directories.
I tested by populating two test directories as follows:
echo "dir one only" > "$DIR1/dir one only.txt"
echo "dir two only" > "$DIR2/dir two only.txt"
echo "in both, same" > $DIR1/"in both, same.txt"
echo "in both, same" > $DIR2/"in both, same.txt"
echo "in both, and different" > $DIR1/"in both, different.txt"
echo "in both, but different" > $DIR2/"in both, different.txt"
My output was:
'dir one only.txt' in './one' only
'in both, different.txt' different between './one' and './two'
'in both, same.txt' same in './one' and './two'
Use -q flag and avoid the for loop:
diff -rq /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/
If you only want the files that differs:
diff -rq /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/ |grep -Po '(?<=Files )\w+'|while read file; do
echo $file
done
-q --brief
Output only whether files differ.
But defitnitely you should check rsync: http://linux.die.net/man/1/rsync

Creating a pathname to check a file doesn't exist there / Permission denied error

Hello from a Linux Bash newbie!
I have a list.txt containing a list of files which I want to copy to a destination($2). These are unique images but some of them have the same filename.
My plan is to loop through each line in the text file, with the copy to the destination occurring when the file is not there, and a mv rename happening when it is present.
The problem I am having is creating the pathname to check the file against. In the code below, I am taking the filename only from the pathname, and I want to add that to the destination ($2) with the "/" in between to check the file against.
When I run the program below I get "Permission Denied" at line 9 which is where I try and create the path.
for line in $(cat list.txt)
do
file=$[ basename $line ]
path=$[ $2$file ]
echo $path
if [ ! -f $path ];
then
echo cp $line $2
else
echo mv $line.DUPLICATE $2
fi
done
I am new to this so appreciate I may be missing something obvious but if anyone can offer any advice it would be much appreciated!
Submitting this since OP is new in BASH scripting no good answer has been posted yet.
DESTINATION="$2"
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || mv "$line" "$path.DUP"
done < list.txt
Don't have logic for counting duplicates at present to keep things simple. (Which means code will take care of one dup entry) As an alternative you get uniq from list.txt beforehand to avoid the duplicate situation.
#anubhava: Your script looks good. Here is a small addition to it to work with several dupes.
It adds a numer to the $path.DUP name
UniqueMove()
{
COUNT=0
while [ -f "$1" ]
do
(( COUNT++ ))
mv -n "$1" "$2$COUNT"
done
}
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || UniqueMove "$line" "$path.DUP"
done < list.txt

preventing scripts from stopping on errors

My issue is likely a rather simple one, I just haven't found a satisfactory answer anywhere I've looked so far.
I have a script which runs a program/script, when it encounters an error it just hangs and will not continue along.
What is the best method to fix this?
To clarify, usually there is no output when it hangs but sometimes there is an error message, depending on which script I am using to work on my data set.
Example script:
#!/bin/bash
# iterate over a NUL-delimited stream of directory names
while IFS='' read -r -d '' dirname; do
# ...then list files in each directory:
for file in "$dirname"/*; do
# ignore directory contents that are not files
[[ -f $file ]] || continue
# run analysis tool
if [[ $file == *.dmp ]]; then
echo $dirname;
tshark -PVx -r "$file" >> $dirname/TEXT_out.txt #with packet summary line, packet details expanded, packet bytes
#ls $dirname;
echo "complete";
continue
fi
done
done < <(find . -type d -print0)
In nutshell, you are asking - how to handle if the child process hangs (from parent perspective). If that is the basic issue your are facing, then you can check this link.
HTH!

Looping through files in different directory given command line argument

I'm trying to extend a script that implements something like a recycling bin for files on Linux. I have the code that I'm extending at the bottom.
In my extension, when the script is presented with the command line argument -cleanup I want to loop through files that are in the /home/7/bearm/.garbage directory, and have the user decide whether they want to delete the file or not.
However, I don't know how to detect when the command line argument is there. The command line can have other parameters, I just want to loop through the files when -cleanup is used.
I also do not know how to loop through files that are in a different directory (/home/7/bearm/.garbage).
How would I go around doing these things?
set directory = '/home/7/bearm/.garbage/'
if(! -d "$directory") then
mkdir .garbage
mv .garbage /home/7/bearm/
endif
set n = 1
while ($n <= $#argv)
set file = $argv[$n]
if(-d $file) then
#do nothing
echo "Cannot trash directory $file"
else
mv $file /home/7/bearm/.garbage
echo "Trashed $file"
endif
# n++
end
du -h /home/7/bearm/.garbage
To test if arguments contains -cleanup, you can do that (tested with ash on Minix3):
if echo "$#" | grep -- "-cleanup" >/dev/null 2>&1; then
echo "-cleanup is present..."
fi
Moreover, if you want a proper solution to use long GNU style options, see http://www.sputnick-area.net/scripts/getopts_long_example.sh and http://www.sputnick-area.net/scripts/getopts_long.sh
A bash version of your pseudo script :
#!/bin/bash
directory='/home/7/bearm/.garbage/'
mkdir -p "$directory"
for arg; do
if [[ -d $arg ]]; then
#do nothing
echo "Cannot trash directory $arg" >&2
else
mv "$arg" "$directory"
echo "Trashed $arg"
fi
done
du -sh "$directory"
Feel free to improve it with -cleanup switch.

Resources