Logging inside a bash script file - linux

I have a pretty large script ( functions contains around 4000 lines of code) . Here is a part of it :
#!/bin/bash
. ./functions > /dev/null 2>&1
some_function(){
while true
do
CHOICE=$(whiptail --menu "\n\n\n\n\n\n\n\n" --title "Tools" --nocancel $window 20 \
"1" "Option1" \
"2" "Option2" \
"3" "Option3" 3>&1 1>&2 2>&3)
case $CHOICE in
1)echo $1 $2 $3;;
2)echo $2 $1 $3;;
3)echo $3 $2 $1;;
esac
done
}
while true; do
arr=()
for ((n=1; n<=$node_num; n++))
do
node+=($n NODE$n)
done
OPTION=$(whiptail --menu "\n\n\n\nPlease choose:\n\n\n" --title "tools" $window 20 "${node[#]}" \
case $OPTION in
1) some_function 1 2 3 ;;
2) some_function 2 1 3 ;;
3) some_function 3 1 2 ;;
esac
done
I want to log the commands executed in the script.
What I have tried so far is :
#!/bin/bash -x --> this will log all the output , but will also "spam" the logs with unneeded information like variable values etc. However this seems to be the best way so far...
I have tried #!/bin/bash -i , enabling history with set -o history . The disadvantage of this is it will log everything . When I call the function file for example it will log every single line as if it was executed .
I have tried creating a log function :
logthis(){
## print the command to the logfile
echo "$(date) $#" >> $historyfile
## run the command and redirect it's error output
## to the logfile
eval "$#" 2>> $historyfile
}
This seems to work most of the time. But when I do, for example:
case $OPTION in
1) logthis "some_function 1 2 3" ;;
2) some_function 2 1 3 ;;
3) some_function 3 1 2 ;;
esac
it will not work as I will lose the arguments 1 2 3
Do you have any other ideas of doing an elegant logging system inside a bash script?

Get rid of the eval in your log function. Just write "$#" to execute the passed command.
logthis() {
echo "$(date): $#" >> "$historyfile"
"$#" 2>> "$historyfile"
}
Then you can log a command by simply prepending logthis. No need for extra quotes.
logthis some_function 1 2 3
This will very nicely preserve all the arguments--even if they have whitespace or other special characters.
I'd recommend a minor improvement to the echo command as well. If you use printf %q it'll log arguments with whitespace better.
echo "$(date):$(printf ' %q' "$#")" >> "$historyfile"

Try set -v
That doesn't parse the commands like set -x, just outputs what gets executed.
set -v
: all the things 'in' $0, 'yay!'
outputs exactly : all the things 'in' $0, 'yay!'
Doesn't even parse $0.
Arguments recorded, minimal spam. ;)
Consider wrapping curlies around the main block of code to manage output logging.
{ # after setup to parse args, set vars & traps, declare funcs, etc
your bulk code here
} 2>&1 | tee /some/file.log
You can save the set -x spam for --verbose mode. ;)

Related

Reading variables from config file in Shell

I have question about Linux shell scripts. My question is realy abstract, so may not make sense. The idea is having 1 script and 2 config files.
Script can be like (drinkOutput.sh):
#!/bin/bash
echo -e " $1 \n"
echo -e " $2 \n"
First Config file contain (beer.conf):
drink1="heineken"
drink2="argus"
Second Config file contain (vine.conf):
drink1="chardonnay"
drink2="hibernal"
The key thing is calling the script. It has to be in next format (or with parameter)
./drinkOutput.sh beer.conf
In this case I need to have in $1 heineken and in $2 argus (inside of drinkOutput script). For
./drinkOutput.sh vine.conf
I need to get back into drinkOutput.sh chardonnay and hibernal.
Does anybody know? Thanks for any tips
You can source the config files if they are in the right format (and it seems it is in your example).
drinkOutput()
{
echo "$1"
echo "$2"
}
conf="$1"
source "$conf"
drinkOutput "$drink1" "$drink2"
If is possible if your script calls itself with the proper arguments after having parsed them from the conf file:
if [ $# == 2 ] ; then
# The arguments are correctly set in the sub-shell.
# 2 arguments: do something with them
echo magic happens: $1 $2
elif [ $# == 1 ] ; then
# 1 argument: conf file: parse conf file
arg1=`sed -n -e 's#drink1="\(.*\)"#\1#p' $1`
arg2=`sed -n -e 's#drink2="\(.*\)"#\1#p' $1`
$0 $arg1 $arg2
else
# error
echo "wrong args"
fi
test:
$ drinkOutput.sh beer.conf
magic happens: heineken argus

Command line arguments validation with GetOpts and mandatory parameters

I'm creating a basic script that should take 3 mandatory command line options and each one must be followed by a value. Like this:
$ myscript.sh -u <username> -p <password> -f <hosts.txt>
I'm trying to make sure the user is passing those exact 3 options and their values and nothing else, otherwise I want to print the usage message and exit.
I've been reading on getopts and came up with this:
usage () { echo "Usage : $0 -u <username> -p <password> -f <hostsFile>"; }
if [ $# -ne 6 ]
then
usage
exit 1
fi
while getopts u:p:f: opt ; do
case $opt in
u) USER_NAME=$OPTARG ;;
p) USER_PASSWORD=$OPTARG ;;
f) HOSTS_FILE=$OPTARG ;;
*) usage; exit 1;;
esac
done
echo "USERNAME: $USER_NAME"
echo "PASS: $USER_PASSWORD"
echo "FILE: $HOSTS_FILE"
I was hoping that if I do not pass any of my 3 "mandatory" options (i.e: -u -p -f) Optargs validation would catch that via the "*)" case. While that is true for other options such "-a","-b", etc.. does not seem to be the case in this particular case:
$ myscript.sh 1 2 3 4 5 6
Getops does not treat that as invalid input and the script moves on executing the echo commands showing 3 empty variables.
How can I capture the input above as being invalid as it is not in the form of:
$ myscript.sh -u <username> -p <password> -f <hosts.txt>
Thanks!
getopts has no concept of "mandatory" options. The colons in u:p:f: mean that, if one of those options happens to be supplied, then an argument to that option is mandatory. The option-argument pairs, however, are always optional.
You can require that the user provide all three though with code such as:
if [ ! "$USER_NAME" ] || [ ! "$USER_PASSWORD" ] || [ ! "$HOSTS_FILE" ]
then
usage
exit 1
fi
Place this code after the while getopts loop.
The Role of *)
I was hoping that if I do not pass any of my 3 "mandatory" options (i.e: -u -p -f) Optargs validation would catch that via the "*)" case.
The *) case is executed only if an option other than -u, -p, or -f is supplied. Thus, if someone supplied, for example a -z argument, then that case would run.

Splitting text files in folder

I have been trying to make a shell script that will split text files one after the other through an entire folder and deposit every split chunk into another designated folder.
Here is what I have so far, I know its probably clunky(have never tried writing a .sh before):
#!/bin/bash
#File Split Automation
echo "Usage: split [Folder w/ Input] [Folder For Outputs] [Options] [PREFIX]
Options: -b [sizeMB]: Split by size
-l [No. of Lines]: Split by Lines
If No Output Folder is Defined Default is Set To: /Desktop/splitter-parts
If No Options Are Selected Default is Size=100MB"
inputdirc=$1
outputdirc=$2
spltion=$3
meastick=$4
prefixture=$5
if [ -d $1 ]
then
echo "You Picked The Folder $1 To Split Files From"
ls $1
else
exit
fi
if [ -d $2 ]
then
echo "Please Confirm Folder Path For Output $outputdirc"
else
cd /root/Desktop/
mkdir -p splitter-parts
fi
read -t 10 -p "Press Enter Or Wait 5 Sec. To Continue"
cd $2
for swordfile in $( ls $1);
do
command -p split $3 $4 -a 3 -d $swordfile $5
done
Anything you see going wrong? Because I am not getting the output I desired, though it functioned fine when I just had a file and a folder in the split-command string.
EDIT::::
Sorry, I apologize. Just getting a bit ahead of myself.
This is what I am seeing when I run it:
root#kali:~/Desktop/Wordlists# ./splitter.sh '/root/Desktop/Wordlists' ' /root/Desktop/Untitled Folder' s 100MB
Usage: split [Folder w/ Input] [Folder For Outputs] [Options] [PREFIX]
Options: -b [sizeMB]: Split by size
-l [No. of Lines]: Split by Lines
If No Output Folder is Defined Default is Set To: /Desktop/splitter-parts
If No Options Are Selected Default is Size=100MB
You Picked The Folder /root/Desktop/Wordlists To Split Files From
10dig10milup2.txt mixed.txt
10dig10miluplow2.txt movie-characters.txt
10dig10miluplow3.txt name1s.txt
((------------------CUT------------)
lower.lst xae2.txt
lower.txt xaf2.txt
mangled.lst xag2.txt
mangled.txt xah6.txt
misc-dictionary.txt
./splitter.sh: line 24: [: /root/Desktop/Untitled: binary operator expected
Press Enter Or Wait 5 Sec. To Continue
./splitter.sh: line 37: cd: /root/Desktop/Untitled: No such file or directory
split: extra operand `10dig10milup2.txt'
Try `split --help' for more information.
split: extra operand `10dig10miluplow2.txt'
Try `split --help' for more information.
split: extra operand `10dig10miluplow3.txt'
Try `split --help' for more information.
split: extra operand `10dig10miluplow4.txt'
Try `split --help' for more information.
...................MORE OF THE SAME.......
As far as what I am supposed to see, I haven't gotten that far yet, clearly I am missing some steps.
A quick rewrite with some notes to follow:
#!/bin/bash
#File Split Automation
usage="Usage: split [Options] [Folder w/ Input] [Folder For Outputs] [PREFIX]
Options: -b [sizeMB]: Split by size
-l [No. of Lines]: Split by Lines
If No Output Folder is Defined Default is Set To: /Desktop/splitter-parts
If No Options Are Selected Default is Size=100MB"
split_opt="-b 100MB"
while getopts hb:l: opt; do
case $opt in
h) echo "$usage"; exit ;;
b) split_opt="-b $OPTARG" ;;
l) split_opt="-l $OPTARG" ;;
esac
done
shift $((OPTIND - 1))
if [[ $# -eq 0 ]]; then
echo "$usage"
exit 1
fi
inputdirc=$1
if [[ -d $inputdirc ]]; then
ls $1
else
echo "no such directory: $inputdirc" >&2
exit 1
fi
if [[ -n $2 ]]; then
outputdirc=$2
else
outputdirc=/root/Desktop/splitter-parts
fi
prefixture=$3
mkdir -p "$outputdirc"
cd "$outputdirc"
for swordfile in "$inputdirc"/*; do
command -p split $split_opt -a 3 -d "$swordfile" $prefixture
done
Notes:
you generally want to quote all your variables. This is the cause of your error, because there was a file with whitespace and square brackets in the name.
I did not quote a couple in the split command because I specifically want the shell to perform word splitting on the values
since options are, well, optional, use getopts to collect them.
you store the positional parameters in variables, but you continue to use the positional parameters. Pick one or the other.

Remove single quotes from string in shell script

This is my shell script-
if ! options=$(getopt -o : -l along:,blong:,clong: -- "$#")
then
# something went wrong, getopt will put out an error message for us
exit 1
fi
set -- $options
while [ $# -gt 0 ]
do
case $1 in
--along) echo "--along selected :: $2" ;;
--blong) echo "--blong selected :: $2" ;;
--clong) echo "--clong selected :: $2" ;;
esac
shift
done
when i run the script i get the following output-
./test.sh --along hi --blong hello --clong bye
--along selected :: 'hi'
--blong selected :: 'hello'
--clong selected :: 'bye'
The problem is I don't want to display the arguments with single quotes ('hi', 'hello', 'bye'). What should I do to get rid of those quotes?
Use the option -u or --unquoted for getopt, i.e.
if ! options=$(getopt -u -o : -l along:,blong:,clong: -- "$#")
The manpage of getopt says for -u:
Do not quote the output. Note that whitespace and special
(shell-dependent) characters can cause havoc in this mode (like they
do with other getopt(1) implementations).

looking for a command to tentatively execute a command based on criteria

I am looking for a command (or way of doing) the following:
echo -n 6 | doif -criteria "isgreaterthan 4" -command 'do some stuff'
The echo part would obviously come from a more complicated string of bash commands. Essentially I am taking a piece of text from each line of a file and if it appears in another set of files more than x (say 100) then it will be appended to another file.
Is there a way to perform such trickery with awk somehow? Or is there another command.. I'm hoping that there is some sort of xargs style command to do this in the sense that the -I% portion would be the value with which to check the criteria and whatever follows would be the command to execute.
Thanks for thy insight.
It's possible, though I don't see the reason why you would do that...
function doif
{
read val1
op=$1
val2="$2"
shift 2
if [ $val1 $op "$val2" ]; then
"$#"
fi
}
echo -n 6 | doif -gt 3 ls /
if test 6 -gt 4; then
# do some stuff
fi
or
if test $( echo 6 ) -gt 4; then : ;fi
or
output=$( some cmds that generate text)
# this will be an error if $output is ill-formed
if test "$output" -gt 4; then : ; fi

Resources