Context
I finally fixed my issue to have stdout & stderr to screen and a file plus having a separate file for the errors. See: Output stdout and stderr to file and screen and stderr to file in a limited environment
Issue
The problem is the ordering of lines.
Some times it is OK:
FIRST
ERROR
LAST
ERROR2
LAST2
Some times the errors are at the end:
FIRST
LAST
LAST2
ERROR
ERROR2
I can't figure out why (except maybe a semaphore underneath that... but... not sure. And if it is the case, there is no solution exception adding line numbers to each echo).
Part of the code where the problem occurs
{ "$0" "${mainArgs[#]}" 2>&1 1>&3 | tee -a "$logPath/$logFileName.err" 1>&3 ; } 3>&1 | tee -a "$logPath/$logFileName.log" &
Full testable code
m=0
declare -a mainArgs
if [ ! "$#" = "0" ]; then
for arg in "$#"; do
mainArgs[$m]=$arg
m=$(($m + 1))
done
fi
function containsElement()
# $1 string to find
# $2 array to search in
# return 0 if there is a match, otherwise 1
{
local e match="$1"
shift
for e; do [[ "$e" == "$match" ]] && return 0; done
return 1
}
function hasMainArg()
# $1 string to find
# return 0 if there is a match, otherwise 1
{
local match="$1"
containsElement "$1" "${mainArgs[#]}"
return $?
}
function activateLogs()
# $1 = logOutput: What is the output for logs: SCREEN, DISK, BOTH. Default is DISK. Optional parameter.
{
local logOutput=$1
if [ "$logOutput" != "SCREEN" ] && [ "$logOutput" != "BOTH" ]; then
logOutput="DISK"
fi
if [ "$logOutput" = "SCREEN" ]; then
echo "Logs will only be output to screen"
return
fi
hasMainArg "--force-log"
local forceLog=$?
local isFileDescriptor3Exist=$(command 2>/dev/null >&3 && echo "Y")
if [ "$isFileDescriptor3Exist" = "Y" ]; then
echo "Logs are configured"
elif [ "$forceLog" = "1" ] && ([ ! -t 1 ] || [ ! -t 2 ]); then
# Use external file descriptor if they are set except if having "--force-log"
echo "Logs are configured externally"
else
echo "Relaunching with logs files"
local logPath="logs"
if [ ! -d $logPath ]; then mkdir $logPath; fi
local logFileName=$(basename "$0")"."$(date +%Y-%m-%d.%k-%M-%S)
exec 4<> "$logPath/$logFileName.log" # File descriptor created only to get the underlying file in any output option
if [ "$logOutput" = "DISK" ]; then
# FROM: https://stackoverflow.com/a/45426547/214898
exec 3<> "$logPath/$logFileName.log"
"$0" "${mainArgs[#]}" 2>&1 1>&3 | tee -a "$logPath/$logFileName.err" 1>&3 &
else
# FROM: https://stackoverflow.com/a/70790574/214898
{ "$0" "${mainArgs[#]}" 2>&1 1>&3 | tee -a "$logPath/$logFileName.err" 1>&3 ; } 3>&1 | tee -a "$logPath/$logFileName.log" &
fi
exit
fi
}
#activateLogs "DISK"
#activateLogs "SCREEN"
activateLogs "BOTH"
echo "FIRST"
echo "ERROR" >&2
echo "LAST"
echo "ERROR2" >&2
echo "LAST2"
Stdout 1 and errout 2 pass by different file descriptors. If is very busy then very close calls like these can get mixed up.
You could use sleep 1 between your calls which would give the system time to process each call in order, but would slow down your script.
You can sleep less than a second : for example sleep 0.5.
Related
I want to check for file in directory if there then push it to ssh server checing server connection if file not there then try 3 times with each 1min interval and in between if it comes ( on 2nd attend for example) then try again to connect ssh and push. else check for 3 attempts and exit
Please check my below code it is halting after 1st attempt ( during 2nd attempt I am making file available)
#!/bin/sh
echo "OK, start pushing the Userdetails to COUPA now..."
cd /usr/App/ss/outbound/usrdtl/
n=0
until [ $n -ge 3 ] || [ ! -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ]
do
if [ -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ] ;
then
pushFiles()
else
n=$[$n+1]
sleep 60
echo " trying " $n "times "
fi
done
pushFiles()
{
echo "File present Now try SSH connection"
while [ $? -eq 0 ];
do
echo $(date);
scpg3 -v /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv <sshHost>:/Incoming/Users/
if [ $? -eq 0 ]; then
echo "Successfull"
echo $(date);
echo "Successfull" >> /usr/App/ss/UserApproverDetails.log
exit 1;
else
echo $(date);
echo "Failed" >> /usr/App/ss/UserApproverDetails.log
echo "trying again to push file.."
scpg3 -v /usr/App/sg/outbound/usrdtl/USERS_APPROVERS_*.csv <ssh Host>:/Incoming/Users/
echo $(date);
exit 1;
fi
done
}
I've tried to simplify this code for you. I hope it helps:
#!/bin/bash
outdir="/usr/App/ss/outbound/usrdtl"
logfile="/usr/App/ss/UserApproverDetails.log"
file_prefix="USERS_APPROVERS_"
function push_files() {
echo "File present now try SSH connection"
local attempts=1
local retries=2
date
while [[ ${attempts} -lt ${retries} ]]; do
if scp ${outdir}/${file_prefix}*.csv <sshHost>:/Incoming/Users/ ; then
echo "Successful" | tee -a ${logfile}
date
exit 0
else
echo "Failed" >> ${logfile}
fi
attempts=$((attempts+1))
do
echo "scp failed twice" | tee -a ${logfile}
exit 2
}
echo "OK, start pushing the Userdetails to COUPA now..."
cd ${outdir}
attempts=1
retries=3
while [[ ${attempts} -lt ${retries} ]]; do
echo "looking for files...attempt ${attempts}"
if test -n "$(shopt -s nullglob; echo ${outdir}/${file_prefix}*.csv)"; then
push_files()
fi
attempts=$((attempts+1))
sleep 60
done
echo "Files were never found" | tee -a ${logfile}
exit 1
Look at this code and tell me how it's not doing what you're trying to do. The most complicated part here is the nullglob stuff, which is a handy trick to see if any file in a glob matches
Also, I generally used bashisms.
have some problem with shell script.
In our office we set up only few commands, that available for devs when they are trying ssh to server. It is configured with help of .ssh/authorized_keys file and available command for user there is bash script:
#!/bin/sh
if [[ $1 == "--help" ]]; then
cat <<"EOF"
This script has the purpose to let people remote execute certain commands without logging into the system.
For this they NEED to have a homedir on this system and uploaded their RSA public key to .ssh/authorized_keys (via ssh-copy-id)
Then you can alter that file and add some commands in front of their key eg :
command="/usr/bin/dev.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
The user will do the following : ssh testuser#server tail testserver.example.com/2017/01/01/user.log
EOF
exit 0;
fi
# set global variable
set $SSH_ORIGINAL_COMMAND
# set the syslog path where the files can be found
PATH="/opt/syslog/logs"
# strip ; or any other unwanted signs out of the command, this prevents them from breaking out of the setup command
if [[ $1 != "" ]]; then
COMMAND=$1
COMMAND=${COMMAND//[;\`]/}
fi
if [[ $2 != "" ]]; then
ARGU1=$2
ARGU1=${ARGU1//[;\`]/}
fi
if [[ $3 != "" ]]; then
ARGU2=$3
ARGU2=${ARGU2//[;\`]/}
fi
if [[ $4 != "" ]]; then
ARGU3=$4
ARGU3=${ARGU3//[;\`]/}
fi
# checking for the commands
case "$COMMAND" in
less)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
#echo " --------------------------------- LESS $FILE"
/usr/bin/less $FILE
;;
grep)
if [[ $ARGU2 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
if [[ $ARGU1 == "" ]]; then
echo "Pls give a string to search for"
exit 1
fi
ARGU2=${ARGU2//\.\./}
FILE=$PATH/$ARGU2
/usr/bin/logger -t restricted-command -- "------- $USER Executing grep $ARGU1 \"$ARGU2\" $FILE"
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
exit 1;
fi
/bin/grep $ARGU1 $FILE
;;
tail)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
ARGU1=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $# ($FILE)"
exit 1;
fi
/usr/bin/tail -f $FILE
;;
cat)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
/bin/cat $FILE
;;
help)
/bin/cat <<"EOF"
# less LOGNAME (eg less testserver.example.com/YYYY/MM/DD/logfile.log)
# grep [ARGUMENT] LOGNAME
# tail LOGNAME (eg tail testserver.example.com/YYYY/MM/DD/logfile.log)
# cat LOGNAME (eg cat testserver.example.com/YYYY/MM/DD/logfile.log)
In total the command looks like this : ssh user#testserver.example.com COMMAND [ARGUMENT] LOGFILE
EOF
/usr/bin/logger -t restricted-command -- "$USER HELP requested $#"
exit 1
;;
*)
/usr/bin/logger -s -t restricted-command -- "$USER Invalid command $#"
exit 1
;;
esac
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
The problem is next:
when i try to exec some command, it takes only first argument, if i do recursion in files by using {n,n1,n2} - it doesn't work:
[testuser#local ~]$ ssh testuser#syslog.server less srv1838.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv2010.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
11591
[testuser#local ~]$ ssh testuser#syslog.server less srv{1838,2010}.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv{2010,1838}.example.com/2017/02/21/local1.log |grep 'srv2010' | wc -l
11591
Could someone help me, how can i parse\count command arguments to make it work?
Thank you and have a nice day!
The number of arguments for a bash script would be $#. As a quick example:
#!/bin/bash
narg=$#
typeset -i i
i=1
while [ $i -le $narg ] ; do
echo " $# $i: $1"
shift
i=$i+1
done
gives, for bash tst.sh a b {c,d}
4 1: a
3 2: b
2 3: c
1 4: d
In your script, the command to execute (cat, less, ...) gets explicitly only the second argument to the script. If you want to read all arguments, you should do something like this (note: only a hint, removed all sorts of checks etc..)
command="$1"
shift
case $command in
(grep) pattern="$1"
shift
while [ $# -gt 0 ] ; do
grep "$pattern" "$1"
shift
done
;;
esac
note: added some quotes as comment suggested, but, being only a hint, you should carefully look at quoting and your checks in your own script.
Less command working now:
case "$COMMAND" in
less)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
FILES_LIST=${#:2}
FILE=(${FILES_LIST//\.\./})
for v in "${FILE[#]}";do
v=${v//[;\']/}
if [ ! -f $v ]; then
echo "File doesn't exist"
fi
/usr/bin/less $PATH/$v
done;;
tail command works too with 2 and more files, but i can't execute tail -f command on two files unfortunately.
I can't get my bash script (a logging file) to detect any other exit code other than 0, so the count for failed commands isn't being incremented, but the successes is incremented regardless of whether the command failed or succeeded.
Here is the code:
#!/bin/bash
#Script for Homework 8
#Created by Greg Kendall on 5/10/2016
file=$$.cmd
signal() {
rm -f $file
echo
echo "User Aborted by Control-C"
exit
}
trap signal 2
i=0
success=0
fail=0
commands=0
read -p "$(pwd)$" "command"
while [ "$command" != 'exit' ]
do
$command
((i++))
echo $i: "$command" >> $file
if [ "$?" -eq 0 ]
then
((success++))
((commands++))
else
((fail++))
((commands++))
fi
read -p "$(pwd)" "command"
done
if [ "$command" == 'exit' ]
then
rm -f $file
echo commands:$commands "(successes:$success, failures:$fail)"
fi
Any help would be greatly appreciated. Thanks!
That's because echo $i: "$command" is succeeding always.
The exit status $? in if [ "$?" -eq 0 ] is actually the exit status of echo, the command that is run immediately before the checking.
So do the test immediate after the command:
$command
if [ "$?" -eq 0 ]
and use echo elsewhere
Or if you prefer you don't need the $? check at all, you can run the command and check status within if alone:
if $command; then .....; else ....; fi
If you do not want to get the STDOUT and STDERR:
if $command &>/dev/null; then .....; else ....; fi
** Note that, as #Charles Duffy mentioned in the comment, you should not run command(s) from variables.
Your code is correctly counting the number of times that the echo $i: "$command" command fails. I presume that you would prefer to count the number of times that $command fails. In that case, replace:
$command
((i++))
echo $i: "$command" >> $file
if [ "$?" -eq 0 ]
With:
$command
code=$?
((i++))
echo $i: "$command" >> $file
if [ "$code" -eq 0 ]
Since $? captures the exit code of the previous command, it should be placed immediately after the command whose code we want to capture.
Improvement
To make sure that the value of $? is captured before any other command is run, Charles Duffy suggests placing the assignment on the same line as the command like so:
$command; code=$?
((i++))
echo $i: "$command" >> $file
if [ "$code" -eq 0 ]
This should make it less likely that any future changes to the code would separate the command from the capture of the value of $?.
I have written the following Linux shell script through snippits gleaned from the web (I'm very new to shell scripts), its purpose is to ensure only a single instance of a programme is run with the added option of specifying which workspace a programme will open to.
I'm sure much of the code could be better constructed, however it works with one bug, when some, like Thunderbird, are opened they ignore the workspace switch unless the workaround I've added is used, but why? and is there a better way?
The script uses wmctrl: sudo apt-get install wmctrl
Usage: single-switch programme_name [-ws(int)] where (int) is number of workspace (must exist), the -ws param must be the last listed
#!/bin/bash
if ! [ $1 ] ; then exit ; fi
if [ "?" = "$1" ] ; then
FILE=$(echo "$0" | rev | cut -d "/" -f1 | rev) # extract filename from path
echo "usage $FILE <program name> [-ws(int)]"
exit 1;
fi
TITLE=$1
NAME=""
for var in "$#"; do [ "$(echo ${var} | head -c3)" != '-ws' ] && NAME="$NAME $var" ; done # remove our param from command
ntharg() { shift $1 ; echo $1 ; }
PARAM=`ntharg $# "$#"` # get the last param
if [ "-ws" != "$(echo ${PARAM} | head -c3)" ]; then PARAM=-1 ; # check its ours
else
PARAM=$( echo "$PARAM" | egrep -o '[0-9]+' ) # get the number
PARAM=$((PARAM-1)) # decrement
fi
if [ $PARAM -ge 0 ] ; then
wmctrl -x -a "$TITLE" || ( wmctrl -s $PARAM && zenity --title="Launch $TITLE" --warning --text="$TITLE is starting" --timeout="1" ; $NAME )
# dummy message otherwise some (ie thunderbird) ignore switch
else
wmctrl -x -a "$TITLE" || $NAME & # no switch, just raise or run programme
fi
# Done.
#
I have the following code:
#!/bin/bash -x
# Arguments:
# $1 - command to run
# $2 - time limit (in milliseconds)
# $3 - memory limit (in kilobytes)
# NOTE TO SELF: $! is the pid of last process
# NOTE TO SELF: Appending & creates new process
dir=$(mktemp -d)
ulimit -m $3
{ $1 ; "$?" > "$dir/retc" } &
pid=$!
./sleep.pl $2
if [ ps -p $pid > /dev/null ]
then
kill -9 $pid
echo "0Time Limit Exceeded"
else
echo "NAH"
ret=$(cat "$dir/retc")
if [ $ret = 9 ]
then
echo "0Memory Limit Exceeded"
else
if [ $ret = 0 ]
then
echo "1" # If it only returns one then it must be passed through final phase of verifying if result is correct
else
echo "0Received signal $ret"
fi
fi
fi
rm -r $dir
exit 0
However, it returns an error "Unexpected end of file", without executing anything in the if/else blocks.
You missed the semicolon, the redirection is wrong also:
{ $1 ; "$?"; } >"$dir/retc" &
From bash man: list must be terminated with a newline or semicolon
Also:
if [ ps -p $pid > /dev/null ]
Should be:
if [[ $(ps -p $pid >/dev/null) -eq 0 ]]
I'd recommend using () over {} to explicitly show that you're wanting to summon a subshell. Also I think you missed echo to show the value of $?.
( "$1"; echo "$?" > "$dir/etc"; ) &
With () you wouldn't need to add a semicolon in the end unlike with {} however it's still a good practice.