file check 3 times and exit shell script - linux

I want to check for file in directory if there then push it to ssh server checing server connection if file not there then try 3 times with each 1min interval and in between if it comes ( on 2nd attend for example) then try again to connect ssh and push. else check for 3 attempts and exit
Please check my below code it is halting after 1st attempt ( during 2nd attempt I am making file available)
#!/bin/sh
echo "OK, start pushing the Userdetails to COUPA now..."
cd /usr/App/ss/outbound/usrdtl/
n=0
until [ $n -ge 3 ] || [ ! -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ]
do
if [ -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ] ;
then
pushFiles()
else
n=$[$n+1]
sleep 60
echo " trying " $n "times "
fi
done
pushFiles()
{
echo "File present Now try SSH connection"
while [ $? -eq 0 ];
do
echo $(date);
scpg3 -v /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv <sshHost>:/Incoming/Users/
if [ $? -eq 0 ]; then
echo "Successfull"
echo $(date);
echo "Successfull" >> /usr/App/ss/UserApproverDetails.log
exit 1;
else
echo $(date);
echo "Failed" >> /usr/App/ss/UserApproverDetails.log
echo "trying again to push file.."
scpg3 -v /usr/App/sg/outbound/usrdtl/USERS_APPROVERS_*.csv <ssh Host>:/Incoming/Users/
echo $(date);
exit 1;
fi
done
}

I've tried to simplify this code for you. I hope it helps:
#!/bin/bash
outdir="/usr/App/ss/outbound/usrdtl"
logfile="/usr/App/ss/UserApproverDetails.log"
file_prefix="USERS_APPROVERS_"
function push_files() {
echo "File present now try SSH connection"
local attempts=1
local retries=2
date
while [[ ${attempts} -lt ${retries} ]]; do
if scp ${outdir}/${file_prefix}*.csv <sshHost>:/Incoming/Users/ ; then
echo "Successful" | tee -a ${logfile}
date
exit 0
else
echo "Failed" >> ${logfile}
fi
attempts=$((attempts+1))
do
echo "scp failed twice" | tee -a ${logfile}
exit 2
}
echo "OK, start pushing the Userdetails to COUPA now..."
cd ${outdir}
attempts=1
retries=3
while [[ ${attempts} -lt ${retries} ]]; do
echo "looking for files...attempt ${attempts}"
if test -n "$(shopt -s nullglob; echo ${outdir}/${file_prefix}*.csv)"; then
push_files()
fi
attempts=$((attempts+1))
sleep 60
done
echo "Files were never found" | tee -a ${logfile}
exit 1
Look at this code and tell me how it's not doing what you're trying to do. The most complicated part here is the nullglob stuff, which is a handy trick to see if any file in a glob matches
Also, I generally used bashisms.

Related

Terminal Freezing after Bash Script Succeeded

I have a pretty simple bash script that coordinates running a couple python scripts. What I am having trouble figuring out is why after running the bash script (. bash_script.sh), the terminal hangs. I can't ctrl+c, ctrl+z or do anything except restart the SSH session. All I see is just a blinking cursor. Checking all the log files indicates a 0 status exit code with no errors in the scripts themselves. Running ps aux | grep bash_script.sh does not show any anything running either. Is there anyway to debug this?
#!/bin/bash
exec >> <DIR>/logfile.log 2>&1
script_message () {
status_arg=$1
if [[ $status_arg = "pass" ]]; then
printf "Script Done\n"
printf '=%.0s' {1..50}
printf "\n"
elif [[ $status_arg = "fail" ]]; then
printf "Script Failed\n"
printf '=%.0s' {1..50}
printf "\n"
else
:
fi
}
current_date=$(date '+%Y-%m-%d %H:%M:%S')
day=$(date +%u)
hour=$(date +%H)
printf "RUN DATE: $current_date\n"
# activate virtual env
source /<VENV DIR/bin/activate>
python <PYTHON SCRIPT>.py >> <DIR>/logfile2.log 2>&1
retVal=$?
if [[ $retVal -eq 0 && $day -eq 4 ]]; then
python <PYTHON SCRIPT 2>.py >> <DIR>/logfile3.log 2>&1
script_message pass
elif [[ $retVal -eq 0 ]]; then
script_message pass
else
#:
script_message fail
fi
echo $?

Shell script to find the process state while excluding 'ps grep command'

can someone guide me writing shell script to find if the process is active or not? I have to exclude my own grep process filtering from ps command. I want to pass the process as a parameter,
script: (this is currently catching my own process)
#!/bin/sh
SERVICE=$1
echo $1
if ps ax | grep $SERVICE > /dev/null
then
echo "ok"
else
echo "not ok"
fi
example input tried: (though the process is dead I'm getting status as "ok")
./processchecker.sh '/usr/sbin/mysqld'
./processchecker.sh '[/usr/sbin/]mysqld' (i tried using square brackets using online suggestions but failed)
Please help.
You can use pgrep as well - which is a little more efficient:
#!/bin/sh
service=$1
status=0
if [ ! -z "$service" ]; then
pgrep "$service" >/dev/null; status=$?
if [ "$status" -eq 0 ]; then
echo "ok"
else
echo "not ok"
fi
fi
exit "$status"
It's better to have an appropriate exit value as well.
What you have is close, but you want to save the status of the grep command (via $?) and then if/else off of that value.
#!/bin/sh
SERVICE=$1
echo $1
ps ax | grep $SERVICE | grep -v ${0} > /dev/null
status=${?}
if [ "${status}" = "0" ]; then
echo "ok"
else
echo "not ok"
fi

Can't parse a string with brace expansion operations into a command

have some problem with shell script.
In our office we set up only few commands, that available for devs when they are trying ssh to server. It is configured with help of .ssh/authorized_keys file and available command for user there is bash script:
#!/bin/sh
if [[ $1 == "--help" ]]; then
cat <<"EOF"
This script has the purpose to let people remote execute certain commands without logging into the system.
For this they NEED to have a homedir on this system and uploaded their RSA public key to .ssh/authorized_keys (via ssh-copy-id)
Then you can alter that file and add some commands in front of their key eg :
command="/usr/bin/dev.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
The user will do the following : ssh testuser#server tail testserver.example.com/2017/01/01/user.log
EOF
exit 0;
fi
# set global variable
set $SSH_ORIGINAL_COMMAND
# set the syslog path where the files can be found
PATH="/opt/syslog/logs"
# strip ; or any other unwanted signs out of the command, this prevents them from breaking out of the setup command
if [[ $1 != "" ]]; then
COMMAND=$1
COMMAND=${COMMAND//[;\`]/}
fi
if [[ $2 != "" ]]; then
ARGU1=$2
ARGU1=${ARGU1//[;\`]/}
fi
if [[ $3 != "" ]]; then
ARGU2=$3
ARGU2=${ARGU2//[;\`]/}
fi
if [[ $4 != "" ]]; then
ARGU3=$4
ARGU3=${ARGU3//[;\`]/}
fi
# checking for the commands
case "$COMMAND" in
less)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
#echo " --------------------------------- LESS $FILE"
/usr/bin/less $FILE
;;
grep)
if [[ $ARGU2 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
if [[ $ARGU1 == "" ]]; then
echo "Pls give a string to search for"
exit 1
fi
ARGU2=${ARGU2//\.\./}
FILE=$PATH/$ARGU2
/usr/bin/logger -t restricted-command -- "------- $USER Executing grep $ARGU1 \"$ARGU2\" $FILE"
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
exit 1;
fi
/bin/grep $ARGU1 $FILE
;;
tail)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
ARGU1=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $# ($FILE)"
exit 1;
fi
/usr/bin/tail -f $FILE
;;
cat)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
/bin/cat $FILE
;;
help)
/bin/cat <<"EOF"
# less LOGNAME (eg less testserver.example.com/YYYY/MM/DD/logfile.log)
# grep [ARGUMENT] LOGNAME
# tail LOGNAME (eg tail testserver.example.com/YYYY/MM/DD/logfile.log)
# cat LOGNAME (eg cat testserver.example.com/YYYY/MM/DD/logfile.log)
In total the command looks like this : ssh user#testserver.example.com COMMAND [ARGUMENT] LOGFILE
EOF
/usr/bin/logger -t restricted-command -- "$USER HELP requested $#"
exit 1
;;
*)
/usr/bin/logger -s -t restricted-command -- "$USER Invalid command $#"
exit 1
;;
esac
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
The problem is next:
when i try to exec some command, it takes only first argument, if i do recursion in files by using {n,n1,n2} - it doesn't work:
[testuser#local ~]$ ssh testuser#syslog.server less srv1838.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv2010.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
11591
[testuser#local ~]$ ssh testuser#syslog.server less srv{1838,2010}.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv{2010,1838}.example.com/2017/02/21/local1.log |grep 'srv2010' | wc -l
11591
Could someone help me, how can i parse\count command arguments to make it work?
Thank you and have a nice day!
The number of arguments for a bash script would be $#. As a quick example:
#!/bin/bash
narg=$#
typeset -i i
i=1
while [ $i -le $narg ] ; do
echo " $# $i: $1"
shift
i=$i+1
done
gives, for bash tst.sh a b {c,d}
4 1: a
3 2: b
2 3: c
1 4: d
In your script, the command to execute (cat, less, ...) gets explicitly only the second argument to the script. If you want to read all arguments, you should do something like this (note: only a hint, removed all sorts of checks etc..)
command="$1"
shift
case $command in
(grep) pattern="$1"
shift
while [ $# -gt 0 ] ; do
grep "$pattern" "$1"
shift
done
;;
esac
note: added some quotes as comment suggested, but, being only a hint, you should carefully look at quoting and your checks in your own script.
Less command working now:
case "$COMMAND" in
less)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
FILES_LIST=${#:2}
FILE=(${FILES_LIST//\.\./})
for v in "${FILE[#]}";do
v=${v//[;\']/}
if [ ! -f $v ]; then
echo "File doesn't exist"
fi
/usr/bin/less $PATH/$v
done;;
tail command works too with 2 and more files, but i can't execute tail -f command on two files unfortunately.

Bash Script not detecting failed exit codes

I can't get my bash script (a logging file) to detect any other exit code other than 0, so the count for failed commands isn't being incremented, but the successes is incremented regardless of whether the command failed or succeeded.
Here is the code:
#!/bin/bash
#Script for Homework 8
#Created by Greg Kendall on 5/10/2016
file=$$.cmd
signal() {
rm -f $file
echo
echo "User Aborted by Control-C"
exit
}
trap signal 2
i=0
success=0
fail=0
commands=0
read -p "$(pwd)$" "command"
while [ "$command" != 'exit' ]
do
$command
((i++))
echo $i: "$command" >> $file
if [ "$?" -eq 0 ]
then
((success++))
((commands++))
else
((fail++))
((commands++))
fi
read -p "$(pwd)" "command"
done
if [ "$command" == 'exit' ]
then
rm -f $file
echo commands:$commands "(successes:$success, failures:$fail)"
fi
Any help would be greatly appreciated. Thanks!
That's because echo $i: "$command" is succeeding always.
The exit status $? in if [ "$?" -eq 0 ] is actually the exit status of echo, the command that is run immediately before the checking.
So do the test immediate after the command:
$command
if [ "$?" -eq 0 ]
and use echo elsewhere
Or if you prefer you don't need the $? check at all, you can run the command and check status within if alone:
if $command; then .....; else ....; fi
If you do not want to get the STDOUT and STDERR:
if $command &>/dev/null; then .....; else ....; fi
** Note that, as #Charles Duffy mentioned in the comment, you should not run command(s) from variables.
Your code is correctly counting the number of times that the echo $i: "$command" command fails. I presume that you would prefer to count the number of times that $command fails. In that case, replace:
$command
((i++))
echo $i: "$command" >> $file
if [ "$?" -eq 0 ]
With:
$command
code=$?
((i++))
echo $i: "$command" >> $file
if [ "$code" -eq 0 ]
Since $? captures the exit code of the previous command, it should be placed immediately after the command whose code we want to capture.
Improvement
To make sure that the value of $? is captured before any other command is run, Charles Duffy suggests placing the assignment on the same line as the command like so:
$command; code=$?
((i++))
echo $i: "$command" >> $file
if [ "$code" -eq 0 ]
This should make it less likely that any future changes to the code would separate the command from the capture of the value of $?.

Unix shell script errors [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
Below script is to push file to remote location through sftp,i faced lot of issues to write below code.But still i am facing some issue,Please guid me to resolve the issues.It's not working with sh.it is only working with ksh.
#test script
#-------------------------------------------------------------------
#!/bin/sh
#------------------------------------------------------------------------
# sftp_file_uploads.sh
#------------------------------------------------------------------------
export REMOTE_SERVER_PROD='192.168.0.1'
export REMOTE_SERVER_FAILOVER='192.168.0.2'
export SFTP_PORT='0001'
export SOURCE_FUNCTIONAL_ID='testusr'
export SOURCE_FILE_DIRECTORY='/var/temp/files/'
export SOURCE_ARCHIVE_DIRECTORY='/var/temp/files/archive'
export DATE_FORMAT=`date "+%Y%m%d"`
export LOG_DIRECTORY='/var/temp/logs'
export DESTINATION_FILE_DIRECTORY='/dest'
export LOG_FILE='$LOG_DIRECTORY/test_$DATE_FORMAT.log'
export SFTP_BATCH_FILE='/var/tmp/SFTP_BATCH_FILE'
#------------------------------------------------------------------------
# Find if the files are available at the source directory.
#------------------------------------------------------------------------
cd $SOURCE_FILE_DIRECTORY
export FILE_TO_UPLOAD_TESTD=`ls -lrt TESTD$DATE_FORMAT.csv | awk '/TESTD/{ f=$NF };END{ print f }'`
export FILE_TO_UPLOAD_TESTDF=`ls -lrt TESTDF$DATE_FORMAT.csv | awk '/TESTDF/{ f=$NF };END{ print f }'`
#------------------------------------------------------------------------
# Try 2 times and Sleep for 5 mins if either of the files is not present
#------------------------------------------------------------------------
counter=0
flag_file_found_TESTD=0
flag_file_found_TESTDF=0
while [ $counter –lt 2 ]
do
#---------------------------
# Check TESTD file arrived
#---------------------------
if [ -z $FILE_TO_UPLOAD_TESTD ] then
echo “No TESTD file to transfer. Sleeping for 5 mins” >> $LOG_FILE
sleep 300
else
echo “TESTD file found to transfer.” >> $LOG_FILE
flag_file_found_TESTD=1
fi
#---------------------------
# Check TESTDF file arrived
#---------------------------
if [ -z $FILE_TO_UPLOAD_TESTDF ] then
echo “No TESTDF file to transfer. Sleeping for 5 mins” >> $LOG_FILE
sleep 300
else
echo “TESTDF file found to transfer.” >> $LOG_FILE
flag_file_found_TESTDF =1
fi
if [[ flag_file_found_TESTD == 1 &&
flag_file_found_TESTDF == 1 ]] then
echo “Both files are found.” >> $LOG_FILE
break
else
echo “At least one of the files is not found. Retrying now.” >> $LOG_FILE
fi
counter=`expr $counter + 1`
done
if [[ flag_file_found_TESTD == 1 &&
flag_file_found_TESTDF == 1 ]] then
echo “Both files are found.”
break
else
if [ flag_file_found_TESTD == 0 ] then
echo “test file is not found and two attempts completed. Cannot transfer the file for today.” >> $LOG_FILE
fi
if [flag_file_found_TESTDF == 0 ] then
echo “test1 file is not found and two attempts completed. Cannot transfer the file for today.” >> $LOG_FILE
fi
fi
#------------------------------------------------------------------------
# Create sftp script
#------------------------------------------------------------------------
rm -f $SFTP_BATCH_FILE
echo "lcd $SOURCE_FILE_DIRECTORY " > $SFTP_BATCH_FILE
echo "cd $DESTINATION_FILE_DIRECTORY " >> $SFTP_BATCH_FILE
if [ -z $FILE_TO_UPLOAD_TESTD ] then
echo "put $FILE_TO_UPLOAD_TESTD " >> $SFTP_BATCH_FILE
fi
if [ -z $FILE_TO_UPLOAD_TESTDF ] then
echo "put $FILE_TO_UPLOAD_TESTDF " >> $SFTP_BATCH_FILE
fi
echo "bye" >> $SFTP_BATCH_FILE
#------------------------------------------------------------------------
# Do sftp
#------------------------------------------------------------------------
echo " Before SFTP " >> $LOG_FILE
if [[ -z $ FILE_TO_UPLOAD && -z $ FILE_TO_UPLOAD1 ]] then
echo “No files to transfer” >> $LOG_FILE
mv $LOG_FILE $LOG_DIRECTORY
exit 1
else
echo “Attempting to connect to Remote Server $REMOTE_SERVER_PROD” >> $LOG_FILE
/usr/bin/sftp –v -oPort=$SFTP_PORT -b $SFTP_BATCH_FILE $SOURCE_FUNCTIONAL_ID#$REMOTE_SERVER_PROD >> $LOG_FILE 2 >> $LOG_FILE
fi
result=$?
errorConnectToProd=0
if [ $result -eq 0 ]
then
echo "SFTP completed successfully to Prod Remote Server" >> $LOG_FILE
else
errorConnectToProd=1
if [[ $result -eq 4 || $result -eq 5 ]]
echo "FAILED to connect to Server. " >> $LOG_FILE
else
echo "FAILED to SFTP to Remote Server. " >> $LOG_FILE
fi
fi
if [ errorConnectToProd == 1 ] then
echo “Attempting to connect to FAILOVER Remote Server $REMOTE_SERVER_FAILOVER” >> $LOG_FILE
/usr/bin/sftp –v -oPort=$SFTP_PORT -b $SFTP_BATCH_FILE $SOURCE_FUNCTIONAL_ID#$REMOTE_SERVER_FAILOVER >> $LOG_FILE 2 >> $LOG_FILE
fi
result=$?
if [ $result -eq 0 ]
then
echo "SFTP completed successfully to Failover Remote Server" >> $LOG_FILE
else
echo "FAILED to SFTP to Failover Remote Server. " >> $LOG_FILE
mv $LOG_FILE $LOG_DIRECTORY
exit 1
fi
fi
cd $SOURCE_FILE_DIRECTORY
mv $FILE_TO_UPLOAD_TESTD $SOURCE_ARCHIVE_DIRECTORY
echo “Moved $FILE_TO_UPLOAD_TESTD to archive direcotry.” >> $LOG_FILE
mv $FILE_TO_UPLOAD_TESTDF $SOURCE_ARCHIVE_DIRECTORY
echo “Moved $FILE_TO_UPLOAD_TESTDF to archive direcotry.” >> $LOG_FILE
rm -f $SFTP_BATCH_FILE
echo “Deleted the SFTP Batch file.” >> $LOG_FILE
echo “Upload completed.” >> $LOG_FILE
mv $LOG_FILE $LOG_DIRECTORY
exit 0
Getting below Errors:
test.ksh[41]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[55]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[56]: flag_file_found_TESTDF: not found
test.ksh[65]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[41]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[55]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[56]: flag_file_found_TESTNDF: not found
test.ksh[65]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[79]: [flag_file_found_TESTDF: not found
rm: /var/tmp/SFTP_BATCH_FILE is a directory
test.ksh[89]: /var/tmp/SFTP_BATCH_FILE: cannot create
test.ksh[90]: /var/tmp/SFTP_BATCH_FILE: cannot create
test.ksh[97]: B: not found
test.ksh[98]: B: not found
test.ksh[99]: B: not found
test.ksh[100]: B: not found
test.ksh[101]: B: not found
test.ksh[102]: B: not found
test.ksh[106]: /var/tmp/SFTP_BATCH_FILE: cannot create
test.ksh[113]: $LOG_DIRECTORY/test_$DATE_FORMAT.log: cannot create
test.ksh[114]: syntax error at line 114 : `FILE_TO_UPLOAD' unexpected
Regards,
Chai
This line is wrong:
export LOG_FILE='$LOG_DIRECTORY/test_$DATE_FORMAT.log'
It should use double quotes, so that the variables will be expanded:
export LOG_FILE="$LOG_DIRECTORY/test_$DATE_FORMAT.log"
Another error:
if [flag_file_found_TESTDF == 0 ] then
needs a space after [. [ is a command (it's a synonym for test), and all commands are separated from their arguments by spaces.
The whole section labeled "Create sftp script" is failing because /var/tmp/SFTP_BATCH_FILE already exists and is a directory; rm -f won't delete a directory, you need to use rm -rf.
if [[ flag_file_found_TESTD == 1 &&
flag_file_found_TESTDF == 1 ]] then
is missing the $ before the variable names.
if [[ -z $ FILE_TO_UPLOAD && -z $ FILE_TO_UPLOAD1 ]] then
Get rid of the space after $.
UPDATE 2:
In all your if statements, you're missing the ; (or newline) before then.
I'm not sure what's causing all the "B: not found" errors. But after you fix all the other errors, maybe it will go away or be easier to find.

Resources