I have written a shell script with name process.sh which calls the sftp script del.sh inside the loop. But once the sftp script(del.sh) is called, it is not coming back to process.sh
Process.sh
#!/bin/bash
for i in $(ls *.csv)
do
fbname=$(basename "$i" .csv)
echo "$fbname"
if [ -f $fbname.done ]
then
gzip $fbname.csv
exec /home/amithsa/del.sh $fbname.csv
else
echo "File Not Found"
fi
done;
del.sh
#!/usr/bin/expect
set arg1 [lindex $argv 0]
spawn sftp xyz#10.10.10.10 22
expect "password:"
send "xxxxxxxxx\n"
expect "sftp>"
send "cd sftp\n"
expect "sftp>"
send "rm $arg1\n"
expect "sftp>"
send "rm *.html\n"
expect "sftp>"
send "exit\n"
interact
Don't use exec if you want the script to continue. exec will completely replace the current process with the command you give it. It never returns1.
Just call your script like you call gzip and echo and the other commands, i.e. directly.
gzip $fbname.csv
/home/amithsa/del.sh $fbname.csv
1 Unless the command cannot be executed and, and you have the execfail option enabled. Or you're running an interactive shell.
Related
Below is the script I have. Basically I just want to copy files from the other server by calling out this script. Some files are large and what happens is that it will kill the first rsync command before it completes and proceed with the next. I tried to use screen command but I'm not sure how to code Ctrl+a d (to detach) in shell/bash.
HFDIR=/var/opt/ubkp/data/local/prework/hotfixes
RODIR=/var/opt/ubkp/data/local/prework/rollouts
THFDIR=$(ls -t /var/opt/ubkp/data/local | grep hotfix | head -1)
TRODIR=$(ls -t /var/opt/ubkp/data/local | grep rollout | grep -v check | head -1)
user=$(/usr/seos/bin/sewhoami)
if [ $user = "root" ]; then
echo "This script should not be run as the TRUE root user"
echo "Log in so that \"sewhoami\" does not display \"root\" and then execute this script."
exit
else
#list of ROs and HFs
list=/tmp/list.txt
echo -n "Enter Password: "
read -s PWD
# first rsync command
/usr/bin/expect<<EOD
spawn rsync -a $user#server:$HFDIR/* /var/opt/ubkp/data/local/$THFDIR
expect "assword"
send "$PWD\r"
wait $!
expect eof
EOD
# second rsync command
/usr/bin/expect<<EOD
spawn rsync -a $user#server:$RODIR/* /var/opt/ubkp/data/local/$TRODIR
expect "assword"
send "$PWD\r"
expect eof
EOD
fi
exit
Your second rsync will be killed after 10 seconds as that is the default timeout for expect eof. You should add a wait after the send, to wait forever until the process ends.
Also, your should remove the $! in the wait. It is a shell variable, not an expect variable. Fortunately in this case $! is empty because you have not run any commands in the shell in the background with &.
my script
#!/bin/ksh
ysdt=`date -d '-1 day' '+%Y%m%d'`
expect << 'EOS'
spawn sftp id#192.168.61.240:/data01/msc/huawei
expect "Password:"
send "password\n"
expect "sftp>"
send "mkdir $ysdt\n"
expect "sftp>"
send "cd /data01/msc/huawei/$ysdt\n"
expect "sftp>"
send "put /home/med/cdr/bak/hw/mss/prm/$ysdt/b*"
expect "sftp>"
send "bye\n"
EOS
-----error show-----------
can't read "ysdt": no such variable
while executing
"send "mkdir $ysdt\n" "
Your shell isn't expanding the variable ysdt.
If any part of the word passed as delimiter for a here-document is quoted, no expansion occurs.
remove the quoting to have expansion occur.
i.e.
expect << EOS
I have a bash script that is supposed to run periodically. the script should connect to a remote SFTP server and get a file from there.
Since this is a SFTP server I had to use expect with the bash script.
the script runs well when I run it manually but fails when running via crontab.
the problematic function is the get_JSON_file()
please advise...
this is the code:
#!/bin/bash
export xxxxx
export xxxxx
export PATH=xxxxx
check_if_file_is_open(){
while :
do
if ! [[ `lsof | grep file.txt` ]]
then
break
fi
sleep 1
done
}
get_JSON_file(){
/usr/bin/expect -f <(cat << EOF
spawn sftp -P port user#ip
expect "Password:"
send "password\r"
expect "$ "
send "get path/to/file/file.json\r"
send "exit\r"
interact
EOF
)
}
get_JSON_file
check_if_file_is_open
cp file.txt /path/to/destination/folder
Expect's interact works only when stdin is on a tty/pty but cron job is not running on tty/pty. So replace interact with expect eof (or expect -timeout 12345 eof if necessary).
That's a very awkward way to pass expect commands to the expect interpreter. Use a (quoted) heredoc instead, and you would drop the -f option for expect
get_JSON_file(){
/usr/bin/expect <<'EOF'
spawn sftp -P port user#ip
expect "Password:"
send "password\r"
expect "$ "
send "get path/to/file/file.json\r"
send "exit\r"
expect eof
EOF
}
The most important tip for debugging expect scripts is to invoke expect's debug output. While you're working out the kinks, use
expect -d <<'EOF'
and in the crontab, you'd want to redirect stderr to stdout so you get the debugging output
* * * * * /path/to/script.sh 2>&1
To run a function within a shell script, no parentheses should be used.
Your code then becomes:
#!/bin/bash
export xxxxx
export xxxxx
export PATH=xxxxx
function check_if_file_is_open(){
while :
do
if ! [[ `lsof | grep file.txt` ]]
then
break
fi
sleep 1
done
}
function get_JSON_file(){
/usr/bin/expect -f <(cat << EOF
spawn sftp -P port user#ip
expect "Password:"
send "password\r"
expect "$ "
send "get path/to/file/file.json\r"
send "exit\r"
interact
EOF
)
}
get_JSON_file
check_if_file_is_open
cp file.txt /path/to/destination/folder
create a new script with screen command and add it in crontab
new_script.sh
#!/bin/bash
cd script_path
screen -dm -S screen_name ./your_script.sh
I have an expect script like this
#!/usr/bin/expect -f
set timeout 30
log_user 0
set PASSWORD $::env(PASSWORD)
set USERNAME $::env(USERNAME)
set TOKEN $::env(TOKEN)
puts stderr "Generating OTP"
spawn oathtool --totp $TOKEN
expect -re \\d+
set otp $expect_out(0,string)
puts stderr "Connecting to VPN server"
spawn -ignore HUP env openconnect -b https://vpn
expect "GROUP:"
send "Tech\n"
expect "Username:"
send "$USERNAME\n"
expect "Password:"
send "$PASSWORD\n"
expect "Password:"
send "$otp\n"
expect EOF
This simple script provides user and password to openconnect to make a new VPN connection in background, but it wont work because the children spawned processes are killed by expect. As you may know, expect will send SIGHUP signal before finish, I was trying to workaround it but even when I put the -ignore HUP flag, it is killing the underlying process, I would like to end my script but the underlying openconnect in background survive.
Do you know what is lacking here?
Take into account that openconnect -b will spawn other PID by its own.
The following method using 2 batch files worked for me:
The -b flag in openconnect is not used and kill command is used instead to send openconnect to background.
contents of file named vpn2:
#!/usr/bin/expect -f
set timeout -1
spawn -ignore HUP -noecho /root/bin/v2vpn2
expect "password"
sleep 3
send -- "my_password\r"
expect "SMS OTP"
interact
expect "Established"
expect eof
contents of file named v2vpn2:
rm /var/log/vpn2.log > /dev/null 2>&1
touch /var/log/vpn2.log
# the word password is printed twice and so filtering here
tail -f /var/log/vpn2.log | grep -m2 -wo "password" | sed '2q;d' &
tail -f /var/log/vpn2.log | grep --color=never -wo "SMS OTP" &
while /bin/true; do
grep -q "Established" /var/log/vpn2.log
if (( $? == 0 )); then
kill -STOP `pgrep openconnect`
kill -CONT `pgrep openconnect`
pkill vpn2
exit
fi
done &
openconnect -u "my_user_name" my_vpn_url >> /var/log/vpn2.log 2>&1
After spending too much time on this, I solved it by adding
expect -timeout -1 -ex "Client killed"
and calling script with &
./vpn.exp &
I need to iterate on a sequence of servers(many type of servers, each type of servers stored in separate files), run on them some commands(stored in different files accordingly to server type) and log the output on the local machine, using ssh with password. Since sshpass, ssh key authentication is not a solution for my case please don't recommend them.
Here is my code:
#!/usr/local/bin/expect -f
#Set path to nodes files
NODES=nodes/*
#Set path to commands files
CMD=commands/*
for fn in $NODES
do
echo "Working in $fn"
for fc in $CMD
do
echo "Working in $fc"
if [ ${fn:6:3} = ${fc:9:3} ]
then
# read Nodes from file
while read fn_line; do
#extracting substrings of user, host, password separated by comma
IFS=', ' read -a uhp <<< $fn_line
#establish ssh session to the node
eval spawn ssh -oStrictHostKeyChecking=no -oCheckHostIP=no ${uhp[0]}#${uhp[1]}
echo ${uhp[0]} ${uhp[1]} ${uhp[2]}
#use correct prompt
set prompt "assword:*"
interact -o -nobuffer -re $prompt return
send "${uhp[2]}\r"
set prompt ":|#|\\\$"
interact -o -nobuffer -re $prompt return
#execute and logging HC commands on the node
while read fc_line; do
#set prompt ":|#|\\\$"
#interact -o -nobuffer -re $prompt return
echo "$fc_line\r" >> logs/${fn:6:3}.log
$fc_line\r >> logs/${fn:6:3}.log
#interact -o -nobuffer -re $prompt return
done < $fc
done < $fn
fi
#cat $f
done
done
I know in my code the problem is combination of bash and expect interpreter. Please help me to do it only in expect style or show me how can i combine bash with expect. Other problem is the while after establishing of ssh connection, but i think it can be solved by storing it previously in an array and looping through it after establishing of ssh connection.
How about writing a small utility in expect which spawns ssh command:
#!/usr/bin/expect
set HOST [lindex $argv 0]
set PORT [lindex $argv 1]
set USER [lindex $argv 2]
set PASSWORD [lindex $argv 3]
set COMMAND [join [lrange $argv 4 end] " "]
spawn ssh -oStrictHostKeyChecking=no -oCheckHostIP=no -p $PORT $USER#$HOST $COMMAND
expect "assword:"
send "$PASSWORD\r"
expect eof
exit
and using it in a Bash script like this:
ssh-util <host> <port> <user> <pass> <command>
e.g.
ssh-util 10.0.0.10 22 root s3cr3t ls -la