I need to parse resulting data from a telnet/ssh command and act on the data.
As an example, I want to interact with a spawn session (ssh here), list files in current dir and collect only file of a certain extension to later execute a command on those files only.
What I've got so far:
#!/usr/bin/expect
set timeout 3
match_max 10000
set prompt {$ }
spawn ssh $user#$host
expect "password: "
send $pw\r
expect $prompt
# here's the command I need to parse resulting data
send "ls -1\r"
expect -re {(.*)\.log} {
set val $expect_out(1,string)
puts "LOG file: $val"
exp_continue
}
That script opens a ssh session, sends the command and displays all the files in current dir (log and others) but I need to process each file matching a given pattern, how can I do this?
script output:
$ DATA:
0_system.log
1_system.log
2_system.log
3_system.log
a.log
a.sh
blah
b.sh
data.csv
First of all... wouldn't ls -1 *.log be easier, and do the trick?
That said, I have found that usually you have to be very careful when using (.*), it can have at times unexpected results.
I would go with "any alphanumeric character plus underscore" (if that works for your filenames) - see my suggested expect block below. Also, keep in mind that by using $expect_out(1,string) you are saving only the filename, without the extension - not sure if that is what you want. If you want the whole thing, $expect_out(0,string) is the way to go in this case.
This will do it:
expect -re "(\[a-zA-Z0-9_\]*)\.log" {
set val $expect_out(1,string)
puts "LOG file: $val"
exp_continue
}
Hope that helps!
James
Corrected Expect probably should be:
expect {
-re {(.*)\.log} {
set val $expect_out(1,string)
puts "LOG file: $val"
exp_continue;
}
}
Related
So I have been trying to learn bash scripting with expect and I'm running into an annoying problem where even after I turn off output by setting log_user to 0, the logout message for when I exit the ssh server is still printed as "logout
Connection to blah.blah.blah closed". Going the hackish way and issuing the exit command inside the script to expect is undesirable, and neglects to print a new line in the terminal, leaving an unsightly terminal prompt behind. How do I suppress or turn off this connection closed message in my expect script? Here's a template of the code for my expect script:
#!/usr/bin/expect -f
#-f flag tells expect to read commands from file
# Set Password Prompt and Password
set PASSP "password:"
set PASSW "your_password_here\n"
# Set SFTP Prompt
set SFTP "sftp> "
# Set Remote Prompt and Project Directories
# The GL server is super weird, so make sure RPROMPT ends in the
# name of the project compilation directory and PROJDIR is the
# relative path (from home but without the tilde) of the main
# project directory.
set RPROMPT "projdir_here]"
set PROJDIR "path_to_projdir_here"
# Set SSH Address
set ADDRESS your_username_here#your_address_here
# Annoying long output turned off
log_user 0
send_user "Logging in...\n"
spawn sftp $ADDRESS
expect $PASSP
send $PASSW
expect $SFTP
send_user "Uploading necessary files...\n"
send "cd $PROJDIR\n"
expect $SFTP
send "put *.cpp .\n"
expect $SFTP
send "put *.h .\n"
expect $SFTP
send "put Makefile .\n"
expect $SFTP
send "exit\n"
spawn ssh $ADDRESS
expect $PASSP
send $PASSW
expect "]"
send "cd $PROJDIR\n"
expect $RPROMPT
send_user "Cleaning remote directory...\n"
send "make clean\n"
# Output turned on temporarily so compiler feedback can be determined
expect $RPROMPT
log_user 1
send_user "Compiling on server...\n"
send "make\n"
# Turn off output and submit (this assumes that your makefile has a 'submit'
# target with the command to submit your file there)
expect $RPROMPT
log_user 0
send_user "\nSubmitting Requisite Files...\n"
send "make submit\n"
expect $RPROMPT
send_user "Quitting...\n"
# I can't figure out how to turn off stderr in expect so that an annoying 'connection closed'
# message is printed even the log_user has been set to 0
send "logout\n"
# Transfer control to user
interact
You can see that at the end of the program, before I use the interact command, that the offending message is printed after "logout\n" is sent and the connection to the server is closed. This happens despite log_user being set to 0 above. How on earth do I turn off stderr output inside my script? I've even looked at the manual pages for expect at expect man page but there didn't seem to be much useful there. I don't understand how to turn off stderr inside an expect script so that this message isn't printed, help would be much appreciated! I really just want the script to be quiet when I'm logging out. Because it's pretty obvious that I issued a logout command, I'm not sure why this output goes to stderr. Is there a command in ssh that won't output a "Connection to...closed" message when the user exits? It would be better if I could just pipe stderr to oblivion or suppress it somehow. The normal ways don't seem to work because this is an expect script.
It is just the incorrect use of log_user. It should be used as,
log_user 0
send "logout\r"
expect eof
log_user 1
Note that I am expecting for eof pattern after sending logout\r, because it will lead to the closure of the connection. Always use \r in place of \n while using the send command.
Read here to know more about the log_user to suppress the output.
Instead of having a prompt to defined as a static string, it can be generalized as,
# We escaped the `$` symbol with backslash to match literal '$'
# The last dollar sign represents line-end.
set prompt "#|>|%|\\\$ $";
While the expect is used, we have to accompany with -re flag to specify that as a regular expression.
expect -re $prompt
SshDemo.exp
#!/usr/bin/expect
set prompt "#|>|\\\$ $"
spawn ssh dinesh#myserver
expect {
"(yes/no)" { send "yes\r";exp_continue}
"password"
}
send "welcome123\r"
expect -re $prompt
# Simply executing 'ls -l' command...
send "ls -l\r"
expect -re $prompt
log_user 0
send "logout\r"
expect eof
log_user 1
Here's my new, improved script based on the suggestions above. I also decided to use scp instead of sftp because it's much more concise and I'm only uploading files.
#!/usr/bin/expect -f
#-f flag tells expect to read commands from file
# Set Password Prompt and Password
set PASSP "password: "
set PASSW "your_password_here"
# Set Remote Prompt and Project Directories
# The ssh server is super weird, so make sure PROJDIR is the
# directory name of the main project directory and PATH is
# the relative path to the project directory from home (with the tilde)
set PROJDIR "your_projdir_name_here"
set PATH "your_projdir_path_here"
# Set SSH Address
set ADDRESS your_username_here#source_hostname_here
# Turn off annoying output
log_user 0
send_user "Logging in and uploading necessary files...\n"
# upload files via scp, (shows upload prompt for user, concise)
spawn bash -c "scp *.cpp *.h Makefile $ADDRESS:$PATH"
expect $PASSP
send "$PASSW\r"
interact
spawn ssh $ADDRESS
expect $PASSP
send "$PASSW\r"
expect "]"
send "cd $PATH\r"
expect "$PROJDIR]"
send_user "\nCleaning remote directory...\n"
send "make clean\r"
# Output turned on temporarily so compiler feedback can be determined
expect "$PROJDIR]"
log_user 1
send_user "Compiling on server...\n\n"
send "make\r"
# Turn off output and submit (this assumes that your makefile has a 'submit'
# target with the command to submit your file there)
expect "$PROJDIR]"
log_user 0
send_user "\n\nSubmitting Requisite Files...\n"
send "make submit\r"
expect "$PROJDIR]"
send_user "Quitting...\n"
send "logout\r"
expect eof
# Exit
exit
I am trying to use a FOR loop to iterate over IP addresses (in a bash array), logs in, runs a script and then exits. The array is called ${INSTANCE_IPS[#]}. The following code doesn't work though, as expect doesn't seem to be able to accept the variable $instance.
for instance in ${INSTANCE_IPS[#]}
do
echo $instance
/usr/bin/expect -c '
spawn ssh root#$instance;
expect "?assword: ";
send "<password>\r";
expect "# ";
send ". /usr/local/bin/bootstrap.sh\r";
expect "# ";
send "exit\r" '
done
However, expect complains with:
can't read "instance": no such variable
while executing
"spawn ssh root#$instance"
There is another question on stackoverflow located here, that uses environmental variables to achieve this, however it doesn't allow me to iterate through different IP addresses like I can in an array.
Any help is appreciated.
Cheers
The problem is with quoting. Single quotes surrounding the whole block don't let Bash expand variables ($instance).
You need to switch to double quotes. But then, double quotes inside double quotes are not allowed (unless you escape them), so we are better off using single quotes with expect strings.
Try instead:
for instance in ${INSTANCE_IPS[#]}
do
echo $instance
/usr/bin/expect -c "
spawn ssh root#$instance;
expect '?assword: ';
send '<password>\r';
expect '# ';
send '. /usr/local/bin/bootstrap.sh\r';
expect '# ';
send 'exit\r' "
done
for instance in ${INSTANCE_IPS[&]} ; do
echo $instance
/usr/bin/expect -c '
spawn ssh root#'$instance' "/usr/local/bin/bootstrap.sh"
expect "password:"
send "<password>\r"
expect eof'
done
From the ssh man page:
If command is specified, it is executed on the remote host instead of a login shell.
Specifying a command means expect doesn't have to wait for # to execute your program, then wait for another # just to send the command exit. Instead, when you specify a command to ssh, it executes that command; it exits when done; and then ssh automatically closes the connection.
Alternately, put the value in the environment and expect can find it there
for instance in ${INSTANCE_IPS[&]} ; do
echo $instance
the_host=$instance /usr/bin/expect -c '
spawn ssh root#$env(the_host) ...
Old thread, and one of many, but I've been working on expect for several days. For anyone who comes across this, I belive I've found a doable solution to the problem of passing bash variables inside an expect -c script:
#!/usr/bin/env bash
password="TopSecret"
read -d '' exp << EOF
set user "John Doe"
puts "\$user"
puts "$password"
EOF
expect -c "$exp"
Please note that escaping quotations are typically a cited issue (as #Roberto Reale stated above), which I've solved using a heredoc EOF method, before passing the bash-variable-evaluated string to expect -c. In contrast to escaping quotes, all native expect variables will need to be escaped with \$ (I'm not here to solve all first-world problems--my afternoon schedule is slightly crammed), but this should greatly simplify the problem with little effort. Let me know if you find any issues with this proof of concept.
tl;tr: Been creating an [expect] daemon script with user authentication and just figured this out after I spent a whole day creating separated bash/expect scripts, encrypting my prompted password (via bash) with a different /dev/random salt each iteration, saving the encrypted password to a temp file and passing the salt to the expect script (highly discouraging anyone from easily discovering the password via ps, but not preventative since the expect script could be replaced). Now I should be able to effectively keep it in memory instead.
I have a very small expect script . The flow is as follows
ssh to remote machine
Execute a command and after executing the command you will be in a different prompt ( prompt name is Enter cmd>.
On the Enter cmd> prompt, I need to run many commands
Hence I designed this script
#!/usr/bin/expect
set FULL_CMD { #cmd1 #cmd2 #cmd3 }
puts "STARTING......."
log_file myfile.log ;# <<< === append output to a file
spawn ssh tempuser\#dummyserver
match_max 100000000
expect "password:"
send "temppasswd\r"
expect "*temp*"
send "cd \/home\r"
expect "*temp*"
send "new cmd prompt\r"
expect "Enter cmd>"
foreach tempcmd $FULL_CMD {
send "${tempcmd} \r "
expect -exact "Enter cmd>\r"
send -- "\r"
expect eof }
send "q"
send "exit\r"
puts "I HAVE ENDED......."
My problem: Actually the O/P of first command cmd#1 is very long and I can see that the expect script is not waiting for the O/P of the first command complete and is sending the second command cmd#2 after some time.
This script works fine on systems where O/P of cmd#1 is small, but on systems where the the O/P of cmd#1 is very large (say 1000000 lines), it has trouble, i.e. it issues the first command cmd#1 and the O/P follows, but before it get back to Enter cmd> prompt, the script issues the second command cmd#2
How can I ensure that command cmd#2 is sent only after the output of cmd#1 completes?
Try setting a timeout at the start:
set timeout 60
or use -1 to wait forever.
The problem lies in your foreach tempcmd $FULL_CMD loop:
foreach tempcmd $FULL_CMD {
send "${tempcmd} \r "
expect -exact "Enter cmd>\r"
send -- "\r"
expect eof }
The problem is that you're expecting "Enter cmd>\r", but you will never get the \r in your output.
What you should expect is "Enter cmd>$", where the $ implies the end of the current command output (and not '\nor '\r' oreof`).
foreach tempcmd $FULL_CMD {
send "${tempcmd} \r "
expect -exact "Enter cmd>$" # Check for spaces after '>' and fix as required
send -- "\r"
# expect eof
}
As for expect eof, it matches the end of the input stream which is connected to the output of the spawned process. It will only match once the spawned process terminates, thus it is of no use here
Why do you have "expect eof" in your for loop?
foreach tempcmd $FULL_CMD {
send "$tempcmd\r"
expect "Enter cmd>"
send -- "\r" # why do you need this?
}
It's smart, while developing your program, to enable debugging: add this near the top: exp_internal 1
I create the following script (expect under ksh ) ,
in order to copy the file data.txt from Linux machine to windows machine
scp process will be automatic by expect , so password question will be answered by expect
the problem is that expect can’t ignore the "\" backslash from the following line
spawn scp /tmp/data.txt ADMIN#192.9.200.17:'c:\'
so from output I see that expect send the line without the "\" backslash
what need to change in my syntax/script so expect will ignore the backslash ?
my script
#!/bin/ksh
PASSORD=secret123
SCP=`cat << EOF
set timeout -1
spawn scp /var/tmp/data.txt ADMIN#192.9.200.17:'c:\'
expect {
")?" { send "yes\r" ; exp_continue }
word: {send $PASSORD\r}
}
expect eof
EOF`
expect -c "$SCP"
.
results ( output )
spawn scp /var/tmp/data.txt ADMIN#192.9.200.17:'c:'
gcs#198.202.183.97's password:
scp: c:: Error opening file: The requested operation failed.
Killed by signal 1.
Use spawn scp /tmp/data.txt ADMIN#192.9.200.17:'c:/' instead as it should evaluate to the same path.
You need to escape \.
Try
spawn scp /tmp/data.txt ADMIN#192.9.200.17:'c:\\'
I'd like to be able to pass in a long command to expect. It's a multiple command somehow. First here's my expect script
#!/usr/bin/expect -f
set timeout -1
spawn telnet xxx.xxx.xxx.xxx
expect "*?username:*"
send "someusername\r"
expect "*?assword:*"
send "somepassword\r"
# Here's the command I'd like to pass from the command prompt
set command [lindex $argv 0]
send "$command\r"
send "exit\r"
I would then run this script as so:
./expectscript "mkdir /usr/local/dir1\ncd /usr/local/dir1\ntouch testfile"
Notice that I put "\n" to initiate an enter as though I'm processing the command before moving to the next.
I know you could separate the commands with ";", but for this particular exercise, I'd like to be able have expect interpret the "\n" with a "\r" so that, expect would behave as though it were like this:
send "mkdir /usr/local/dir1\r"
send "cd /usr/local/dir1\r"
send "touch testfile\r"
The question then becomes how can expect interpret the "\n" to be "\r"? I've tried putting the "\r" in the argument instead of "\n", but that doesn't work.
Thanks for the input.
When I do a simple experiment, I find that the \n in the argument is not converted by my shell (bash) into a newline; it remains a literal. You can check this out for yourself by just using puts to print out the command line argument, like this:
puts [lindex $argv 0]
Working around this requires a little bit of work to split things. Alas, Tcl's split command does not split on multi-character sequences (it splits on many different characters at once instead) so we'll need a different approach. However, Tcllib has exactly what we need: the splitx command. With that, we do this (based on #tensaix2j's answer):
#!/usr/bin/expect -f
package require Expect; # Good practice to put this explicitly
package require textutil::split; # Part of Tcllib
# ... add your stuff here ...
foreach line [textutil::split::splitx [lindex $argv 0] {\\n}] {
send "$line\r"
# Wait for response and/or prompt?
}
# ... add your stuff here ...
If you don't have Tcllib installed and configured for use with Expect, you can also snarf the code for splitx directly out of the code (find it online here) as long as you internally acknowledge the license it's under (standard Tcl licensing rules).
foreach cmd [ split $command \n ] {
send "$cmd\r\n"
}