I'm using expect to execute a bunch of commands in a remote machine. Then, i'm calling the expect script from a shell script.
I don't want the expect script to log to stdout the sent commands but i want it to log the output of the commands, so my shell script can do other things depending on that results.
log_user 0
Hides both the commands and the results, so it doesn't fit my needs. How can i tell expect to log the results?
Hmm... I'm not sure you can do that, since the reason for seeing the commands you send is because the remote device echoes them back to you. This is standard procedure, and is done so that a user sees what he or she types when interacting with the device.
What I'm trying to say is that both the device output to issued commands, and the echoed-back commands, are part of the spawned process's stdout, therefore I don't believe you can separate one from the other.
Now that I think of it, I think you can configure a terminal to not display echoed commands... but not sure how you would go about doing that with a spawned process that is not using an interactive terminal.
Let us know if you find a way, I'd be interested of knowing if there is one.
Related
I have written a Linux C program, which runs on an embedded processor, and which behaves like a shell -- either interactively, giving a prompt, parsing user commands, and executing them, in an indefinite loop, or non-interactively -- reading and parsing a command off the invocation command. I want to run the same program concurrently on another embedded processor, which is reachable using e/net (e.g. ssh), and use it for some commands because the second processor has access to some hardware that the first processor does not. I usually need to capture and process the outputs from that remote command. Currently, I invoke the program on the second processor for each command -- e.g.
system ("ssh other-cpu my_program "do this command > /vtmp/out_capt");
system ("scp other-cpu:/vtmp/out_capt .")
This works, but it is slow. Is there a better way, perhaps using pipes? If someone could point me to their pick of a best way to do this kind of IPC I would appreciate it.
You could get rid of scp and just save the output from ssh on the local machine. Something like this:
ssh other-cpu '( my_program command )' > file.log
Or if you want to run multiple commands:
ssh other-cpu > file.log << EOF
my_program command
my_program other_command
EOF
There are a few ways to do this, with varying speed and complexity. [Naturally :-)], the fastest requires the most setup.
(1) You can replace your two command sequence with an output pipe.
You create a single pipe via pipe(2). You do a fork(2). The child attaches the output fildes of the pipe to stdout. The child does execvp("ssh","remote","my_program","blah"). The parent reads the results from the input fildes of the pipe. No need for a temp file.
This is similar to what you're currently doing in that you do an ssh for each remote command you want to execute, but eliminating the temp file and the scp.
(2) You can modify my_program to accept various commands from stdin. I believe you already do this in the program's "shell" mode.
You create two pipes via pipe(2). Again, fork a child. Attach the output fildes of the "from_remote" pipe as before to stdout. But, now, attach the input fildes of the "to_remote" pipe to stdin.
In the parent, using the fildes for output of the "to_remote" pipe, send a command line. The remote reads this line [via the input fildes of the "to_remote" pipe, parses it the way a shell would, and fork/execs the resulting command.
After the child program on the remote has terminated, my_program can output a separater line.
The parent reads the data as before until it sees this separater line.
Now, any time the local wants to do something on the remote, the pipes are already set up. It can just write subsequent commands to the output fildes of the "to_remote" pipe and repeat the process.
Thus, no teardown and recreation required. Only one ssh needs to be set up. This is similar to setting up a server with a socket, but we're using ssh and pipes.
If the local wishes to close the connection, it can close the pipe on its end [or send a (e.g.) "!!!stop" command]
If your remote target commands are text based, the separater is relatively easy (i.e. some string that none of your programs would ever output like: _jsdfl2_werou_tW__987_).
If you've got raw binary data, my_program may have to filter/encapsulate the data in some way (e.g. similar to what the PPP protocol does with its flag character)
(3) You can create a version of my_program (e.g. my_program -server) that acts like a server that listens on a socket [in shell mode].
The "protocol" is similar to case (2) above, but may be a bit easier to set up because a network socket is inherently bidirectional (vs. the need for two pipe(2) calls above).
One advantage here is that you're communicating directly over a TCP socket, bypassing the overhead of the encryption layer.
You can either start the remote server at boot time or you can use a [one-time] ssh invocation to kick it off into the background.
There is one additional advantage. Instead of the "separater line" above, the local could make a separate socket connection to the server for each command. This is still slower than above, but faster than creating the ssh on each invocation.
I have a program that takes standard input from the user and runs through the command line. Is there someway to make a program ignore pipes and redirects?
For example: python program.py < input.txt > output.txt would just act as if you put in python program.py
There is no simple way to find the terminal the user launched you with in the general case. There are some techniques you can use, but they will not always work.
You can use os.isatty() to detect whether a file (such as sys.stdin or sys.stdout) appears to be an interactive terminal session. It is possible you are hooked up to a terminal session other than the one the user used to launch your program, so this is not foolproof. Such a terminal session might even be under the control of a program rather than a human.
Under Unix, processes have a notion of a "controlling terminal." You may be able to talk to that via os.ctermid(). But the user can manipulate this value before launching your process. You also may not have a controlling terminal at all, e.g. if running as a daemon.
You can inspect the parent process and see if any of its file descriptors are hooked up to terminal sessions. Unfortunately, I'm not aware of any cross-platform way to do that. On Linux, I'd start with os.getppid() and the /proc filesystem (see proc(5)). If the parent process has exited (e.g. the user ran your_program.py & disown; exit under bash), this will not work. But in that case, there isn't much you can do anyway.
I ever tried to input password by I/O redirection like echo <password> | ssh <user>#<host>, but it didn't work of course. Then I got that ssh actually reads password directly from /dev/tty instead of STDIN, so I/O redirection doesn't work for it.
As far as I know, Expect script is the standard way for this kind of job. I'm curious about how Expect works? I guess it runs the target program in a child process, and it changes the /dev/tty of the child process to refer to another place, but I don't know how.
It uses something called a pseudo-TTY which looks to the called program like a TTY, but allows for programmed control. See e.g. Don Libes' Exploring Expect p498f
I have an embedded application that I want a simple-minded logger for.
The system starts from a script file, which in turn runs the application. There could be various reasons that the script fails to run the application, or the application itself could fail to start. To diagnose this remotely, I need to view the stdout from the script and the application.
I tried writing a tee-like logger that would repeat its stdin to stdout, and save the text in a FIFO for later retrieval via the network. Then I naively tried
./script | ./logger
I ended up with only the script stdout going to the logger, and the application stdout disappearing. I had similar results trying tee.
The system is running kernel 2.4.26, and busybox.
What is going on, and how can I accomplish my desired ends?
It turns out it was working exactly as I thought it should work, with one minor gotcha. stdout was being buffered, and without any fflush(stdout) commands, I never saw it. Had I been really patient, I would have suddenly seen a big gush of output when the stdout buffer filled up. A call to setlinebuf(3) fixed my problem.
Apparently, the application output doesn't end up on stdout...
The output is actually on stderr (which is usually also connected to the terminal)
./script.sh 2>&1 | ./logger
should then work
The application actively disconnects from stdin/stdout (e.g. by closing/reopening file descriptors 0,1(,2) or, using nohup, exec or similar utilities)
the script daemonizes (which also detaches from all standard streams)
Currently I am working with a embedded system that has the Linux OS. I need to run multiple application at the same time, and I would like them to be able to run through one script. A fellow colleague already had implemented this by using a wrapper script and return codes.
wrapperScript.sh $command & > output_log.txt
wrapperScript.sh $command2 & >output_log2.txt
But the problem arises in when exiting the application. Normally all the application that are on the embedded system require a user to press q to exit. But the wrapper script rather than doing that when it gets the kill signal or user signal, it just kill the process. This is dangerous because the wrapper script assumes that the application has the proper facilities to deal with the kill signal (that is not always the case and leads to memory leaks and unwanted socket connections). I have looked into automating programs such as expect but since I am using an embedded board, I am unable to get expect for it. Is there a way in the bash shell or embedded C to deal with multiple process have one single program automatically send the q signal to the programs.
I also would like the capability to maintain log and the output of the files.
EDIT:
Solution:
Okay I found the issue to the problem, Expect is the way to go about it in any situation. There is a serious limitation that it might slower, but the trade off is not bad in this situation. I decided to use Expect Scripting Language to implement the solution. There are certain trade off.
Pros:
* Precise control over embedded application
* Can Make Process Interactive to User
* can Deal with Multiple Process
Cons:
* Performance is slow
Use a pipe
Make the command read input from a named pipe. You'll then be able to send it commands from anywhere.
mkfifo command1.ctrl
{ "$command1" <command1.ctrl >command1.log 2>&1;
rm command1.ctrl; } &
Use screen
Run your applications inside the Screen program. You can run all your commands in separate windows in a single instance of screen (you'll save a little memory that way). You can specify the commands to run from a Screen configuration file:
sessionname mycommands
screen -t command1 command1
screen -t command2 command2
To terminate a program, use
screen -S mycommands -p 1 -X stuff 'q
'
where 1 is the number of the window to send the input to (each screen clause in the configuration file starts a window). The text after stuff is input to send to the program; note the presence of a newline after the q (some applications may require a carriage return instead; you can get one with stuff "q$(printf \\015)" if your shell isn't too featured-starved). If your command expects a q with no newline at all, just stuff q.
For logging, you can use Screen's logging feature, or redirect the output to a file as before.