I found that when I use certain bytes as input to a program in an expect script, then there is an automatic conversion to multibytes when a byte is above 0x7f. For example the following line in the script:
spawn ./myprog [exec perl -e { print "\x7f\x80" }]
sends actually three instead of two bytes to myprog: 0x7f 0xc2 0x80
myprog is a simple test program that prints the input it gets:
int main(int argc, char** argv) {
int i;
for (i=0;i<strlen(argv[1]);i++) {
printf("%x\n", (unsigned char)argv[1][i]);
}
I understand that 0x7f is the magic boundary to unicode-related encodings, but how can I just send a byte like 0x80 to my program? In the expect script I already tried conversions like [encoding convertto iso8859-1 [exec perl ...]] described in https://www.tcl.tk/doc/howto/i18n.html, but nothing works.
On the other hand, when I do the identical thing on the command line, e.g.:
./myprog `perl -e 'print "\x7f\x80"'`
I do get only two bytes - as expected (the differing {} compared to the expect script line is tcl's replacement of '').
How can I force the same behavior in an expect script?
After some more experimentation, I found that the only way to do that is to have the argument handover outside the expect logic, e.g.:
set input [binary format H* 7f80]
exec echo "$input" > input.dat
spawn sh -c "./myprog `cat input.dat`"
Note that using ${...} instead of backticks does not seem to work easily due to the special meaning of $ to expect.
Of course, having spawned a shell instead of the process directly is not the same thing but that does not matter for most of my use cases.
Related
I have a Bash script which generates, stores and modifies values in an array. These values are later used as arguments for a command.
For a MCVE I thought of an arbitrary command bash -c 'echo 0="$0" ; echo 1="$1"' which explains my problem. I will call my command with two arguments -option1=withoutspace and -option2="with space". So it would look like this
> bash -c 'echo 0="$0" ; echo 1="$1"' -option1=withoutspace -option2="with space"
if the call to the command would be typed directly into the shell. It prints
0=-option1=withoutspace
1=-option2=with space
In my Bash script, the arguments are part of an array. However
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2="with space"')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
prints
0=-option1=withoutspace
1=-option2="with space"
which still shows the double quotes (because they are interpreted literally?). What works is
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
which prints again
0=-option1=withoutspace
1=-option2=with space
What do I have to change to make ARGUMENTS+=('-option2="with space"') work as well as ARGUMENTS+=('-option2=with space')?
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
Get rid of the single quotes. Write the options exactly as you would on the command line.
ARGUMENTS+=(-option1=withoutspace)
ARGUMENTS+=(-option2="with space")
Note that this is exactly equivalent to your second option:
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
-option2="with space" and '-option2=with space' both evaluate to the same string. They're two ways of writing the same thing.
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
It's the exact right thing to do. Arrays are perfect for this. Using a flat string would be a mistake.
I want to write an Expect script that will simply input commands into GDB regardless of its output. Then I want to take certain parts of the output of GDB and extract information from it using shell commands such as grep and sed. Then I want to use this information to input more commands into GDB.
For example, I would initiate a back trace by sending the command "bt" to GDB from the expect script. Then I would grep for a word such as "pardrivr" and get the line number associated with it. Then I would input "f lineNumberOfPardrivr" into GDB. This process would be repeated until the correct information is eventually extracted.
Is this possible. If so what is the best way to go about doing this?
Thanks
My $0.02: I'd use a coprocess or named pipe under ksh/bash/zsh. Much easier. See: https://unix.stackexchange.com/questions/86270/how-do-you-use-the-command-coproc-in-bash
Also, consider tee'ing the output of gdb into a named pipe that you cat in another xterm. Makes it much easier to debug what your script is reading if you can see a copy of the gdb output.
Edited to add:
Still can't post comments. *sigh*
gdb in batch mode, or via a simple shell redirect, won't let us define commands on the fly based upon current gdb output. A coprocess or named pipe approach is much the same technique, but it lets us create new input dynamically at will based upon gdb's output processed through grep/sed/awk/perl/whatever. Python or Perl might be even easier to use with their facilities for regular expressions and subprocesses. E.g. (perl) open("|gdb ...")
http://perldoc.perl.org/functions/open.html
Edited again to add:
A named pipe is a FIFO (first in first out) that exists much like a file in the filesystem. It's not really a file of course. It's just something that can be used like a file. Anything that you write to it can be read back out, within the limits of the OS buffering. (Otherwise writes will block.)
FIFO's are available under Unix, Linux, & Macs, but not windows. You create them with mkfifo. Any process can write to it. Any process can read from it. From that link I posted up above:
mkfifo in out
cmd <in >out &
exec 3> in 4< out
echo data >&3
read var <&4
From my own playing around to demo this...
#in BASH
mkfifo IN OUT
#or mkfifo IN OUT ERR
gdb < IN > OUT 2>&1 &
#or gdb < IN > OUT 2> ERR &
#or gdb < IN > OUT &
exec 3> IN
exec 4< OUT
echo "help bt" >&3
while read -t 0.001 var <&4 ; do echo $var; done
echo "help stack" >&3
while read -t 0.001 var <&4 ; do echo $var; done
#don't forget to kill the gdb process when you are done...
echo "quit" >&3
while read -t 0.001 var <&4 ; do echo $var; done
I want to write an Expect script that will simply input commands into GDB regardless of its output.
For non interactive control you don't need expect as gdb has a -batch mode and is able to read (-x) commands from a file.
Moreover, as gdbreads input from stdin and produces output to stdout standard redirection might do the trick.
For example, I wrote a simple C program:
sh$ cat hello.c
#include <stdio.h>
int main() {
char msg[] = "Hello world";
printf("%s\n", msg);
return 0;
}
sh$ gcc -ggdb hello.c -o hello
I'm able to "script" the gdb session like that:
sh$ gdb -q hello | awk '$2=="$1" { print "Var was #" $NF }'
br 6
r
print &msg
c
quit
EOF
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
Var was #0x7fffffffe230
I want my script to read a string either from stdin , if it's piped, or from an argument. So first i want to check if some text is piped and if not it should use an argument as input. My code looks something like this:
value=$(cat) # read from stdin
if [ "$pipe" != "" ]; then #check if pipe is not empty
#Do something with pipe string
else
#Do something with argument string
fi
The problem is when it's not piped, then the script will halt and wait for "ctrl d" and i dont want that. Any suggestions on how to solve this?
Thanks in advance.
/Tomas
What about checking the argument first?
if (($#)) ; then
process "$1"
else
cat | process
fi
Or, just take advantage from the same behaviour of cat:
cat "$#" | process
If you only need to know if it's a pipe or a redirection, it should be sufficient to determine if stdin is a terminal or not:
if [ -t 0 ]; then
# stdin is a tty: process command line
else
# stdin is not a tty: process standard input
fi
[ (aka test) with -t is equivalent to the libc isatty() function.
The above will work with both something | myscript and myscript < infile. This is the simplest solution, assuming your script is for interactive use.
The [ command is a builtin in bash and some other shells, and since [/test with -tis in POSIX, it's portable too (not relying on Linux, bash, or GNU utility features).
There's one edge case, test -t also returns false if the file descriptor is invalid, but it would take some slight adversity to arrange that. test -e will detect this, though assuming you have a filename such as /dev/stdin to use.
The POSIX tty command can also be used, and handles the adversity above. It will print the tty device name and return 0 if stdin is a terminal, and will print "not a tty" and return 1 in any other case:
if tty >/dev/null ; then
# stdin is a tty: process command line
else
# stdin is not a tty: process standard input
fi
(with GNU tty, you can use tty -s for silent operation)
A less portable way, though certainly acceptable on a typical Linux, is to use GNU stat with its %F format specifier, this returns the text "character special file", "fifo" and "regular file" in the cases of terminal, pipe and redirection respectively. stat requires a filename, so you must provide a specially-named file of the form /dev/stdin, /dev/fd/0, or /proc/self/fd/0, and use -L to chase symlinks:
stat -L -c "%F" /dev/stdin
This is probably the best way to handle non-interactive use (since you can't make assumptions about terminals then), or to detect an actual pipe (FIFO) distinct from redirection.
There is a slight gotcha with %F in that you cannot use it to tell the difference between a terminal and certain other device files, for example /dev/zero or /dev/null which are also "character special files" and might reasonably appear. An unpretty solution is to use %t to report the underlying device type (major, in hex), assuming you know what the underlying tty device number ranges are... and that depends on whether you're using BSD style ptys or Unix98 ptys, or whether you're on the actual console, among other things. In the simple case %t will be 0 though for a pipe or a redirection of a normal (non-special) file.
More general solutions to this kind of problem are to use bash's read with a timeout (read -t 0 ...) or non-blocking I/O with GNU dd (dd iflag=nonblock).
The latter will allow you to detect lack of input on stdin, dd will return an exit code of 1 if there is nothing ready to read. However, these are more suitable for non-blocking polling loops, rather than a once-off check: there is a race condition when you start two or more processes in a pipeline as one may be ready to read before another has written.
It's easier to check for command line arguments first and fallback to stdin if no arguments. Shell Parameter Expansion is a nice shorthand instead of the if-else:
value=${*:-`cat`}
# do something with $value
I was trying to log data from RS232 into a file with cat:
cat /dev/ttyS0 > rs232.log
The result was that I had everything in my file except for the last line.
By printing to stdout, I was able to discover, that cat only writes the output if it gets a newline character ('\n'). I discovered the same with:
dd bs=1 if=/dev/ttyS0 of=rs232.log
After reading How can I print text immediately without waiting for a newline in Perl? I was starting to think, if this could be a buffering problem of either the Linux-Kernel or the coreutils package.
According to TJD's comment, I wrote my own program in C but still had the same problems:
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* args[])
{
char buffer;
FILE* serial;
serial = fopen(args[1],"r");
while(1)
{
buffer = fgetc(serial);
printf("%c",buffer);
}
}
As of the results of my own C-Code this seems to be a Linux-Kernel related issue.
You're opening a TTY. When that TTY is in cooked (aka canonical) mode, it performs line processing (e.g. backspace removes the previous character from the buffer). You'll want to put the TTY into raw mode in order to get every single byte when it arrives instead of waiting for the end of line.
From the man page:
Canonical and noncanonical mode
The setting of the ICANON canon flag in c_lflag determines whether
the terminal is operating in canonical mode (ICANON set) or
noncanonical mode (ICANON unset). By default, ICANON set.
In canonical mode:
Input is made available line by line. An input line is available
when one
of the line delimiters is typed (NL, EOL, EOL2; or EOF at the start of
line). Except in the case of EOF, the line delimiter is included in the
buffer returned by read(2).
Line editing is enabled (ERASE, KILL; and if the IEXTEN flag is
set: WERASE,
REPRINT, LNEXT). A read(2) returns at most one line of input; if the
read(2) requested fewer bytes than are available in the current line of
input, then only as many bytes as requested are read, and the remaining
characters will be available for a future read(2).
In noncanonical mode input is available immediately (without the
user having to type a line-delimiter character), and line editing
is disabled.
The simplest thing to do is just to call cfmakeraw.
Does this work?
perl -e 'open(IN, "/dev/ttyS0") || die; while (sysread(IN, $c, 1)) { print "$c" }'
This DOES work:
$ echo -n ccc|perl -e 'while (sysread(STDIN, $c, 1)) { print "$c" } '
ccc$
I'm a little confused about the terminology I should use when referring to Linux command line programs and commands.
If I were to execute a command such as:
mkdir testing_dir
Would "testing_dir" be called an 'operand' to the program mkdir, or an 'argument' or a 'parameter'?
Another question I have is, what terminology would you use to describe the following process?
find *.txt | grep a | grep b
Could I say; the output of the "find" program is piped (redirected) to the input of the grep program?
For the first question, it is an "argument". That is why in C programs, the main prototype is int main(int argc, char** argv). argc means argument count, and argv means argument vector.
For the second, it is "piped". (Because it is done with the pipe | charactor, and/or the data is passed from one program, like it is going through a pipe.) Generally, stdout is only called "redirected" when it is sent to a file with the > operator.
Most people would call it the argument.
Yes, it's piped.
Argument or parameter rather than operand , while mkdir is a command , not a operater
Yes, the output is piped to the input parameter of the next command.
There are no operands to commands, operators are having operands. For commands, they are arguments. Since the mkdir is a command,surely the name is an argument.
And about the second one.. The first output is given into the second command as input. Redirects are usually used with the operators > , >> and 2> 2>>