Two way communication with process - linux

I have given some compiled program. I want to communicate with it from my bash script by program stdin and stdout. I need two way communication. Program cannot be killed between exchange of information. How I can do that?
Simple example:
Let that program be compiled partial summation (C++) and script results will be squares of that sums. Program:
int main() {
int num, sum = 0;
while(true) {
std::cin >> num;
sum += num;
std::cout << sum << std::endl;
}
}
My script should looks like that:
for i in 1 2 3 4; do
echo "$i" > program
read program to line;
echo $((line * line))
done
If in program I have for(int i = 1; i <= 4; ++i), then I can do something like that:
exec 4< <(./program); # Just read from program
for i in 1 2 3 4; do
read <&4 line;
echo "sh: $((line * line))";
done
For more look here. From the other hand, if in program I have std::cout << sum * sum;, then solution could be:
exec &3> >(./program); # Write to program
for i in 1 2 3 4; do
echo "$i" > &3
done
My problem is two way communication with other process / program. I don't have to use exec. I cannot install third party software. Bash-only solution, without files, will be nice.
If I run other process, it will be nice to know pid to kill that at the end of script.
I think about communication with two or maybe three processes in the future. Output of firs program may dependents on output of second program and also in second side. Like communicator of processes.
However, I cannot recompile programs and change something. I have only stdin and stdout communication in programs.

If you have bash which is newer than 4.0, you can use coproc.
However, don't forget that the input/output of the command you want to communicate might be buffered.
In that case you should wrap the command with something like stdbuf -i0 -o0
Reference: How to make output of any shell command unbuffered?
Here's an example
#!/bin/bash
coproc mycoproc {
./a.out # your C++ code
}
# input to "std::cin >> num;"
echo "1" >&${mycoproc[1]}
# get output from "std::cout << sum << std::endl;"
# "-t 3" means that it waits for 3 seconds
read -t 3 -u ${mycoproc[0]} var
# print it
echo $var
echo "2" >&${mycoproc[1]}
read -t 3 -u ${mycoproc[0]} var
echo $var
echo "3" >&${mycoproc[1]}
read -t 3 -u ${mycoproc[0]} var
echo $var
# you can get PID
kill $mycoproc_PID
output will be
1
3
6
If your bash is older than 4.0, using mkfifo can do the same thing like:
#!/bin/bash
mkfifo f1 f2
exec 4<> f1
exec 5<> f2
./a.out < f1 > f2 &
echo "1" >&4
read -t 3 -u 5 var
echo $var
rm f1 f2

Considering that your C++ program reads from standard output, and prints to standard output, it's easy to put it inside a simple chain of pipes:
command_that_writes_output | your_cpp_program | command_that_handle_output
In your specific case you probably need to modify the program to only handle one single input and writing one single output, and remove the loop. Because then you can do it very simple, like this:
for i in 1 2 3 4; do
result=`echo $i | ./program`
echo $((result * result))
done

Related

How to read stdout from a sub process in bash in real time

I have a simple C++ program that counts from 0 to 10 with an increment every 1 second. When the value is incremented, it is written to stdout. This program intentionally uses printf rather than std::cout.
I want to call this program from a bash script, and perform some function (eg echo) on the value when it is written to stdout.
However, my script waits for the program to terminate, and then process all the values at the same time.
C++ prog:
#include <stdio.h>
#include <unistd.h>
int main(int argc, char **argv)
{
int ctr = 0;
for (int i = 0; i < 10; ++i)
{
printf("%i\n", ctr++);
sleep(1);
}
return 0;
}
Bash script:
#!/bin/bash
for c in $(./script-test)
do
echo $c
done
Is there another way to read the output of my program, that will access it in real time, rather than wait for for the process to terminate.
Note: the C++ program is a demo sample - the actual program I am using also uses printf, but I am not able to make changes to this code, hence the solution needs to be in the bash script.
Many thanks,
Stuart
As you correctly observed, $(command) waits for the entire output of command, splits that output, and only after that, the for loop starts.
To read output as soon as is available, use while read:
./script-test | while IFS= read -r line; do
echo "do stuff with $line"
done
or, if you need to access variables from inside the loop afterwards, and your system supports <()
while IFS= read -r line; do
echo "do stuff with $line"
done < <(./script-test)
# do more stuff, that depends on variables set inside the loop
You might be more lucky using a pipe:
#!/bin/bash
./script-test | while IFS= read -r c; do
echo "$c"
done

Insert input to exe program with linux script

I got a script that asks 1000 times for input of 1-5, it looks like this:
insert1:
insert2:
insert3:
insert4:
insert5:
//and again 1-5
insert 1:
...in total it will get 1000 inputs
I want to write a one line script that will run the script I just described, it will insert the input that needed each time.
this is what I tried:
#!/bin/bash
./my_script.exe -l | for i in {1..200}; do for j in {1..5}; do j; done; done
You are nearly there, but do it the other way around:
for ((i=1;i<=200:i++)) ; do
for ((j=1;j<=5;j++)) ; do
echo $j
done
done | ./myscript.exe -l
You can put a # before the | to comment it out and see what the script sends to your program.
You need to differentiate between parameters which are specified after the program name like this:
program param1 param2 param3
and inputs, which a program gets by reading its stdin and are supplied like this:
printf "input1\ninput2\ninput3\n" | program
Alternative version of second command:
{ echo input1; echo input2; echo input3; } | program

how to redirect this perl script's output to file?

I don't have much experience with perl, and would appreciate any/all feedback....
[Before I start: I do not have access/authority to change the existing perl scripts.]
I run a couple perl scripts several times a day, but I would like to begin capturing their output in a file.
The first perl script does not take any arguments, and I'm able to "tee" its output without issue:
/asdf/loc1/rebuild-stuff.pl 2>&1 | tee $mytmpfile1
The second perl script hangs with this command:
/asdf/loc1/create-site.pl --record=${newsite} 2>&1 | tee $mytmpfile2
FYI, the following command does NOT hang:
/asdf/loc1/create-site.pl --record=${newsite} 2>&1
I'm wondering if /asdf/loc1/create-site.pl is trying to process the | tee $mytmpfile2 as additional command-line arguments? I'm not permitted to share the entire script, but here's the beginning of its main routine:
...
my $fullpath = $0;
$0 =~ s%.*/%%;
# Parse command-line options.
...
Getopt::Long::config ('no_ignore_case','bundling');
GetOptions ('h|help' => \$help,
'n|dry-run|just-print' => \$preview,
'q|quiet|no-mail' => \$quiet,
'r|record=s' => \$record,
'V|noverify' => \$skipverify,
'v|version' => \$version) or exit 1;
...
Does the above code provide any clues? Other than modifying the script, do you have any tips for allowing me to capture its output in a file?
It's not hanging. You are "suffering from buffering". Like most programs, Perl's STDOUT is buffered by default. Like most programs, Perl's STDOUT is flushed by a newline when connected to a terminal, and block buffered otherwise. When STDOUT isn't connected to a terminal, you won't get any output until 4 KiB or 8 KiB of output is accumulated (depending on your version of Perl) or the program exits.
You could add $| = 1; to the script to disable buffering for STDOUT. If your program ends with a true value or exits using exit, you can do that without changing the .pl file. Simply use the following wrapper:
perl -e'
$| = 1;
$0 = shift;
do($0);
my $e = $# || $! || "$0 didn\x27t return a true value\n";
die($e) if $e;
' -- prog args | ...
Or you could fool the program into thinking it's connected to a terminal using unbuffer.
unbuffer prog args | ...

Write output of subprocess launched by a `screen` CLI Command to a log file?

I am launching a bunch of the same script (generate_records.php) into screens. I am doing this to easily parallelize the processes. I would like to write the output of each of the PHP processes to a log file using something like &> log_$i (StdOut an StdErr).
My shell scripting is weak sauce, and I can't get the syntax correct. I keep getting the output of the screen, which is empty.
Exmaple: launch_processes_in_screens.sh
max_record_id=300000000
# number of parallel processors to run
total_processors=10
# max staging companies per processor
(( num_records_per_processor = $max_record_id / $total_processors))
i=0
while [ $i -lt $total_processors ]
do
(( starting_id = $i * $num_records_per_processor + 1 ))
(( ending_id = $starting_id + $num_records_per_processor - 1 ))
printf "\n - Starting processor #%s starting at ID:%s and ending at ID: %s" "$i" "$starting_id" "$ending_id"
screen -d -m -S "process_$i" php generate_records.php "$starting_id" "$num_records_per_processor" "FALSE"
((i++))
done
If the only reason you're using screen is to launch many processes in parallel, you can avoid it entirely and use & to start them in the background:
php generate_records.php "$starting_id" "$num_records_per_processor" FALSE &
You may also be able to remove some code by using parallel.

Bash: Split stdout from multiple concurrent commands into columns

I am running multiple commands in a bash script using single ampersands like so:
commandA & commandB & commandC
They each have their own stdout output but they are all mixed together and flood the console in an incoherent mess.
I'm wondering if there is an easy way to pipe their outputs into their own columns... using the column command or something similar. ie. something like:
commandA | column -1 & commandB | column -2 & commandC | column -3
New to this kind of thing, but from initial digging it seems something like pr might be the ticket? or the column command...?
Regrettably answering my own question.
None of the supplied solutions were exactly what I was looking for. So I developed my own command line utility: multiview. Maybe others will benefit?
It works by piping processes' stdout/stderr to a command interface and then by launching a "viewer" to see their outputs in columns:
fooProcess | multiview -s & \
barProcess | multiview -s & \
bazProcess | multiview -s & \
multiview
This will display a neatly organized column view of their outputs. You can name each process as well by adding a string after the -s flag:
fooProcess | multiview -s "foo" & \
barProcess | multiview -s "bar" & \
bazProcess | multiview -s "baz" & \
multiview
There are a few other options, but thats the gist of it.
Hope this helps!
pr is a solution, but not a perfect one. Consider this, which uses process substitution (<(command) syntax):
pr -m -t <(while true; do echo 12; sleep 1; done) \
<(while true; do echo 34; sleep 2; done)
This produces a marching column of the following:
12 34
12 34
12 34
12 34
Though this trivially provides the output you want, the columns do not advance individually—they advance together when all files have provided the same output. This is tricky, because in theory the first column should produce twice as much output as the second one.
You may want to investigate invoking tmux or screen in a tiled mode to allow the columns to scroll separately. A terminal multiplexer will provide the necessary machinery to buffer output and scroll it independently, which is important when showing output side-by-side without allowing excessive output from commandB to scroll commandA and commandC off-screen. Remember that scrolling each column separately will require a lot of screen redrawing, and the only way to avoid screen redraws is to have all three columns produce output simultaneously.
As a last-ditch solution, consider piping each output to a command that indents each column by a different number of characters:
this is something that commandA outputs and is
and here is something that commandB outputs
interleaved with the other output, but visually
you might have an easier time distinguishing one
here is something that commandC outputs
which is also interleaved with the others
from the other
Script print out three vertical rows and a timer each row containing the output from a single script.
Comment on anything you dont understand and ill add answers to my answer as needed
Hope this helps :)
#!/bin/bash
#Script by jidder
count=0
Elapsed=0
control_c()
{
tput rmcup
rm tail.tmp
rm tail2.tmp
rm tail3.tmp
stty sane
}
Draw()
{
tput clear
echo "SCRIPT 1 Elapsed time =$Elapsed seconds"
echo "------------------------------------------------------------------------------------------------------------------------------------------------------"
tail -n10 tail.tmp
tput cup 25 0
echo "Script 2 "
echo "------------------------------------------------------------------------------------------------------------------------------------------------------"
tail -n10 tail2.tmp
tput cup 50 0
echo "Script 3 "
echo "------------------------------------------------------------------------------------------------------------------------------------------------------"
tail -n10 tail3.tmp
}
Timer()
{
if [[ $count -eq 10 ]]; then
Draw
((Elapsed = Elapsed + 1))
count=0
fi
}
main()
{
stty -icanon time 0 min 0
tput smcup
Draw
count=0
keypress=''
MYSCRIPT1.sh > tail.tmp &
MYSCRIPT2.sh > tail2.tmp &
MYSCRIPT3.sh > tail3.tmp &
while [ "$keypress" != "q" ]; do
sleep 0.1
read keypress
(( count = count + 2 ))
Timer
done
stty sane
tput rmcup
rm tail.tmp
rm tail2.tmp
rm tail3.tmp
echo "Thanks for using this script."
exit 0
}
main
trap control_c SIGINT

Resources