Run bash scripts from Perl threads - multithreading

Question

My script should have n subroutines (my_proc) to run simultaneously, each of them runs bash script and one sub (check_procs) checks if subs has finished.
use strict;
use threads;
use threads::shared;
my %proc_status :shared;
my %thr;
foreach my $i (1,2,3,4) {
$proc_status{$i}=0;
}
sub my_proc {
my $arg=shift(#_);
while (1) {
sleep(2);
print "Proc $arg Started\n";
#exec("/bin/bash","sleep_for_10_sec.bash") or die("Can't exec"); # case 1
#`sleep_for_10_sec.bash &`; # case 2
print "Proc $arg Finished\n";
{
lock(%proc_status);
$proc_status{$arg}=1;
}
}
}
sub check_procs {
my $all_finished;
while (! $all_finished) {
sleep 5;
print "CHECK: \n";
$all_finished=1;
foreach my $num (1,2,3,4) {
if ($proc_status{$num} == 1) {
print "CHECK: procedure $num has finished\n";
} else {
$all_finished=0;
}
}
}
print "All jobs finished\n";
}
foreach my $num (1,2,3,4) {
$thr{"$num"} = new threads \&my_proc,$num;
}
my $thr_check= new threads \&check_procs;
$thr_check->join();
And here are the sleep_for_10_sec.bash
ls
# bunch of other stuff
sleep 10
echo "finished sleep"
I don't want my_proc subs to wait "sleep_for_10_sec.bash" command to be executed, after browsing I have found that either #case1 or #case2 should work, but they both fail.
the output of #case1:
Proc 1 Started
[ls result]
finsihed sleep
the output of #case2:
Proc 1 Started
Proc 2 Started
Proc 3 Started
Proc 4 Started
CHECK:
CHECK:
Proc 4 Finished
Proc 2 Finished
Proc 3 Finished
Proc 1 Finished
Proc 3 Started
Proc 1 Started
Proc 2 Started
Proc 4 Started
CHECK:
CHECK: procedure 1 has finished
CHECK: procedure 2 has finished
CHECK: procedure 3 has finished
CHECK: procedure 4 has finished
But I expect something like this :
Proc 1 Started
Proc 2 Started
Proc 3 Started
Proc 4 Started
Proc 1 Finished
Proc 1 Started
Proc 3 Finished
Proc 3 Started
Proc 4 Finished
Proc 4 Started
Proc 2 Finished
Proc 2 Started
CHECK:
CHECK:
CHECK:
CHECK: procedure 1 has finished
CHECK: procedure 2 has finished
CHECK: procedure 3 has finished
CHECK: procedure 4 has finished
Actually I get wanted result in case of redirecting output to " > log", but anyway after:
Proc 1 Started
Proc 2 Started
Proc 3 Started
Proc 4 Started
it waits "sleep_for_10_sec.bash" to finish.
This is my first project where I use "thread" and "exec", could someone help me on this ?

Answer 1

exec shouldn't be combined with threads. exec launches a new program within the current process, so when you call exec from one thread, the program the threads were executing disappears. Since the threads would have no program to execute, exec kills the threads as well.
It's not clear to me why case 2 doesn't work (edit: see ikegami's comment below). I would think it would launch the process, run it in the background, and allow the Perl thread to immediately continue. It doesn't seem to do that, but this code will:
system("/bin/bash sleep_for_10_sec.bash &"); # case 3

Answer 2

exec("/bin/bash","sleep_for_10_sec.bash") or die("Can't exec"); # case 1
exec replaces the program running in the current process with another program. At the same time, the existing threads are terminated (since the program they want to execute is no longer there), replaced with a single thread executing the new program.
This means that exec never returns (except on error). Threads or no threads, exec is not what you want, because you don't want your program to stop running.
But I expect something like this:
Are you sure you want to launch sleep_for_10_sec.bash 4 times every two seconds (meaning you can have up to 20 of them running at a time) as your desired output indicates?
Are you sure you don't care if sleep_for_10_sec.bash completes or not as your desired output indicates?
If so, why are you using threads at all? You could simply use the following:
sub start {
my $num = shift;
say "Proc $num Started";
system('bash -c sleep_for_10_sec.bash &');
say "Proc $num Finished";
}
for my $pass (1..2) {
start($_) for 1..4;
sleep 2;
start($_) for 1..4;
sleep 2;
start($_) for 1..4;
sleep 1;
if ($pass == 1) {
say "CHECK:";
} else {
say "CHECK: procedure $_ has finished" for 1..4;
}
}
I think you want
use threads;
use Thread::Queue qw( ); # 3.01+
use constant NUM_WORKERS => 4;
sub worker {
my $num = shift;
say "Job $num Started\n";
system("sleep_for_10_sec.bash"); # Make sure starts with #! and is executable.
say "Job $num Finished\n";
}
{
my $q = Thread::Queue->new();
for (1..NUM_WORKERS) {
while (defined( my $job = $q->dequeue() )) {
worker($job);
}
}
$q->enqueue(1..4, 1..4);
$q->end();
$_->join() for threads->list;
}