Can I reuse joined threads in perl? - multithreading

Question

I have a module that is running multiple threads and pushing them onto a list of threads.
ex:
#!/usr/bin/perl
#test_module.pm
package test_module;
use strict;
use warnings;
use threads;
sub main {
my $max_threads = 10;
my #threads = ();
# create threads
while (scalar #threads < $max_threads) {
my $thread = threads->new(\&thread_sub);
push #threads, $thread;
}
# join threads
for my $thread (#threads) {
$thread->join();
}
}
sub thread_sub {
my $id = threads->tid();
print "I am in thread $id\n";
}
1;
The problem is that I am calling this module multiple times from one Perl script and instead of eliminating the old threads and creating new ones, the thread ids just keep incrementing. I have heard that if you don't properly get rid of old threads in Perl this will cause a memory leak and slow your program down, is this true? Is the data from my old threads just sitting in memory taking up space?
If so this can become a large problem since my script will be part of a much larger program that may generate hundreds or thousands of threads all of which would just be taking up memory even after they are done being used. How can I stop this from happening? Can my threads be reused?
Here is an example script that will call the module and show how the threads will continue to increment even though I joined the old threads (I thought that "join" was how you cleaned up after them, am I doing something wrong?) The way this script will be used I can't afford to have memory from old threads sitting there taking up space.
ex:
#!/usr/bin/perl
#testing.pl
use strict;
use warnings;
use test_module;
test_module::main();
test_module::main();
test_module::main();
system 'pause';
Thanks!

Answer 1

Don't worry about thread IDs incrementing - that doesn't mean the number of running threads is increasing. Once a thread is joined it has finished executing and been terminated.
However, continuously respawning threads isn't ideal either - creating a thread isn't a particularly lightweight operation in perl. So if you've got to do something like that, and are particularly focussing on efficiency - look to fork() instead.
I find I tend to use a 'worker thread' model, using Thread::Queue:
my $processing_q = Thread::Queue -> new();
sub worker_thread {
while ( my $item = $processing_q -> dequeue() ) {
# do stuff to $item
}
}
for ( 1 .. $num_threads ) {
my $thr = threads -> create ( \&worker_thread );
}
$processing_q -> enqueue ( #generic_list_of_things );
$processing_q -> end;
foreach my $thread ( threads -> list() ) {
$thread -> join();
}
This will feed in a batch of items into a queue, and your worker threads will process them one at a time - means you can have a sensible number running, without having to continuously respawn.
As an alternative though - take a look at Parallel::ForkManager - fork style parallel processing may seem counterintuitive initially, but fork() is a native system call on Unix systems, so it tends to be better optimised.