Recently I'm doing a project on Network connections. I'm making the server side program on Perl which handle the request from client, processing, request data from other server. For the multiuser handling purpose I've to use Multitasking. And for not leaking the resource, each thread/connection from client have a limited time out (5 seconds)
Here are my codes:
while(1)
{
# waiting for new client connection.
$client_socket = $socket->accept();
threads->new(\&gotRequest, $client_socket);
#gotRequest($client_socket);
}
This is for Catching the connection from the client
sub gotRequest
{
$client_socket=$_[0];
# $peer_address = $client_socket->peeraddr();
$th1=threads->new(\&Responce, $client_socket);
sleep(5);
if (!($th1->is_running())) {print "Connection terminated\n";}
else
{
print "Operation time out, killing process and terminating connection\n";
print $client_socket "QUIT\n";
close $client_socket;
print "Closing...\n";
#$thr->set_thread_exit_only();
$th1->detach();
$th1->exit(); #This thing causing thread's death
print "Hello I'm still here!";
}
}
This is the thread that manage the processing thread to quit on time otherwise server cant get new connection
sub Responce
{
$client_socket=$_[0];
$peer_address = $client_socket->peeraddr();
$peer_port = $client_socket->peerport();
sleep (10);
print "I'm still alive";
print "Accepted New Client Connection From : $peeraddress, $peerport\n";#Dont know why but this printed null with 2 null string :(
$client_socket->recv($data,1024000);
$data_decode = decode("utf-16", $data);
print "Received from Client : $data_decode\n";
#Custom code added here
$data = encode("utf-16","DATA from Server");
print $client_socket "$data\n";
#close($sock);
}
I got the error:
Thread 1 terminated abnormally: Usage: threads->exit(status) at server-cotton.pl line 61 thread 1
When the
$th1->exit();
executing.
And one more thing, I can't not disconnect to connection from client.
As the message says, it's a static method call
threads->exit(1); # ok
Not an instance method call
$th1->exit(1); # not ok unless $th1 contains the string "threads"
Related
I am building a multithreaded perl TCP server that uses different threads to handle different clients. For this purpose, I am maintaining a thread pool which keeps track of whether the thread is idle or working.
In the main thread, I open a listening socket and bind to a specific port using:
$socket = new IO::Socket::INET(Localhost => '127.0.0.1',
LocalPort => '5000',
Proto => 'tcp',
Listen => $MAX_THREADS, Reuse => 1) or die "Error in Socket Creation: $!\n";
The main thread also listens to any incoming connections using socket->accept() and if successful passes this return socket to the child thread which handles it and sends an acknowledgement to the client corresponding to the socket. However, I was unable to pass this socket.
I googled a bit and having no luck, later I decided to maintain a global hashmap of incoming sockets which can be later accessed by the child threads(the hashmap is shared) and then work on it.
However perl is giving me an error as to Invalid value for shared scalar. Here is the code bit:
$sochandler->{$tid} = $socket->accept();
#$sochandler is the shared global hashmap with keys as thread IDs
PS: I am newbie to perl and I have tried to explain as much as I can regarding my problem
Here is the subroutine code which is run by every child thread:
sub worker
{
my ($work_q) = #_;
my $tid = threads->tid();
do {
printf("Idle -> %2d\n", $tid);
$IDLE_QUEUE->enqueue($tid);
my $work_tid = $work_q->dequeue();
my $work = $sochandler->{$work_tid};
last if ($work_tid < 0);
printf(" %2d <- Working\n", $tid);
while (($work_tid > 0) && ! $TERM) {
print "Accepted New Client Connection From: $work->peerhost(), $work->peerport()\n";
my $data = "ACK from server";
$work->send($data);
$work->recv($data,1024);
print "Received from Client : $data\n";
}
} while (! $TERM);
printf("Finished -> %2d\n", $tid);
}
The error you are getting is because the $tid is not declared as a shared variable. You need to share 'deep' so that not only is the hash shared, but also its keys.
I'm trying to write a multithreaded server with perl (Windows x64). When trying to connect to it from another computer, I found the memory and handle usage kept going up, even if I maintained only one connection at a time. And after thousands of trials it used up nearly all system memory. I can't figure out the reason.
Here is the server side:
use IO::Socket::INET;
use threads;
sub session_thread
{
my $client_socket=$_[0];
my $client_address = $client_socket->peerhost();
my $client_port = $client_socket->peerport();
print "connection from $client_address:$client_port\n";
my $data = "";
$client_socket->recv($data, 1024);
print "$client_address:$client_port says: $data";
$data = "ok";
$client_socket->send($data);
shutdown($client_socket, 1);
$client_socket->close();
threads->exit();
}
$| = 1;
my $socket = new IO::Socket::INET (
LocalHost => '0.0.0.0',
LocalPort => '7777',
Proto => 'tcp',
Listen => 5,
ReuseAddr => 1
);
die "cannot create socket $!\n" unless $socket;
print "server waiting for client connection on port 7777\n";
while(1)
{
my $client_socket = $socket->accept();
threads->create('session_thread',$client_socket);
}
$socket->close();
Thanks.
You either have to wait for the thread to finish by joining it or tell Perl that you don't care about the thread's return value and that Perl itself should clean up the data once the thread exits. The latter seems to match your use case and is done by detaching.
Also note that using exit is not necessary in your example. You can simply return from the thread's subroutine normally. exit is used for ending a thread from a deeper nesting level within the program.
I have a process that must create and close threads on demand.
Each thread forks a new process using open2. Sometimes after executing the program for a long time open2 fails to fork the process sometimes and gives a "Can not allocate memory error", sometimes this happens for threads too.I know that the Linux has soft and hard limits but the number of the concurrent threads and processes for my server does not exceed those values.
Is there something like a counter for number of processes and threads that eliminates thread and process creation after sometime?
If it is so how servers like Postgres work for a long period of time?
The project has multiple processes that communicate using TCP, but the part that causes the error that i described in a frond end to mplayer, that is written in Perl. The code is as follows:
use strict;
use warnings;
use IO::Socket::INET;
use IO::Select;
use POSIX ":sys_wait_h";
use IPC::Open2;
use 5.010;
use Config;
BEGIN
{
if(!$Config{useithreads})
{
die "Your perl does not compiled with threading support.";
}
}
use threads;
use threads::shared;
use constant
{
SERVER_PORT=>5000,
#Remote request packet fields
PACKET_REQTYPE=>0,
PACKET_FILENAM=>1,
PACKET_VOLMLVL=>2,
PACKET_ENDPOSI=>3,
PACKET_SEEKPOS=>4,
#our request typs
PLAY_REQUEST=>1,
STOP_REQUEST=>2,
INFO_REQUEST=>3,
VOCH_REQUEST=>4,
PAUS_REQUEST=>5,
PLPA_REQUEST=>6,
SEEK_REQUEST=>7,
#Play states
STATE_PAUS=>0,
STATE_PLAY=>1,
STATE_STOP=>2,
};
#The following line must be added because of a bad behavior in the perl thread library that causes a SIGPIPE to be generated under heavy usage of the threads.
$SIG{PIPE} = 'IGNORE';
#This variable holds the server socket object
my $server_socket;
#This array is used to hold objects of our all threads
my #thread_objects;
#create the server socket
$server_socket=IO::Socket::INET->new(LocalPort=>SERVER_PORT,Listen=>20,Proto=>'tcp',Reuse=>1) or
die "Creating socket error ($#)";
#Now try to accept remote connections
print "Server socket created successfully now try to accept remote connections on port: ".SERVER_PORT."\n";
while(my $client_connection=$server_socket->accept())
{
push #thread_objects,threads->create(\&player_thread,$client_connection);
$thread_objects[$#thread_objects]->detach();
}
#This subroutine is used to play something using tcp-based commands
sub player_thread
{
my $client_socket=shift;
#create a new select object
my $selector=IO::Select->new($client_socket);
#this variabe is used to pars our request
my #remote_request;
#getting th thread id of the current thread
my $tid=threads->self()->tid;
#This variable is used to hold the pid of mplayer child
my $mp_pid=-1;
#Mplayer stdin and stdout file descriptors
my ($MP_STDIN,$MP_STDOUT);
#This variable is used to check if we are playing something now or not
my $is_playing=STATE_STOP;
print "Client thread $tid created.\n";
while(1)
{
#check to see if we can read anything from our handler
#print "Before select\n";
#my #ready=$selector->can_read();
#print "After select: #ready\n";
#now the data is ready for reading so we read it here
my $data=<$client_socket>;
#This means if the connection is closed by the remote end
if(!defined($data))
{
print "Remote connection has been closed in thread $tid mplayer id is: $mp_pid and state is: $is_playing.\n";
#if we have an mplayer child when remote connection is closed we must wait for it
#so that is work is done
if($mp_pid!=-1 and $is_playing ==STATE_PLAY)
{
waitpid $mp_pid,0;
$is_playing=STATE_STOP;
}
elsif($is_playing==STATE_PAUS and $mp_pid!=-1)
{
print "thread $tid is in the paused state, we must kill mplayer.\n";
print $MP_STDIN "quit\n";
waitpid $mp_pid,0;
$is_playing=STATE_STOP;
}
last;
}#if
#FIXME:: Here we must validate our argument
#Now we try to execute the command
chomp($data);
#remote_request=split ",",$data;
print "#remote_request\n";
#Trying to reap the death child and change the state of the thread
my $dead_child=-1;
$dead_child=&reaper($mp_pid);
if($dead_child)
{
$is_playing=STATE_STOP;
$mp_pid=-1;
}
given($remote_request[PACKET_REQTYPE])
{
when($_==PLAY_REQUEST)
{
print "Play request\n";
if($is_playing==STATE_STOP)
{
eval{$mp_pid=open2($MP_STDOUT,$MP_STDIN,"mplayer -slave -really-quiet -softvol -volume ".$remote_request[PACKET_VOLMLVL]." -endpos ".$remote_request[PACKET_ENDPOSI]." ./".$remote_request[PACKET_FILENAM]);};
print "Some error occurred in open2 system call: $#\n" if $#;
$is_playing=STATE_PLAY;
print "Mplayer pid: $mp_pid.\n";
}
}
when($_==STOP_REQUEST)
{
print "Stop request\n";
if($is_playing != STATE_STOP)
{
print $MP_STDIN "pausing_keep stop\n";
#FIXME:: Maybe we should use WNOHANG here
my $id=waitpid $mp_pid,0;
print "Mplayer($id) stopped.\n";
$is_playing=STATE_STOP;
$mp_pid=-1;
}
}
when($_==PAUS_REQUEST)
{
print "pause request\n";
if($is_playing !=STATE_STOP)
{
print $MP_STDIN "pausing_keep pause\n";
$is_playing=STATE_PAUS;
}
}
when($_==VOCH_REQUEST)
{
print "volume change request\n";
if($is_playing !=STATE_STOP)
{
print $MP_STDIN "pausing_keep volume ".$remote_request[PACKET_VOLMLVL]." 1\n";
}
}
when($_==INFO_REQUEST)
{
my $id;
$id=&reaper($mp_pid);
if($id > 0)
{
print "Mplayer($id) stopped.\n";
$is_playing=STATE_STOP;
$mp_pid=-1;
}
given($is_playing)
{
when($_==STATE_STOP)
{
print $client_socket "Stopped\n";
}
when($_==STATE_PAUS)
{
print $client_socket "Paused\n";
}
when($_==STATE_PLAY)
{
print $client_socket "Playing\n";
}
}
}
when ($_==PLPA_REQUEST)
{
print "play paused request\n";
if($is_playing==STATE_STOP)
{
eval{$mp_pid=open2($MP_STDOUT,$MP_STDIN,"mplayer -slave -really-quiet -softvol -volume ".$remote_request[PACKET_VOLMLVL]." -endpos ".$remote_request[PACKET_ENDPOSI]." ./".$remote_request[PACKET_FILENAM]);};
print "Some error occurred in open2 system call: $#\n" if $#;
print $MP_STDIN "pausing_keep pause\n";
$is_playing=STATE_PAUS;
}
}
when ($_==SEEK_REQUEST)
{
print "Seek request\n";
if($is_playing != STATE_STOP)
{
my $seek_pos=abs $remote_request[PACKET_SEEKPOS];
print $MP_STDIN "seek $seek_pos 2\n";
$is_playing=STATE_PLAY;
}
}
default
{
warn "Invalid request($_)!!!";
next;
}
}#Given
}#while
$client_socket->close();
print "Thread $tid is exiting now, the child mplayer pid is: $mp_pid and state is: $is_playing.\n";
}
#The following subroutine takes a pid and if that pid is grater than 0 it tries to reap it
#if it is successful returns pid of the reaped process else 0
sub reaper
{
my $pid=shift;
if($pid > 0)
{
my $id=waitpid($pid,WNOHANG);
if($id > 0)
{
return $id;
}
}
return 0;
}
"Can not allocate memory error" is what it says, either the user exceeded its memory quota (check with ulimit -m, compare to ps ux) or you're really out of memory (free).
The limits for max user processes are only indirectly connected - if you fork() more processes then the user's memory quota permits, fork() will fail with ENOMEM.
You also might want to see:
What are some conditions that may cause fork() or system() calls to fail on Linux?
I finally found the problem, it is because of a memory leak in the Perl's thread module that causes the memory to grow after a long time. Then open2 can not allocate memory and fails.
I am trying to write Perl code that does two-way communication over a Unix socket. It needs to do the following things:
Client code sends a request over the socket
Server code reads the request
Server performs any actions that should happen immediately
Server creates a new thread to do additional actions that may take a long time
Server sends response over the socket
Client receives response
The new thread continues to work after the response has been sent
I have gotten most of this to work now, but with one problem. Steps 1 - 5 work fine, but at step 6, the client is unable to read the response until AFTER the thread has exited. (even though the response was sent more-or-less immediately) Any idea what I'm doing wrong?
I'm using Perl 5.10.1 on Ubuntu Lucid.
Here's an example:
Client code:
#!/usr/bin/perl
use strict;
use Socket;
my $socket_name = 'catsock';
my $client_message = "Hello server, this is the client.";
my $SOCK;
socket($SOCK, PF_UNIX, SOCK_STREAM, 0) or die "socket: $!";
connect($SOCK, sockaddr_un($socket_name)) or die "connect: $!";
$| = 1, select $_ for select $SOCK; # turn on autoflush
print $SOCK $client_message; # send the message
shutdown($SOCK,1); # finished writing
print "sent: $client_message\n";
my $server_message = do { local $/; <$SOCK> }; # get the response
print "recieved: $server_message\n\n";
Server code
#!/usr/bin/perl
use strict;
use Socket;
use threads;
use threads::shared;
sub threadfunc
{
print " waiting 5 seconds in new thread...\n";
sleep(5);
print " exiting thread...\n\n";
}
my $server_message = "Hello client, this is the server.";
my $socket_name = 'catsock';
my $SERVER;
my $CLIENT;
socket($SERVER, PF_UNIX, SOCK_STREAM, 0) or die "socket: $!";
unlink($socket_name);
bind($SERVER, sockaddr_un($socket_name)) or die "bind: $!";
chmod(0660, $socket_name);
listen($SERVER, SOMAXCONN) or die "listen: $!";
print "server started on $socket_name\n\n";
while(1) {
accept($CLIENT, $SERVER);
my $client_message = do { local $/; <$CLIENT> }; # grab entire message
print "recieved: $client_message\n";
print "creating new thread...\n";
my $thr = threads->create(\&threadfunc);
$thr->detach();
print $CLIENT $server_message; # send the response
print "sent: $server_message\n\n";
close $CLIENT;
}
When I run this, the following things happen:
Client sends "Hello server, this is the client."
Server receives client's message
Server creates a new thread
Server sends "Hello client, this is the server."
Client should receive the message now, but it does not.
New thread on server sleeps for 5 seconds
Thread exits
Now the client receives the server's message, even though it was sent 5 seconds ago. Why?
my $server_message = do { local $/; <$SOCK> }
in the client script smells funny to me. By setting $/ to undef, you are asking the <$SOCK> statement to read all input from the handle $SOCK until end of file. In this context, end of file means that the other end of the socket has closed the connection.
The server is delivering the message, as you can see if you change the line above to something like
print "Received message: ";
print while $_ = getc($SOCK);
(this will also hang for 5 seconds when the input stream is exhausted, but it will at least display each character in the input in the meantime).
It is a "feature" that one end of a TCP socket connection can never really know whether the other end of the connection is still alive. Network programmers resort to other conventions -- encoding the message length at the beginning of each message, ending each message with a special token, implementing heartbeats -- to determine how much input can safely be read from a socket (you might also want to look into the 4-argument version of select)
For this problem, one possibility is to apply the convention that all messages from the server should end in a newline. Change the server script to say
print $CLIENT $server_message, "\n";
and change the client script to say
my $server_message = do { local $/="\n"; <$SOCK> }; # if you're paranoid about $/ setting
my $server_message = <$SOCK>; # if you're not
and you will get results closer to what you expect.
Here's my situation:
I'm writing a chat client to connect to a chat server. I create the connection using a TcpClient and get a NetworkStream object from it. I use a StreamReader and StreamWriter to read and write data back and forth.
Here's what my read looks like:
public string Read()
{
StringBuilder sb = new StringBuilder();
try
{
int tmp;
while (true)
{
tmp = StreamReader.Read();
if (tmp == 0)
break;
else
sb.Append((char)tmp);
Thread.Sleep(1);
}
}
catch (Exception ex)
{
// log exception
}
return sb.ToString();
}
That works fine and dandy. In my main program I create a thread that continually calls this Read method to see if there is data. An example is below.
private void Listen()
{
try
{
while (IsShuttingDown == false)
{
string data = Read();
if (!string.IsNullOrEmpty(data))
{
// do stuff
}
}
}
catch (ThreadInterruptedException ex)
{
// log it
}
}
...
Thread listenThread = new Thread(new ThreadStart(Listen));
listenThread.Start();
This works just fine. The problem comes when I want to shut down the application. I receive a shut down command from the UI, and tell the listening thread to stop listening (that is, stop calling this read function). I call Join and wait for this child thread to stop running. Like so:
// tell the thread to stop listening and wait for a sec
IsShuttingDown = true;
Thread.Sleep(TimeSpan.FromSeconds(1.00));
// if we've reach here and the thread is still alive
// interrupt it and tell it to quit
if (listenThread.IsAlive)
listenThread.Interrupt();
// wait until thread is done
listenThread.Join();
The problem is it never stops running! I stepped into the code and the listening thread is blocking because the Read() method is blocking. Read() just sits there and doesn't return. Hence, the thread never gets a chance to sleep that 1 millisecond and then get interrupted.
I'm sure if I let it sit long enough I'd get another packet and get a chance for the thread to sleep (if it's an active chatroom or a get a ping from the server). But I don't want to depend on that. If the user says shut down I want to shut it down!!
One alternative I found is to use the DataAvailable method of NetworkStream so that I could check it before I called StreamReader.Read(). This didn't work because it was undependable and I lost data when reading from packets from the server. (Because of that I wasn't able to login correctly, etc, etc)
Any ideas on how to shutdown this thread gracefully? I'd hate to call Abort() on the listening thread...
Really the only answer is to stop using Read and switch to using asynchronous operations (i.e. BeginRead). This is a harder model to work with, but means no thread is blocked (and you don't need to dedicate a thread—a very expensive resource—to each client even if the client is not sending any data).
By the way, using Thread.Sleep in concurrent code is a bad smell (in the Refactoring sense), it usually indicates deeper problems (in this case, should be doing asynchronous, non-blocking, operations).
Are you actually using System.IO.StreamReader and System.IO.StreamWriter to send and receive data from the socket? I wasn't aware this was possible. I've only ever used the Read() and Write() methods on the NetworkStream object returned by the TcpClient's GetStream() method.
Assuming this is possible, StreamReader returns -1 when the end of the stream is reached, not 0. So it looks to me like your Read() method is in an infinite loop.