My server gets stuck by Thread.sleep? - multithreading

I have this code executed as a thread for each client to detect the disconnection of an user in my server:
private void detectDisconnect (Client user)
{
try
{
boolean listening = true;
while (listening)
{
System.out.println("<" + user.getUserName() + "> Sleeping...");
user.setDisconnected(true);
send(user, "heartBeat#");
Thread.sleep(Main.LATENCY);
System.out.println("<" + user.getUserName() + "> I wake up!");
if (user.isDisconnected()) { listening= false; exitClient(user);
}
}
catch (InterruptedException e) { e.printStackTrace(); }
}
The server before of sleeping put user.setDisconnected(true); and if the client send a response before of waking up I change to user.setDisconnected(false); in other method(), so this thread won't exit the Client unless in the latency's period doesn't get any message... This works fine, but sometimes randomly I could check that after the message of "Sleeping..." does not appear the message "I wake up!" (!!!), and the worst thing is that ALL MY PROGRAM GETS FROZEN, so the server doesn't listen or send messages to any client anymore :( even being these in separated Threads executing!....
I don't know why this happen, but I think that all the problems are caused by some eventually fatal error in Thread.sleep(). Could be possible? But I don't receiving any error message in the catch(Exception e){}, so I'm going crazy xD
Help please!!

Related

Akka.net Ask timeout when used in Azure WebJob

At work we have some code in a Azure WebJob where we use Rabbit
The basic workflow is this
A message arrives on RabbitMQ Queue
We have a message handler for the incoming message
Within the message handler we start a top level (user) supervisor actor where we "ask" it to handle the message
The supervisor actor hierarchy is like this
And the relevant top level code is something like this (this is the WebJob code)
static void Main(string[] args)
{
try
{
//Bootstrap akka IoC resolver well ahead of any actor usages
new AutoFacDependencyResolver(ContainerOperations.Instance.Container, ContainerOperations.Instance.Container.Resolve<ActorSystem>());
var system = ContainerOperations.Instance.Container.Resolve<ActorSystem>();
var busQueueReader = ContainerOperations.Instance.Container.Resolve<IBusQueueReader>();
var dateTime = ContainerOperations.Instance.Container.Resolve<IDateTime>();
busQueueReader.AddHandler<ProgramCalculationMessage>("RabbitQueue", x =>
{
//This is code that gets called whenever we have a RabbitMQ message arrive
//This is code that gets called whenever we have a RabbitMQ message arrive
//This is code that gets called whenever we have a RabbitMQ message arrive
//This is code that gets called whenever we have a RabbitMQ message arrive
//This is code that gets called whenever we have a RabbitMQ message arrive
try
{
//SupervisorActor is a singleton
var supervisorActor = ContainerOperations.Instance.Container.ResolveNamed<IActorRef>("SupervisorActor");
var actorMessage = new SomeActorMessage();
var supervisorRunTask = runModelSupervisorActor.Ask(actorMessage, TimeSpan.FromMinutes(25));
//we want to wait this guy out
var supervisorRunResult = supervisorRunTask.GetAwaiter().GetResult();
switch (supervisorRunResult)
{
case CompletedEvent completed:
{
break;
}
case FailedEvent failed:
{
throw failed.Exception;
}
}
}
catch (Exception ex)
{
_log.Error(ex, "Error found in Webjob");
//throw it for the actual RabbitMqQueueReader Handler so message gets NACK
throw;
}
});
Thread.Sleep(Timeout.Infinite);
}
catch (Exception ex)
{
_log.Error(ex, "Error found");
throw;
}
}
And this is the relevant IOC code (we are using Autofac + Akka.NET DI for Autofac)
builder.RegisterType<SupervisorActor>();
_actorSystem = new Lazy<ActorSystem>(() =>
{
var akkaconf = ActorUtil.LoadConfig(_akkaConfigPath).WithFallback(ConfigurationFactory.Default());
return ActorSystem.Create("WebJobSystem", akkaconf);
});
builder.Register<ActorSystem>(cont => _actorSystem.Value);
builder.Register(cont =>
{
var system = cont.Resolve<ActorSystem>();
return system.ActorOf(system.DI().Props<SupervisorActor>(),"SupervisorActor");
})
.SingleInstance()
.Named<IActorRef>("SupervisorActor");
The problem
So the code is working fine and doing what we want it to, apart from the Akka.Net "ask" timeout shown above in the WebJob code.
Annoyingly this seems to work fine if I try and run the webjob locally. Where I can simulate a "ask" timeout by providing a new supervisorActor that simply doesn't EVER respond with a message back to the "Sender".
This works perfectly running on my machine, but when we run this code in Azure, we DO NOT see a Timeout for the "ask" even though one of our workflow runs exceeded the "ask" timeout by a mile.
I just don't know what could be causing this behavior, does anyone have any ideas?
Could there be some Azure specific config value for the WebJob that I need to set.
The answer to this was to use the async rabbit handlers which apparently came out in V5.0 of the C# rabbit client. The offical docs still show the sync usage (sadly).
This article is quite good : https://gigi.nullneuron.net/gigilabs/asynchronous-rabbitmq-consumers-in-net/
Once we did this, all was good

IOException message not printed correctly when using Java 9 on Windows 10 set to Japan locale and language

An Exception is thrown in this particular block.
try
{
transport.m_readListener.onReadTransport(transport);
}
catch (IOException e)
{
->onIOException(e,transport);
}
The onIOException() method puts it on the log:
private void onIOException(IOException e, AbstractConnection connection)
{
String reason = e.getMessage();
...
log.error("Closing ",connection," because ",reason);
}
The reason variable in Java 8 shows a correct japanese phrase:
reason : 既存の接続はリモート ホストに強制的に切断されました。
(meaning: The existing connection was forcibly disconnected to the remote host)
When ran on Java 9, the exception message is now broken:
reason : 譌「蟄倥?ョ謗・邯壹?ッ繝ェ繝「繝シ繝? 繝帙せ繝医↓蠑キ蛻カ逧?縺ォ蛻?譁ュ縺輔l縺セ縺励◆縲?
The code block that checks if the Socket port is still open is inside a try catch block that catches the IOException. The message from the IOException is acquired via
String reason = e.getMessage();
if (null == reason) reason = e.toString();
Tried running the app with java.locale.providers=COMPAT,CLDR,SPI to make it behave like in Java 8 but nothing happens. Anyone has an idea on this issue? Can anyone help? Thanks!

How to handle Connect/ Authentication errors when using imap-idle-channel-adapter

I am able to handle and report connection/ authentication related errors for imap and pop adapter by using error-channel on the poller.
I have one for imap-idle too but its not getting these error on that channel but a warning in the logs only.
what is the way out to handle and report similar errors for email idle adapter?
I guess you mean this peace of code:
catch (Exception e) { //run again after a delay
logger.warn("Failed to execute IDLE task. Will attempt to resubmit in " + ImapIdleChannelAdapter.this.reconnectDelay + " milliseconds.", e);
ImapIdleChannelAdapter.this.receivingTaskTrigger.delayNextExecution();
ImapIdleChannelAdapter.this.publishException(e);
}
And you see that WARN in your logs.
Pay attention, please, to the latest line about publishException(). It is this code:
private void publishException(Exception e) {
if (this.applicationEventPublisher != null) {
this.applicationEventPublisher.publishEvent(new ImapIdleExceptionEvent(e));
}
So, what you need is to add an EventListener for that ImapIdleExceptionEvent. For example you can use ApplicationEventListeningMessageProducer.

C# Handling threads and blocking sockets

In the following thread, UDP packets are read from clients until the boolean field Run is set to false.
If Run is set to false while the Receive method is blocking, it stays blocked forever (unless a client sends data, which will make the thread loop and check for the Run condition again).
while (Run)
{
IPEndPoint remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
byte[] data = udpClient.Receive(ref remoteEndPoint); // blocking method
// process received data
}
I usually get around the problem by setting a timeout on the server. It works fine, but seems to be a patchy solution to me.
udpClient.Client.ReceiveTimeout = 5000;
while (Run)
{
try
{
IPEndPoint remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
byte[] data = udpClient.Receive(ref remoteEndPoint); // blocking method
// process received data
}
catch(SocketException ex) {} // timeout reached
}
How would you handle this problem? Is there any better way?
Use UdpClient.Close(). That will terminate the blocking Receive() call. Be prepared to catch the ObjectDisposedException, it signals your thread that the socket is closed.
You could do something like this:
private bool run;
public bool Run
{
get
{
return run;
}
set
{
run = value;
if(!run)
{
udpClient.Close();
}
}
}
This allows you to close the client once whatever condition is met to stop your connection from listening. An exception will likely be thrown, but I don't believe it will be a SocketTimeoutException, so you'll need to handle that.

TcpClient and StreamReader blocks on Read

Here's my situation:
I'm writing a chat client to connect to a chat server. I create the connection using a TcpClient and get a NetworkStream object from it. I use a StreamReader and StreamWriter to read and write data back and forth.
Here's what my read looks like:
public string Read()
{
StringBuilder sb = new StringBuilder();
try
{
int tmp;
while (true)
{
tmp = StreamReader.Read();
if (tmp == 0)
break;
else
sb.Append((char)tmp);
Thread.Sleep(1);
}
}
catch (Exception ex)
{
// log exception
}
return sb.ToString();
}
That works fine and dandy. In my main program I create a thread that continually calls this Read method to see if there is data. An example is below.
private void Listen()
{
try
{
while (IsShuttingDown == false)
{
string data = Read();
if (!string.IsNullOrEmpty(data))
{
// do stuff
}
}
}
catch (ThreadInterruptedException ex)
{
// log it
}
}
...
Thread listenThread = new Thread(new ThreadStart(Listen));
listenThread.Start();
This works just fine. The problem comes when I want to shut down the application. I receive a shut down command from the UI, and tell the listening thread to stop listening (that is, stop calling this read function). I call Join and wait for this child thread to stop running. Like so:
// tell the thread to stop listening and wait for a sec
IsShuttingDown = true;
Thread.Sleep(TimeSpan.FromSeconds(1.00));
// if we've reach here and the thread is still alive
// interrupt it and tell it to quit
if (listenThread.IsAlive)
listenThread.Interrupt();
// wait until thread is done
listenThread.Join();
The problem is it never stops running! I stepped into the code and the listening thread is blocking because the Read() method is blocking. Read() just sits there and doesn't return. Hence, the thread never gets a chance to sleep that 1 millisecond and then get interrupted.
I'm sure if I let it sit long enough I'd get another packet and get a chance for the thread to sleep (if it's an active chatroom or a get a ping from the server). But I don't want to depend on that. If the user says shut down I want to shut it down!!
One alternative I found is to use the DataAvailable method of NetworkStream so that I could check it before I called StreamReader.Read(). This didn't work because it was undependable and I lost data when reading from packets from the server. (Because of that I wasn't able to login correctly, etc, etc)
Any ideas on how to shutdown this thread gracefully? I'd hate to call Abort() on the listening thread...
Really the only answer is to stop using Read and switch to using asynchronous operations (i.e. BeginRead). This is a harder model to work with, but means no thread is blocked (and you don't need to dedicate a thread—a very expensive resource—to each client even if the client is not sending any data).
By the way, using Thread.Sleep in concurrent code is a bad smell (in the Refactoring sense), it usually indicates deeper problems (in this case, should be doing asynchronous, non-blocking, operations).
Are you actually using System.IO.StreamReader and System.IO.StreamWriter to send and receive data from the socket? I wasn't aware this was possible. I've only ever used the Read() and Write() methods on the NetworkStream object returned by the TcpClient's GetStream() method.
Assuming this is possible, StreamReader returns -1 when the end of the stream is reached, not 0. So it looks to me like your Read() method is in an infinite loop.

Resources