Netty ChannelInboundHandlerAdapter async/multithreading - multithreading

I'm having trouble grasping the concepts behind multithreading in netty, EventLoopGroup (MultithreadEventLoopGroup), MultithreadEventExecutorGroup, DefaultEventExecutorGroup
I am trying to understand how the server handles multiple clients simultaneously sending requests that will execute some business logic and CRUD operations that add to RTT. Below is my netty server code which works, but I am trying to understand exactly how it will work with concurrent users and multiple open channels.
I have a simple ServerBootstrap
#Component
#RequiredArgsConstructor
public class SocketServer {
private final ContextAwareLogger logger;
private final ServerInitializer serverInitializer;
private final NioEventLoopGroup bossGroup;
private final NioEventLoopGroup workerGroup;
private Channel mainChannel;
#PostConstruct
public void start() {
try {
ServerBootstrap bootstrap = init();
mainChannel = bootstrap.bind(8484).sync().channel(); // save the main channel so we can cleanly close it when app is shutdown
logger.info("Netty server started...");
} catch (Exception e) {
e.printStackTrace();
}
}
#PreDestroy
public void stop() throws InterruptedException {
logger.info("Shutting down Netty server");
bossGroup.shutdownGracefully().sync();
workerGroup.shutdownGracefully().sync();
mainChannel.closeFuture().sync();
logger.info("Netty Server shutdown complete.");
}
private ServerBootstrap init() {
return new ServerBootstrap()
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.SO_BACKLOG, 5000)
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_KEEPALIVE, true)
.childHandler(serverInitializer);
}
}
ChannelInitializer:
#Component
#RequiredArgsConstructor
public class ServerInitializer extends ChannelInitializer<SocketChannel> {
private final PacketDecoder packetDecoder;
private final ServerHandler serverHandler;
private final PacketEncoder packetEncoder;
#Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
socketChannel.pipeline()
.addLast("decoder", packetDecoder) // ByteArrayDecoder
.addLast("encoder", packetEncoder) // ByteArrayEncoder
.addLast("inbound", serverHandler); // ChannelInboundHandlerAdapter
}
}
ChannelInboundHandlerAdapter:
#Component
#Sharable
public class ServerHandler extends ChannelInboundHandlerAdapter {
#Autowired
private SomeService someService;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
// contains db access
byte[] accept = someService.validateClient(ctx.channel());
ctx.channel().writeAndFlush(accept);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
// may contain db access
byte[] response = someService.processPacket(ctx.channel(), msg));
ctx.channel().writeAndFlush(response);
}
}
Now when a client connects, I understand that a new Channel will opened and the handlers will be reused. Requirement is each client request/response needs to process immediately without waiting for some other client's CRUD operations to finish.
Are my channelRead and channelActive, etc, async because I am using NioEventLoopGroup (ie will each client's channel operations be run independent of each other)?
If a single client sends multiple requests in series, are they guaranteed to be handled in the same order?
Do I need to specific DefaultEventExecutorGroup for my inbound handler? (https://stackoverflow.com/a/28305019/1738539)

You either would need to use a DefaultEventExecutorGroup for your ServerHandler or dispatch the validateClient(...) / processPacket(...) to your own ThreadPool. Failing todo so will cause the EventLoop thread to block and so no other IO can be processed for this EventLoop until the blocking operation completes.

Related

Hazelcast listeners don't work after client reconnects

My hazelcast client got disconnected because of possible memory outbreak. After the client reconnects, it is not getting getting messages from listeners on maps and topics. Is there a way to make the client active to the listeners again ?
This is how I am initiating a listener in my client/service:
IMap liveMap = hazelcastCacheClientService.getMap("MYMAP");
MyListener myMapListener = new MyListener();
liveMap.addEntryListener(myMapListener,true);
This is how my listener looks like:
public class MyListener implements EntryListener {
#Override
public void entryAdded(EntryEvent entryEvent) {
//do something
}
#Override
public void entryEvicted(EntryEvent entryEvent) {
}
#Override
public void entryRemoved(EntryEvent entryEvent) {
}
#Override
public void entryUpdated(EntryEvent entryEvent) {
//do something
}
#Override
public void mapCleared(MapEvent mapEvent) {
}
#Override
public void mapEvicted(MapEvent mapEvent) {
}
}
I have multiple other services, who put entries into the map.
If your version is 3.9 or older it could be a bug. We improved the client reconnect logic in 3.10 and further improved it in 3.11. Here are the 3.11 documentation for client reconnect configuration: https://docs.hazelcast.org/docs/3.11/manual/html-single/index.html#configuring-client-connection-retry
Now having said that I always recommend an additional circuit breaker pattern such as the one from netflix: https://github.com/Netflix/Hystrix/wiki/How-it-Works
In order to restore the client listeners it is best to destroy the hazelcastinstance and create a new one. This forces all the sockets to perform an init during startup. Hazelcast is a socket application so if you suffer from 1/2 open sockets then there is not much Hazelcast can do about restoring the connections.
Hope this helps,

On servlet 3.0 webserver, is it good to make all servlets and filters async?

I am confused with Async feature introduced in Servlet 3.0 spec
From Oracle site (http://docs.oracle.com/javaee/7/tutorial/doc/servlets012.htm):
To create scalable web applications, you must ensure that no threads
associated with a request are sitting idle, so the container can use
them to process new requests.
There are two common scenarios in which a thread associated with a
request can be sitting idle.
1- The thread needs to wait for a resource to become available or process data before building the response. For example, an application
may need to query a database or access data from a remote web service
before generating the response.
2- The thread needs to wait for an event before generating the response. For example, an application may have to wait for a JMS
message, new information from another client, or new data available in
a queue before generating the response.
The first item happens a lot (nearly always, we always query db or call a remote webservice to get some data). And calling an external resource will always consume some time.
Does it mean that we should ALWAYS use servelt async feature for ALL our servelts and filter ?!
I can ask this way too, if I write all my servelts and filters async, will I lose anything (performance)?!
If above is correct the skeleton of ALL our servlets will be:
public class Work implements ServletContextListener {
private static final BlockingQueue queue = new LinkedBlockingQueue();
private volatile Thread thread;
#Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
thread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
ServiceFecade.doBusiness();
AsyncContext context;
while ((context = queue.poll()) != null) {
try {
ServletResponse response = context.getResponse();
PrintWriter out = response.getWriter();
out.printf("Bussiness done");
out.flush();
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
} finally {
context.complete();
}
}
} catch (InterruptedException e) {
return;
}
}
}
});
thread.start();
}
public static void add(AsyncContext c) {
queue.add(c);
}
#Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
thread.interrupt();
}
}

Running windows service in separate thread and use autofac for DI

I'm trying to create a long running windows service, so I need to run the actual worker class on a separate thread, to avoid the "service did not respond in a timely fashion" error when I right click and select start in Windows Service Manager.
The worker class ("NotificationProcess") has a whole raft of dependencies and I'm using Autofac to satisfy these.
I'm really not sure how to set up Autofac for the worker class. At the moment I'm getting errors telling me that the DbContext has been disposed when I go to use it in the "Execute" method of the worker class.
I guess I'm looking for how to write a windows service and use a new thread for the worker class with dependencies satisfied by autofac.
I've googled and can't find any examples of this.
Any suggestions would be awesome.
Here's what I've got so far...
Program.cs:
static class Program
{
static void Main()
{
using (var container = ServiceStarter.CreateAutoFacContainer())
{
var service = container.Resolve<NotificationService>();
if (Environment.UserInteractive)
{
service.Debug();
}
else
{
ServiceBase.Run(container.Resolve<NotificationService>());
}
}
The Service class:
public partial class NotificationService : ServiceBase
{
private NotificationProcess _app;
readonly ILifetimeScope _lifetimeScope;
public NotificationService(ILifetimeScope lifetimeScope)
{
_lifetimeScope = lifetimeScope;
InitializeComponent();
}
protected override void OnStart(string[] args)
{
_app = _lifetimeScope.Resolve<NotificationProcess>();
_app.Start();
}
The worker class:
public class NotificationProcess
{
private Thread _thread;
private readonly IBankService _bankService;
private readonly IRateService _rateService;
private readonly IEmailService _emailService;
private readonly IRateChangeSubscriberService _rateChangeSubscriberService;
private readonly IRateChangeNotificationService _rateChangeNotificationService;
private readonly ILogManager _logManager;
public NotificationProcess(IBankService bankService, ILogManager logManager, IRateService rateService, IEmailService emailService,
IRateChangeSubscriberService rateChangeSubscriberService, IRateChangeNotificationService rateChangeNotificationService)
{
_bankService = bankService;
_rateService = rateService;
_emailService = emailService;
_rateChangeSubscriberService = rateChangeSubscriberService;
_rateChangeNotificationService = rateChangeNotificationService;
_logManager = logManager;
}
public void Start()
{
_thread = new Thread(new ThreadStart(Execute));
_thread.Start();
}
public void Execute()
{
try
{
var rateChangeToNotify = _rateService.GetRateChangesForNotification();
foreach (var rateChange in rateChangeToNotify)
{
//do whatever business logic.....
}
}
}
The answer is actually simple: use scoping! You should do the following:
Register all services (such as DbContext) that should live for the duration of a request or action with the LifetimeScope lifestyle. You'll usually have a timer in your windows service. Each 'pulse' can be considered a request.
On the beginning of each request begin a lifetime scope.
Within that scope, resolve the root object from the object graph and call its method.
Dispose the scope.
In your case that means you need to change your design, since NotificationService is resolved once and its dependencies are reused on another thread. This is a no-no in dependency injection land.
Here's an alternative design:
// This method is called on a background thread
// (possibly in a timely manner)
public void Run()
{
try
{
using (var scope = container.BeginLifetimeScope())
{
var service = scope.Resolve<NotificationService>();
service.Execute();
}
}
catch (Exception ex)
{
// IMPORTANT: log exception.
// Not logging an exception will leave us in the dark.
// Not catching the exception will kill our service
// because we run in a background thread.
}
}
Using a lifetime scope allows you to get a fresh DbContext for every request and it would even allow you to run requests in parallel (with each request its own DbContext).

Websocket Servlet thread-safe

I'am playing around with WebSocketServlet (tomcat) and I have some question about doing it properly without race condition problems.
I have an instance variable (so non thread-safe) that will keep track of all the websocket connections
HashMap<String,MyServers> myNonThreadSafeVariable = HashMap<String,MyServers>
This is what the HashMap will contain (roughly...)
private final class MyServers extends MessageInbound {
final Set<MyClients> clients = new CopyOnWriteArraySet<MyClients>();
private String serverName;
#Override
protected void onOpen(WsOutbound outbound) {}
#Override
protected void onClose(WsOutbound outbound) {}
#Override
protected void onText(WsOutbound outbound) {}
}
private final class Clients extends MessageInbound {
private int clientID;
#Override
protected void onOpen(WsOutbound outbound) {}
#Override
protected void onClose(WsOutbound outbound) {}
#Override
protected void onText(WsOutbound outbound) {}
}
So now.. during my servlet life time, I am looping through myNonThreadSafeVariable and then maybe also will loop through myNonThreadSafeVariable.clients and then maybe also modified or add a clients or server etc...
For example when a server connect, in his onOpen there will be something like
myNonThreadSafeVariable.put(key,this);
or When a client connects in his onOpen (quit concern about this one)
server = myNonThreadSafeVariable,get(key);
sever.clients.add(this);
Or sometimes when I have to ping all the clients of all the servers:
for (Entry<String, MyServers> entry : myNonThreadSafeVariable.entrySet()) {
MyServers server = entry.getValue();
server.sendMessage("ping||");
for (MyClients member : entry.getValue().clients) {
client.sendMessage("")
}
}
So If I undertand correctly as myNonThreadSafeVariable is global so will myNonThreadSafeVariable.clients etc..
So my question is what is a good practice to avoid race condition in this scenario ?
Using mutex and synchronized on them when access ether the myNonThreadSafeVariable and myNonThreadSafeVariable.clients ? Or should I avoid using an instance variable at all ? But how ?
thanks !
You could use a ReadWriteLock: you block readers and writers when writing, you block only writers when reading:
private final ReadWriteLock lock = new ReentrantReadWriteLock();
private final Lock readLock = lock.readLock();
private final Lock writeLock = lock.writeLock();
...
writeLock.lock();
try
{
myNonThreadSafeVariable.put(key,this);
}
finally
{
writeLock.unlock();
}
...
writeLock.lock();
try
{
server = myNonThreadSafeVariable,get(key);
sever.clients.add(this);
}
finally
{
writeLock.unlock();
}
...
readLock.lock();
try
{
for (Entry<String, MyServers> entry : myNonThreadSafeVariable.entrySet())
{
MyServers server = entry.getValue();
server.sendMessage("ping||");
for (MyClients member : entry.getValue().clients)
{
client.sendMessage("")
}
}
}
finally
{
readLock.unlock();
}
Moreover if you want to avoid the read lock you can copy the whole collection and scan the copy, letting the possibility for the original collection to be changed while notifying.

Timeout individual threads of cached thread pool

I want to use a thread pool in a web application which should support large number of users concurrently (~3000 users). I am invoking a web service in a separate thread which I am executing using a thread pool. Whenever the web service is not able to send the response, the thread gets stuck. So I want to stop/timeout the thread after 150 Milli-seconds. This is what I am doing right now:
the custom thread:
public class RetrieveDocTask implements Runnable {
public void run() {
//gather variables
//invoke webservice
}}
The filter which executes the threads:
public class DocFilter implements Filter {
private static ExecutorService executor = Executors.newCachedThreadPool();
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
RetrieveDocTask task=new RetrieveDocTask();
executor.execute(task);
}}
I browsed the internet for solutions and none of them worked for me. Some said use Future and callable while some asked to create ThreadPoolExecutor and specify the timeout. Don't know why it was not working.
Also, will it be fine to use cached pool executor for large number of users. I am new to this and need to implement it as soon as possible.
Indeed a Future is what you need here. Suppose that your class RetriveDoc returns a string actually.
private static final class RetrieveDoc implements Callable<String>{
#Override
public String call() throws Exception {
//do some computation and retirieve doc
return "DocAsString";
}
}
ExecutorService service = Executors.newFixedThreadPool(1);
Future<String> futureResponse = service.submit(new RetrieveDoc());
//this will blokc for only 150 milliseconds
String response = null;
try{
response = futureResponse.get(150, TimeUnit.MILLISECONDS);
} catch(TimeoutException e){
System.out.println("TimeoutException happended");
}
if(response == null){
//do something
}

Resources