I have a Java servlet that acts as a facade to other webservices deployed on the same Tomcat instance. My wrapper servlet creates N more threads, each which invokes a webservice, collates the response and sends it back to the client. The webservices are deployed all on the same Tomcat instance as different applications.
I am seeing thread blocking on this facade wrapper service after a few hours of deployment which brings down the Tomcat instance. All blocked threads are endpoints to this facade webservice (like http://domain/appContext/facadeService)
Is there a way to control such thread-blocking, due to starvation of available threads that actually do the processing? What are the best practices to prevent such deadlocks?
The common solution to this problem is to use the Executor framework. You need to express your web service call as Callable and pass it to the executor either as it stands, or as a Collection<Callable> (see the Javadoc for complete list of options).
You have two choices to control the time. First is to use parameters of an appropriate method of the Executor class where you specify the max web service timeout. Another option is to do get the result (which is expressed as Future<T>) and use .get(long, TimeUnit) to specify the maximum amount of time you can wait for a result.
Related
we are facing an issue with initializing our cache at server startup or application deployment. Initializing the cache involves
Querying a database to get the list of items
Making an rmi call for each item
Listening to the data on a JMS queue/topic
Constructing the cache
This initialization process is in startup code. All this is taking lot of time due to which the deployment is taking lot of time or server start time is increasing.
So what I proposed is to create a thread in the startup and run the initialization code in it. I wrote a sample application to demonstrate it.
It involves a ServletContextListener, a filter. In the listener I am creating a new thread in which the HeavyProcess will run. When it finishes an event will be fired which the filter will be listening. On receiving the event the filter will allow incoming http requests. Until then the filter redirects all clients to a default page which shows a message that the application is initializing.
I presented this approach and few concerns were raised.
We should not ideally create a thread because handling the thread will be difficult.
My question is why cant we create a thread like these in web applications.
If this is not good, then what is the best approach?
If you can use managed threads, avoid unmanaged ones. The container has no control over unmanaged threads, and unmanaged threads survive redeployments, if you do not terminate these properly. So you have to register unmanaged threads, and terminate these somehow (which is not easy as well, because you have to handle race-conditions carefully).
So one solution is to use #Startup, and something like this:
#Schedule(second = "*/45", minute = "*", hour = "*")
protected void asyncInit(final Timer timer) {
timer.cancel();
// Do init here
// Set flag that init has been completed
}
I have learned about this method here: Executing task after deployment of Java EE application
So this gives you an async managed thread, and deployment will not be delayed by #PostConstruct. Note the timer.cancel().
Looking at your actual problem: I suggest using a cache which supports "warm starts".
For example, Infinispan supports cache stores so that the cache content survives restarts. If you have a cluster, there are distributed or replicated caching modes as well.
JBoss 7 embeds Infinispan (it's an integrated service in the same JVM), but it can be operated independently as well.
Another candidate is Redis (and any other key/value store with persistence will do as well).
In general, creating unmanaged threads in a Java EE environment is a bad idea. You will loose container managed transactions, user context and many more Java EE concepts within your unmanaged thread. Additionally unmanaged threads may block the conainer on shutdown if your thread handling isn't appropriate.
Which Java EE Version are you using? Perhaps you can use Servlet 3.0's async feature?
Or call a asynchronous EJB for doing the heavy stuff at startup (#PostConstruct). The call will then set a flag when its job is done.
My understanding is that in Tomcat, each request will take up one Java/(and thus OS) thread.
Imagine I have an app with lots of long-running requests (eg a poker game with multiple players,) that involves in-game chat, and AJAX long-polling etc.
Is there a way to change the tomcat configuration/architecture for my webapp so that I'm not using a thread for each request but 'intercept' the request and response so they can be processed as part of a queue?
I think you're right about tomcat likes to handle each request in its own thread. This could be problematic for several concurrent threads. I have the following suggestions:
Configure maxThreads and acceptCount attributes of the Connector elements in server.xml. In this way you limit the number of threads that can get spawned to a threshold. Once that limit is reached, requests get queued. The acceptCount attribute is to set this queue size. Simplest to implement but not a good long term solution
Configure multiple Connector elements in server.xml and make them share a threadpool by adding an Executor element in server.xml. You probably want to point tomcat to your own implementation of Executor interface.
If you want finer grain control no how requests are serviced, consider implementing your own connector. The 'protocol' attribute of the Connector element in server.xml should point to your new connector. I have done this to add a custom SSL connector and this works great.
Would you reduce this problem to a general requirement to make tomcat more scalable in terms of the number of requests/connections? The generic solution to that would be configuring a loadbalancer to handle multiple instances of tomcat.
I have written a web service using spring, cxf and jax ws implementation and I have a basic question on WS. How does a Web Service endpoint handles concurrent requests? Does it creates a new thread for each and every request similar to a servlet or it ia a single threaded model? As we are expecting a huge volume for each web service, Does it makes any difference to slipt WSDL to multiple WSDLs to have different end points?
The web service is, of course, hosted by a web server (like Glassfish for example), which is multithreaded when receiving multiple simultaneous requests.
From the perspective of both your client and your service, there's no such thing as "multithreading". Your client invokes a request, and gets a response (possibly a fault response). Your server receives a request, and services that request. Period.
How the request is dispatched is an implementation detail.
And the WSDL is simply a "contract". The service "publishes" what operations it supports and what data types it uses with the WSDL; the client packs and unpacks his request and response SOAP messages accordingly. But a WSDL plays no direct role in any given web service invocation.
Its late but might help.
Endpoint.publish(Url, ServiceImplObj) publishes a webservice at a given url. The no. of threads assigned for request handling truly is under control of the jvm because this is a light weight deployment which is handled by jvm itself.
For better clarification you can print the current thread name at service side and you can see that the service threads are being assigned from a thread pool which is managed by jvm.
[pool-1-thread-1]: Response[57]:
[pool-1-thread-5]: Response[58]:
[pool-1-thread-4]: Response[59]:
[pool-1-thread-3]: Response[60]:
[pool-1-thread-6]: Response[61]:
[pool-1-thread-6]: Response[62]:
This i tried on jdk 1.6.0_35.
xjc -version
xjc version "JAXB 2.1.10 in JDK 6"
JavaTM Architecture for XML Binding(JAXB) Reference Implementation, (build JAXB
2.1.10 in JDK 6)
When an EJB application receives several requests (work load) it can manage this work load just POOLING the EJBs, so when each EJB object is being used by a thread, the next threads will have to wait queued until some EJB ends up the work (avoiding overloading and efficiency degradation of the system).
Spring is using stateless singletons (not pooling at all) that are used by an "out of control" number of threads.
Is there a way to do something to control the way the work load is going to be delivered? (equivalent to the EJB instance pooling).
Thank you!
In the case of the web app, the servlet container has a pool of threads that determine how many incoming HTTP requests it can handle simultaneously. In the case of the message driven POJO the JMS configuration defines a similar thread pool handing incoming JMS messages. Each of these threads would then access the Spring beans.
Googling around for RMI threading it looks like there is no way to configure thread pooling for RMI. Each RMI client is allocated a thread. In this case you could use Spring's Task Executor framework to do the pooling. Using <task:executor id="executor" pool-size="10"/> in your context config will set up a executor with 10 threads. Then annotate the methods of your Spring bean that will be handling the work with #Async.
Using the Spring task executor you could leave the Servlet and JMS pool configuration alone and configure the pool for your specific work in one place.
To achieve a behaviour similar to the EJB pooling, you could define your own custom scope. Have a look at SimpleThreadScope and the example referenced from this class' javadoc.
The difference between Spring and EJB is, that Spring allows multiple threads on an single instance of an bean, while in EJB you have only one tread per bean (at one point in time).
So you do not need any pooling in Spring for this topic. But on the other hand you need take care that you implement your beans in a threadsave way.
From the comments:
Yes I need it if I want to limit the number of threads that can use my beans simultaneously
One (maybe not the best) way to handle this is to implement the application in normal spring style (no limits). And than have a "front-controller" that accept the client request. But instead of invoking the service directly, it invokes the service asyncron (#Async). May you use some kind of async proxy instead of making the service itselfe asyncron.
class Controller{...
Object doStuff() {return asyncProxy.doStuffAsync().get();}
}
class AsyncProxy{...
#Async Future<Object> duStuffAscny{return service.doStuff();
}
class Service{...
Object doStuff{return new Object();}
}
Then you only need to enable springs Async Support, and there you can configure the Pool used for the Threads.
In this case I would use some kind of front controller, that starts an new Async
I'm using multithreaded wcf maxConcurrentCalls = 10. By logging calls to my service I see that 10 different threads are executing in my service class and that they are reused in the following calls.
Can I tell WCF to destroy/delete a thread so it will create a new one on the next call?
This is because I have thread-static state that I sometimes want to be cleared (on unexpected exceptions). I am using the thread-static scope to gain performance.
WCF doesn't create new threads. It uses threads from a thread pool to service requests. So when a request begins it draws a thread from this pool to execute the request and after it finishes it returns the thread to the pool. The way that WCF uses threads underneath is an implementation detail that you should not rely on. You should never use Thread Static in ASP.NET/WCF to store state.
In ASP.NET you should use HttpContext.Items and in WCF OperationContext to store some state that would be available through the entire request.
Here's a good blog post you may take a look at which illustrates a nice way to abstract this.