Is this possible to apply bucketisation in Spring Boot (TOMCAT) - multithreading

Question

I exposed 2 api's
/endpoint/A and /endpoint/B .
#GetMapping("/endpoint/A")
public ResponseEntity<ResponseA> controllerA() throws InterruptedException {
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
#GetMapping("/endpoint/B")
public ResponseEntity<ResponseA> controllerB() throws InterruptedException {
ResponseA responseB = serviceB.responseClient();
return ResponseEntity.ok().body(responseB);
}
Services implemented regarding endpoint A internally call /endpoint/C and endpoint B internally call /endpoint/D.
As external service /endpoint/D taking more time i.e getting response from /endpoint/A takes more time hence whole threads are stucked that is affecting /endpoint/B.
I tried to solve this using executor service having following implementation
#Bean(name = "serviceAExecutor")
public ThreadPoolTaskExecutor serviceAExecutor(){
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(100);
taskExecutor.setMaxPoolSize(120);
taskExecutor.setQueueCapacity(50);
taskExecutor.setKeepAliveSeconds(120);
taskExecutor.setThreadNamePrefix("serviceAExecutor");
return taskExecutor;
}
Even after implementing this if I received more than 200 request on /endpoint/A simultaneously (greater than default max number of threads in Tomcat server) then I am not getting responses from /endpoint/B as all threads are busy for getting response from endpoint A or in queue.
Can someone plz suggest is there any way to apply bucketization on each exposed endpoint level and allow only limited request to process at a time & put remaining into bucket/queue so that request on other endpoints can work properly ?
Edit:- following is solution approach
#GetMapping("/endpoint/A")
public CompletableFuture<ResponseEntity<ResponseA>> controllerA() throws InterruptedException {
return CompletableFuture.supplyAsync(()->controllerHelperA());
}
#GetMapping("/endpoint/B")
public CompletableFuture<ResponseEntity<ResponseB>> controllerB() throws InterruptedException {
return CompletableFuture.supplyAsync(()->controllerHelperB());
}
private ResponseEntity<ResponseA> controllerHelperA(){
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
private ResponseEntity<ResponseB> controllerHelperB(){
ResponseB responseB = serviceB.responseClient();
return ResponseEntity.ok().body(responseB);
}

Answer 1

Spring MVC supports the async servlet API introduced in Servlet API 3.0. To make it easier when your controller returns a Callable, CompletableFuture or DeferredResult it will run in a background thread and free the request handling thread for further processing.
#GetMapping("/endpoint/A")
public CompletableFuture<ResponseEntity<ResponseA>> controllerA() throws InterruptedException {
return () {
return controllerHelperA();
}
}
private ResponseEntity<ResponseA> controllerHelperA(){
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
Now this will be executed in a background thread. Depending on your version of Spring Boot and if you have configured your own TaskExecutor it will either
use the SimpleAsycnTaskExecutor (which will issue a warning in your logs),
the default provided ThreadPoolTaskExecutor which is configurable through the spring.task.execution namespace
Use your own TaskExecutor but requires additional configuration.
If you don't have a custom TaskExecutor defined and are on a relatively recent version of Spring Boot 2.1 or up (IIRC) you can use the following properties to configure the TaskExecutor.
spring.task.execution.pool.core-size=20
spring.task.execution.pool.max-size=120
spring.task.execution.pool.queue-capacity=50
spring.task.execution.pool.keep-alive=120s
spring.task.execution.thread-name-prefix=async-web-thread
Generally this will be used to execute Spring MVC tasks in the background as well as regular #Async tasks.
If you want to explicitly configure which TaskExecutor to use for your web processing you can create a WebMvcConfigurer and implement the configureAsyncSupport method.
#Configuration
public class AsyncWebConfigurer implements WebMvcConfigurer {
private final AsyncTaskExecutor taskExecutor;
public AsyncWebConfigurer(AsyncTaskExecutor taskExecutor) {
this.taskExecutor=taskExecutor;
}
public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
configurer.setTaskExecutor(taskExecutor);
}
}
You could use an #Qualifier on the constructor argument to specify which TaskExecutor you want to use.