ExecutorService: calling future.get(long, TimeUnit) does not cause the queued Callable to run - executorservice

I try to implement an asynchronous DNS resolver by calling all the routines that perform a DNS query in a separate thread using ThreadPoolExecutor.
I define Callable object like this:
public class SocketAddressCreator extends DnsCallable<String, InetSocketAddress> {
private static final Logger log = Logger.getLogger(SocketAddressCreator.class);
private int port;
public SocketAddressCreator(String host, int port) {
super(host);
this.port = port;
}
public InetSocketAddress call() throws Exception {
log.info("Starting to resolve. Host is: " + target + " .Port is: " + port);
long start = System.currentTimeMillis();
**InetSocketAddress addr = new InetSocketAddress(target, port);**
log.info("Time waiting: " + (System.currentTimeMillis() - start));
return addr;
}
}
Basically the callable object will attempt to resolve the hostname into an InetAddress.
Then I define an ExecutorService:
executor = new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(), new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = Executors.defaultThreadFactory()
.newThread(r);
t.setName("DnsResolver");
t.setDaemon(true);
return t;
}
});
And I submit the Callable task:
..............
**Future<V> f = executor.submit(task);**
try {
log.info("Query will be made");
log.info("Queue size: " + executor.getQueue().size());
**result = f.get(timeout, TimeUnit.MILLISECONDS);**
log.info("Queue size: " + executor.getQueue().size());
log.info("Query is finished");
} catch (TimeoutException e) {
boolean isCancelled = f.cancel(true);
log.info("Task was cancelled: " + isCancelled);
log.info("Queue size: " + executor.getQueue().size());
..........
}
..............
Then I watch the logs that are thrown by my program and they are quite strange.
This is where I have a timeout in resolving the DNS:
DnsResolver : Queue size: 1
DnsResolver : Task was cancelled: true
DnsResolver : Queue size: 1
So after submitting my Callable object but before calling future.get(long, TimeUnit) the queue size is 1. But that's ok for me.
However after I catch the TimeoutException and I cancel the Future, the queue size is the same (one). In my program there is only one thread which submits the Callable tasks to the ExecutorService and the same thread will also retrieve the results.
More than that, there is a even stranger issue here: the Callable.call() method is not called because if it were called I would get a log message:
log.info("Starting to resolve. Host is: " + target + " .Port is: " + port);
So how it is possible for the future.get(long, TimeUnit) method to throw a TimeoutException when the Callable is not called?

The following calls that make DNS queries:
1/ new InetSocketAddress(String, int) - name lookup
2/ InetAddress.getByName(String) - name lookup
3/ InetAddress.getHostName() - reverse name lookup
are NON-INTERRUPTIBLE blocking calls!
As I said before I use a thread pool composed from a single thread. I did not realized that it is necessary to have multiple threads
So if I catch the TimeoutException from future.get(long, TimeUnit) call, and after that I try to cancel the tasks in progress by calling future.cancel(boolean)... I do not stop the single running thread from what it is doing.
I try to simulate a long running DNS query and I modified resolv.conf like this:
nameserver X.X.X.X // this address does not have a valid DNS server!
options timeout:30
I want for the DNS client to block for some time before returning me a negative/positive response.
I have done a load testing on my applicaation and...it's a total disaster! That is because I have a single thread that resolves these DNS queries and calling future.get(long, TimeUnit) does not make it stop!
Of course, I can increase the thread pool size. I have done that and it fixes my issue.
But...it seems silly to have more than one thread in my pool size to resolve these DNS queries because there is only one thread that submits the Callables that are supposed to resolve the DNS queries and the same thread will also get also the results.

Related

AsyncHttpClient creates how much threads?

I use async http client in my code to asynchronously handle GET responses
I can run simultaneously 100 requests in the same time.
I use just on instance of httpClient in container
#Bean(destroyMethod = "close")
open fun httpClient() = Dsl.asyncHttpClient()
Code looks like
fun method(): CompletableFuture<String> {
return httpClient.prepareGet("someUrl").execute()
.toCompletableFuture()
.thenApply(::getResponseBody)
}
It works fine functionally. In my testing I use mock endpoint with the same url address. But my expectation was that all the requests are handled in several threads, but in profiler I can see that 16 threads are created for AsyncHttpClient, and they aren't destroyed, even if there are no requests to send.
My expectation was that
it will be less threads for async client
threads will be destroyed after some configured timeout
is there some option to control how much threads can be created by asyncHttpClient?
Am I missing something in my expectations?
UPDATE 1
I saw instruction on https://github.com/AsyncHttpClient/async-http-client/wiki/Connection-pooling
I found no info on thread pool
UPDATE 2
I also created method to do the same, but with handler and additional executor pool
Utility method look like
fun <Value, Result> CompletableFuture<Value>.handleResultAsync(executor: Executor, initResultHandler: ResultHandler<Value, Result>.() -> Unit): CompletableFuture<Result> {
val rh = ResultHandler<Value, Result>()
rh.initResultHandler()
val handler = BiFunction { value: Value?, exception: Throwable? ->
if (exception == null) rh.success?.invoke(value) else rh.fail?.invoke(exception)
}
return handleAsync(handler, executor)
}
The updated method look like
fun method(): CompletableFuture<String> {
return httpClient.prepareGet("someUrl").execute()
.toCompletableFuture()
.handleResultAsync(executor) {
success = {response ->
logger.info("ok")
getResponseBody(response!!)
}
fail = { ex ->
logger.error("Failed to execute request", ex)
throw ex
}
}
}
Then I can see that result of GET method is executed in the threads provided by thread pool (previously result was executed in "AsyncHttpClient-3-x"), but additional thread for AsyncHttpClient are still created and not destroyed.
AHC has two types of threads:
For I/O operation.
On your screen, it's AsyncHttpClient-x-x
threads. AHC creates 2*core_number of those.
For timeouts.
On your screen, it's AsyncHttpClient-timer-1-1 thread. Should be
only one.
Source: issue on GitHub: https://github.com/AsyncHttpClient/async-http-client/issues/1658

Using thenAccept after supplyAsync blocks the main thread

I am developing a web application which communicates with other web applications. From time to time, my system sends HTTP request as notification to other systems. Since their responses are not essential to me, I send the requests with Java 8 CompletableFuture supplyAsync and prints their responses with thenAccept so that my main thread will not get blocked. However, I found the CompletableFuture function chains took around 100 to 200 ms each time, which confused me because from my understanding thenAccept() should run in the same thread with supplyAsync()'s.
I mocked my process with below codes
public static void run() {
long start = System.currentTimeMillis();
log.info("run start -> " + new Timestamp(start));
CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
return 42;
}).thenAccept(res -> log.info("run result -> " + res + ", time -> " + new Timestamp(System.currentTimeMillis())));
log.info("run duration ->" + (System.currentTimeMillis() - start));
}
public static void runAsync() {
long start = System.currentTimeMillis();
log.info("runAsync start -> " + new Timestamp(start));
CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
return 42;
}).thenAcceptAsync(res -> log.info("runAsync result -> " + res + ", time ->" + new Timestamp(System.currentTimeMillis())));
log.info("runAsync duration ->" + (System.currentTimeMillis() - start));
}
public static void main(String[] args) throws InterruptedException {
Test.run();
Test.runAsync();
Thread.sleep(1000);
}
the run() method uses thenAccept() with supplyAsync() while runAsync() uses thenAcceptAsync(). I expected both of them should take just a few milliseconds. However, the real outputs are:
10:04:54.632 [main] INFO Test - run start -> 2017-12-08 10:04:54.622
10:04:54.824 [main] INFO Test - run duration ->202
10:04:54.824 [main] INFO Test - runAsync start -> 2017-12-08 10:04:54.824
10:04:54.826 [main] INFO Test - runAsync duration ->2
10:04:55.333 [ForkJoinPool.commonPool-worker-1] INFO Test - run result -> 42, time -> 2017-12-08 10:04:55.333
10:04:55.333 [ForkJoinPool.commonPool-worker-3] INFO Test - runAsync result -> 42, time ->2017-12-08 10:04:55.333
We can see run() takes 202 ms which is 100 times of the duration of runAsync() which uses only 2 ms.
I don't understand where is the 202 ms overhead comes from, and obviously it is not the lambda function in supplyAysnc() which sleeps 500 ms.
Could anyone explain why the run() method blocks, and should I always use thenAcceptAsync() over thenAccept() ?
Many thanks.
…because from my understanding thenAccept() should run in the same thread with supplyAsync()'s
Your understanding is wrong.
From the documentation of CompletableFuture:
Actions supplied for dependent completions of non-async methods may be performed by the thread that completes the current CompletableFuture, or by any other caller of a completion method.
The most obvious consequence is that when a future is already completed, the function passed to thenAccept() will be evaluated directly in the caller’s thread, as the future has no possibility to command the thread which completed it. In fact, there is no association of a CompletableFuture with a thread at all, as anyone could call complete on it, not just the thread executing the Supplier you passed to supplyAsync. That’s also the reason why cancel does not support interruption. The future doesn’t know which thread(s) could potentially try to complete it.
The not so obvious consequence is that even the behavior described above is not guaranteed. The phrase “or by any other caller of a completion method” does not restrict it to the caller of the completion method registering the dependent action. It could also be any other caller registering a dependent action on the same future. So if two threads are calling thenApply concurrently on the same future, either of them could end up evaluating both functions or even weirder, each thread could end up executing the other thread’s action. The specification does not preclude it.
For the test case you have provided in your question, you are more likely to measure the initial­ization overhead, as described in this answer. But for the actual problem in your web application where the framework will be initialized only once, you’re likely stumbling over the wrong under­standing of thenApply’s behavior (or any non-async chaining method in general). If you want to be sure that the evaluation does not happen in the caller’s thread, you must use thenApplyAsync.
The 200 ms are startup time for the thread pool and all the classes supporting it.
It becomes obvious if you swap the statements in your main class:
public static void main(String[] args) throws InterruptedException {
Test.runAsync();
Test.run();
Thread.sleep(1000);
}
now Test.runAsync(); is the call that needs 200 ms and Test.run(); completes in 2 ms

Worker stuck in a Sandbox?

Trying to figure out why I can login with my rest API just fine on the main thread but not in a worker. All communication channels are operating fine and I am able to load it up no problem. However, when it tries to send some data it just hangs.
[Embed(source="../bin/BGThread.swf", mimeType="application/octet-stream")]
private static var BackgroundWorker_ByteClass:Class;
public static function get BackgroundWorker():ByteArray
{
return new BackgroundWorker_ByteClass();
}
On a test script:
public function Main()
{
fBCore.init("secrets", "my-firebase-id");
trace("Init");
//fBCore.auth.addEventListener(FBAuthEvent.LOGIN_SUCCES, hanldeFBSuccess);
fBCore.auth.addEventListener(AuthEvent.LOGIN_SUCCES, hanldeFBSuccess);
fBCore.auth.addEventListener(IOErrorEvent.IO_ERROR, handleIOError);
fBCore.auth.email_login("admin#admin.admin", "password");
}
private function handleIOError(e:IOErrorEvent):void
{
trace("IO error");
trace(e.text); //Nothing here
}
private function hanldeFBSuccess(e:AuthEvent):void
{
trace("Main login success.");
trace(e.message);//Complete success.
}
When triggered by a class via an internal worker channel passed from Main on init:
Primordial:
private function handleLoginClick(e:MouseEvent):void
{
login_mc.buttonMode = false;
login_mc.play();
login_mc.removeEventListener(MouseEvent.CLICK, handleLoginClick);
log("Logging in as " + email_mc.text_txt.text);
commandChannel.send([BGThreadCommand.LOGIN, email_mc.text_txt.text, password_mc.text_txt.text]);
}
Worker:
...
case BGThreadCommand.LOGIN:
log("Logging in with " + message[1] + "::" + message[2]); //Log goes to a progress channel and comes to the main thread reading the outputs successfully.
fbCore.auth.email_login(message[1], message[2]);
fbCore.auth.addEventListener(AuthEvent.LOGIN_SUCCES, loginSuccess); //Nothing
fbCore.auth.addEventListener(IOErrorEvent.IO_ERROR, handleLoginIOError); //Fires
break;
Auth Rest Class: https://github.com/sfxworks/FirebaseREST/blob/master/src/net/sfxworks/firebaseREST/Auth.as
Is this a worker limitation or a security sandbox issue? I have a deep feeling it is the latter of the two. If that's the case how would I load the worker in a way that also gives it the proper permissions to act?
Completely ignored the giveAppPrivelages property in the createWorker function. Sorry Stackoverflow. Sometimes I make bad questions when I get little (or none in this case) sleep the night before.

Does Google Guava Cache load on same thread?

Does Google Guava Cache load the cache on the same thread by default?
Code:
cache = CacheBuilder
.newBuilder()
.refreshAfterWrite(2, TimeUnit.SECONDS)
.build(new CacheLoader<String,String>() {
#Override
public String load(String s) throws Exception {
return addCache(s);
}
});
Will the call to addCache be made on a different thread? As far as I know, it is a synchronous call but I am not sure.
Here's a simple test allowing to know:
System.out.println("Thread.currentThread() = " + Thread.currentThread());
LoadingCache<String, String> cache = CacheBuilder
.newBuilder()
.refreshAfterWrite(2, TimeUnit.SECONDS)
.build(new CacheLoader<String, String>() {
#Override
public String load(String s) throws Exception {
System.out.println("Thread.currentThread() = " + Thread.currentThread());
return "world";
}
});
cache.get("hello");
Output:
Thread.currentThread() = Thread[main,5,main]
Thread.currentThread() = Thread[main,5,main]
Of course, as the documentation indicates, if another thread has already started loading the value for the key, the current thread won't reload it: it will wait for the value to be loaded by the other one:
If another call to get(K) or getUnchecked(K) is currently loading the value for key, simply waits for that thread to finish and returns its loaded value.
To add to JB Nizet's answer, you can find out why Guava avoids making the cache multi-threaded by default here:
The reason for this is as follows: if we wanted to perform Cache maintenance continuously, we would need to create a thread, and its operations would be competing with user operations for shared locks. Additionally, some environments restrict the creation of threads, which would make CacheBuilder unusable in that environment.

How to parallelize an azure worker role?

I have got a Worker Role running in azure.
This worker processes a queue in which there are a large number of integers. For each integer I have to do processings quite long (from 1 second to 10 minutes according to the integer).
As this is quite time consuming, I would like to do these processings in parallel. Unfortunately, my parallelization seems to not be efficient when I test with a queue of 400 integers.
Here is my implementation :
public class WorkerRole : RoleEntryPoint {
private readonly CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
private readonly ManualResetEvent runCompleteEvent = new ManualResetEvent(false);
private readonly Manager _manager = Manager.Instance;
private static readonly LogManager logger = LogManager.Instance;
public override void Run() {
logger.Info("Worker is running");
try {
this.RunAsync(this.cancellationTokenSource.Token).Wait();
}
catch (Exception e) {
logger.Error(e, 0, "Error Run Worker: " + e);
}
finally {
this.runCompleteEvent.Set();
}
}
public override bool OnStart() {
bool result = base.OnStart();
logger.Info("Worker has been started");
return result;
}
public override void OnStop() {
logger.Info("Worker is stopping");
this.cancellationTokenSource.Cancel();
this.runCompleteEvent.WaitOne();
base.OnStop();
logger.Info("Worker has stopped");
}
private async Task RunAsync(CancellationToken cancellationToken) {
while (!cancellationToken.IsCancellationRequested) {
try {
_manager.ProcessQueue();
}
catch (Exception e) {
logger.Error(e, 0, "Error RunAsync Worker: " + e);
}
}
await Task.Delay(1000, cancellationToken);
}
}
}
And the implementation of the ProcessQueue:
public void ProcessQueue() {
try {
_queue.FetchAttributes();
int? cachedMessageCount = _queue.ApproximateMessageCount;
if (cachedMessageCount != null && cachedMessageCount > 0) {
var listEntries = new List<CloudQueueMessage>();
listEntries.AddRange(_queue.GetMessages(MAX_ENTRIES));
Parallel.ForEach(listEntries, ProcessEntry);
}
}
catch (Exception e) {
logger.Error(e, 0, "Error ProcessQueue: " + e);
}
}
And ProcessEntry
private void ProcessEntry(CloudQueueMessage entry) {
try {
int id = Convert.ToInt32(entry.AsString);
Service.GetData(id);
_queue.DeleteMessage(entry);
}
catch (Exception e) {
_queueError.AddMessage(entry);
_queue.DeleteMessage(entry);
logger.Error(e, 0, "Error ProcessEntry: " + e);
}
}
In the ProcessQueue function, I try with different values of MAX_ENTRIES: first =20 and then =2.
It seems to be slower with MAX_ENTRIES=20, but whatever the value of MAX_ENTRIES is, it seems quite slow.
My VM is a A2 medium.
I really don't know if I do the parallelization correctly ; maybe the problem comes from the worker itself (which may be it is hard to have this in parallel).
You haven't mentioned which Azure Messaging Queuing technology you are using, however for tasks where I want to process multiple messages in parallel I tend to use the Message Pump Pattern on Service Bus Queues and Subscriptions, leveraging the OnMessage() method available on both Service Bus Queue and Subscription Clients:
QueueClient OnMessage() - https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.queueclient.onmessage.aspx
SubscriptionClient OnMessage() - https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.subscriptionclient.onmessage.aspx
An overview of how this stuff works :-) - http://fabriccontroller.net/blog/posts/introducing-the-event-driven-message-programming-model-for-the-windows-azure-service-bus/
From MSDN:
When calling OnMessage(), the client starts an internal message pump
that constantly polls the queue or subscription. This message pump
consists of an infinite loop that issues a Receive() call. If the call
times out, it issues the next Receive() call.
This pattern allows you to use a delegate (or anonymous function in my preferred case) that handles the receipt of the Brokered Message instance on a separate thread on the WaWorkerHost process. In fact, to increase the level of throughput, you can specify the number of threads that the Message Pump should provide, thereby allowing you to receive and process 2, 4, 8 messages from the queue in parallel. You can additionally tell the Message Pump to automagically mark the message as complete when the delegate has successfully finished processing the message. Both the thread count and AutoComplete instructions are passed in the OnMessageOptions parameter on the overloaded method.
public override void Run()
{
var onMessageOptions = new OnMessageOptions()
{
AutoComplete = true, // Message-Pump will call Complete on messages after the callback has completed processing.
MaxConcurrentCalls = 2 // Max number of threads the Message-Pump can spawn to process messages.
};
sbQueueClient.OnMessage((brokeredMessage) =>
{
// Process the Brokered Message Instance here
}, onMessageOptions);
RunAsync(_cancellationTokenSource.Token).Wait();
}
You can still leverage the RunAsync() method to perform additional tasks on the main Worker Role thread if required.
Finally, I would also recommend that you look at scaling your Worker Role instances out to a minimum of 2 (for fault tolerance and redundancy) to increase your overall throughput. From what I have seen with multiple production deployments of this pattern, OnMessage() performs perfectly when multiple Worker Role Instances are running.
A few things to consider here:
Are your individual tasks CPU intensive? If so, parallelism may not help. However, if they are mostly waiting on data processing tasks to be processed by other resources, parallelizing is a good idea.
If parallelizing is a good idea, consider not using Parallel.ForEach for queue processing. Parallel.Foreach has two issues that prevent you from being very optimal:
The code will wait until all kicked off threads finish processing before moving on. So, if you have 5 threads that need 10 seconds each and 1 thread that needs 10 minutes, the overall processing time for Parallel.Foreach will be 10 minutes.
Even though you are assuming that all of the threads will start processing at the same time, Parallel.Foreach does not work this way. It looks at number of cores on your server and other parameters and generally only kicks off number of threads it thinks it can handle, without knowing too much about what's in those threads. So, if you have a lot of non-CPU bound threads that /can/ be kicked off at the same time without causing CPU over-utilization, default behaviour will not likely run them optimally.
How to do this optimally:
I am sure there are a ton of solutions out there, but for reference, the way we've architected it in CloudMonix (that must kick off hundreds of independent threads and complete them as fast as possible) is by using ThreadPool.QueueUserWorkItem and manually keeping track number of threads that are running.
Basically, we use a Thread-safe collection to keep track of running threads that are started by ThreadPool.QueueUserWorkItem. Once threads complete, remove them from that collection. The queue-monitoring loop is indendent of executing logic in that collection. Queue-monitoring logic gets messages from the queue if the processing collection is not full up to the limit that you find most optimal. If there is space in the collection, it tries to pickup more messages from the queue, adds them to the collection and kick-start them via ThreadPool.QueueUserWorkItem. When processing completes, it kicks off a delegate that cleans up thread from the collection.
Hope this helps and makes sense

Resources