Generally the events in my disruptor need to be handle sequentially. However, we sometimes receive high priority events which needs to be prioritize to be handled first. Is there such a functionality in Disruptor (like a priority queue in Java?)
Related
I'm just wondering if there would be any reason I might want to lock a queue. I am working on an application that has several threads that reads and writes to a database. In order to reduce traffic, I want to reduce the amount of calls to that database at any given point (I know many databases can handle some traffic already). Would it make any sense to make a queue for the read/write requests and only the request at the top executes and then protect the queue's push and pop commands with a lock? Is having a lock on each read/write call enough? Isn't a lock implemented as a "queue" by the OS anyways? Could size of this "queue" be an issue or would there be any other reason I wouldn't use a lock by itself?
Thanks!
You could limit the number of threads that are engaged in database requests or if that's not feasible due to the nature of your app, you could use a more granular approach to limit access to the shared resource. In python, you can use the built-in semaphore objects for inter-thread synchronization. For inter-process synchronization (or inter-thread), you'd use posix_ipc. It depends what your service's execution model is.
Most database clients wouldn't require any application-level throttling. In a typical system, the database connections would be pooled and the connection manager would be responsible for acquiring an available connection. Internally this usually involves a queue of some sort with timeouts to prevent waiting indefinitely. The database itself would then handle the scheduling of individual operations made by each connection.
However, a semaphore is a signalling primitive that can be used to limit the number of concurrent operations: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Semaphore.html
Tasks can also be modeled as a producer-consumer problem which involves a shared queue, however you'll have to deal with the added complexity of managing the consumer threads in addition to the producers.
Im using VisualVM to profile an application that uses akka-streams-kafka.
It shows a lot of Kafka coordinator blocking threads
Who are these coordinators?
I also have three Kafka consumers that are also blocking threads
Do I need to create a separate Execution context for them?
From Java Kafka client API
One Consumer Per Thread A simple option is to give each thread its own consumer instance. Here are the pros and cons of this approach:
PRO: It is the easiest to implement PRO: It is often the fastest as no
inter-thread co-ordination is needed PRO: It makes in-order processing
on a per-partition basis very easy to implement (each thread just
processes messages in the order it receives them). CON: More consumers
means more TCP connections to the cluster (one per thread). In general
Kafka handles connections very efficiently so this is generally a
small cost. CON: Multiple consumers means more requests being sent to
the server and slightly less batching of data which can cause some
drop in I/O throughput. CON: The number of total threads across all
processes will be limited by the total number of partitions.
In my embedded system running Linux (Ubuntu armhf) I have to communicate between processes.
I'm doing it with TCP sockets. It works great but due the high frequency of my requests I have a very high processor usage (94% average measured whit nmon).
There is a way to lower it using that kind of communication in a more efficient manner?
shared memory and message queues can be used to exchange information between processes. The difference is in how they are used. both have some advantage and disadvantage.
Shared memory
it's an area of storage that can be read and written by more than one process. It provides no inherent synchronization; in other words, it's up to the programmer to ensure that one process doesn't clobber another's data. But it's efficient in terms of throughput: reading and writing are relatively fast operations.
A message queue is a one-way pipe:
one process writes to the queue, and another reads the data in the order it was written until an end-of-data condition occurs. When the queue is created, the message size (bytes per message, usually fairly small) and queue length (maximum number of pending messages) are set. Access is slower than shared memory because each read/write operation is typically a single message. But the queue guarantees that each operation will either processes an entire message successfully or fail without altering the queue. So the writer can never fail after writing only a partial message, and the reader will either retrieve a complete message or nothing at all.
If you wish to stick with your basic architecture, you can switch from TCP sockets to Unix domain sockets (AF_UNIX/AF_LOCAL). Since it's a strictly local protocol, it doesn't have the overhead of TCP.
Can we re-purpose the completion port thread (for async I/O operation) as worker thread in CLR process ThreadPool ?
If this is naïve then can someone suggest me how to maximize the use of thread pool threads in order to reduce number of work item stacked in the worker queue.
The IOCP threads are already sorta 'workers' - they take input from a queue and act on the received items. If you wish to avoid using another thread pool for processing items other than 'normal' IOCP completion objects rx from the network drivers, there is nothing stopping you from 'manually' queueing up objects to the IOCP queue that ask the IOCP pool threads to perform other actions. I forget the actual APIs now, but AFAIK there should be no problem.
I remember using such a mechanism for server tuning - reducing the number of IOCP threads by queueing an item that instructed the receiving IOCP pool thread to terminate.
That said, I'm not sure that such a mechansim will improve throughput significantly - the work has to be done somewhere and it may be that avoiding an extra thread pool would not help much. Empirically, as a general-purpose inter-thread comms mechanism, an IOCP queue has a worse performance than Windows message queues, (useless for thread pools anyway since only one thread can wait), and user-space CS/semaphore-based P-C queues.
Rgds,
Martin
As a followup on this question discussing the use of the ThreadPool vs a dedicated thread:
When would you use a dedicated thread (with lowered priority) for applicationwide logging and when would you use the ThreadPool?
What I would do is completely dependent on the requirements of my app and its logging component.
If logging is mission-critical (maybe you need the ability to replay recent traffic based on the log, for example) then a dedicated thread is more likely the right approach.
If logging is 'best effort', then ThreadPool would be fine subject to other constraints on your app's required performance and latency. Async I/O for the logger would be fine here. Since you suggest lower priority for your putative logger thread, this may match your app's profile.
If more critical work is happening on the ThreadPool then I would not overload it to do logging, esp. if logging itself is important - you could well be doing synchronous, flushed I/O to write out the logs and that's a possible bottleneck depending on the volume of stuff that you wish to log.
If logging is not critical and you want to do it asynchronously then I would recommend using a single background thread for logging and a producer/consumer queue to send log messages. This can achieve improved performance over threadpool since you have a single thread performing I/O on less-critical logs which would have less of a likelyhood to block higher-priority I/O on other threads.
You can also use this mechanism to make sure critical logs are committed before logging. Add them to the queue and then have a mechanism to wait until that particular message is committed.