I have an application with Many Producers and consumers.
From my understanding, RingBuffer creates objects at start of RingBuffer init and you then copy object when you publish in Ring and get them from it in EventHandler.
My application LogHandler buffers received events in a List to send it in Batch mode further once the list has reached a certain size. So EventHandler#onEvent puts the received object in the list , once it has reached the size , it sends it in RMI to a server and clears it.
My question, is do I need to clone the object before I put in list, as I understand, once consumed they can be reused ?
Do I need to synchronize access to the list in my EventHandler#onEvent ?
Yes - your understanding is correct. You copy your values in and out of the ringbuffer slots.
I would suggest that yes you clone the values as you extract it from the ring buffer and into your event handler list; otherwise the slot can be reused.
You should not need to synchronise access to the list as long as it is a private member variable of your Event Handler and you only have one event handler instance per thread. If you have multiple event handlers adding to the same (eg static) List instance then you would need synchronisation.
Clarification:
Be sure to read the background in OzgurH's comments below. If you stick to using the endOfBatch flag on disruptor and use that to decide the size of your batch, you do not have to copy objects out of the list. If you are using your own accumulation strategy (such as size - as per the question), then you should clone objects out as the slot could be reused before you have had the chance to send.
Also worth noting that if you are needing to synchronize on the list instance, then you have missed a big opportunity with disruptor and will destroy your performance anyway.
It is possible to use slots in the Disruptor's RingBuffer (including ones containing a List) without cloning/copying values. This may be a preferable solution for you depending on whether you are worried about garbage creation, and whether you actually need to be concerned about concurrent updates to the objects being placed in the RingBuffer. If all the objects being placed in the slot's list are immutable, or if they are only being updated/read by a single thread at a time (a precondition which the Disruptor is often used to enforce), there will be nothing gained from cloning them as they are already immune to data races.
On the subject of batching, note that the Disruptor framework itself provides a mechanism for taking items from the RingBuffer in batches in your EventHandler threads. This is approach is fully thread-safe and lock-free, and could yield better performance by making your memory access patterns more predictable to the CPU.
Related
I'm successfully using a mpsc::channel() to send messages from a producer thread to a consumer.
The consumer is only ever interested in the latest message. (It uses the message from the previous check if there is no new message.)
In consequence, I'm running the consumer's try_recv() in a loop until it fails to get a new message, and then using the last received message, or the old one if no new messages were found.
Memory is being wasted storing old messages which the consumer will throw away.
How would I build a one-element variant of mpsc::channel()?
(I've considered using sync::Mutex<Option<MyMessage>> but it is critical that the consuming thread blocks for as little time as possible. Also, I want ownership to pass from the producer to the consumer.)
You can do it with an AtomicPtr, whose compare_exchange method should compile to a simple cmpxchg instruction, allowing you to store either std::ptr::null or an actual message.
There's quite a few possibilities, with various trade-offs.
I'd recommend the arc-swap crate (see below) for a safe and fast interface, and the DIY Double Buffering approach if performance is that critical.
std::mpsc
There's a second option for std::mpsc: the sync_channel function creates a bounded channel, where the sender blocks when the channel is full, until the receiver picks off a message.
I do not think that it is ideal for your usecase.
Tokio Watch channel
The Tokio ecosystem has the watch channel designed for the purpose of propagating configuration changes.
Unfortunately it is designed for multiple consumers, so the consumers borrow the messages: there is no transfer of ownership.
Arc Swap
I believe the arc-swap crate may be closer to what you need. As the name implies, it provides the moral equivalent of an Atomic<Arc<T>>.
You can use the ArcSwapOption<T> to have the equivalent of an Atomic<Option<Arc<T>>>, and the consumer can simply perform a let new = atomic.swap(None); then check if new is None (nothing new) or Some(Arc<T>) in which case it received an updated configuration.
Do be mindful of the cost of the dropping the previous Arc<T> when swapping a new one in: free is typically more expensive than malloc.
Back to std
You could use an AtomicPtr<T>. It'll require you to use unsafe, and would be a smidgen faster than ArcSwap by virtue of avoiding the reference counting.
It would suffer from the same drop issue, though.
DIY Double Buffering
You could also simply Do It Yourself. A simple double-buffering storage would work.
By storing a plain Option<T>, you avoid the additional extra allocation (and thus extra de-allocation), at the cost of making the check itself slower -- as you may now need to check both buffers. It may be possible to check a single buffer, not clear.
I want to implement ring buffer for classic Producer--Consumer interaction. In the future both P and C will be implemented as permanent threads running during data processing task, and GUI will be the third thread only for displaying actual data and coordinate starts and stops of data processing by user interaction. C can be quite slow to be able to fully process all incoming data, but only a bit and for a short periods of time. So I want to just allocate ring buffer of several P's MTUs in size, but in any case, if C will be too slow to process existing data it's okay to loose old data in favor of new one (overwrite policy).
I've read QSemaphore example in Qt help and realized that by usage of semaphore's acquires and releases I can only implement discard policy, because acquiring of specified chunk in queue will block until there are no free space.
Are there any ways of implementing overwrite policy together with QSemaphore or I just need to go and implement another approach?
I've came to this solution. If we should push portion of the src data to the ring buffer at any costs (it's ok to drop possible newly incoming data) we should use acquire() in Producer part - that would provide us discard policy. In case we need overwrite policy we should use tryAcquire() in Producer - thus at the very first possible moment of time only the newest data will be pushed to the ring buffer
I'm experimenting with the System.Collections.Concurrent namespace but I have a problem implementing my design.
My input queue (ConcurrentQueue) is getting populated fine from a Thread which is doing some I/O at startup to read and parse.
Next I kick off a Parallel.ForEach() on the input queue. I'm doing some I/O bound work on each item.
A log item is created for each item processed in the ForEach() and is dropped into a result queue.
What I would like to do is kick off the logging I start reading the input because I may not be able to fit all of the log items in memory. What is the best way to wait for items to land in the result queue? Are there design patterns or examples that I should be looking at?
I think the pattern you're looking for is the producer/consumer pattern. More specifically, you can have a producer/consumer implementation built around TPL and BlockingCollection.
The main concepts you want to read about are:
Task,
BlockingCollection,
TaskFactory.ContinueWhenAll(will allow you to perform some action when a set of tasks/threads is finished running).
Bounding and Blocking in BlockingCollection. This allows you to set a maximum size for your output collection (for memory reasons) and producer thread(s) will wait for consumers to pick up elements in case the maximum size you specify is reached.
BlockingCollection.CompleteAdding and BlockingCollection.IsCompleted which can be used to synchronize producers and consumers (producer can say when it's finished, consumer can check for that and keep running until the producer(s) are finised).
A more complete sample is in the second article I linked.
In your case I think you want the consumer to just pick up things from the result queue and dispose of them as soon as possible (write them to a logging store, or similar).
So your final collection, where you dump log items should be a BlockingCollection, not a ConcurrentQueue.
In our scenario,
the consumer takes at least half-a-second to complete a cycle of process (against a row in a data table).
Producer produces at least 8 items in a second (no worries, we don't mind about the duration of a consuming).
the shared data is simply a data table.
we should never ask producer to wait (as it is a server and we don't want it to wait on this)
How can we achieve the above without locking the data table at all (as we don't want producer to wait in any way).
We cannot use .NET 4.0 yet in our org.
There is a great example of a producer/consumer queue using Monitors at this page under the "Producer/Consumer Queue" section. In order to synchronize access to the underlying data table, you can have a single consumer.
That page is probably the best resource for threading in .NET on the net.
Create a buffer that holds the data while it is being processed.
It takes you half a second to process, and you get 8 items a second... unless you have at least 4 processors working on it, you'll have a problem.
Just to be safe I'd use a buffer at least twice the side needed (16 rows), and make sure it's possible with the hardware.
There is no magic bullet that is going to let you access a DataTable from multiple threads without using a blocking synchronization mechanism. What I would do is to hold the lock for as short a duration as possible. Keep in mind that modifying any object in the data table's hierarchy will require locking the whole data table. This is because modifying a column value on a DataRow can change the internal indexing structures inside the parent DataTable.
So what I would do is from the producer acquire a lock, add a new row, and release the lock. Then in the conumser you will acquire the same lock, copy data contained in a DataRow into a separate data structure, and then release the lock immediately. Now, you can operate on the copied data without synchronization mechanisms since it is isolated. After you have completed the operation on it you will again acquire the lock, merge the changes back into the DataRow, and then release the lock and start the process all over again.
I have four threads in a C++/CLI GUI I'm developing:
Collects raw data
The GUI itself
A background processing thread which takes chunks of raw data and produces useful information
Acts as a controller which joins the other three threads
I've got the raw data collector working and posting results to the controller, but the next step is to store all of those results so that the GUI and background processor have access to them.
New raw data is fed in one result at a time at regular (frequent) intervals. The GUI will access each new item as it arrives (the controller announces new data and the GUI then accesses the shared buffer). The data processor will periodically read a chunk of the buffer (a seconds worth for example) and produce a new result. So effectively, there's one producer and two consumers which need access.
I've hunted around, but none of the CLI-supplied stuff sounds all that useful, so I'm considering rolling my own. A shared circular buffer which allows write-locks for the collector and read locks for the gui and data processor. This will allow multiple threads to read the data as long as those sections of the buffer are not being written to.
So my question is: Are there any simple solutions in the .net libraries which could achieve this? Am I mad for considering rolling my own? Is there a better way of doing this?
Is it possible to rephrase the problem so that:
The Collector collects a new data point ...
... which it passes to the Controller.
The Controller fires a GUI "NewDataPointEvent" ...
... and stores the data point in an array.
If the array is full (or otherwise ready for processing), the Controller sends the array to the Processor ...
... and starts a new array.
If the values passed between threads are not modified after they are shared, this might save you from needing the custom thread-safe collection class, and reduce the amount of locking required.