Does someone know of a good resource for the implementation (meaning source code) of lock-free usual data types. I'm thinking of Lists, Queues and so on?
Locking implementations are extremely easy to find but I can't find examples of lock free algorithms and how to exactly does CAS work and how to use it to implement those structures.
Check out Julian M Bucknall's blog. He describes (in detail) lock-free implementations of queues, lists, stacks, etc.
http://www.boyet.com/Articles/LockfreeQueue.html
http://www.boyet.com/Articles/LockfreeStack.html
http://www.liblfds.org
Written in C.
If C++ is okay with you, take a look at boost::lockfree. It has lock-free Queue, Stack, and Ringbuffer implementations.
In the boost::lockfree::details section, you'll find a lock-free freelist and tagged pointer (ABA prevention) implementation. You will also see examples of explicit memory ordering via boost::atomic (an in-development version of C++0x std::atomic).
Both boost::lockfree and boost::atomic are not part of boost yet, but both have seen attention from the boost-development mailing list and are on the schedule for review.
Related
As question says it all I was looking for in depth explanation of data-dependency barrier in SMP especially with respect to Linux Kernel. I have the definition and brief description handy in this link here.
Linux Kernel Memory Barriers Documentation
However I was attempting to get a profound understanding of this concept. Your thoughts and inputs are highly appreciated.
Actually, at least in terms of C++11, this is more closely related to consume semantics. You can read more about it e.g. here. In short, they provide weaker guarantees than acquire semantics, which makes them more efficient on certain platforms that support data dependency ordering.
In C++ terms, it is called memory_order_consume. See this presentation from Paul Mckenney.
Old answer: I believe "acquire semantics" is the more commonly used term for what the document is calling a "data-dependency barrier". See for example this presentation or the C++11 memory_order_acquire.
Update: per comments, the Linux description for Data Dependency Barriers sounds more like C++ "consume semantics".
I am trying to implement the Michael-Scott FIFO queue from here. I'm unable to implement their solution for the ABA problem. I get this error.
error: incompatible type for argument 1 of '__sync_val_compare_and_swap'
For reference, I am using a linux box to compile this on an intel architecture. If you need more information on my setup please ask.
It seems that sync_val_CAS handles only up to 32 bit values. So when I remove their counter which is used to eliminate the ABA problem, everything compiles and runs fine.
Does anyone know of the relevant 64-bit CAS instruction I should be using here?
As an additional question, are there better (faster) implementations of lock-free fifo queues out there? I came across this by Nir Shavit et al which seems interesting. I am wondering if others have seen similar efforts? Thanks.
Assuming gcc, try using the "march" switch. Something like this: -march=i686
There is also the __sync_bool_compare_and_swap. I don't know if its faster or not.
GCC, last I looked in 2009, does not support contigious double-word CAS. I had to implement in-line assembly.
You can find my implemenation of the M&S queue (including in the abstraction layer the assembly implementation of DCAS) and other lock-free data structures here;
http://www.liblfds.org
Briefly looking at the Nir Shavit et al paper, the queue requires Safe Memory Reclaimation, which I suspect you'll need to implement - it won't be built into the queue. An SMR API will be available in the next release (couple weeks).
Lock-free may not be what you want, since lock-free is not necessarily wait-free. If you need a fast thread-safe queue (not lock-free!), then consider using Threading Building Blocks concurrent_queue.
I've read a lot of articles about distributed Haskell. Much work has been done but seems to be in the area of distributing computations. I saw the remote package which seems to implement Erlang-style messaging passing but it is 0.1 and early stage.
I'd like to implement a system where there are many separate processes that provide distinct services, and are tied together by several main processes. This seems to be a natural fit for Erlang, but not so for Haskell. But I like Haskell's type safety.
Has there been any recent adoption of Erlang-style process management in Haskell?
If you want to learn more about the remote package, a.k.a CloudHaskell, see the paper as well as Jeff Epstein's thesis. It aims to provide precisely the actor abstraction you want, but as you say it is in the early stages. There is active discussion regarding improvements on the parallel-haskell mailing list, so if you have specific needs that remote doesn't provide, we'd be happy for you to jump in and help us decide its future directions.
More mature but lower-level than remote is the haskell-mpi package. If you stick to the Simple interface, messages can be sent containing arbitrary Serialize instances, but the abstraction is still way lower than remote.
There are some experimental systems, such as described in Implementing a High-level Distributed-Memory Parallel Haskell in Haskell (Patrick Maier and Phil Trinder, IFL 2011, can't find a pdf online). It blends a monad-par approach of deterministic dataflow parallelism with a limited ability to make the I-structures serializable over the network. These sorts of abstraction have promise for doing distributed computation, but since the focus is on computing purely-functional values rather than providing Erlang-style processes, they probably wouldn't be a good fit for your application.
Also, for completeness, I should point out the Haskell wiki page on cloud and HPC Haskell, which covers what I describe here, as well as the subsection on distributed Haskell, which seems in need of a refresh.
I frequently get the feeling that IPC and actors are an oversold feature. There are plenty of attractive messaging systems out there that have Haskell bindings e.g. MessagePack, 0MQ or Thrift. IMHO the only thing you have to add is proper addressing of processes and decide who/what is managing this addressing capability.
By the way: a number of coders adopt e.g. 0MQ into their Erlang environments, simply because it offers the possibility to structure messaging via message brokers rather then relying on pure process to process messaging in super scale.
In a "massively multicore world" I personally assume that shared memory approaches will eventually be outperforming messaging. Someone can then always come and argue with asynchrony of course. But already when you write that you want to "tie together" your processes by "several main processes" you in fact speak about synchronization. Also, you can of course challenge whether a single function, process or thread is the right level of parallelization.
In short: I would probably see whether MessagePack or 0MQ could fit my needs in Haskell and care for the rest in my code.
I've been searching for information for a common kernel implementation of queues, that is, first-in-first-out data structures. I thought there may be one since it's likely something that's common to use, and there's a standard for linked lists (in the form of the list_head structure). Is there some standard queue implementation I can't find, or is it perhaps common practice to just use linked lists as queues and hope for the best?
Are you looking for include/linux/kfifo.h?
From the heading:
A simple kernel FIFO implementation.
It's rather new anyway, so it's not hard to find direct usages of linked lists. Also, they have a quite different implementation (FIFOs are implemented as circular buffers), so they have different applications.
Note also they are designed with multithreaded usage in mind (think to producer/consumer queues), but you can use them without locking with __kfifo_put/__kfifo_get.
Btw: I remember I learned about them on lwn.net - bookmark this: lwn.net/Kernel/Index, and read the entry about kfifo :-).
From your ex-kernel developer,
Blaisorblade
You're right, the Linux kernel typically uses linked lists to implement queues. This makes sense, because linked lists offer the required behavior. See this example from kernel/workqueue.c:
INIT_LIST_HEAD(&wq->list);
// ...
case CPU_UP_CANCELED:
list_for_each_entry(wq, &workqueues, list) {
if (!per_cpu_ptr(wq->cpu_wq, hotcpu)->thread)
You seem to confusing an abstraction (a fifo queue) with an implementation (a linked list).
They are not mutually exclusive - in fact queues are most commonly implemented as linked lists - there is no "hoping for the best".
I'm about to tackle what I see as a hard problem, I think. I need to multi-thread a pipeline of producers and consumers.
So I want to start small. What are some practice problems, in varying levels of difficulty, that would be good for multi-threading practice? (And not contrived, impractical examples you see in books not dedicated to concurrency).
What books or references would you recommend that focus on concurrency and give in-depth problems and cases?
(I'd rather not focus on the problem I want to solve. I just want to ask for good references and sample problems. This would be more useful to other users. I'm not stuck on the problem.)
The little book of semaphores is a good free book. The author takes a unique approach of first asking a problem and then presenting hints before answering. The problems increase in difficulty level gradually, and the book isn't written for any language in particular but covers general multithreading concepts.
If you have enough time to invest I would recommend the book "Concurrency: State Models & Java Programs, 2nd Edition" by Jeff Magee and Jeff Kramer, John Wiley&Sons 2006
You can ignore the Java part if you are using some other language
There's a language used to model processes and concurrent processes called FSP. It needs some time and energy to be invested in order to be proficient in the language. There's a tool (LTSA, both are free and supported by an Eclipse plugin or stand alone app) which verifies your models and make you pretty shure that your model is correct from the standpoint of concurrent execution.
Translating this models to your language constructs is then just a question of programming technique and few design patterns.
Most text book problems, like readers-writers, producers-consumers or dinning philosophers are all illustrations of the mutex. I would prefer to model a prototype which is a simplistic approximation the bigger problem and go ahead.
I have some times seen situations where dead-lock avoidance is what is needed and dead-lock prevention measures are being used. It is always a good idea to analyse if Banker's algorithm would suit the case or not.
Completely ignoring your request, I'll suggest that you should look at SEDA (staged event driven architecture) as a way to think about setting up a multi-threaded pipeline of producers and consumers.
I'm not sure what you are looking for. But in real world enterprise situation, we usually use some kind of messaging framework when doing producers consumers stuff. Tipically in Java, that's JMS. And you can use the excellent Spring Framework to help you along.
If you're working with Java at all (and possibly even if you're not), you should definitely read Java Concurrency In Practice.
To be honest, many real-world multithreading programs are not doing much more than reading/writing some value (whether string or int) -- circular buffers (as a network connection might need), readers/writers of log files, etc.
In fact, I'd say that if you implement (or find) a solid (and generic) circular buffer, and then run all thread-to-thread communication through those buffers as the only contact point, that'll cover a very large portion of any multithread syncing you might need to do. (Unless you're working in a buzzword-compliant environment, and need to tack "enterprise", "messaging", or whatever onto the buzzword list... or you're writing a database or operating system.)
(Note that "circular buffer" is a fairly C-centric term, being rooted in the relatively direct manipulation of a block of memory. Python's Queue class implements the same basic principle in a list-centric way, and I'm sure that numerous other languages have conceptually similar constructs under slightly different names...)