I am reading 'The Art of Multiprocessor Programming'. Now, i get stuck at third chapter , because do not understand the notion of Quiescent Consistency.Can you give me a example to expain what is Quiescent Consistency?Please as clearly as possible!
In distributed programming, there is an approach to describe data
structure behavior known as quiescent consistency. There is a number
of consistency conditions, sequential consistency, linearizability and
others. These conditions describe how an object behaves when there are
several threads calling its methods.
A data structure possesses quiescent consistency if it is consistent
between its states of quiescence, i.e. when there are no methods
currently in progress. As soon as a quiescently consistent structure
has no operations pending (i.e. reaches the quiescence), we may be
sure that the executions of methods before this state and after this
state is never interpositioned.
An execution is quiescently consistent if the method calls can be
correctly arranged retaining the mutual order of calls separated by
quiescence, a period of time where no method is being called in any
thread.
Sources:
http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/88
http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/72
Related
Here are the definitions by Wikipedia:
Asynchrony, in computer programming, refers to the occurrence of events independently of the main program flow and ways to deal with such events. These may be "outside" events such as the arrival of signals, or actions instigated by a program that take place concurrently with program execution, without the program blocking to wait for results.
And:
Concurrent computing is a form of computing in which several computations are executed during overlapping time periods—concurrently—instead of sequentially (one completing before the next starts).
In the context of single-threaded computation, do 'asynchronous', 'non-blocking', and 'concurrent' imply one another?
If not, could you give me a counter-example?
Note that I have excluded the word 'parallel' as it implies multiple threads.
Non-blocking operations are based on two approaches:
by simply returning without data (when no data is available - in such cases the caller has to "come back" by itself and "read" again)
by using callbacks. In that context "blocking" means that you wait for an operation to reach a certain state - whereas "non-blocking" means that you trigger the operation - and when that state is reached, you are notified.
Please note: both options do not imply concurrency or multiple threads on the client side. You absolutely can implement such a system using a single process ( think coroutines or node.js for example ).
In that sense: a non-blocking operation is always asynchronous - as you don't know when it will have results for you - or when it will call you back. Both concepts can be be implemented using concurrency, but there is absolute need for doing it that way.
Non-blocking and concurrent don't really apply to single threaded programs, due to the fact that they refer to ways of managing multiple threads. Non-blocking means that a program doesn't wait for all threads to finish before moving on, and concurrent computation can only happen if you have multiple threads doing the calculation. (Someone please correct me if I'm wrong.)
Asynchrony is the only term that applies to single threaded programming, in the form of human input, communication with other programs, etc. Because of this, no, they don't imply each other in the context of single threaded programs.
I'd like to clarify something about ConcurrentHashMap vs ConcurrentSkipListMap based on the API documentation.
From my understanding ConcurrentHashMap gaurantees thread safety for insertions by multiple threads. So if you have a map that will only be populated concurrently by multiple threads then there are no issues. The API however goes on to suggest that it does not gaurantee locking for retrieval so you may get misleading results here?
In contrast, for the ConcurrentSkipListMap it is stated that: "Insertion, removal, update, and access operations safely execute concurrently by multiple threads". So I assume this does not have the aforementioned retrieval issue that the hash map has, but obviously this would generally come with a performance cost?
In practice, has anyone found the need to use the ConcurrentSkipListMap because of this particular behaviour, or does it generally not matter that retrievals may give an out of date view?
ConcurrentHashMap
Retrievals reflect the results of the most recently completed update
operations holding upon their onset. For aggregate operations such as
putAll and clear, concurrent retrievals may reflect insertion or
removal of only some entries.
it uses volatile semantics for get(key). In case when Thread1 calls put(key1, value1) and right after that Thread2 calls get(key1), Thread2 wouldn't wait Thread1 to finish its put, they are not synchronized with each other and Thread2 can get old associated value. But if put(key1, value1) was finished in Thread1 before Thread2 tries to get(key1) then Thread2 is guaranteed to get this update (value1).
ConcurrentSkipListMap is sorted and provides
expected average log(n) time cost for the containsKey, get,
put and remove operations and their variants
ConcurrentSkipListMap isn't so fast, but is useful when you need sorted thread-safe map.
The API however goes on to suggest that it does not gaurantee locking for retrieval so you may get misleading results here?
Interestingly enough, neither does the ConcurrentSkipListMap, infact the CSLM is completely non-blocking.
In Java 7 The CHM, for all intents and purposes, is non-blocking when executing reads. In fact, Java 8's updated CHM implementation has completely non-blocking reads.
The point here is that the CHM and CSLM have similar read semantics, the difference is time complexity.
From your question, you seem to have come to the conclusion that only insertions into ConcurrentHashMap are thread safe.
From my understanding ConcurrentHashMap gaurantees thread safety for insertions by multiple threads. So if you have a map that will only be populated concurrently by multiple threads then there are no issues.
How did you come to this conclusion? The first line of the documentation for ConcurrentHashMap implies that all operations are thread safe:
A hash table supporting full concurrency of retrievals and adjustable expected concurrency for updates.
Additionally, it implies that get() operations can sustain a higher level of concurrency than put() operations.
Simply put ConcurrentHashMap does not have the retrieval issue that you think it has. In most cases you should be using ConcurrentHashMap instead of ConcurrentSkipListMap since performance of ConcurrentHashMap is generally better than ConcurrentSkipListMap. You should only be using CurrentSkipListMap when you need a ConcurrentMap that has predictable iteration order or if you need the facilities of a NavigableMap.
Concurrent data structures:
[1] http://www.cs.tau.ac.il/~shanir/concurrent-data-structures.pdf
[2] http://en.wikipedia.org/wiki/Concurrent_data_structure
Are concurrent data structures and thread safe data structures interchangeable terms?
My understanding differs from #xxa (the answer is: No). Though I have no a regious definition either. Concurrent implies thread-safety. But nowadays, it implies simultaneous access as well. While thread-safety gives no such assumptions. See this quote form the mentioned Wiki-article:
Today, as multiprocessor computer architectures that provide parallelism become the dominant computing platform (through the proliferation of multi-core processors), the term has come to stand mainly for data structures that can be accessed by multiple threads which may actually access the data simultaneously because they run on different processors that communicate with one another.
For example, STL containers are claimed to be thread-safe for the given conditions (read-only), moreover, it allows simultaneous reads by a number of threads (STL says them are "as safe as int"), but only one thread can modify them and in absence of readers. Can we name them 'concurrent'? No. While practical concurrent containers (see tbb for example) allow at least two or more threads to work with the container (including modification) at the same time.
And one more point. You could implement std::queue so that methods push() and pop() will not trigger a failure when used by different threads. But does this make it a concurrent_queue? No. Because, queue::front() and queue::pop() do not provide a way to get elements simultaneously by two or more threads without external synchronization. To become a concurrent_queue it needs different interface, which takes care of atomicity by combining pop() with returning the value.
These terms do not have a rigorous definition, but in general it can be asserted that the answer to your questions is yes.
Both refer to data structures that are stored in shared memory (http://en.wikipedia.org/wiki/Shared_memory) and manipulated by several threads or processes.
In fact [1] asserts the same when it refers to threads explicitly:
The primary source of this additional difficulty is concurrency: Because threads are executed concurrently on different processors, ..."
I often find these terms being used in context of concurrent programming . Are they the same thing or different ?
No, they are not the same thing. They are not a subset of one another. They are also neither the necessary, nor the sufficient condition for one another.
The definition of a data race is pretty clear, and therefore, its discovery can be automated. A data race occurs when 2 instructions from different threads access the same memory location, at least one of these accesses is a write and there is no synchronization that is mandating any particular order among these accesses.
A race condition is a semantic error. It is a flaw that occurs in the timing or the ordering of events that leads to erroneous program behavior. Many race conditions can be caused by data races, but this is not necessary.
Consider the following simple example where x is a shared variable:
Thread 1 Thread 2
lock(l) lock(l)
x=1 x=2
unlock(l) unlock(l)
In this example, the writes to x from thread 1 and 2 are protected by locks, therefore they are always happening in some order enforced by the order with which the locks are acquired at runtime. That is, the writes' atomicity cannot be broken; there is always a happens before relationship between the two writes in any execution. We just cannot know which write happens before the other a priori.
There is no fixed ordering between the writes, because locks cannot provide this. If the programs' correctness is compromised, say when the write to x by thread 2 is followed by the write to x in thread 1, we say there is a race condition, although technically there is no data race.
It is far more useful to detect race conditions than data races; however this is also very difficult to achieve.
Constructing the reverse example is also trivial. This blog post also explains the difference very well, with a simple bank transaction example.
According to Wikipedia, the term "race condition" has been in use since the days of the first electronic logic gates. In the context of Java, a race condition can pertain to any resource, such as a file, network connection, a thread from a thread pool, etc.
The term "data race" is best reserved for its specific meaning defined by the JLS.
The most interesting case is a race condition that is very similar to a data race, but still isn't one, like in this simple example:
class Race {
static volatile int i;
static int uniqueInt() { return i++; }
}
Since i is volatile, there is no data race; however, from the program correctness standpoint there is a race condition due to the non-atomicity of the two operations: read i, write i+1. Multiple threads may receive the same value from uniqueInt.
TL;DR: The distinction between data race and race condition depends on the nature of problem formulation, and where to draw the boundary between undefined behavior and well-defined but indeterminate behavior. The current distinction is conventional and best reflects the interface between processor architect and programming language.
1. Semantics
Data race specifically refers to the non-synchronized conflicting "memory accesses" (or actions, or operations) to the same memory location. If there is no conflict in the memory accesses, while there is still indeterminate behavior caused by operation ordering, that is a race condition.
Note "memory accesses" here have specific meaning. They refer to the "pure" memory load or store actions, without any additional semantics applied. For example, a memory store from one thread does not (necessarily) know how long it takes for the data to be written into the memory, and finally propagates to another thread. For another example, a memory store to one location before another store to another location by the same thread does not (necessarily) guarantee the first data written in the memory be ahead of the second. As a result, the order of those pure memory accesses are not (necessarily) able to be "reasoned" , and anything could happen, unless otherwise well defined.
When the "memory accesses" are well defined in terms of ordering through synchronization, additional semantics can ensure that, even if the timing of the memory accesses are indeterminate, their order can be "reasoned" through the synchronizations. Note, although the ordering between the memory accesses can be reasoned, they are not necessarily determinate, hence the race condition.
2. Why the difference?
But if the order is still indeterminate in race condition, why bother to distinguish it from data race? The reason is in practical rather than theoretical. It is because the distinction does exist in the interface between the programming language and processor architecture.
A memory load/store instruction in modern architecture is usually implemented as "pure" memory access, due to the nature of out-of-order pipeline, speculation, multi-level of cache, cpu-ram interconnection, especially multi-core, etc. There are lots of factors leading to indeterminate timing and ordering. To enforce ordering for every memory instruction incurs huge penalty, especially in a processor design that supports multi-core. So the ordering semantics are provided with additional instructions like various barriers (or fences).
Data race is the situation of processor instruction execution without additional fences to help reasoning the ordering of conflicting memory accesses. The result is not only indeterminate, but also possibly very weird, e.g., two writes to the same word location by different threads may result with each writing half of the word, or may only operate upon their locally cached values. -- These are undefined behavior, from the programmer's point of view. But they are (usually) well defined from the processor architect's point of view.
Programmers have to have a way to reason their code execution. Data race is something they cannot make sense, therefore should always avoid (normally). That is why the language specifications that are low level enough usually define data race as undefined behavior, different from the well-defined memory behavior of race condition.
3. Language memory models
Different processors may have different memory access behavior, i.e., processor memory model. It is awkward for programmers to study the memory model of every modern processor and then develop programs that can benefit from them. It is desirable if the language can define a memory model so that the programs of that language always behave as expected as the memory model defines. That is why Java and C++ have their memory models defined. It is the burden of the compiler/runtime developers to ensure the language memory models are enforced across different processor architectures.
That said, if a language does not want to expose the low level behavior of the processor (and is willing to sacrifice certain performance benefits of the modern architectures), they can choose to define a memory model that completely hide the details of "pure" memory accesses, but apply ordering semantics for all their memory operations. Then the compiler/runtime developers may choose to treat every memory variable as volatile in all processor architectures. For these languages (that support shared memory across threads), there are no data races, but may still be race conditions, even with a language of complete sequential consistence.
On the other hand, the processor memory model can be stricter (or less relaxed, or at higher level), e.g., implementing sequential consistency as early-days processor did. Then all memory operations are ordered, and no data race exists for any languages running in the processor.
4. Conclusion
Back to the original question, IMHO it is fine to define data race as a special case of race condition, and race condition at one level may become data race at a higher level. It depends on the nature of problem formulation, and where to draw the boundary between undefined behavior and well-defined but indeterminate behavior. Just the current convention defines the boundary at language-processor interface, does not necessarily mean that is always and must be the case; but the current convention probably best reflects the state-of-the-art interface (and wisdom) between processor architect and programming language.
No, they are different & neither of them is a subset of one or vice-versa.
The term race condition is often confused with the related term data
race, which arises when synchronization is not used to coordinate all
access to a shared nonfinal field. You risk a data race whenever a
thread writes a variable that might next be read by another thread or
reads a variable that might have last been written by another thread
if both threads do not use synchronization; code with data races has
no useful defined semantics under the Java Memory Model. Not all race
conditions are data races, and not all data races are race conditions,
but they both can cause concurrent programs to fail in unpredictable
ways.
Taken from the excellent book - Java Concurrency in Practice by Brian Goetz & Co.
Data races and Race condition
[Atomicity, Visibility, Ordering]
In my opinion definitely it is two different things.
Data races is a situation when same memory is shared between several threads(at least one of them change it (write access)) without synchronoization
Race condition is a situation when not synchronized blocks of code(may be the same) which use same shared resource are run simultaneously on different threads and result of which is unpredictable.
Race condition examples:
//increment variable
1. read variable
2. change variable
3. write variable
//cache mechanism
1. check if exists in cache and if not
2. load
3. cache
Solution:
Data races and Race condition are problem with atomicity and they can be solved by synchronization mechanism.
Data races - When write access to shared variable will be synchronized
Race condition - When block of code is run as an atomic operation
According to an article on IBM.com, "a race condition is a situation in which two or more threads or processes are reading or writing some shared data, and the final result depends on the timing of how the threads are scheduled. Race conditions can lead to unpredictable results and subtle program bugs." . Although the article concerns Java, I have in general been taught the same definition.
As far as I know, simple operation of reading from RAM is composed of setting the states of specific input lines (address, read etc.) and reading the states of output lines. This is an operation that obviously cannot be executed simultaneously by two devices and has to be serialized.
Now let's suppose we have a situation when a couple of threads access an object in memory. In theory, this access should be serialized in order to prevent race conditions. But e.g. the readers/writers algorithm assumes that an arbitrary number of readers can use the shared memory at the same time.
So, the question is: does one have to implement an exclusive lock for read when using multithreading (in WinAPI e.g.)? If not, why? Where is this control implemented - OS, hardware?
Best regards,
Kuba
Reading memory at hardware level is done sequentially - you don't need to worry about concurrency at this level. Two threads issue read instructions and all the necessary stuff - setting addresses on the address bus and actual reads are implemented by the memory access hardware in such way that reads will always work right.
In fact the same is true for read/write scenarios except that when read and write requests are interleaved you will get different results depending on timing and this is why you need synchronization.
As long as there's nothing changing the data, it's perfectly safe to be reading it from several threads. Even if two CPUs (or cores) race to access the memory for reading at the exact same clock cycle, their accesses will be serialized by the memory controller and they won't interfere with each other. This feature is essential for HW working correctly.
There is not a simple answer to this question. Different API's (and different environments) will have different levels of multhreaded-awareness and multithreaded safety.