Lets say I have two threads reading and modifying a bool / int "state". The reads and writes are guaranteed to be atomic by the processor.
Thread 1:
if (state == ENABLED)
{
Process_Data()
}
Thread 2:
state = DISABLED
In this case yes the thread 1 can read the state and go into it's "if" to Process_Data and then Thread2 can change state. But it isn't incorrect at that point to still go on to Process_Data. Yes if we peek into the hood we have an inconsistency of state being DISABLED and us entering the Process_Data function. But after its executed the next time Thread1 executes it will get state = DISABLED and not Process_Data.
My question is do I still need a lock in both these threads to make Thread1's check-state-and-process atomic and Thread2's write atomic (wrt to Thread 1) ?
You've addressed the atomicity concerns. However, in modern processors, you have to worry not just about atomicity, but also memory visibility.
For example, thread 1 is executing on one processor, and reads ENABLED from state - from its processor's cache.
Meanwhile, thread 2 is executing on a different processor, and writes DISABLED to state on its processor's cache.
Without further code - in some languages, for example, declaring state volatile - the DISABLED value may not get flushed to main memory for a long time. It may never get flushed to main memory if thread 2 changes the value back to ENABLED eventually.
Meanwhile, even if the DISABLED value is flushed to main memory, thread 1 may never pick it up, instead continuing to use its cached value of ENABLED indefinitely.
Generally if you want to share values between threads, it's better to do so explicitly using the appropriate mechanisms for the programming language and environment that you're using.
There's no way to answer your question generically. If the specification for the language, compiler, threading library and/or platform you are using says you need protection, then you do. If it says you don't, then you don't. I believe every threading library or multi-threading implementation specifies rules for sane use and sharing of data. If yours doesn't, it's a piece of junk that is impossible to use reliably and you should get a better one.
Do not make the mistake of thinking, "This is safe because I can't think of any way it can go wrong." Or "I tested this, and I couldn't get it to fail, so it's safe." That kind of thinking produces fragile code that tends to fail when you change compiler options, upgrade your CPU, or run the program on a different platform. Follow the specifications for the tools you are using.
Related
There is lot of discussion on thread synchronization on SO as well as on many forums all-over the Internet. However, I could not find precise information as to how exactly it happens at OS level conceptually.
As we all know there are these types of thread synchronization objects:
Mutex
Semaphore
Critical section
And as I understand it fully, allowing multiple threads at a time to modify a resource (for example, two threads simultaneously changing bits of a variable in memory) is not a good idea and so we use these objects. But then that's what exactly same should happen when multiple threads try to access these objects as well.
What really happens at the core? How exactly does OS achieve this?
How can we explain this to someone at conceptual level (rather than going in hardware or assembly level details)?
First let's sum up what the fundamental problem of threading really is-- two threads try to access the same piece of memory at the same time. You can imagine that when this happens we can't guarantee that a piece of memory is in a valid state, and our program might be incorrect.
Trying to keep this very high level, part of the way processors work is by throwing interrupts which basically tell a thread to stop what is doing and do something else. This is where much of the problem of threading lies. A thread can be interrupted in the middle of task. Imagine one thread is interrupted in the middle of an operation and some intermediate garbage value exists because the thread hasn't finished its task. Another thread could come along and read this value and destroy the correctness of your program.
The OS achieves this with Atomic instructions. Without getting into the details, image that there were some instructions that were guaranteed to either be completed or not completed. This means that if a thread checks the result of an instruction it won't see an intermediate results. So an atomic add method would either show the value before the add or after the add, but not during the add when their might be some intermediate state.
Now if you have a few atomic instructions you might be able to imagine that you could build higher level abstractions that deal with threads and thread safety on the back of these. Maybe the most basic example in a lock created with the test and set primitive. Take a look at this wikipedia article https://en.wikipedia.org/wiki/Test-and-set. Now that was probably a lot because these things get pretty complex. But I will attempt to given an example that clarifies. If you have two processes running that are trying to access some section of code, a very naive solution would be to create a lock variable
boolean isLocked = false;
Anytime a process tried to acquire this lock you could merely check isLocked==false and wait until isLocked ==true before executing some code. For example...
while(isLocked){
//wait for isLocked == false
}
isLocked = true;
// execute the code you want to be locked
isLocked = false;
Of course, we know that something as simple as setting or reading a boolean can be interrupted and cause threading mayhem. So, the good folks that developed kernels and processors and hardware created an atomic test and set operation which returns the old value of a boolean and sets the new value to true. So of course you can implement our lock above by doing something like.
while(testAndSet(isLocked)){ //wait until the old value returned is
false so the lock is unlocked } //do some critical stuff
//unlock after you did the critical stuff lock = false;
I only show the implementation of a basic lock above to prove the point that it is possible to build higher level abstractions on atomic instructions. Atomic instruction are about as low level as you can get conceptually, in my opinion, without delving into hardware specifics. You can imagine though that within hardware, the hardware must somehow set a flag of some sort when memory is being read that precludes another thread from accessing the same memory.
Hope that helps!
Hi I am writing kernel code which intends to do process scheduling and multi-threaded execution. I've studied about locking mechanisms and their functionality. Is there a thumb rule regarding what sort of data structure in critical section should be protected by locking (mutex/semaphores/spinlocks)?
I know that where ever there is chance of concurrency in part of code, we require lock. But how do we decide, what if we miss and test cases don't catch them. Earlier I wrote code for system calls and file systems where I never cared about taking locks.
Is there a thumb rule regarding what sort of data structure in critical section should be protected by locking?
Any object (global variable, field of the structure object, etc.), accessed concurrently when one access is write access requires some locking discipline for access.
But how do we decide, what if we miss and test cases don't catch them?
Good practice is appropriate comment for every declaration of variable, structure, or structure field, which requires locking discipline for access. Anyone, who uses this variable, reads this comment and writes corresponded code for access. Kernel core and modules tend to follow this strategy.
As for testing, common testing rarely reveals concurrency issues because of their low probability. When testing kernel modules, I would advice to use Kernel Strider, which attempts to prove correctness of concurrent memory accesses or RaceHound, which increases probability of concurrent issues and checks them.
It is always safe to grab a lock for the duration of any code that accesses any shared data, but this is slow since it means only one thread at a time can run significant chunks of code.
Depending on the data in question though, there may be shortcuts that are safe and fast. If it is a simple integer ( and by integer I mean the native word size of the CPU, i.e. not a 64 bit on a 32 bit cpu ), then you may not need to do any locking: if one thread tries to write to the integer, and the other reads it at the same time, the reader will either get the old value, or the new value, never a mix of the two. If the reader doesn't care that he got the old value, then there is no need for a lock.
If however, you are updating two integers together, and it would be bad for the reader to get the new value for one and the old value for the other, then you need a lock. Another example is if the thread is incrementing the integer. That normally involves a read, add, and write. If one reads the old value, then the other manages to read, add, and write the new value, then the first thread adds and writes the new value, both believe they have incremented the variable, but instead of being incremented twice, it was only incremented once. This needs either a lock, or the use of an atomic increment primitive to ensure that the read/modify/write cycle can not be interrupted. There are also atomic test-and-set primitives so you can read a value, do some math on it, then try to write it back, but the write only succeeds if it still holds the original value. That is, if another thread changed it since the time you read it, the test-and-set will fail, then you can discard your new value and start over with a read of the value the other thread set and try to test-and-set it again.
Pointers are really just integers, so if you set up a data structure then store a pointer to it where another thread can find it, you don't need a lock as long as you set up the structure fully before you store its address in the pointer. Another thread reading the pointer ( it will need to make sure to read the pointer only once, i.e. by storing it in a local variable then using only that to refer to the structure from then on ) will either see the new structure, or the old one, but never an intermediate state. If most threads only read the structure via the pointer, and any that want to write do so either with a lock, or an atomic test-and-set of the pointer, this is sufficient. Any time you want to modify any member of the structure though, you have to copy it to a new one, change the new one, then update the pointer. This is essentially how the kernel's RCU ( read, copy, update ) mechanism works.
Ideally, you must enumerate all the resources available in your system , the related threads and communication, sharing mechanism during design. Determination of the following for every resource and maintaining a proper check list whenever change is made can be of great help :
The duration for which the resource will be busy (Utilization of resource) & type of lock
Amount of tasks queued upon that particular resource (Load) & priority
Type of communication, sharing mechanism related to resource
Error conditions related to resource
If possible, it is better to have a flow diagram depicting the resources, utilization, locks, load, communication/sharing mechanism and errors.
This process can help you in determining the missing scenarios/unknowns, critical sections and also in identification of bottlenecks.
On top of the above process, you may also need certain tools that can help you in testing / further analysis to rule out hidden problems if any :
Helgrind - a Valgrind tool for detecting synchronisation errors.
This can help in identifying data races/synchronization issues due
to improper locking, the lock ordering that can cause deadlocks and
also improper POSIX thread API usage that can have later impacts.
Refer : http://valgrind.org/docs/manual/hg-manual.html
Locksmith - For determining common lock errors that may arise during
runtime or that may cause deadlocks.
ThreadSanitizer - For detecting race condtion. Shall display all accesses & locks involved for all accesses.
Sparse can help to lists the locks acquired and released by a function and also identification of issues such as mixing of pointers to user address space and pointers to kernel address space.
Lockdep - For debugging of locks
iotop - For determining the current I/O usage by processes or threads on the system by monitoring the I/O usage information output by the kernel.
LTTng - For tracing race conditions and interrupt cascades possible. (A successor to LTT - Combination of kprobes, tracepoint and perf functionalities)
Ftrace - A Linux kernel internal tracer for analysing /debugging latency and performance related issues.
lsof and fuser can be handy in determining the processes having lock and the kind of locks.
Profiling can help in determining where exactly the time is being spent by the kernel. This can be done with tools like perf, Oprofile.
The strace can intercept/record system calls that are called by a process and also the signals that are received by a process. It shall show the order of events and all the return/resumption paths of calls.
I think, no matter the whole lot of documentation available, I don't understand why one have to wait for a spin lock in a kernel context.
Why isn't there a specific queue with process requiring a lock with an atomic counter/index and , with preempt disabled, treat them as they come in this list and when the counter is down to 0 on thislist, go back to the main schedule list ?
Two situations :
system underloaded, maybe the spinlock is faster (depends on the lock concurrency at this moment);
system heavily loaded, maybe this strategy is faster (no more wait).
I may miss something very smart here, and I would like to understand it, please.
Thank you
Spinlocks are primarily for use in (or to interoperate with) contexts that cannot block / reschedule. They should only be used where the likelihood of actually waiting for them is relatively low and the lock will not be held long. For example, assume an interrupt handler (and/or other contexts as well) has created a data structure and needs to link it into a doubly-linked list. That will only take nanoseconds to complete and the likelihood of colliding with another process is low, yet it must have an atomic effect: no other cpu/thread should see the list in an intermediate (partially linked) state.
I'm reading the book Crack Code Interview recently, but there's one paragraph confusing me a lot on page 257:
A thread is a particular execution path of a process; when one thread modifies a process resource, the change is immediately visible to sibling threads.
IIRC, if one thread make a change to a variable, the change will firstly save in the CPU cache (say, L1 cache), and will not guarantee to synchronize to other threads unless the variable is declared as volatile.
Am I right?
Nope, you're wrong. But this is a very common misunderstanding.
Every modern multi-core CPU has hardware cache coherence. The L1, and similar caches, are invisible. CPU caches like the L1 cache have nothing to do with memory visibility.
Changes are visible immediately when a thread modifies a process resource. The issue is optimizations that cause process resources not to be modified in precisely the order the code specifies.
If your code has k = j; i = 4; if (j == 2) foo(); an optimizer might see that your first assignment reads the value of j. So it might not bother reading it again when you compare it to 2 since it "knows" that it can't have changed. However, another thread might have changed it. So optimizations of some kinds need to be disabled when synchronization between threads is required. That's what things like volatile do.
If compilers and CPUs made no optimizations and executed a program precisely as it was written, volatile would never be needed. Memory visibility is about optimizations in code (some done by the compiler, some by the CPU), not caches.
I think the text you are quoting is incorrect. The whole idea of the Java Memory Model is to deal with the complex optimizations by modern software & hardware, so that programmers can determine what writes are visible by the respective reads in other threads.
Unless a program in Java is properly synchronized, you can't guarantee that changes by one thread are immediately visible to other threads. Maybe the text refers to a very specific (and weak) memory model.
Usage of volatile variables is just one way to synchronize threads, and it's not suitable for all scenarios.
--Edit--
I think I understand the confusion now... I agree with David Schwartz, assuming that:
1) "modifies a process resource" means the actual change of the resource, not just the execution of a write instruction written in some high level computer language.
2) "is immediately visible to sibling threads" means that other threads are able to see it; it doesn't mean that a thread in your program will necessarily see it. You may still need to use synchronization tools in order to disable optimizations that bypass the actual access to the resource.
i have following code:
while(lock)
;
lock = 1;
// critical section
lock = 0;
As reading or changing lock value is in itself a multi-instruction
read lock
change value
write it
If it happens like:
1) One thread reads the lock and stops there
2) Another thread reads it and sees it is free; lock it and do something untill half
3) First thread wakes up and goes into CS
SO how would locking would be implmented in system ?
Placing variables over top of another variables is not right : it would be like Guarding the guard ?
Stopping other processors threads is also not right ?
It is 100% platform specific. Generally, the CPU provides some form of atomic operation such as exchange or compare and swap. A typical lock might work like this:
1) Create: Store 0 (unlocked) in the variable.
2) Lock: Atomically attempt to switch the value of the variable from 0 (unlocked) to 1 (locked). If we failed (because it wasn't unlocked to begin with), let the CPU rest a bit, and then retry. Use a memory barrier to ensure no future memory operations sneak behind this one.
3) Unlock: Use a memory barrier to ensure previous memory operations don't sneak past this one. Atomically write 0 (unlocked) to the variable.
Note that you really don't need to understand this unless you want to design your own synchronization primitives. And if you want to do that, you need to understand an awful lot more. It's certainly a good idea for every programmer to have a general idea of what he's making the hardware do. But this is an area filled with seriously heavy wizardry. There are so many, many ways this can go horribly wrong. So just use the locking primitives provided by the geniuses who made your platform, compiler, and threading library. Here be dragons.
For example, SMP Pentium Pro systems have an erratum that requires special handling in the unlock operation. A naive implementation of the lock algorithm will cause the branch prediction logic to expect the operation to keep spinning, incurring a massive performance penalty at the worst possible time -- when you first acquire the lock. A naive implementation of the lock algorithm may cause two cores each waiting for the same lock to saturate the bus, slowing the CPU that needs to get work done in order to release the lock to a crawl. These all require heavy wizardry and deep understanding of the hardware to deal with.
In a course I studied at Uni, a possible firmware solution for implementing locks was presented in the form of the "atomicity bit" associated to a memory operation initiated by a processor.
Basically, when locking, you'll notice that you have a sequence of operations that need to be executed atomically: test the value of the flag and, if not set, set it to locked, otherwise try again. This sequence can be made atomic by associating a bit with each memory request send by the CPU. The first N-1 operations will have the bit set, while the last one will have it unset, to mark the end of the atomic sequence.
When the memory module (there can be several modules) where the flag data is stored receives the request for the first operation in the sequence (whose bit is set), it will serve it and not take requests from any other CPU until the CPU that initiated the atomic sequence sends a request with an unset atomicity bit (since these transactions are usually short, a coarse-grain approach like this is acceptable). Note that this is usually made easier by the assembler providing specialized instructions of type "compare-and-set", that do exactly what I mentioned before.