In this pseudocode block:
atomically $ do
if valueInLocalStorage key
then readValueFromLocalStorage key
else do
value <- unsafeIOToSTM $ fetchValueFromDatabase key
writeValueToLocalStorage key value
Is it safe to use unsafeIOToSTM? The docs say:
The STM implementation will often run transactions multiple times, so you need to be prepared for this if your IO has any side effects.
Basically, if a transaction fails it is because some other thread wroteValueToLocalStorage and when the transaction is retried it will return the stored value instead of fetching from the database again.
The STM implementation will abort transactions that are known to be invalid and need to be restarted. This may happen in the middle of unsafeIOToSTM, so make sure you don't acquire any resources that need releasing (exception handlers are ignored when aborting the transaction). That includes doing any IO using Handles, for example. Getting this wrong will probably lead to random deadlocks.
This worries me the most. Logically, if fetchValueFromDatabase doesn't open a new connection (i.e. an existing connection is used) everything should be fine. Are there other pitfalls I am missing?
The transaction may have seen an inconsistent view of memory when the IO runs. Invariants that you expect to be true throughout your program may not be true inside a transaction, due to the way transactions are implemented. Normally this wouldn't be visible to the programmer, but using unsafeIOToSTM can expose it.
key is a single value, no invariants to break.
I would suggest that doing I/O from an STM transaction is just a bad idea.
Presumably what you want is to avoid two threads doing the DB lookup at the same time. What I would do is this:
See if the item is already in the cache. If it is, we're done.
If it isn't, mark it with an "I'm fetching this" flag, commit the STM transaction, go get it from the DB, and do a second STM transaction to insert it into the cache (and remove the flag).
If the item is already flagged, retry the transaction. This blocks the calling thread until the first thread inserts the value from the DB.
Related
I'm reading C++ Concurrency in Action by Anthony Williams, and don't understand its push implementation of the lock_free_stack class. Listing 7.12 to be precise
void push(T const& data)
{
counted_node_ptr new_node;
new_node.ptr=new node(data);
new_node.external_count=1;
new_node.ptr->next=head.load(std::memory_order_relaxed)
while(!head.compare_exchange_weak(new_node.ptr->next,new_node, std::memory_order_release, std::memory_order_relaxed));
}
So imagine 2 threads (A, B) calling push function. Both of them reach while loop but not start it. So they both read the same value from head.load(std::memory_order_relaxed).
Then we have the following things going on:
B thread gets swiped out for any reason
A thread starts the loop and obviously successfully adds a new node to the stack.
B thread gets back on track and also starts the loop.
And this is where it gets interesting as it seems to me.
Because there was a load operation with std::memory_order_relaxed and compare_exchange_weak(..., std::memory_order_release, ...) in case of success it looks like there is no synchronization between threads whatsoever.
I mean it's like std::memory_order_relaxed - std::memory_order_release and not std::memory_order_acquire - std::memory_order_release.
So B thread will simply add a new node to the stack but to its initial state when we had no nodes in the stack and reset head to this new node.
I was doing my research all around this subject and the best i could find was in this post Does exchange or compare_and_exchange reads last value in modification order?
So the question is, is it true? and all RMW functions see the last value in modification order? No matter what std::memory_order we used, if we use RMW operation it will synchronize with all threads (CPU and etc) and find the last value to be written to the atomic operation upon it is called?
So after some research and asking a bunch of people I believe I found the proper answer to this question, I hope it'll be a help to someone.
So the question is, is it true? and all RMW functions see the last
value in modification order?
Yes, it is true.
No matter what std::memory_order we used, if we use RMW operation it
will synchronize with all threads (CPU and etc) and find the last
value to be written to the atomic operation upon it is called?
Yes, it is also true, however there is something that needs to be highlighted.
RMW operation will synchronize only the atomic variable it works with. In our case, it is head.load
Perhaps you would like to ask why we need release - acquire semantics at all if RMW does the synchronization even with the relaxed memory order.
The answer is because RMW works only with the variable it synchronizes, but other operations which occurred before RMW might not be seen in the other thread.
lets look at the push function again:
void push(T const& data)
{
counted_node_ptr new_node;
new_node.ptr=new node(data);
new_node.external_count=1;
new_node.ptr->next=head.load(std::memory_order_relaxed)
while(!head.compare_exchange_weak(new_node.ptr->next,new_node, std::memory_order_release, std::memory_order_relaxed));
}
In this example, in case of using two push threads they won't be synchronized with each other to some extent, but it could be allowed here.
Both threads will always see the newest head because compare_exchange_weak
provides this. And a new node will be always added to the top of the stack.
However if we tried to get the value like this *(new_node.ptr->next) after this line new_node.ptr->next=head.load(std::memory_order_relaxed) things could easily turn ugly: empty pointer might be dereferenced.
This might happen because a processor can change the order of instructions and because there is no synchronization between threads the second thread could see the pointer to a top node even before that was initialized!
And this is exactly where release-acquire semantic comes to help. It ensures that all operations which happened before release operation will be seen in acquire part!
Check out and compare listings 5.5 and 5.8 in the book.
I also recommend you to read this article about how processors work, it might provide some essential information for better understanding.
memory barriers
According to the documentation in beginWriteTransaction method
Only one write transaction can be open at a time. Write transactions
cannot be nested, and trying to begin a write transaction on a
RLMRealm which is already in a write transaction will throw an
exception. Calls to beginWriteTransaction from RLMRealm instances in
other threads will block until the current write transaction
completes.
but when I looked at code I found next one:
void Realm::begin_transaction()
{
check_read_write(this);
verify_thread();
if (is_in_transaction()) {
throw InvalidTransactionException("The Realm is already in a write transaction");
}
// make sure we have a read transaction
read_group();
transaction::begin(*m_shared_group, m_binding_context.get());
}
Could you explain when a condition is met ?
Calls to beginWriteTransaction from RLMRealm instances in other
threads will block until the current write transaction completes.
The last call will lead to a method which will leave this intermediate cross-platform C++ API level and goes one level deeper into our internal storage engine, where it uses a mutex to coordinate exclusive access between processes and threads.
Once this mutex is acquired, it is hold until the write transaction is committed or cancelled.
In a multiple threaded server application, I use the type Client to represent a client. The nature of Client is quite mutable: clients send UDP heartbeat messages to keep registered with the server, the message may also contain some realtime data (think of a sensor). I need to keep track of many things such as the timestamp and source address of the last heartbeat, the realtime data, etc. The result is a pretty big structure with many states. Each client has a client ID, and I use a HashMap wrapped in an MVar to store the clients, so lookup is easy and fast.
type ID = ByteString
type ClientMap = MVar (HashMap ID Client)
There's a "global" value of ClientMap which is made available to each thread. It's stored in a ReaderT transformer along with many other global values.
The Client by itself is a big immutable structure, using strict fields to prevent from space leaks:
data Client = Client
{
_c_id :: !ID
, _c_timestamp :: !POSIXTime
, _c_addr :: !SockAddr
, _c_load :: !Int
...
}
makeLenses ''Client
Using immutable data structures in a mutable wrapper in a common design pattern in Concurrent Haskell, according to Parallel and Concurrent Programming in Haskell. When a heartbeat message is received, the thread that processes the message would construct a new Client, lock the MVar of the HashMap, insert the Client into the HashMap, and put the new HashMap in the MVar. The code is basically:
modifyMVar hashmap_mvar (\hm ->
let c = Client id ...
in return $! M.insert id c hm)
This approach works fine, but as the number of clients grows (we now have tens of thousands of clients), several problems emerge:
The client sends heartbeat messages pretty frequently (around every 30 seconds), resulting in access contention of the ClientMap.
Memory consumption of the program seems to be quite high. My understanding is that, updating large immutable structures wrapped in MVar frequently will make the garbage collector very busy.
Now, to reduce the contention of the global hashmap_mvar, I tried to wrap the mutable fields of Client in an MVar for each client, such as:
data ClientState = ClientState
{
_c_timestamp :: !POSIXTime
, _c_addr :: !SockAddr
, _c_load :: !Int
...
}
makeLenses ''ClientState
data Client = Client
{
c_id :: !ID
, c_state :: MVar CameraState
}
This seems to reduce the level of contention (because now I only need to update the MVar in each Client, the grain is finer), but the memory footprint of the program is still high. I've also tried to UNPACK some of the fields, but that didn't help.
Any suggestions? Will STM solve the contention problem? Should I resort to mutable data structures other than immutable ones wrapped in MVar?
See also Updating a Big State Fast in Haskell.
Edit:
As Nikita Volkov pointed out, a shared map smells like bad design in a typical TCP-based server-client application. However, in my case, the system is UDP based, meaning there's no such thing as a "connection". The server uses a single thread to receive UDP messages from all the clients, parses them and performs actions accordingly, e.g., updating the client data. Another thread reads the map periodically, checks the timestamp of heartbeats, and deletes those who have not sent heartbeats in the last 5 minutes, say. Seems like a shared map is inevitable? Anyway I understand that using UDP was a poor design choice in the first place, but I would still like to know how can I improve my situation with UDP.
First of all, why exactly do you need that shared map at all? Do you really need to share the private states of clients with anything? If not (which is the typical case with a client-server application), then you can simply get around without any shared map.
In fact, there is a "remotion" library, which encompasses all the client-server communication, allowing you to create services simply by extending it with your custom protocol. You should take a look.
Secondly, using multiple MVars over fields of some entity is always potentially a race condition bug. You should use STM, when you need to update multiple things atomically. I'm not sure if that's the case in your app, nonetheless you should be aware of that.
Thirdly,
The client sends heartbeat messages pretty frequently (around every 30 seconds), resulting in access contention of the ClientMap
Seems like just a job for Map of the recently released "stm-containers" library. See this blog post for introduction to the library. You'll be able to get back to the immutable Client model with this.
This is a continuation from here: Golang: Shared communication in async http server
Assuming I have a hashmap w/ locking:
//create async hashmap for inter request communication
type state struct {
*sync.Mutex // inherits locking methods
AsyncResponses map[string]string // map ids to values
}
var State = &state{&sync.Mutex{}, map[string]string{}}
Functions that write to this will place a lock. My question is, what is the best / fastest way to have another function check for a value without blocking writes to the hashmap? I'd like to know the instant a value is present on it.
MyVal = State.AsyncResponses[MyId]
Reading a shared map without blocking writers is the very definition of a data race. Actually, semantically it is a data race even when the writers will be blocked during the read! Because as soon as you finish reading the value and unblock the writers - the value may not exists in the map anymore.
Anyway, it's not very likely that proper syncing would be a bottleneck in many programs. A non-blocking lock af a {RW,}Mutex is probably in the order of < 20 nsecs even on middle powered CPUS. I suggest to postpone optimization not only after making the program correct, but also after measuring where the major part of time is being spent.
I'm attempting to learn Clojure from the API and documentation available on the site. I'm a bit unclear about mutable storage in Clojure and I want to make sure my understanding is correct. Please let me know if there are any ideas that I've gotten wrong.
Edit: I'm updating this as I receive comments on its correctness.
Disclaimer: All of this information is informal and potentially wrong. Do not use this post for gaining an understanding of how Clojure works.
Vars always contain a root binding and possibly a per-thread binding. They are comparable to regular variables in imperative languages and are not suited for sharing information between threads. (thanks Arthur Ulfeldt)
Refs are locations shared between threads that support atomic transactions that can change the state of any number of refs in a single transaction. Transactions are committed upon exiting sync expressions (dosync) and conflicts are resolved automatically with STM magic (rollbacks, queues, waits, etc.)
Agents are locations that enable information to be asynchronously shared between threads with minimal overhead by dispatching independent action functions to change the agent's state. Agents are returned immediately and are therefore non-blocking, although an agent's value isn't set until a dispatched function has completed.
Atoms are locations that can be synchronously shared between threads. They support safe manipulation between different threads.
Here's my friendly summary based on when to use these structures:
Vars are like regular old variables in imperative languages. (avoid when possible)
Atoms are like Vars but with thread-sharing safety that allows for immediate reading and safe setting. (thanks Martin)
An Agent is like an Atom but rather than blocking it spawns a new thread to calculate its value, only blocks if in the middle of changing a value, and can let other threads know that it's finished assigning.
Refs are shared locations that lock themselves in transactions. Instead of making the programmer decide what happens during race conditions for every piece of locked code, we just start up a transaction and let Clojure handle all the lock conditions between the refs in that transaction.
Also, a related concept is the function future. To me, it seems like a future object can be described as a synchronous Agent where the value can't be accessed at all until the calculation is completed. It can also be described as a non-blocking Atom. Are these accurate conceptions of future?
It sounds like you are really getting Clojure! good job :)
Vars have a "root binding" visible in all threads and each individual thread can change the value it sees with out affecting the other threads. If my understanding is correct a var cannot exist in just one thread with out a root binding that is visible to all and it cant be "rebound" until it has been defined with (def ... ) the first time.
Refs are committed at the end of the (dosync ... ) transaction that encloses the changes but only when the transaction was able to finish in a consistent state.
I think your conclusion about Atoms is wrong:
Atoms are like Vars but with thread-sharing safety that blocks until the value has changed
Atoms are changed with swap! or low-level with compare-and-set!. This never blocks anything. swap! works like a transaction with just one ref:
the old value is taken from the atom and stored thread-local
the function is applied to the old value to generate a new value
if this succeeds compare-and-set is called with old and new value; only if the value of the atom has not been changed by any other thread (still equals old value), the new value is written, otherwise the operation restarts at (1) until is succeeds eventually.
I've found two issues with your question.
You say:
If an agent is accessed while an action is occurring then the value isn't returned until the action has finished
http://clojure.org/agents says:
the state of an Agent is always immediately available for reading by any thread
I.e. you never have to wait to get the value of an agent (I assume the value changed by an action is proxied and changed atomically).
The code for the deref-method of an Agent looks like this (SVN revision 1382):
public Object deref() throws Exception{
if(errors != null)
{
throw new Exception("Agent has errors", (Exception) RT.first(errors));
}
return state;
}
No blocking is involved.
Also, I don't understand what you mean (in your Ref section) by
Transactions are committed on calls to deref
Transactions are committed when all actions of the dosync block have been completed, no exceptions have been thrown and nothing has caused the transaction to be retried. I think deref has nothing to do with it, but maybe I misunderstand your point.
Martin is right when he say that Atoms operation restarts at 1. until is succeeds eventually.
It is also called spin waiting.
While it is note really blocking on a lock the thread that did the operation is blocked until the operation succeeds so it is a blocking operation and not an asynchronously operation.
Also about Futures, Clojure 1.1 has added abstractions for promises and futures.
A promise is a synchronization construct that can be used to deliver a value from one thread to another. Until the value has been delivered, any attempt to dereference the promise will block.
(def a-promise (promise))
(deliver a-promise :fred)
Futures represent asynchronous computations. They are a way to get code to run in another thread, and obtain the result.
(def f (future (some-sexp)))
(deref f) ; blocks the thread that derefs f until value is available
Vars don't always have a root binding. It's legal to create a var without a binding using
(def x)
or
(declare x)
Attempting to evaluate x before it has a value will result in
Var user/x is unbound.
[Thrown class java.lang.IllegalStateException]