Context
I need to do computations using multi-threading. I use SBCL and portability is not a concern. I am aware that bordeaux-threads and lparallel exist but I want to implement something at the relatively low level provided by the specific SBCL threading implementation. I need maximal speed, even at the expense of readability/programming effort.
Example of computation intensive operation
We can define a sufficiently computation-intensive function that will benefit from multi-threading.
(defun intensive-sqrt (x)
"Dummy calculation for intensive algorithm.
Approx 50 ms for 1e6 iterations."
(let ((y x))
(dotimes (it 1000000 t)
(if (> y 1.01d0)
(setf y (sqrt y))
(setf y (* y y y))))
y))
Mapping each computation to a thread and execute
Given a list of argument-lists llarg and a function fun, we want to compute nthreads results and return the list of results res-list. Here is what I came up with using the resources I found (see below).
(defmacro splice-arglist-help (fun arglist)
"Helper macro.
Splices a list 'arglist' (arg1 arg2 ...) into the function call of 'fun'
Returns (funcall fun arg1 arg2 ...)"
`(funcall ,fun ,#arglist))
(defun splice-arglist (fun arglist)
(eval `(splice-arglist-help ,fun ,arglist)))
(defun maplist-fun-multi (fun llarg nthreads)
"Maps 'fun' over list of argument lists 'llarg' using multithreading.
Breaks up llarg and feeds it to each thread.
Appends all the result lists at the end."
(let ((thread-list nil)
(res-list nil))
;; Create and run threads
(dotimes (it nthreads t)
(let ((larg-temp (elt llarg it)))
(setf thread-list (append thread-list
(list (sb-thread:make-thread
(lambda ()
(splice-arglist fun larg-temp))))))))
;; Join threads
;; Threads are joined in order, not optimal for speed.
;; Should be joined when finished ?
(dotimes (it (list-length thread-list) t)
(setf res-list (append res-list (list (sb-thread:join-thread (elt thread-list it))))))
res-list))
nthreads does not necessarily match the length of llarg, but I avoid the extra book-keeping just for the example simplicity's sake. I also omitted the various declare used for optimization.
We can test the multi-threading and compare timings using :
(defparameter *test-args-sqrt-long* nil)
(dotimes (it 10000 t)
(push (list (+ 3d0 it)) *test-args-sqrt-long*))
(time (intensive-sqrt 5d0))
(time (maplist-fun-multi #'intensive-sqrt *test-args-sqrt-long* 100))
The number of threads is quite high. I think the optimum would be to use as many threads as the CPU has, but I noticed the performance drop-off is barely noticeable in terms of time/operations. Doing more operations would involve breaking up the input lists into smaller pieces.
The above code outputs, on a 2 cores/4 threads machine :
Evaluation took:
0.029 seconds of real time
0.015625 seconds of total run time (0.015625 user, 0.000000 system)
55.17% CPU
71,972,879 processor cycles
22,151,168 bytes consed
Evaluation took:
1.415 seconds of real time
4.703125 seconds of total run time (4.437500 user, 0.265625 system)
[ Run times consist of 0.205 seconds GC time, and 4.499 seconds non-GC time. ]
332.37% CPU
3,530,632,834 processor cycles
2,215,345,584 bytes consed
What's bugging me
The example I've given works very well and is robust (ie results don't get mixed up between threads, and I experience no crash). The speed gain is also there and the computations do use several cores/threads on the machines I've tested this code on. But there are a few things that I'd like an opinion/help on :
The use of the argument list llarg and larg-temp. Is this really necessary ? Is there any way to avoid manipulating potentially huge lists ?
Threads are joined in the order in which they are stored in the thread-list. I imagine this would not be optimal if operations each took a different time to complete. Is there a way to join each thread when it is finished, instead of waiting ?
The answers should be in the resources I already found, but I find the more advanced stuff hard to grapple with.
Resources found so far
http://www.sbcl.org/manual/#Threading
http://cl-cookbook.sourceforge.net/process.html
https://lispcookbook.github.io/cl-cookbook/process.html
Stylistic issues
The splice-arglist helpers are not needed at all (so I'll also skip details in them). Use apply in your thread function instead:
(lambda ()
(apply fun larg-temp))
You don't need to (and should not) index into a list, because that is O(n) for each lookup—your loops are quadratic. Use dolist for simple side-effective loops, or loop when you have e. g. parallel iteration:
(loop :repeat nthreads
:for args :in llarg
:collect (sb-thread:make-thread (lambda () (apply fun args))))
For going over a list while creating a new list of the same length where each element is calculated from the corresponding element in the source list, use mapcar:
(mapcar #'sb-thread:join-thread threads)
Your function thus becomes:
(defun map-args-parallel (fun arglists nthreads)
(let ((threads (loop :repeat nthreads
:for args :in arglists
:collect (sb-thread:make-thread
(lambda ()
(apply fun args))))))
(mapcar #'sb-thread:join-thread threads)))
Performance
You are right that one usually creates only as many threads as ca. the number of cores available. If you test performance by always creating n threads, then joining them, then going to the next batch, you will indeed have not much difference in performance. That is because the inefficiency lies in creating the threads. A thread is about as resource intensive as a process.
What one usually does is to create a thread pool where the threads do not get joined, but instead reused. For that, you need some other mechanism to communicate arguments and results, e. g. channels (e. g. from chanl).
Note however that e. g. lparallel already provides a pmap function, and it does things right. The purpose of such wrapper libraries is not only to give the user (programmer) a nice interface, but also to think really hard about the problems and optimize sensibly. I am quite confident that pmap will be significantly faster than your attempt.
Related
I am looking for some advice or good practice about Threads in common lisp. Basically, I am trying to synchronise some threads with a global variable +clock+ (set as a thread too). I am a bit confused about different concepts as join-process, process-wait, make-mutex/make-lock, condition-variable, etc.
I am using ccl and sbcl, so I should probably use bordeaux-threads, but this is just a convenient way to manage both.
In short, my code works but when I add a thread, instead to share the cpu resources, this one increases beyond 150%.
;; for instance using CCL64 Version 1.12 DarwinX8664
;; --- THREADS-SET-1
(defvar +buffer+ nil)
(defvar +buffer-size+ 30)
(defparameter +compute+
(ccl:process-preset (ccl:make-process "+compute+")
#'(lambda ()
(loop do
(push (do-some-computation) +buffer+)
(sleep 0.1)))
'+compute+))
(defparameter +osc-send+
(ccl:process-preset (ccl:make-process "+osc-send+")
#'(lambda ()
(loop do
(when +buffer+
(OSCsend (car (last +buffer+)))
(setf +buffer+ (butlast +buffer+))
(if (> (length +buffer+) +buffer-size+)
(ccl:process-suspend +compute+)
(ccl:process-resume +compute+)))
(sleep (some-time))))
'+osc-send+))
;; commands:
(progn
(ccl:process-enable +compute+)
(sleep 1)
(ccl:process-enable +osc-send+))
(progn
(ccl:process-suspend +compute+)
(ccl:process-suspend +osc-send+))
(progn
(ccl:process-resume +compute+)
(ccl:process-resume +osc-send+))
(progn
(ccl:process-kill +compute+)
(ccl:process-kill +osc-send+))
;; when these threads as THREADS-SET-1 are 'playing' dx86cl64 takes more than 90% of cpu
;; and when I add some other threads as THREADS-SET-2, the cpu goes beyond 150%
;; needless to say that I did not try to add a third one before to solve this issue...
The final idea is to synchronise THREADS-SET-1 and THREADS-SET-2 with +clock+.
Maybe lparralel can be relevant in that context, if so thanks to illustrate how to use it in that case. Thanks in advance for any help or any reference book to learn about this topic.
Finally I solved the issue by making one single thread for each 'routine' plus one for the 'clock'. Then the cpu activity decreased until around 50% for 3 routines.
Anyway, I will be pleased if somebody could explain why in the first example the threads take all that cpu resources and not when I do one thread by routine. I suspect the use of process-suspend/resume instead of mutex... An affair to follow! :)
I'd like to create some futures in Clojure and run them all on a specific thread, to make sure they run one-at-a-time. Is this possible?
It's not hard to wrap the Java libraries to do this, but before I do that I want to make sure I'm not missing a Clojure way of doing it. In Java I can do this by implementing FutureTask and submitting those tasks to a single-threaded executor.
Clojure's future macro calls future-call function which uses a dedicated executor service. This means that you have no control to enforce a sequential execution.
On the other hand you can use promise instead of future objects and one future thread to sequentially deliver the results. Promise's API is similar to what futures provide. They have deref and realized? too.
The following code example has the subtasks executed sequentially on a new thread in the background while the immediately returned result of the function contains the promises to the computed values.
(defn start-async-calc []
(let [f1 (promise)
f2 (promise)
f3 (promise)]
(future
(deliver f1 (task-1))
(deliver f2 (task-2))
(deliver f3 (task-3)))
{:task1 f1
:task2 f2
:task3 f3}))
if you want to sequentialize the calls to future you can use it manually like this:
(do #(future 1)
#(future 2)
#(future 3))
they would still possibly called in different threads, but the next one won't be called until the previous has finished. This is guaranteed by the # (or deref function). This means that the thread in which you execute do form would be blocked with prev promise before it completes, and then spawn next one.
you can prettify it with macro like this:
(defmacro sequentialize [& futures]
`(do ~#(map #(list `deref %) futures)))
user> (let [a (atom 1)]
(sequentialize
(future (swap! a #(* 10 %)))
(future (swap! a #(+ 20 %)))
(future (swap! a #(- %))))
#a)
;;=> -30
this does exactly the same as manual do. Notice that mutations to a atom are in-order even if some threads run longer:
user> (let [a (atom 1)]
(sequentialize
(future (Thread/sleep 100)
(swap! a #(* 10 %)))
(future (Thread/sleep 200)
(swap! a #(+ 20 %)))
(future (swap! a #(- %))))
#a)
;;=> -30
Manifold provides a way to create future with specific executor. It's not part of core Clojure lib, but it's still a high quality lib and probably a best option in case you need more flexibility dealing with futures than core lib provides (without resorting to Java interop).
In addition the promises mentioned, you can use a delay. Promises have the problem that you can accidentally not deliver them, and create a deadlock scenario that's not possible with futures and delays. The difference between a future and a delay is only the thread that the work is executed on. With a future, the work is done in the background, and with a delay the work is done by the first thread that tries to deref it. So if future's are a better fit than promises, you could always do something like:
(def result-1 (delay (long-calculation-1)))
(def result-2 (delay (long-calculation-2)))
(def result-3 (delay (long-calculation-3)))
(defn run-calcs []
#(future
#result-1
#result-2
#result-3))
My question may seem weird but I think I'm facing an issue with volatile objects.
I have written a library implemented like this (just a scheme, not real content):
(def var1 (volatile! nil))
(def var2 (volatile! nil))
(def do-things [a]
(vreset! var1 a)
(vswap! var2 (inc #var2))
{:a #var1 :b #var2})
So I have global var which are initialized by external values, others that are calculated and I return their content.
i used volatile to have better speed than with atoms and not to redefine everytime a new var for every calculation.
The problem is that this seems to fail in practice because I map do-things to a collection (in another program) with inner sub-calls to this function occasionaly, like (pseudo-code) :
(map
(fn [x]
(let [analysis (do-things x)]
(if blabla
(do-things (f x))
analysis)))) coll)
Will inner conditionnal call spawn another thread under the hood ? It seems yes because somethimes calls work, sometimes not.
is there any other way to do apart from defining volatile inside every do-things body ?
EDIT
Actually the error was another thing but the question is still here : is this an acceptable/safe way to do without any explicit call to multithreading capabilities ?
There are very few constructs in Clojure that create threads on your behalf - generally Clojure can and will run on one or more threads depending on how you structure your program. pmap is a good example that creates and manages a pool of threads to map in parallel. Another is clojure.core.reducers/fold, which uses a fork/join pool, but really that's about it. In all other cases it's up to you to create and manage threads.
Volatiles should only be used with great care and in circumstances where you control the scope of use such that you are guaranteed not to be competing with threads to read and write the same volatile. Volatiles guarantee that writes can be read on another thread, but they do nothing to guarantee atomicity. For that, you must use either atoms (for uncoordinated) or refs and the STM (for coordinated).
From http://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io/ :
To get a bit more concrete let's see what happens when we try to issue
some HTTP GET request using core.async. Let's start with the naive
solution, using blocking IO via clj-http.
(defn blocking-get [url]
(clj-http.client/get url))
(time
(def data
(let [c (chan)
res (atom [])]
;; fetch em all
(doseq [i (range 10 100)]
(go (>! c (blocking-get (format "http://fssnip.net/%d" i)))))
;; gather results
(doseq [_ (range 10 100)]
(swap! res conj (<!! c)))
#res
)))
Here we're trying to fetch 90 code snippets (in parallel) using go
blocks (and blocking IO). This took a long time, and that's because
the go block threads are "hogged" by the long running IO operations.
The situation can be improved by switching the go blocks to normal
threads.
(time
(def data-thread
(let [c (chan)
res (atom [])]
;; fetch em all
(doseq [i (range 10 100)]
(thread (>!! c (blocking-get (format "http://fssnip.net/%d" i)))))
;; gather results
(doseq [_ (range 10 100)]
(swap! res conj (<!! c)))
#res
)))
What does it mean that "go block threads are hogged by the long running IO operations"?
Go blocks are intended to be a sort of light-weight cooperative threads; they provide thread-like behaviour with less overhead than full JVM threads by using a few threads in a pool and switching go blocks when they park - for instance, when waiting on a channel using <!. The thread-switching cannot work when you call a method in the block that blocks the JVM thread, so you quickly run out of JVM threads. Most standard Java (and Clojure) IO operations will block the current thread when waiting.
What does it mean that "go block threads are hogged by the long running IO operations"?
There are a limited number of threads dedicated to serving go blocks*. If you perform a blocking I/O operation on one of those threads, then it cannot be used for any other purpose until that operation completes (unless the thread is interrupted). This is also true for non-go block threads (i.e., threads that are returned from the thread function), but non-go block threads do not come from the limited go block thread pool. So if you do blocking I/O in a go block, you are "hogging" that go block's thread from being used by other go blocks, even though the thread isn't doing any actual work (it's just waiting for the I/O operation).
* That number currently happens to be 42 + the number of processors available to the JVM.
I have some non-thread-safe code (a writer to shared data) that can only be called from multiple threads in a serialised manner, but I don't want to block any other thread-safe work (multiple readers) when this code is not being called.
This is essentially a multiple reader / single writer type locking situation where writers need to exclude both readers and other writers.
i.e. I have two functions:
(defn reader-function [] ....) // only reads from shared data
(defn writer-function [] ....) // writes to shared data
And a number of threads that are running (possibly in a loop) the following:
(do
(reader-function)
...
(writer-function))
If any single thread is executing the writer function, all the other threads must block. i.e. at any one time either:
one thread is executing the
writer and all
others are blocked
multiple threads are
executing the reader function, possibly some threads are
blocked waiting to execute
the writer once all readers are completed
What's the best way to achieve this kind of synchronisation in Clojure?
Put your data in a ref. The data should be a Clojure data structure (not a Java class). Use dosync to create a transaction around the read and write.
Example. Because you split your writer into a separate function, that function must modify a ref with something like an alter. Doing so requires a transaction (dosync). You could rely on writer being called only in a dosync but you can also put a dosync inside the write and rely on nested transactions doing what you want - this makes writer safe to call either in or out of a transaction.
(defn reader [shared]
(println "I see" #shared))
(defn writer [shared item]
(dosync
(println "Writing to shared")
(alter shared conj item)))
;; combine the read and the write in a transaction
(defn combine [shared item]
(dosync
(reader shared)
(writer shared item)))
;; run a loop that adds n thread-specific items to the ref
(defn test-loop [shared n]
(doseq [i (range n)]
(combine shared (str (System/identityHashCode (Thread/currentThread)) "-" i))
(Thread/sleep 50)))
;; run t threads adding n items in parallel
(defn test-threaded [t n]
(let [shared (ref [])]
(doseq [_ (range t)]
(future (test-loop shared n)))))
Run the test with something like (test-threaded 3 10).
More info here: http://clojure.org/refs
You didn't ask about this case, but it's important to note that anyone can read the shared ref by derefing it at any time. This does not block concurrent writers.
Take a look at java.util.concurrent.locks.ReentrantReadWriteLock. This class allow you to have multiple readers that do not contend with each other on one writer at a time.