Common lisp: calling a class method in a separate thread - multithreading

I am trying to build a common lisp implementation of the channel construct of Golang for a personal project (also to learn lisp). So far I've implemented the channels as objects of a class, containing a queue, a lock and a condition variable to signal listening functions that a new message has been added to the queue. I'm using bordeaux threads to create threads, locks, condition variables and join the executions (from the lisp cookbook).
This is the channel class and the recive function:
(defclass channel ()
((messages :initform '()
:accessor messages
:documentation "Messages in the channel")
(lock :initform (bt:make-lock)
:accessor lock
:documentation
"Lock to push/pop messages in the channel")
(cv :initarg :cv
:initform (bt:make-condition-variable)
:accessor cv
:documentation
"Condtional variable to notify the channel of a new message")))
(defmethod recive-loop ((self channel))
(with-slots (lock cv messages) self
(let ((to-ret nil))
(loop
(bt:with-lock-held (lock)
(if (not (null messages))
(setf to-ret (car (pop messages)))
(bt:condition-wait cv lock))
(if to-ret (return to-ret)))))))
(defmethod recive ((self channel))
(with-slots (name thread) self
(let ((thread
(bt:make-thread #'(lambda() (recive-loop self))
:name name)))
(bt:join-thread thread))))
(defmacro gorun (f &rest args)
(flet ((fn () (apply f args)))
(bt:make-thread #'fn
:initial-bindings (list args)
:name "gorun worker")))
gorun should be the equivalent of go routine() for go (without the light threading). To test the setup I've built a printer function over a channel
(defvar printch (channel))
(defun printover (ch)
(let ((x (recive ch)))
(format t "Recived variable x: ~d~%" x)))
but when I run
(gorun printover printch)
The interpreter (using sbcl, but with clisp the same happens) gives back an error:
There is no applicable method for the generic function
#<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::RECIVE (1)>
when called with arguments
(PRINTCH).
[Condition of type SB-PCL::NO-APPLICABLE-METHOD-ERROR]
See also:
Common Lisp Hyperspec, 7.6.6 [:section]
Restarts:
0: [RETRY] Retry calling the generic function.
1: [ABORT] abort thread (#<THREAD "gorun worker" RUNNING {100293E9F3}>)
Backtrace:
0: ((:METHOD NO-APPLICABLE-METHOD (T)) #<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::RECIVE (1)> PRINTCH) [fast-method]
Locals:
SB-PCL::ARGS = (PRINTCH)
GENERIC-FUNCTION = #<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::RECIVE (1)>
1: (SB-PCL::CALL-NO-APPLICABLE-METHOD #<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::RECIVE (1)> (PRINTCH))
Locals:
ARGS = (PRINTCH)
GF = #<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::RECIVE (1)>
2: (PRINTOVER PRINTCH)
Locals:
CH = PRINTCH
3: ((LAMBDA NIL :IN BORDEAUX-THREADS::BINDING-DEFAULT-SPECIALS))
[No Locals]
I'm confused, since the method to run over the channel printch should be the one I've defined.
Trying to call a class method inside of a new thread, but got no applicable method

A macros is supposed to return code to run in place of the original call. Your macro is creating the thread at expansion time.
If you're not using backtick in a macro definition, there's usually something wrong with it. You should figure out what the code would look like without the macro, then define a macro that returns code with that same structure in a backticked list, replacing the places that need to vary with the parameters, using comma to expand them.
(defmacro gorun (f &rest args)
`(bt:make-thread (function ,f)
:initial-bindings (list ,#args)
:name "gorun worker"))
In the above, you need to substitute the function name into a (function ...) expression, and the args list as the :initial-bindings argument.

In multi-threaded environments it is often the case that special variables are thread-local. The global binding is visible from all threads, but if you bind one locally its value will not automatically be transferred to a thread created in that context. It has to be done explicitly, and I wrote a pair of macros recently just for that.
The first one captures bindings into lexically-scoped variables; the other bind the original variables back to the values captured in a different context.
I am using an intermediate data structure in the code to store bindings:
(defstruct bindings data)
The first macro is with-captured-bindings:
(defmacro with-captured-bindings ((&rest symbols) as name &body body)
(assert (eq as :as))
(loop for s in (alexandria:flatten
(sublis
'((:stdio *standard-output* *error-output* *standard-input*)
(:path *default-pathname-defaults*))
symbols))
for g = (gensym)
collect (list g s) into capture
collect (list s g) into rebind
finally
(return
`(let ,capture
,#(subst (make-bindings :data rebind)
name
body)))))
The capture variable holds a list of bindings to initialize the lexically-scoped variables. The rebind variables is a list of bindings to set back special variables to their values in another thread.
I inject with subst an instance of the bindings struct in the code. It helps to have a dedicated data structure, but the crude search-and-replace approach means the symbols name will not be usable as a function, local macro, etc. in body. I don't think it is too much of a problem.
Also, I define aliases like :stdio and :path for commonly used variables.
The second macro is with-bindings:
(defmacro with-bindings (bindings &body body)
(check-type bindings bindings)
`(let ,(bindings-data bindings)
,#body))
This replaces the intermediate struct with the proper code. The final code does not have this struct anymore and can be processed as usual.
For example:
(defvar *my-var* "hello")
(with-captured-bindings (:stdio :path *my-var*) :as <bindings>
(sb-thread:make-thread
(lambda ()
(with-bindings <bindings>
(print *var*)))))
A first application of macroexpand gives:
(LET ((#:G3882 *STANDARD-OUTPUT*)
(#:G3883 *ERROR-OUTPUT*)
(#:G3884 *STANDARD-INPUT*)
(#:G3885 *DEFAULT-PATHNAME-DEFAULTS*)
(#:G3886 *MY-VAR*))
(SB-THREAD:MAKE-THREAD
(LAMBDA ()
(WITH-BINDINGS #S(BINDINGS
:DATA ((*STANDARD-OUTPUT* #:G3882)
(*ERROR-OUTPUT* #:G3883)
(*STANDARD-INPUT* #:G3884)
(*DEFAULT-PATHNAME-DEFAULTS* #:G3885)
(*MY-VAR* #:G3886)))
(PRINT *MY-VAR*)))))
Notice that there is #S(BINDINGS ...) object in the tree.
The full expansion is:
(LET ((#:G3887 *STANDARD-OUTPUT*)
(#:G3888 *ERROR-OUTPUT*)
(#:G3889 *STANDARD-INPUT*)
(#:G3890 *DEFAULT-PATHNAME-DEFAULTS*)
(#:G3891 *MY-VAR*))
(SB-THREAD:MAKE-THREAD
(LAMBDA ()
(LET ((*STANDARD-OUTPUT* #:G3887)
(*ERROR-OUTPUT* #:G3888)
(*STANDARD-INPUT* #:G3889)
(*DEFAULT-PATHNAME-DEFAULTS* #:G3890)
(*MY-VAR* #:G3891))
(PRINT *MY-VAR*)))))

Related

WITH-OUTPUT-TO-STRING with multithreading in Common Lisp

I want to do something that means the following:
(with-output-to-string (*standard-output*)
(bt:join-thread
(bt:make-thread
(lambda ()
(format *standard-output* "Hello World")))))
;=> "" (actual output)
;=> "Hello World" (expected output)
In my understanding, this does not work because the *standard-output* that gets dynamically rebound by with-output-to-string outside the thread does not take effect inside the thread. What are the possible and recommedable ways?
In essence, I want to capture the output that was written to *standard-output* by another thread.
One can rebind the special variable to be thread local:
(with-output-to-string (*standard-output*)
(bt:join-thread
(bt:make-thread
(lambda ()
(format *standard-output* "Hello World"))
:initial-bindings `((*standard-output* . ,*standard-output*)))))
*initial-bindings* is an alist with (variable . value) elements.
A previous idea was to mutate the original binding of *standard-output* itself. As #coredump suggested, this has the downside that the binding will be mutated in all the threads - other threads that are sending their output to *standard-output* would also send their output to the string-output-stream.
Another idea is to let the thread itself decide whether to send the output to *standard-output* or to some other stream:
(let ((in-with-output-to-string nil)
(output-stream-string nil))
(unwind-protect
(progn
(setq output-stream-string (make-string-output-stream))
(setq in-with-output-to-string t)
(bt:join-thread
(bt:make-thread
(lambda ()
(format (if in-with-output-to-string
output-stream-string
*standard-output*)
"Hello World"))))
(get-output-stream-string output-stream-string))
(setq in-with-output-to-string nil)))
;=> "Hello World"
A more involved example is illustrated in the following. The general situation I was interested in involved a thread reading some stream and sending the contents of that stream to *standard-output*. However, in certain cases, I was interested in capturing the output of that thread into a string.
Even before that, drawing inspiration from [1], we define a helper macro which captures the variable bindings that were present before executing the body and then restores them once the body has completed execution.
(deftype list-of (&rest types)
(if types
`(cons ,(first types) (list-of ,#(rest types)))
'null))
(defmacro thread-global-let (bindings &body body)
(let* ((bindings (mapcar (lambda (binding)
;; Normalize the bindings
(etypecase binding
(symbol
(list binding nil))
((list-of symbol)
(list (first binding) nil))
((list-of symbol t)
binding)))
bindings))
(variables (mapcar #'first bindings))
(gensyms (alexandria:make-gensym-list (length variables))))
`(let (,#(mapcar (lambda (var gensym)
`(,gensym ,var))
variables gensyms))
(unwind-protect
(progn
,#(mapcar (lambda (binding)
`(setq ,#binding))
bindings)
,#body)
,#(mapcar (lambda (var gensym)
`(setq ,var ,gensym))
variables gensyms)))))
The main example then is the following:
(defvar *input-wait-condition* (bt:make-condition-variable))
(defvar *input-wait-lock* (bt:make-lock))
(defvar *stream-input-string* nil)
(defvar *thread*)
(let ((in-with-thread-output nil)
(stream-output-string nil))
(when (and (boundp '*thread*)
(bt:threadp *thread*))
(bt:destroy-thread *thread*))
(setq *thread*
(bt:make-thread
(lambda ()
(bt:with-lock-held (*input-wait-lock*)
(loop :do (bt:condition-wait *input-wait-condition* *input-wait-lock*)
(loop :while (listen *stream-input-string*)
:do (write-char (read-char *stream-input-string*)
(if in-with-thread-output
stream-output-string
*standard-output*))))))))
(defun thread-output-thunk (thunk)
(thread-global-let ((stream-output-string (make-string-output-stream))
(in-with-thread-output t))
(funcall thunk)
(get-output-stream-string stream-output-string))))
(defmacro with-thread-output (&body body)
`(thread-output-thunk (lambda () ,#body)))
What it essentially achieves is the following:
CL-USER> (setq *stream-input-string* (make-string-input-stream "Hello World"))
#<SB-IMPL::STRING-INPUT-STREAM {100D0D47A3}>
CL-USER> (bt:condition-notify *input-wait-condition*)
NIL
Hello World
CL-USER> (with-thread-output
(thread-global-let
((*stream-input-string*
(make-string-input-stream "Output from a thread")))
(bt:condition-notify *input-wait-condition*)
(loop :while (listen *stream-input-string*))))
"Output from a thread"
CL-USER> (with-thread-output
(thread-global-let
((*stream-input-string*
(make-string-input-stream "Output from a thread")))
(print (with-thread-output
(thread-global-let
((*stream-input-string*
(make-string-input-stream "Output from a thread 2")))
(bt:with-lock-held (*input-wait-lock*)
(bt:condition-notify *input-wait-condition*))
(loop :while (listen *stream-input-string*)))))
(bt:with-lock-held (*input-wait-lock*)
(bt:condition-notify *input-wait-condition*))
(loop :while (listen *stream-input-string*))))
"Output from a thread 2"
"Output from a thread"
The following code illustrates the previous idea of mutating the original binding of *standard-output*. This has the downside of the mutation affecting all the threads.
(let ((original-stdout *standard-output*))
(with-output-to-string (stdout)
(unwind-protect
(progn
(setq *standard-output* stdout)
(bt:join-thread
(bt:make-thread
(lambda ()
(format *standard-output* "Hello World")))))
(setq *standard-output* original-stdout))))
This all seem a bit complex, and it looks like you are calling bt:join-thread to wait for the thread to finish. Unfortunately, this means that your main thread is blocked until the worker thread is finished, something that typically is a code smell in multi-threaded applications (if you block the main thread, you might as well call the function directly in the same thread).
In fact, when threads are involved, lexically scoped macros like with-X are usually either better done inside the thread, or not at all. I am going to use the lparallel library because it provides queues datastructures.
(defpackage :so (:use :cl))
(in-package :so)
(ql:quickload :lparallel)
Let's define a sample test function that uses standard input and output streams:
(defun test-function ()
"A test function that acts on standard input/output streams"
(format t "input is: ~s" (read-line)))
The following make-thread/redirect-io function takes a function to execute as well as its input as a string. It returns another function that will block until the thread terminates and return the accumulated output as a string. In practice you would also need to handle exceptions:
(defun make-thread/redirect-io (fn input-as-string)
(let ((queue (lparallel.queue:make-queue)))
(values (lambda () (lparallel.queue:pop-queue queue))
(bt:make-thread
(lambda ()
(lparallel.queue:push-queue
(with-output-to-string (*standard-output*)
(with-input-from-string (*standard-input* input-as-string)
(funcall fn)))
queue))))))
This allows you to really run things in parallel, for example you can spawn two threads with different inputs:
(let ((join-a (make-thread/redirect-io #'test-function "Hello"))
(join-b (make-thread/redirect-io #'test-function "World")))
;; do something else in parallel, do not join the thread otherwise
;; it would just be blocking as-if you called the function in the
;; same thread
;; then, get the results
(list (funcall join-a)
(funcall join-b)))
This returns the following list:
("input is: \"Hello\"" "input is: \"World\"")
In fact, I'd suggest using having a look at lparallel.org to see if you can use it directly, it can greatly simplify working with threads.

Does SBCL for lisp handle scope differently? It does not seem to pass scope into called functions?

When using emacs or my android app I run
(defun big (num) (setf num2 5)(little num)))
(defun little (num)(+ num2 num))
Little happily accepts num2 but when I run it in my SBCL repl (with sublimetext3) it does not.
Is this correct?
What is a workaround without creating a global variable for num2?
I could just pass a second argument (little num num2)
But this wont work when I am trying to mapcar little over a list. Because I can only have one argument when mapcaring correct?
Please read ยง6. Variables from Practical Common Lisp.
Unlike Emacs Lisp, Common Lisp relies on lexical scope by default (Emacs Lisp is dynamic by default). Dynamic scope (i.e. indefinite scope and dynamic extent) is provided by declaring variables special, and by convention, they are written with asterisks around their names (named "earmuffs"), like *standard-output*. You use defparameter or defvar to declare those variables. Since it has a global effect, you should never use them from inside functions; likewise, your usage of setf is not defined in Common Lisp: no variable named num2 was declared previously in the scope; besides, even if it did, using a global/special variable for local variable is bad style.
Dynamic scope
With special variables, you can for example locally rebind the standard output: the new value is only visible while the code is inside the body of the let binding:
(let ((*standard-output* *error-output*))
(print "Stream redirection"))
By default, print writes to the stream bound to *standard-output*; here, the stream is locally bound to the one given by *error-output*. As soon as you escape the let, *standard-output* reverts to its previous value (imagine there is a stack).
Lexical scope
With lexical scope, your code can only access the bindings that are visible in the text surrounding your code (and the global scope), and the extent is indefinite: it is possible to access a binding (sometimes indirectly) even after the code returns from the let:
(let ((closure
(let ((count 0))
(lambda () (print (incf count))))))
(funcall closure)
(funcall closure))
;; prints:
;; 1
;; 2
The lambda expression creates a closure, which captures the variable named count. Every time you call it, it will increase the count variable and print it. If you evaluate the same code another time, you define another closure and create another variable, with the same name.
Mapcar
Because I can only have one argument when mapcaring correct?
Not exactly; the function called by mapcar should be able to accept at least as many elements as the number of lists that are given to it (and it should also not require more mandatory arguments):
(mapcar (lambda (x y) (* x y))
'(1 2 3)
'(0 3 6))
=> (0 6 18)
(mapcar #'list '(1 2) '(a b) '(+ /))
=> ((1 a +) (2 b /))
The function can also be a closure, and can use special variables.
... with a closure
(defun adder (x)
(lambda (y) (+ x y)))
(mapcar (adder 10) '(0 1 2))
=> (10 11 12)
The adder functions takes a number x and returns a closure which accepts a number y and returns (+ x y).
... with a special variable
If you prefer dynamic scope, use earmuffs and give it a meaningful name:
(defparameter *default-offset* 0)
... and define:
(defun offset (x)
(+ x *default-offset*))
You can then mapcar too:
(let ((*default-offset* 20))
(mapcar #'offset '(1 2 3)))
=> (21 22 23)
As said by jkiiski in comments, you can also declare special variables with (declare (special ...)) where you usually put declarations (when entering a let, a defun, ...). You could also use the special operator progv. This can be useful to have "invisible" variables that are only known by a set of functions to exchange information. You rarely need them.

When to use a Var instead of a function?

I am going through the book Web Development with Clojure and it tells me to pass the handler (defined bellow) as a Var object instead of as the function itself, since the function can then change dynamically (this is what wrap-reload does).
The book says:
"Note that we have to create a var from the handler in order for this middleware
to work. This is necessary to ensure that the Var object containing the current
handler function is returned. If we used the handler instead then the app would
only see the original value of the function and changes would not be reflected."
I don't really understand what this means, are vars similar to c pointers?
(ns ring-app.core
(:require [ring.adapter.jetty :as jetty]
[ring.util.response :as response]
[ring.middleware.reload :refer [wrap-reload]]))
(defn handler [request]
(response/response
(str "<html>/<body> your IP is: " (:remote-addr request)
"</body></html>")))
(defn wrap-nocache [handler]
(fn [request]
(-> request
handler
(assoc-in [:headers "Pragma"] "no-cache"))))
Here is the handler call:
(defn -main []
(jetty/run-jetty
(wrap-reload (wrap-nocache (var handler)))
{:port 3001
:join? false}))
Yes, a Var in Clojure is similar to a C pointer. This is poorly documented.
Suppose you create a function fred as follows:
(defn fred [x] (+ x 1))
There are actually 3 things here. Firstly, fred is a symbol. There is a difference between a symbol fred (no quotes) and the keyword :fred (marked by the leading : char) and the string "fred" (marked by a double-quote at both ends). To Clojure, each of them is composed of 4 characters; i.e. neither the colon of the keyword nor the double-quotes of the string are included in their length or composition:
> (name 'fred)
"fred"
> (name :fred)
"fred"
> (name "fred")
"fred"
The only difference is how they are interpreted. A string is meant to represent user data of any sort. A keyword is meant to represent control information for the program, in a readable form (as opposed to "magic numbers" like 1=left, 2=right, we just use keywords :left and :right.
A symbol is meant to point to things, just like in Java or C. If we say
(let [x 1
y (+ x 1) ]
(println y))
;=> 2
then x points to the value 1, y points to the value 2, and we see the result printed.
the (def ...) form introduces an invisible third element, the Var. So if we say
(def wilma 3)
we now have 3 objects to consider. wilma is a symbol, which points to a Var, which in turn points to the value 3. When our program encounters the symbol wilma, it is evaluated to find the Var. Likewise, the Var is evaluated to yield the value 3. So it is like a 2-level indirection of pointers in C. Since both the symbol and the Var are "auto-evaluated", this happens automatically and invisibly and you don't have to think about the Var (indeed, most people aren't really aware the invisible middle step even exists).
For our function fred above, a similar situation exists, except the Var points to the anonymous function (fn [x] (+ x 1)) instead of the value 3 like with wilma.
We can "short-circuit" the auto-evaluation of the Var like:
> (var wilma)
#'clj.core/wilma
or
> #'wilma
#'clj.core/wilma
where the reader macro #' (pound-quote) is a shorthand way of calling the (var ...) special form. Keep in mind that a special form like var is a compiler built-in like if or def, and is not the same as a regular function. The var special form returns the Var object attached to the symbol wilma. The clojure REPL prints the Var object using the same shorthand, so both results look the same.
Once we have the Var object, auto-evaluation is disabled:
> (println (var wilma))
#'clj.core/wilma
If we want to get to the value that wilma points to, we need to use var-get:
> (var-get (var wilma))
3
> (var-get #'wilma)
3
The same thing works for fred:
> (var-get #'fred)
#object[clj.core$fred 0x599adf07 "clj.core$fred#599adf07"]
> (var-get (var fred))
#object[clj.core$fred 0x599adf07 "clj.core$fred#599adf07"]
where the #object[clj.core$fred ...] stuff is Clojure's way of representing a function object as a string.
With regard to the web server, it can tell via the var? function or otherwise if the supplied value is the handler function or the var which points to the handler function.
If you type something like:
(jetty/run-jetty handler)
the double auto-evaluation will yield the handler function object, which is passed to run-jetty. If, instead, you type:
(jetty/run-jetty (var handler))
then the Var which points to the handler function object will be passed to run-jetty. Then, run-jetty will have to use an if statement or equivalent to determine what it has received, and call (var-get ...) if it has received a Var instead of a function. Thus, each time through (var-get ...) will return the object to which the Var currently points. So, the Var acts like a global pointer in C, or a global "reference" variable in Java.
If you pass a function object to run-jetty, it saves a "local pointer" to the function object and there is no way for the outside world to change what the local pointer refers to.
You can find more details here:
http://clojure.org/reference/evaluation
http://clojure.org/reference/vars
Update
As OlegTheCat has pointed out, Clojure has yet another trick up its sleeve regarding Var objects that point to Clojure functions. Consider a simple function:
(defn add-3 [x] (+ x 3))
; `add-3` is a global symbol that points to
; a Var object, that points to
; a function object.
(dotest
(let [add-3-fn add-3 ; a local pointer to the fn object
add-3-var (var add-3)] ; a local pointer to the Var object
(is= 42 (add-3 39)) ; double deref from global symbol to fn object
(is= 42 (add-3-fn 39)) ; single deref from local symbol to fn object
(is= 42 (add-3-var 39))) ; use the Var object as a function
; => SILENT deref to fn object
If we treat a Var object as a function, Clojure will SILENTLY deref it into the function object, then invoke that function object with the supplied args. So we see that all three of add-3, add-3-fn and add-3-var will work. This is what is occurring in Jetty. It never realizes that you have given it a Var object instead of a function, but Clojure magically patches up that mismatch without telling you.
Sidebar: Please note this only works since our "jetty" is actually
the Clojure wrapper code ring.adapter.jetty, and not the actual Java
webserver Jetty. If you tried to depend on this trick with an
actual Java function instead of a Clojure wrapper, it would fail. Indeed, you must use a Clojure wrapper like proxy in order to pass a Clojure function to Java code.
You have no such guardian angel to save you if you use the Var object as anything other than a function:
(let [wilma-long wilma ; a local pointer to the long object
wilma-var (var wilma)] ; a local pointer to the Var object
(is (int? wilma-long)) ; it is a Long integer object
(is (var? wilma-var)) ; it is a Var object
(is= 4 (inc wilma)) ; double deref from global symbol to Long object
(is= 4 (inc wilma-long)) ; single deref from local symbol to Long object
(throws? (inc wilma-var)))) ; Var object used as arg => WILL NOT deref to Long object
So, if you are expecting a function and someone gives you a Var object that points to a function, then you are OK since Clojure silently fixes the problem. If you are expecting anything other than a function and someone gives you a Var object that points to that thing, then you are on your own.
Consider this helper function:
(defn unvar
"When passed a clojure var-object, returns the referenced value (via deref/var-get);
else returns arg unchanged. Idempotent to multiple calls."
[value-or-var]
(if (var? value-or-var)
(deref value-or-var) ; or var-get
value-or-var))
Now you can safely use the thing you were given:
(is= 42 (+ 39 (unvar wilma))
(+ 39 (unvar wilma-long))
(+ 39 (unvar wilma-var)))
Appendix
Notice that there are three dualities that can confuse the issue:
Both var-get and deref do the same thing with a Clojure Var
The reader macro #'xxx is translated into (var xxx)
The reader macro #xxx is translated into (deref xxx)
So we have (confusingly!) many ways of doing the same thing:
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(def wilma 3)
; `wilma` is a global symbol that points to
; a Var object, that points to
; a java.lang.Long object of value `3`
(dotest
(is= java.lang.Long (type wilma))
(is= 3 (var-get (var wilma)))
(is= 3 (var-get #'wilma))
; `deref` and `var-get` are interchangable
(is= 3 (deref (var wilma)))
(is= 3 (deref #'wilma))
; the reader macro `#xxx` is a shortcut that translates to `(deref xxx)`
(is= 3 #(var wilma))
(is= 3 ##'wilma)) ; Don't do this - it's an abuse of reader macros.
Another note
The (def ...) special form returns the clojure.lang.Var object it creates. Normally, a (def ...) form is used only at the top level in a Clojure source file (or at the REPL), so the return value is silently discarded. However, a reference to the created Var object can also be captured:
(let [p (def five 5)
q (var five)]
(is= clojure.lang.Var
(type p)
(type q))
(is= 6
(inc five)
(inc (var-get p))
(inc (deref q)))
(is (identical? p q)))
Here we create a global Var five pointing to the number 5. The return value of the def form is captured in the local value p. We use the (var ...) special form to get a reference q pointing to the same Var object.
The first test shows that p and q are both of type clojure.lang.Var. The middle test shows three ways of accessing the value 5. As expected, all retrieve the value 5 which is incremented to yield 6. The last test verifies that p and q both point to the same Java object (i.e. there is only one clojure.lang.Var object that points to the integer 5).
It is even possible for a Var to point to another Var instead of a data value:
(def p (def five 5)) ; please don't ever do this
While it works, I cannot think of a legitimate reason for ever doing this.
There are a couple of good answers already. Just wanted to add this caveat:
(defn f [] 10)
(defn g [] (f))
(g) ;;=> 10
(defn f [] 11)
;; -Dclojure.compiler.direct-linking=true
(g) ;;=> 10
;; -Dclojure.compiler.direct-linking=false
(g) ;;=> 11
So, when direct linking is on, the indirection via a var is replaced with a direct static invocation. Similar to the situation with the handler, but then with every var invocation, unless you explicitly refer to a var, like:
(defn g [] (#'f))
Hopefully this small example will get you on track:
> (defn your-handler [x] x)
#'your-handler
> (defn wrap-inc [f]
(fn [x]
(inc (f x))))
> #'wrap-inc
> (def your-app-with-var (wrap-inc #'your-handler))
#'your-app-with-var
> (def your-app-without-var (wrap-inc your-handler))
#'your-app-without-var
> (your-app-with-var 1)
2
> (your-app-without-var 1)
2
> (defn your-handler [x] 10)
#'your-handler
> (your-app-with-var 1)
11
> (your-app-without-var 1)
2
The intuition for this is when you use a var when creating your handler you are actually passing a "container" with some value, content of which can be changed in future by defining var with the same name. When you don't use var (like in your-app-without-var) you are passing a current value of this "container", which cannot be redefined in any way.

Pass an object argument to a macro

I encounter errors when passing an object argument to a macro. Must I quote the argument, put it in a list, or not quote it?
I wish to use Clozure Common Lisp to generate and run multiple processes in parallel using a read-write-lock to control data output to another process. With-write-lock is a macro that waits until the given lock is available for write access, then executes its body with the lock held. However, I get errors no matter how I try to pass the lock to with-write-lock. I'm having trouble I think because I fail to understand how to pass a lock object to the with-write-lock macro. If I bind the lock to a symbol I get destructuring errors:
(let ((l (make-read-write-lock)))
(with-write-lock l (1+ 1)))
==>
> Error: L can't be destructured against the lambda list (LOCK), because it is not a proper list.
While executing: (:INTERNAL CCL::NX1-COMPILE-LAMBDA), in process Listener(4).
but if I pass the call to make-read-write-lock as the lock argument to with-write-lock then I get an undeclared free variable error:
(with-write-lock (make-read-write-lock) (1+ 1))
==>
;Compiler warnings for "/Users/frank/Documents/Lisp/threaded/act-parallel.lisp" :
;In an anonymous lambda form at position 18: Undeclared free variable MAKE-READ-WRITE-LOCK
Error: Unbound variable: MAKE-READ-WRITE-LOCK
While executing: #, in process Listener(4).
Am I failing because I misunderstand how to pass an object to a macro or am I going awry because or something more particular to with-write-lock?
Here's the with-write-lock macro that comes with Clozure Common Lisp (macros.lisp):
(defmacro with-write-lock ((lock) &body body)
(let* ((locked (gensym))
(p (gensym)))
`(with-lock-context
(let* ((,locked (make-lock-acquisition))
(,p ,lock))
(declare (dynamic-extent ,locked))
(unwind-protect
(progn
(write-lock-rwlock ,p ,locked)
,#body)
(when (lock-acquisition.status ,locked) (unlock-rwlock ,p)))))))
The lambda list for that macro is destructuring its arguments.
((lock) &body body)
Means that it wants the first argument as a list which contains the lock and then the body form. This is a pretty standard macro argument list in CL, you can use it like this:
(with-write-lock (lock)
..... body forms ....)
So your examples would be
(let ((l (make-read-write-lock)))
(with-write-lock (l) (1+ 1)))
And:
(with-write-lock ((make-read-write-lock)) (1+ 1))
respectively. Note the extra parens around the first argument.
For similar macros see with-open-file or destructuring-bind
(with-open-file ("path/to/file" :open :args)
.... body ...)
(destructuring-bind (one two three) '(1 2 3)
.... body forms ...)

Is there a generic method for cloning CLOS objects?

I'm looking for a way to clone CLOS objects in a shallow manner, so the created object would be of the same type with the same values in each slot, but a new instance. The closest thing I found is a standard function copy-structure which does this for structures.
There is no standard predefined way to copy CLOS objects in general. It is not trivial, if possible at all, to provide a reasonable default copy operation that does the right thing (at least) most of the time for arbitrary objects, since the correct semantics change from class to class and from application to application. The extended possibilities the MOP provides make it even harder to provide such a default. Also, in CL, being a garbage collected language, copying of objects is not really needed very often, e.g. when passed as parameters or being returned. So, implementing your copy operations as needed would probably be the cleanest solution.
That being said, here is what I found in one of my snippet files, which might do what you want:
(defun shallow-copy-object (original)
(let* ((class (class-of original))
(copy (allocate-instance class)))
(dolist (slot (mapcar #'slot-definition-name (class-slots class)))
(when (slot-boundp original slot)
(setf (slot-value copy slot)
(slot-value original slot))))
copy))
You will need some MOP support for class-slots and slot-definition-name.
(I probably adopted this from an old c.l.l thread, but I can't remember. I never really needed something like this, so it's utterly untested.)
You can use it like this (tested with CCL):
CL-USER> (defclass foo ()
((x :accessor x :initarg :x)
(y :accessor y :initarg :y)))
#<STANDARD-CLASS FOO>
CL-USER> (defmethod print-object ((obj foo) stream)
(print-unreadable-object (obj stream :identity t :type t)
(format stream ":x ~a :y ~a" (x obj) (y obj))))
#<STANDARD-METHOD PRINT-OBJECT (FOO T)>
CL-USER> (defparameter *f* (make-instance 'foo :x 1 :y 2))
*F*
CL-USER> *f*
#<FOO :x 1 :y 2 #xC7E5156>
CL-USER> (shallow-copy-object *f*)
#<FOO :x 1 :y 2 #xC850306>
Here's a slightly different version of the function submitted by danlei. I wrote this a while ago and just stumbled across this post. For reasons that I don't entirely recall, this calls REINITIALIZE-INSTANCE after copying. I think it's so you could make some changes to the new object by passing additional initargs to this function
e.g.
(copy-instance *my-account* :balance 100.23)
This is also defined as generic function over objects that are 'standard-object's. Which might or might not be the right thing to do.
(defgeneric copy-instance (object &rest initargs &key &allow-other-keys)
(:documentation "Makes and returns a shallow copy of OBJECT.
An uninitialized object of the same class as OBJECT is allocated by
calling ALLOCATE-INSTANCE. For all slots returned by
CLASS-SLOTS, the returned object has the
same slot values and slot-unbound status as OBJECT.
REINITIALIZE-INSTANCE is called to update the copy with INITARGS.")
(:method ((object standard-object) &rest initargs &key &allow-other-keys)
(let* ((class (class-of object))
(copy (allocate-instance class)))
(dolist (slot-name (mapcar #'sb-mop:slot-definition-name (sb-mop:class-slots class)))
(when (slot-boundp object slot-name)
(setf (slot-value copy slot-name)
(slot-value object slot-name))))
(apply #'reinitialize-instance copy initargs))))
This solution does not require sl-mob:
(defun copy-slot (s d slot)
`(setf (,slot ,d) (,slot ,s)))
(defun copy-by-slots (s d slots)
(assert (eql (class-of s) (class-of d)))
(let ((f (lambda (s$) (eval (copy-slot s d s$)))))
(mapcar f slots)))
(copy-by-slots src dest quoted-list-of-slots)
I mention a dirty trick producing a clone of a CLOS instance.
(defclass cl () ((sl1 :initarg :sl1) (sl2 :initarg :sl2)))
(defmethod update-instance-for-different-class ((copy cl) (original cl) &key)
(setf clone copy))
(setf a (make-instance 'cl :sl1 111 :sl2 222))
(change-class a 'cl)
(eq clone a) -> NIL
(eql (slot-value a 'sl1) (slot-value clone 'sl1)) -> T
Implies CLOS itself needs a notion of clone.

Resources