Using the Lparallel library for Common Lisp, you can start a parallel thread running by calling (submit-task channel function). This executes the given function in a new parallel thread. However, the function always seems to run in the :cl-user package, as opposed to the package in which it is defined. I’m just wondering why Lparallel was designed this way. You can override by including (in-package :my-package) in the function, but is there some advantage to this predictability?
#-:lparallel
(ql:quickload :lparallel)
(defpackage :test (:use :cl :lparallel))
(in-package :test)
(defun main ()
(setf lparallel:*kernel* (lparallel:make-kernel 1))
(print *package*)
(let ((c (make-channel)))
(submit-task c (lambda () (print *package*))))
(lparallel:end-kernel))
gives:
* (load "d:\\test.lisp")
To load "lparallel":
Load 1 ASDF system:
lparallel
; Loading "lparallel"
T
* (in-package :test)
#<PACKAGE "TEST">
* (main)
#<PACKAGE "TEST">
#<PACKAGE "COMMON-LISP-USER">
(#<SB-THREAD:THREAD "lparallel kernel shutdown manager" FINISHED values: NIL {1005873953}>
#<SB-THREAD:THREAD "lparallel" FINISHED values: NIL {10058735F3}>)
The default behaviour for dynamic variables is that only a binding in the global environment (i. e. not a dynamic re-binding) is shared between threads.
The value of *package* is mostly irrelevant at runtime, except when reading (which is unusual at runtime, and you'd explicitly bind *package* for that anyway). There is thus no reason to deviate from the standard behaviour described above.
There is also no connection between code (e. g. a function) and the package that was the value of *package* when it was read, compiled, or loaded. The only effect of *package* is the default package when interning, finding, or printing symbols.
Related
For rapid prototyping purposes in common-lisp it would be convenient to be able to easily function-modify an object in an arbitrary data structure. This would seem to involve calling an arbitrary function on a place in the data structure, replacing the object at that place with the result of the function call. Common-lisp has a number of specialized modification macros (eg, incf, push, getf, etc) for particular types of objects, and setf for generalized place modification (eg, setf-second, setf-aref, setf-gethash, etc). But rather than inventing new specialized macros for other object types, or having to mentally consider the characteristics of each macro (slowing down development), it might be nice to have a generalized setf-like modification capability that was simpler to use than setf. For example, instead of (setf (second (getf plist indicator)) (1+ (second (getf plist indicator)))) or (incf (second (getf plist indicator))), one might write (callf (second (getf plist indicator)) #'1+), using any of the normal one argument functions (or lambda-expression) provided by common-lisp or the user. Here is an attempt at code:
(defun call (object function) (funcall function object))
(define-modify-macro callf (&rest args) call)
Will something like this work for all general cases, and can it actually simplify code in practice?
callf
I think what you are looking for is _f from OnLisp 12.4:
(defmacro _f (op place &rest args)
"Modify place using `op`, e.g., (incf a) == (_f a 1+)"
(multiple-value-bind (vars forms var set access)
(get-setf-expansion place)
`(let* (,#(mapcar #'list vars forms)
(,(car var) (,op ,access ,#args)))
,set)))
This uses get-setf-expansion - the workhorse of generalized reference handling.
Note that _f only works with single-value places.
IOW, (_f (values a b c) 1+) will not increment all 3 variables.
It is not all that hard to fix that though...
can it actually simplify code in practice?
This really depends on your coding style and the specific problem you are solving.
test.lisp:
(defvar test
#'(lambda (var1)
#'(lambda (var2)
`((var1 . ,var1)
(var2 . ,var2)))))
(defvar var1 'wrong)
(defvar var2 'wrong)
And in the REPL:
$ clisp -q -i test.lisp
;; Loading file test.lisp ...
;; Loaded file test.lisp
[1]> (funcall (funcall test 'right) 'right)
((VAR1 . WRONG) (VAR2 . RIGHT))
I thought that common lisp was supposed to be lexically scoped these days, so why is the value of var1 not captured by the inner lambda in test? How can I make sure it is captured?
This is visible when using an interpreter.
Let's look at a compiler first:
? (load "/tmp/test.lisp")
#P"/private/tmp/test.lisp"
? (funcall (funcall test 'right) 'right)
((VAR1 . RIGHT) (VAR2 . RIGHT))
First the functions are getting compiled. The compiler assumes lexical binding. Then DEFVAR declares the variables VAR1 and VAR2 to be special (-> not lexical). In the executed code then, the code is still using lexical binding.
You were using an Interpreter:
First the functions are loaded. Nothing gets compiled. Then DEFVAR declares VAR1 and VAR2 as special.
In the executed code then, the Interpreter is using dynamic binding - like you declared it. The Interpreter looks at the variables at runtime and sees that they are declared to be special.
Difference:
The compiler has generated the machine code before the special declaration. Thus at runtime it uses lexical binding.
The interpreter looks at runtime at the existing declarations.
Style
If you want to avoid dynamic binding, don't declare variables to be special.
defvar (and defparameter) declare the variable to be special (dynamic). Give your special variables *earmuffs* to prevent surprises about whether a binding is lexical or dynamic.
I'm having some trouble understanding the scope of variables and functions when defined inside a function call. I tried searching for this scope, but could not find a suitable answer (or maybe was searching for the wrong thing), so I decided to write a few functions to test things myself:
(defun test-scope1 ()
(setf myvar 1)
(defun set-var1 ()
(setf myvar 2))
(set-var1))
With this function I just wanted to see if anything gets set global. I would expect myvar and set-var to be defined globally because I have no scope here. As expected, before calling (test-scope1) the commands myvar and (set-var) give me errors. After calling (test-scope1), I can run myvar and (set-var) in the interpreter to yield 2.
Thinking to myself it would be nice to encapsulate my variables, I decided to add a let inside my function, therefore getting my next test:
(defun test-scope2 ()
(let ((myvar 10))
(defun set-var2 ()
(setf myvar 20))
(set-var2)))
I would expect myvar to be stuck in the scope of the let block, but wouldn't be able to guess for set-var2. It could be stuck in the let block or it could be defined globally. After running (test-scope2) I try to access myvar and get 2. That means this new function has its own myvar since it's still 2 from the previous function. I try running (set-var2)and get 20.
I'm not completely surprised that the function is defined globally after being run in the let block, but now I'm very confused what myvar variable its accessing. Since it didn't change my global copy of myvar, it would seem there's some variable floating around it still refers to.
Now I want to see if I can manipulate that floating variable, so I create this third function.
(defun test-scope3 ()
(let ((myvar (if (ignore-errors myvar)
myvar
100)))
(defun set-var3 ()
(setf myvar (+ myvar 100)))
(set-var3)))
Instead of just setting the variable to a fixed value, I want to increment it based on the previous value. I check two things here. The first is when test-scope3 is called. I wanted to see if I could pull up a "previous value" of myvar, since if it's floating around somewhere maybe I can access it again. It probably wouldn't be good practice, but that's not the point here. Ignore-errors is there in case there wasn't really a previous value floating around, in which case I choose a default of 100.
The second thing I test is to have set-var3 add 100 to the previous the value of myvar. I want to see if this will adjust that floating variable, or if somehow is something static. I have no idea what my function "should" return.
After running (test-scope3) I'm completely surprised to see 102. So apparently my test-scope3 found the value of myvar from running test-scope1. But, after checking the value of myvar in the interpreter, it is still 2. Next I run (set-var3) and get a return value of 202. Okay, so it added 100 again to my previous value. Calling it again returns 302 and so on. But, calling (test-scope3) again resets this value back down to 102.
I wrote one more function as a double nested let command. I just ran this in two ways: with no myvar definition for the let arguments. This function returns 10002. Then, I tried setting the local myvar to 50 and had 1050 return.
(defun test-scope4 ()
(let () ; or ((myvar 50))
(let ((myvar (if (ignore-errors myvar)
myvar
2000)))
(defun set-var4 ()
(setf myvar (+ myvar 1000)))
(set-var4))))
So, with all this together, here are some of my specific questions:
When I do a defun inside another defun, even in a let block, why does that function become accessible globally?
When I call set-var#, what variables (or within what scope) is this accessing? Or maybe more appropriately, when I define a function inside another statement, what am I actually binding?
When I use the variable myvarin a function, where does this pull from? From my 4th example conjecture is that it looks for the symbol myvar in the current scope, then checks up one level higher until it finds a value for it.
Sorry if everything was very wordy and my questions ill-formed. I tried to investigate things to the best of my ability. Really this all leads to my real question, which after writing everything up I realize might be beyond the scope (no pun intended) of this question as I've set it up so far.
The issue of hiding my inner functions as given above could be handled with a lambda expression; however, what I'd really love to do is to have a recursive function inside of a bigger block that uses the block to store values without feeding them directly into the function. To my knowledge, this is not possible with a lambda expression. For example, consider the following function which does this.
(defun outer-function (start)
(let ((x start))
(defun increment-to-ten ()
(setf x (+ x 1))
(if (< x 10)
(increment-to-ten)))
(increment-to-ten)
(print x)))
Which could be instead implemented recursively with arguments as
(defun increment-to-ten-recursive (x)
(if (< x 10)
(increment-to-ten-recursive (+ x 1))
10))
If there's a solution for this, that would be great, or if my thinking is completely wrong and there's a better way to do this that would be great to. It just seems convenient to have a block store data for you and then just call a recursive function with no arguments to work on that data.
Why it is difficult to discuss the effects you see:
You are doing things which are undefined in Common Lisp: setting undeclared variables in test-scope1: myvar. It's from then on unclear how the following code behaves.
Undeclared Variables
It is undefined what effect it has when an undeclared variable foo is set. Implementations allow it. Some will warn. SBCL:
* (setf foo 10)
; in: SETF FOO
; (SETF FOO 10)
; ==>
; (SETQ FOO 10)
;
; caught WARNING:
; undefined variable: FOO
;
; compilation unit finished
; Undefined variable:
; FOO
; caught 1 WARNING condition
Global variables are defined with DEFPARAMETER and DEFVAR. Local variables are defined by LET, LET* and by functions parameters. Since DEFPARAMETER and DEFVAR define global special (using dynamic binding) variables, it is common to write them as *some-variable* - note the * around it, which are part of the symbol name and not special syntax. It's a convention for special variables.
Nested DEFUNs
Nested DEFUNs are not used in Common Lisp. DEFUN is a top-level form, which sets a global function. To define local functions use FLET and LABELS. You can nest DEFUN forms, but it is bad style and you won't find it used in actual Lisp code. Don't nest DEFUN forms. Note: this is different from Scheme, where one can nest DEFINE forms. Since nested DEFUNs are not used, it makes little sense to discuss the effects. Convert the examples to use local functions defined by FLET or LABELS.
When I do a defun inside another defun, even in a let block, why does that function become accessible globally?
Because DEFUN defines global functions. That's its purpose.
You need to rewrite your examples to make it possible to discuss the effects:
declare all variables
don't use nested DEFUNs
We could try to understand the code you have in your current examples, but much depends on the implementation and the sequence of actions. We are either not in the realm of portable Common Lisp or doing stuff (nested DEFUNs), which has no practical use in Common Lisp.
As Rainer explained, the answer to your first question is that defun defines global functions. Use labels to define local functions.
Rainer is also right about much being dependent on the configuration of your lisp. Many Lisps know about two types of scope, lexical and dynamic. Lexical scope is the one you can see when you look at a page of code, and what you're used to from most other programming languages. Dynamic scope is what you get when you a lisp looks for the value of a variable in the environment where the function was called instead of the one where it was defined.
Finally, your code makes usse of closures. When you put a lambda (defun) inside a let, you create a closure, e.g. your variables "hang or float around" for the function to use in the future even when the function that contains them returns to it's caller.
So here's what might be going on (as Rainer said, it's hard to know since you're doing things that usually aren't done, such as nesting defuns):
So... probably, test-scope1 uses a global myvar and it's set-var references the global myvar since we are dealing with lexical scope. test-scope2 builds a closure, which creates a myvar with lexical scope only accessible to set-var2. Since this myvar has lexical scope, the call to set-var in test-scope2 will not use this myvar but still the global one.
I'm not sure why calling test-scope3 gets you 102, since test-scope3 just calls set-var, which should set the global myvar to 2, not 102. The only code that could explain 102 is in set-var3, and this isn't being called when you call test-scope3. Here's what might be going on if you got 2 instead of 102 when calling test-scope3:
(defun test-scope3 ()
(let ((myvar (if (ignore-errors myvar)
myvar
100)))
(defun set-var3 ()
(setf myvar (+ myvar 100)))
(set-var)))
With the let, you create a local binding for myvar with lexical scope, but you refer to the global myvar in it's binding form - the (if (ignore-errors myvar) myvar). Later on you call set-var, which should again reference the global myvar since test'scope3's let uses lexical scope for the binding of myvar. Then, wen you call set-var3 for the first time, it takes the myvar bound by the above let, which currently has a value of 2, and adds 100 to it. When you call it the second time, you again add 100 to the lexically bound myvar which still hangs around in the closure you built in test-scope3, and so on. But when you call test-scope3 again, you create a new closure (but replace the old one because you use defun to give the same name to it...) which again sets the initial value of the closed-over myvar to the value of the global myvar, which is 2.
I suggest you read Paul Graham's excellent ANSI Common Lisp and On Lisp, which explain special variables (dynamic scope), lexical scope and closures in depth.
Here's how you might have several closures share variables:
(let ((myvar 0))
(defun reset ()
(setf myvar 0)
(defun inc ()
(setf myvar (+ myvar 1))))
This builds two closures and makes them share myvar, and since myvar has lexical scope you can't access this copy of myvar from anywhere else. So you've basically given exclusive control of myvar to reset and inc.
This could be one way to build your recursive function that doesn't need a value passed into it with each iteration.
To implement a domain specific language, within lua,
I want to add barewords to the language.
So that
print("foo") could be written as print(foo)
The way I have done this is by changing the metatable of the enviroment table _G.
mt = {__index = function(tbl,key) return key end}
setmetatable(_G, mt)
And that works, because retrieving the value of variable foo is done by _G.foo which is equivalent to _G["foo"]
Is this a good approach?
Are there hidden downsides?
Is there a better way?
Can I do it so that barewords only work inside a certain file?
(Perhaps by executing that file, from another luascript, using loadstring)
As soon as someone declares a local with the same name of your "keywords" it will shadow your "keyword/global var" and your mechanism will fail:
print(foo) -- does what you think
local foo = 2
print(foo) -- argh!
and note that you cannot prevent the definition of local variables in a script.
Edit: (answering to a comment).
Are you using a customized Lua engine? You cannot prevent entering local scope, because you are always in local scope. It is exactly the other way around: technically there is no global scope in Lua (with the same meaning as in C, for example). There is a global namespace (implemented as a table), instead. The mechanisms for accessing globals differs between Lua 5.1 (function environments) and Lua 5.2 (implicit _ENV prefixing), but the concept is almost the same.
In particular, when a Lua script is loaded, whether by the interpreter, or by load, loadstring, dofile, etc., it is interpreted as the body of an anonymous function (a closure), usually referred to as the "main chunk". Thus there is always a local scope. In standard Lua you cannot prevent the definition of a local variable when the parser encounters a local statement in the script being loaded.
Z3 2.x had the feature (well, probably rather the bug) that symbol declarations were not popped away, e.g. the following code is accepted by Z3 2.x:
(push)
(declare-fun x () Int)
; Assumptions made in this scope will be popped correctly
(pop)
(assert (= x x))
Z3 3.x no longer accepts this code ("unknown constant").
Is there a way to restore the old behaviour? Or another way how one could declare symbols inside scopes such that the declaration (and only the declaration, not the assumptions) is global, i.e. not popped?
Yes, in Z3 2.x all declarations were global. We changed this behavior in Z3 3.x because the SMT-LIB 2.0 standard states that all declarations should be scoped.
You can restore the old behavior using the option :global-decls.
(set-option :global-decls true)
(push)
(declare-fun x () Int)
(pop)
(assert (= x x))