I have a Common Lisp program that behaves differently depending on how I use *standard-input*. Here are the details:
(if input-stream?
(process)
(with-open-file (*standard-input* up :element-type 'unsigned-byte)
(process)))
The process function starts multiple threads. Each thread reads part of the standard input, writes it in a file (all within a lock) and processes the resulting files in parallel (out of the lock). In fact it processes the resulting files in parallel only in case input-stream? is false otherwise it processes them sequentially.
(defun process ()
(let ((psize 4194304)
(stream *standard-input*)
(result-lock (bt:make-lock))
(input-stream-lock (bt:make-lock))
eof)
(flet ((add-job (fname)
(make-thread
#'(lambda ()
(do () (eof)
(when (bt:with-lock-held (input-stream-lock)
(unless eof
(setq eof (write-input-stream-to-file stream fname psize))
t))
(sleep 0.1)
(bt:with-lock-held (result-lock)
(display-progress))))))))
(mapcar
#'join-thread
(loop for i from 1 to 10
collect (add-job
(make-pathname :directory "/tmp"
:name "test"
:type (princ-to-string i))))))))
(let ((counter 0))
(defun display-progress ()
(if (zerop (mod (incf counter) 10))
(format t " ~a " counter)
(write-char #\+))))
(defun write-input-stream-to-file (stream fname psize-bytes)
(with-open-file (out fname
:direction :output
:element-type 'unsigned-byte
:if-exists :supersede)
(do ((byte (read-byte stream nil nil)
(read-byte stream nil nil))
(offset 0 (1+ offset)))
((or (= offset psize-bytes) (null byte)) (not byte))
(write-byte byte out))))
If we create a FIFO (with mkfifo), copy the file to it and run the program with it instead, we again observe parallelism.
The above program is built as a command line utility with ECL and runs on Linux. I run it in one of the following ways:
cat "bigfile" | my-program
my-program "bigfile"
Parallelism happens only in case 2.
The question is why the difference?
Update:
I had mistake in my question. Now it is OK.
Added the process function and described how I run the program
Related
How do I detect if a lexical variable is bound in a scope? I basically want boundp for lexical variables.
Concretely, say I have:
(defvar *dynamic* 1)
(defconstant +constant+ 2)
(let ((lexical 3))
(when (boundp '*dynamic*) ; t
(print "*dynamic* bound."))
(when (boundp '+constant+) ; t
(print "+constant+ bound."))
(when (boundp 'lexical) ; nil
(print "lexical bound.")))
So boundp correctly checks for dynamic variables (and constants), and as the hyperspec says, doesn't cover lexical bindings.
But I can't find any equivalent of boundp for lexical bindings. So how do I check them then? (Implementation-specific code for say SBCL is fine if there isn't anything portable.)
There is nothing like that in ANSI Common Lisp. There is no access to a lexical environment.
You only can check it this way:
CL-USER 8 > (let ((lexical 3))
(when (ignore-errors lexical)
(print "lexical bound."))
(values))
"lexical bound."
CL-USER 9 > (let ((lexical 3))
(when (ignore-errors lexxxical)
(print "lexical bound."))
(values))
<nothing>
There is no way to take a name and see if it is lexically bound at all. There is an extension to CL, where the function variable-information would give some information, but even in this case it would probably not work:
* (require "sb-cltl2")
("SB-CLTL2")
* (apropos "variable-information")
VARIABLE-INFORMATION
SB-CLTL2:VARIABLE-INFORMATION (fbound)
* (let ((lexical 3))
(sb-cltl2:variable-information 'lexical))
; in: LET ((LEXICAL 3))
; (LET ((LEXICAL 3))
; (SB-CLTL2:VARIABLE-INFORMATION 'LEXICAL))
;
; caught STYLE-WARNING:
; The variable LEXICAL is defined but never used.
;
; compilation unit finished
; caught 1 STYLE-WARNING condition
NIL
NIL
NIL
for cltl2:variable-information to work, it should be done in macro expansion time.
(ql:quickload :introspect-environment)
(use-package :introspect-environment) ;; also exports cltl2 functions.
(defmacro in-compile-time ((environment) &body body &environment env)
(check-type environment symbol)
(eval `(let ((,environment ,env)) (progn ,#body)))
nil) ; does not affect the expansion
(defun fn ()
(let ((lexical 2))
(in-compile-time (env)
(print (introspect-environment:variable-information 'lexical env))
(print (introspect-environment:variable-information 'lexxxxical env)))))
; compiling (DEFUN FN ...)
:LEXICAL
NIL
I have some common lisp code that is behaving oddly. It's a TCP client/server application.
Unless I add (sleep 0.01) or similar at the end of my code, I get the following utterly unhelpful error message after my program completes. It is a very short-lived program, simply hosting a TCP server and testing that it can be connected to.
;; compilation unit aborted; caught 1 fatal ERROR condition
This doesn't happen every time, maybe 80% of the runs cause this. There is no context, no explanation.
Code to reproduce the problem:
(defmacro with-gensyms ((&rest names) &body body)
`(let ,(loop for n in names collect `(,n (gensym)))
,#body))
(defmacro kilobytes (qty)
(* qty 1024))
(defun is-sequence (sequence)
(or (listp sequence) (vectorp sequence)))
(defmacro append-to (seq values)
(with-gensyms (cached-values)
`(let ((,cached-values ,values))
(cond
((is-sequence ,cached-values)
(setf ,seq (append ,seq (coerce ,cached-values 'list))))
(t
(setf ,seq (append ,seq (list ,cached-values))))))))
(defmacro remove-from (seq value)
(with-gensyms (cached-value)
`(let ((,cached-value ,value))
(delete-if (lambda (value) (equalp value ,cached-value)) ,seq))))
(defclass tcp-server ()
((server-socket :initform nil)
(server-threads :initform (list))))
(defgeneric start-server (this &key port bind-address buffer-length))
(defmethod start-server ((this tcp-server) &key (port 0) (bind-address #(127 0 0 1)) (buffer-length (kilobytes 10)))
(with-slots (server-socket server-threads) this
(when server-socket
(error "Server already running"))
(let ((backlog 5))
(setf server-socket (make-instance 'sb-bsd-sockets:inet-socket :type :stream :protocol :tcp))
(sb-bsd-sockets:socket-bind server-socket bind-address port)
(sb-bsd-sockets:socket-listen server-socket backlog)
(flet ((handle-connection (client-socket)
(let ((buffer (make-array buffer-length :element-type '(unsigned-byte 8) :fill-pointer t)))
(catch 'eof
(loop
while (sb-bsd-sockets:socket-open-p client-socket)
do (let ((length (nth-value 1 (sb-bsd-sockets:socket-receive client-socket buffer nil))))
(when (eq 0 length)
(throw 'eof nil)))))
(sb-bsd-sockets::socket-close client-socket)
(remove-from server-threads sb-thread:*current-thread*))))
(sb-thread:make-thread
(lambda ()
(loop
while (and server-socket (sb-bsd-sockets:socket-open-p server-socket))
do
(let ((client-socket (sb-bsd-sockets:socket-accept server-socket))) ;; Listen for incoming connections
(append-to server-threads
(sb-thread:make-thread #'handle-connection :name "Connection handler" :arguments client-socket)))) ;; Spawn a process to handle the connection))
(remove-from server-threads sb-thread:*current-thread*))
:name "Server")))
nil))
(defun start-tcp-server (&key (port 0) (bind-address #(127 0 0 1)) (buffer-length (kilobytes 10)))
(let ((server (make-instance 'tcp-server)))
(start-server server :port port :bind-address bind-address :buffer-length buffer-length)
server))
(defgeneric stop-server (this))
(defmethod stop-server ((this tcp-server))
(with-slots (server-socket server-threads) this
(unless server-socket
(error "Server not running"))
(sb-bsd-sockets:socket-close server-socket)
(setf server-socket nil)
(loop for thread in (reverse server-threads)
; do (sb-thread:interrupt-thread thread 'sb-thread:abort-thread))
do (sb-thread:terminate-thread thread))
(loop for thread in (reverse server-threads)
do (sb-thread:join-thread thread :default nil))))
(defgeneric server-running? (this))
(defmethod server-running? ((this tcp-server))
(if (slot-value this 'server-socket) t nil))
(defgeneric server-port (this))
(defmethod server-port ((this tcp-server))
(nth-value 1 (sb-bsd-sockets:socket-name (slot-value this 'server-socket))))
(let ((server-instance nil))
(defun deltabackup-start-server (&key (port 0) (bind-address #(127 0 0 1)) (buffer-length (kilobytes 10)))
(setf server-instance (start-tcp-server :port port
:bind-address bind-address
:buffer-length buffer-length))
nil)
(defun deltabackup-stop-server ()
(unless server-instance
(error "Server not running"))
(stop-server server-instance)
(setf server-instance nil))
(defun deltabackup-server-running? ()
(server-running? server-instance))
(defun deltabackup-server-port ()
(server-port server-instance)))
(defmacro with-tcp-client-connection (address port socket-var &body body-forms)
(with-gensyms (client-socket)
`(let* ((,client-socket (make-instance 'sb-bsd-sockets:inet-socket :type :stream :protocol :tcp))
(,socket-var ,client-socket)) ; duplicate this, to prevent body-form modifying the original
(sb-bsd-sockets:socket-connect ,client-socket ,address ,port)
(unless ,client-socket
(error "Failed to connect"))
,#body-forms
(sb-bsd-sockets:socket-close ,client-socket))))
(defmacro with-running-server ( (&optional (port 0)) &body body-forms)
`(progn
(deltabackup-start-server :port ,port)
(unless (deltabackup-server-running?)
(error "Server did not run"))
,#body-forms
(deltabackup-stop-server)))
(with-running-server ()
(with-tcp-client-connection #(127 0 0 1) (deltabackup-server-port) client-socket
client-socket))
Using SBCL common lisp.
The reason you get such a vague error message is that you're running directly from the command line and the error is happening in a thread. If you can get the error to happen in EMACS under SLIME, you'll get more detailed error information.
I ran your program from SLIME like this:
CL-USER> (loop repeat 100 do (load "/tmp/stackoverflow.lisp"))
...and got the following error in SLIME:
Socket error in "accept": EBADF (Bad file descriptor)
[Condition of type SB-BSD-SOCKETS:BAD-FILE-DESCRIPTOR-ERROR]
So, something is going wrong on the server's end when you try to accept the connection. My theory is you have a race condition. In the server thread, you have this:
(loop
while (and server-socket (sb-bsd-sockets:socket-open-p server-socket))
do
(let ((client-socket (sb-bsd-sockets:socket-accept server-socket))) ;; Listen for incoming connections
....))
...and in the client thread, you do this:
(defmethod stop-server ((this tcp-server))
(with-slots (server-socket server-threads) this
(unless server-socket
(error "Server not running"))
(sb-bsd-sockets:socket-close server-socket)
It is possible for sb-bsd-sockets:socket-close to be called and to finish between the call to socket-open-p and socket-accept in the server thread, so that socket-accept gets called on a closed socket.
I wrote this code after reading the rsound documentation in Racket:
#lang racket
(provide (all-defined-out))
(require rsound)
(define (sunet)
(sine-wave 880))
(signal-play sunet)
(stop)
I used the "sine-wave" function on a 880 Hz frequency and stored it into the function "sunet". Then I applied on it the function "signal-play" that plays signals as sound. I got this error:
application: not a procedure;
expected a procedure that can be applied to arguments
given: #<network/s>
arguments.:
880
>
Isn't "sine-wave" a function and isn't it taking frequency as an argument?
I read more of the documentation of rsound and read about networks. Apparently you need to create a network with the input signal and the output signal that gets out to something like signal-play.
This would be the code that I needed yesterday:
(define sunet
(network ()
[A5 <= sine-wave 880]
[out = A5]))
(signal-play sunet)
(sleep 3)
(stop)
And with two signals to create a powerchord:
(define sunet
(network ()
[E2 <= sine-wave 82]
[B2 <= sine-wave 123]
[out = (+ E2 B2)]))
(signal-play sunet)
(sleep 3)
(stop)
So, I just answered my own question.
This was harder than I expected. Turns out sine-wave is not a function but a network. The following will play a sine wave.
#lang racket
(require rsound)
(define s (network ()
[a <= sine-wave 880]
[out = a]))
(signal-play s)
I do not know why the following simpler program does not work:
#lang racket
(require rsound)
(define s (network ()
[out = sine-wave 880]))
(signal-play s)
(defvar firstnumber)
(defvar secondnumber)
(defvar length1)
(defvar mulb)
(defvar var1)
(defvar num1)
(defvar num2)
(defvar addition)
(defvar subs)
(defvar muls)
(defvar divs)
(defun acceptnumber()
(write-line "Enter the first number:")
(force-output t)
(setf firstnumber(read))
(write-line "Enter the second number:")
(force-output t)
(setf secondnumber(read))
(setf length1(integer-length secondnumber))
)
(defun add(num1 num2)
(setf addition(+ num1 num2))
(write-line "Addition of two numbers:")
(write addition)
(write-line " " )
)
(defun sub(num1 num2)
(setf subs(- num1 num2))
(write-line "Substraction of two numbers:")
(write subs)
(write-line " ")
)
(defun mul(num1 num2)
(setf muls(* num1 num2))
(write-line "Multiplication of two numbers:")
(write muls)
(write-line " ")
)
(defun div(num1 num2)
(setf divs(/ num1 num2))
(write-line "Division of two numbers:")
(write divs)
(write-line " ")
)
(defun binarymul(num1 num2)
(setf mulb 0)
(
loop for i from 1 to length1 do
(
if(= 1 (logand num2 1))
(setf mulb(+ mulb num1))
)
(setf num2(ash num2 -1))
(setf num1(ash num1 1))
)
(write-line "Binary Multiplication ::")
(write mulb)
(write-line " ")
(write-line "In binary form::")
(setf var1(format nil "~B" mulb ))
(write-line var1)
)
(acceptnumber)
(sb-thread:make-thread(lambda()(binarymul firstnumber secondnumber)))
(sb-thread:make-thread(lambda()(add firstnumber secondnumber)))
(sb-thread:make-thread(lambda()(sub firstnumber secondnumber)))
(sb-thread:make-thread(lambda()(mul firstnumber secondnumber)))
(sb-thread:make-thread(lambda()(div firstnumber secondnumber)))
I started learning with LISP concurrent programming. I wrote a simple calculation program with sb-thread. Now i want to set the priority for each thread. Ex like set priority higher ti binarymul function etc.
Can I set priority of sb-thread ? If yes How?
SBCL's manual says (emphasis added):
12 Threading
SBCL supports a fairly low-level threading interface that maps onto
the host operating system’s concept of threads or lightweight
processes. This means that threads may take advantage of hardware
multiprocessing on machines that have more than one CPU, but it does
not allow Lisp control of the scheduler. This is found in the
SB-THREAD package.
I think that thread priority would typically require some access to the scheduler, so you won't have the ability to set thread priority with SBCL.
I have been looking for a solution to execute several task (atleast 2) simultaneously. I found something like coroutines in lua. Can anybody please clarify me in detail how to handle 2 or more than one task? Basically what I am trying to do is to run execute some event and measure the memory consumption for that process using lua script. Any quick solution or ideas would be highly appreciated.
Thanks
Take a look a io.popen(prog [, mode]).
From the documentation for io.popen, it "starts prog in an other process".
Here is how I would implement this:
-- launch the prog you want to measure. I assume it does not quit immediately.
-- lua will not block here: it does not wait for prog to exit.
io.popen('script-you-want-to-measure')
-- launch the second process to measure the first.
local measure = io.popen('measuring-process', 'r')
-- read all will block until the measure program exits
local measure_output = measure:read('*a')
f:close()
-- do what you want with the output
I don't know if this can help:
-- task-like example --
local tasks = { } -- queue
local task = coroutine.wrap -- alias
local function suspend ( )
return coroutine.yield (true)
end
local function shift (xs)
return table.remove (xs, 1) -- removes the first element
end
local function push (xs, x)
return table.insert (xs, x) -- inserts in the last position
end
local function enqueue (f)
return push (tasks, task (f))
end
-- begin to run the tasks --
local function start ( )
local curr
while #tasks > 0 do -- while the tasks queue isn't empty
curr = shift (tasks)
if curr ( ) then push (tasks, curr) end -- enqueue the task if still alive
end
end
-- create a task and begin to run the tasks --
local function spawn (f)
local curr = task (f) --
if curr ( ) then push (tasks, curr) end
return start ( )
end
-- adds task to tasks queue --
enqueue (function ( )
for i = 1, 3 do
print ("Ping. <<", i)
-- os.execute "sleep 1" -- uncomment if you're using *nix
suspend( )
end
end)
-- yet another task --
enqueue (function ( )
for i = 1, 5 do
print "..."
suspend( )
end
end)
-- begins to run the tasks --
spawn (function ( )
for i = 1, 5 do
print ("Pong. >>", i)
-- os.execute "sleep 1" -- uncomment if you're using *nix
suspend( )
end
end)
-- end of script --