I'm using the race function from the async package, exported by Control.Concurrent.Async.
The subtasks I fire off using race themselves invoke runInteractiveProcess to run (non-Haskell) executables. The idea is to run different external programs and take the result of the first one to finish. In a sense, Haskell "orchestrates" a bunch of external programs.
What I'm observing is that while race works correctly by killing the Haskell level "slower" thread; the sub-processes spawned from the slow thread itself continue to run.
I suspect expecting race to kill processes spawned this way is a bit unrealistic, as they probably become zombies and get inherited by init. For my purposes however, keeping the external processes running defeats the whole purpose of using race in the first place.
Is there an alternative way of using race so the subprocesses created this way get killed as well? While I don't have a use case yet, it'd be best if the entire chain of processes created from the raced tasks get killed; as one can imagine those external programs themselves creating a bunch of processes as well.
As already mentioned in the comments, you could use a combination of onException and terminateProcess.
My process-streaming library (which contains helper functions built on top of process and pipes) already does this. Asynchronous exceptions trigger the termination of the external process.
For example, the following code does not create file toolate.txt.
import qualified Pipes.ByteString as B
import System.Process.Streaming
import Control.Concurrent.Async
main :: IO ()
main =
do
result <- race (runProgram prog1) (runProgram prog2)
putStrLn $ show $ result
where
-- collecting stdout and stderr as bytestrings
runProgram = simpleSafeExecute $ pipeoe $
separated (surely B.toLazyM) (surely B.toLazyM)
prog1 = shell "{ echo aaa ; sleep 2 ; }"
prog2 = shell "{ echo bbb ; sleep 7 ; touch toolate.txt ; }"
The result is:
Left (Right ("aaa\n",""))
Related
I have a function that spawns a process as follows:
(_, Just outh, _, ph) <- createProcess $
(proc "someproc" []) { std_out = CreatePipe }
line <- hGetLine outh
doSomeComputationOn line
-- ... 'outh' is not read anymore from here onwards
So "someproc" is created, and then the parent reads a line from it to obtain some information, and then it forgets about the pipe whose handle is outh.
Now the problem is that if the pipe is not read, "someproc" will block as soon as it is full. So this requires that the parent process reads outh even if it does not do anything with it. So my questions are:
Is this this a good way to get the first line of a child process' output and then forget about additional output?
Is there any way in Haskell in which I can automatically discard the input to the pipe (or even redirect it to a file)?.
So far the only way I can see to woraround this problem is spawning a new thread that constantly tries to read from outh (and just discards the output), which indicates that I'm doing something wrong ...
As additional background, this question is related to this one.
Now the problem is that if the pipe is not read, "someproc" will block as soon as it is full. [...] Is there any way in Haskell in which I can automatically discard the
input to the pipe (or even redirect it to a file)?
There is a helper library for process called process-streaming (written by the author of this answer) that tries to do just that: even if the user passes a stream-consuming function that doesn't exhaust a standard stream, it drains the stream automatically under the hood to avoid potential deadlocks.
The library doesn't work directly with handles, but accepts pipe-consuming functions and foldl folds through an adapter type.
An example of reading the first line:
import Data.Text.Lazy
import qualified Pipes.Prelude
import System.Process.Streaming (CreateProcess,shell,
piped,execute,foldOut,transduce1,withCont)
import System.Process.Streaming.Text (utf8x,foldedLines)
program :: CreateProcess
program = piped $ shell "{ echo aaa ; echo bbbb ; }"
firstLine :: IO (Maybe Text)
firstLine = execute program streams
where
streams = foldOut
. transduce1 utf8x
. transduce1 foldedLines
$ withCont Pipes.Prelude.head
The library has a much bigger dependency footprint than process though.
The alternative to use depends on the behavior of the external command.
If you simply want to interrupt that, you can hClose outh. This will close the pipe, and further writes to the pipe by the external command will fail with a "broken pipe" error. Most processes terminate upon receiving this.
If you instead want to read and discard the output, you can do that as well. Perhaps the easiest way is
do c <- hGetContents outh
evaluate (length c) -- force this to fetch all data
doStuff -- now we are sure that the remote end closed its output
which should run in constant space.
If you don't want to wait for the process to end before performing doStuff, wrap everything in forkIO.
How could I watch several files/sockets from Haskell and wait for these to become readable/writable?
Is there anything like the select/epoll/... in Haskell? Or I am forced to spawn one thread per file/socket and always use the blocking resource from within that thread?
The question is wrong: you aren't forced to spawn one thread per file/socket and use blocking calls, you get to spawn one thread per file/socket and use blocking calls. This is the cleanest solution (in any language); the only reason to avoid it in other languages is that it's a bit inefficient there. GHC's threads are cheap enough, however, that it is not inefficient in Haskell. (Additionally, behind the scenes, GHC's IO manager uses an epoll-alike to wake up threads as appropriate.)
There's a wrapper for select(2): https://hackage.haskell.org/package/select
Example usage here: https://github.com/pxqr/udev/blob/master/examples/monitor.hs#L36
There's a wrapper for poll(2):
https://hackage.haskell.org/package/poll
GHC base comes with functionality that wraps epoll on Linux (and equivalent on other platforms) in the GHC.Event module.
Example usage:
import GHC.Event
import Data.Maybe (fromMaybe)
import Control.Concurrent (threadDelay)
main = do
fd <- getSomeFileDescriptorOfInterest
mgr <- fromMaybe (error "Must be compiled with -threaded") <$> getSystemEventManager
registerFd mgr (\fdkey event -> print event) fd evtRead OneShot
threadDelay 100000000
More documentation at http://hackage.haskell.org/package/base-4.11.1.0/docs/GHC-Event.html
Example use of an older version of the lib at https://wiki.haskell.org/Simple_Servers#Epoll-based_event_callbacks
Though, the loop in that example has since been moved to the hidden module GHC.Event.Manager, and is not exported publicly as far as I can tell. GHC.Event itself says "This module should be considered GHC internal."
In Control.Concurrent there's threadWaitRead and threadWaitWrite.
So, to translate the above epoll example:
import Control.Concurrent (threadWaitRead)
main = do
fd <- getSomeFileDescriptorOfInterest
threadWaitRead fd
putStrLn "Got a read ready event"
You can wrap the threadWaitRead and subsequent IO action in Control.Monad.forever to run them repeatedly. You can also wrap the thing in forkIO to run it in the background while your program does something else.
I've got a program which uses several threads. As I understand it, when thread 0 exits, the entire program exits, regardless of any other threads which might still be running.
The thing is, these other threads may have files open. Naturally, this is wrapped in exception-handling code which cleanly closes the files in case of a problem. That also means that if I use killThread (which is implemented via throwTo), the file should also be closed before the thread exits.
My question is, if I just let thread 0 exit, without attempting to stop the other threads, will all the various file handles be closed nicely? Does any buffered output get flushed?
In short, can I just exit, or do I need to manually kill threads first?
You can use Control.Concurrent.MVar to achieve this. An MVar is essentially a flag which is either ''empty'' or "full". A thread can try to read an MVar and if it is empty it blocks the thread. Wherever you have a thread which performs file IO, create an MVar for it, and pass it that MVar as an argument. Put all the MVars you create into a list:
main = do
let mvars = sequence (replicate num_of_child_threads newEmptyMVar)
returnVals <- sequence (zipWith (\m f -> f m)
mvars
(list_of_child_threads :: [MVar -> IO a]))
Once a child thread has finished all file operations that you are worried about, write to the MVar. Instead of writing killThread you can do
mapM_ takeMVar mvars >> killThread
and where-ever your thread would exit otherwise, just take all the MVars.
See the documentation on GHC concurrency for more details.
From my testing, I have discovered a few things:
exitFailure and friends only work in thread 0. (The documentation actually says so, if you go to the trouble of reading it. These functions just throw exceptions, which are silently ignored in other threads.)
If an exception kills your thread, or your whole program, any open handles are not flushed. This is excruciatingly annoying when you're desperately trying to figure out exactly where your program crashed!
So it appears it if you want your stuff flushed before the program exits, then you have to implement this. Just letting thread 0 die doesn't flush stuff, doesn't throw any exception, just silently terminates all threads without running exception handlers.
I am currently writing a Haskell program that does some initialization work and then calls ncmpcpp. What I am trying to do is start ncmpcpp and terminate the Haskell program, so that only ncmpcpp is left (optionally, the program can keep running in the background, as long as it's unintrusive)
However, even though I am able to start ncmpcpp, I cannot interact with it. I see its output, but input appears to be impossible.
What I am currently doing is:
import System.Process (createProcess, proc)
...
spawnCurses :: [String] -> IO ()
spawnCurses params = do
_ <- createProcess (proc "ncmpcpp" params)
return ()
What am I doing wrong/What should I do differently?
What you are trying to achieve sounds like what the exec family of functions does. Take a look at the executeFile function.
If you want your parent Haskell process to be still running after the child process is started read about fork-exec and the forkProcess function.
A complete example of using forkProcess together with executeFile can be found at http://therning.org/magnus/archives/727.
Many times it happens that while I try to compile an .hs file, the Emacs 23 Haskell mode goes into an unending process and does not show any response. The only thing that happens is that the cursor turns into a circle and within the circle the dots start circling.
What is the best approach to avoid this situation? If I get into this situation, how can I get out? At present, I restart my computer. Please guide.
Recently, it happened for the following code:
import Control.Monad
import Data.Char
main = do
putStrLn "This is an addition"
contents <- getContents
putStrLn $ (shortLinesOnly contents)
shortLinesOnly :: String -> String
shortLinesOnly input =
allLines = lines input
shortLines = filter (\line -> length line < 10) allLines
res = unlines shortLines
in result
Maybe it would make sense to find the function that does the compilation and wrap it into something like:
(with-local-quit
(with-timeout (60 (error "The process didn't finish in 60 seconds"))
;; do the calls to the compiler here
))
If this has not been done already. If this has been done, then, as illusionoflife suggested, you should be able to interrupt the process with C-g.
Sometimes, however, Emacs may become too slow or too unresponsive (the system may give very high priority to the process it launched, so that you will have to wait very long until the control will return to Emacs from a blocking process with high priority). Still, this is not yet the reason to restart the computer. On Linux, you are probably running in runlevel 3, in which case you could do Ctrl+Alt+1 or any number less then the shell running X-server, then do
$ ps aux | grep "my process name"
this will print you some details about the process, starting with its id, given the id you can do:
$ kill id
Alternatively, you could just kill all instances of emacs and all processes spawned from it:
$ killall emacs
Use C-g.
You can use it to stop any sync process, internal or external.