Haskell createProcess and read from Handle - haskell

when i use Haskell createProcess do i need to fork before as if i was using exec in c?
From the example's i have seen and for what i have tried i don't think i do but if i read from the output Handle once i get the expected result but if i try to read twice it doesn't read even once.
For example:
beginProcess is the same as createProcess and z3 is a smt solver that reads from stdin and writes to stdout.
execute :: Process -> String -> IO String
execute (Just std_in, Just std_out,_,_) cmd = do
hPutStr std_in cmd
hFlush std_in
hGetLine std_out
main :: IO()
main = do
proc <- beginProcess "z3" ["-smt2","-in"]
execute proc "(set-option :print-success true)" >>= print
execute proc "(set-option :print-success true)" >>= print
If i do one execute i get the expected result, but if i do both nothing happens.
Am i reading or writing wrong to the handles ?
Thank you for any help you can give.

I have a hunch that you are "suffering from buffering".
Note that you are not emitting any newlines to the z3 process. Also, z3 may not be flushing its output after every command.
The best way to interact with a terminal program is through a pseudo tty. Here is an explanation of how such a set up would work: http://www.rkoucha.fr/tech_corner/pty_pdip.html

Related

How can I fork standard output?

I have a program that I want to run and wait for it to initialize before proceeding, but I also want the program to write to the terminal as usual, which it would do if I simply called it with callProcess. To this end, I have devised the following hack:
(In this case I listen to the standard error, but I suppose this is inessential.)
…
withCreateProcess applicationProcessDescription \ _ _ maybeApplicationStdErr _ -> do
case maybeApplicationStdErr of
Nothing -> throwIO $ DeploymentException $ "Unable to grab application's standard error stream!"
Just applicationStdErr -> do
maybeTimeout <- timeout waitingTime (waitForApplication applicationStdErr)
when (isNothing maybeTimeout) $ throwIO $ DeploymentException $ "Application took too long initializing!"
redirectHandle applicationStdErr stderr
…
waitForApplication :: Handle -> IO ( )
waitForApplication stdErr = do
line <- hGetLine stdErr
hPutStrLn stderr line
if line == "Application initialized."
then return ( )
else waitForApplication stdErr
redirectHandle :: Handle -> Handle -> IO ( )
redirectHandle from to = void $ forkFinally
do forever do
line <- hGetLine from
hPutStrLn to line
\ _ -> return ( )
I intercept whatever the subprocess is saying, line by line, analyze the lines in some way and put them to the main process's standard error. When I am done, I still have to keep listening and relaying. This seems like an absurd way of going about it.
What I would like to do is fork the subprocess's standard error stream in some way, so that I may eavesdrop on it passively while I need to, and then simply disconnect. It is important for me that whatever the subprocess is saying was delivered to the human as faithfully as possible — without delay and omission.
Is there a nice way to do it?

typed-process withProcessWait_ and getExitCode race condition?

import System.Process.Typed
import Control.Monad.STM
import Control.Concurrent.STM.TVar
processConf = setStderr byteStringOutput . setStdout byteStringOutput
main :: IO ()
main = do
withProcessWait_ (processConf $ proc "sleep" ["1"])
$ \p -> do
atomically (getStdout p) >>= print
atomically (getStderr p) >>= print
getExitCode p >>= print
print "test"
The above code mostly returns Nothing for the exit code, while other times it'll return Just ExitSuccess, so seemingly random / race condition.
Why might this occur?
The function in question is: http://hackage.haskell.org/package/typed-process-0.2.6.0/docs/System-Process-Typed.html#v:withProcessWait_
withProcessWait_ :: MonadUnliftIO m => ProcessConfig stdin stdout stderr -> (Process stdin stdout stderr -> m a) -> m a
My understanding of what the function will do is, run the process, wait till it's terminated and then run the IO a. However... I just noticed there is a readProcess function which sounds like what I should actually be using instead http://hackage.haskell.org/package/typed-process-0.2.6.0/docs/System-Process-Typed.html#v:readProcess
None the less, it would be useful to know what is actually happening in the above / withProcessWait_.
The race condition is that three separate threads are consuming all available standard output (first thread) and standard error (second thread) and waiting on the exit code (third thread). The standard output and standard error thread can complete and publish the full output to STM before the third thread publishes the exit code.
Since getExitCode checks the exit code TMVar but doesn't wait on it, the exit code may not be available. (In fact, if the child process closes standard output and error before exiting, it may genuinely still be running at this point!)
If you replace getExitCode with waitExitCode, you reliably get ExitSuccess. Of course, withProcessWait_ is already waiting on the exit code and will raise an exception if it's not ExitSuccess, so there's no particular reason to do this anyway.

How does hWaitForInput work?

This bit of code gets characters from stdin and converts them to a String. There is a timeout on the input implemented with hWaitForInput. I have included the tryIOError function to capture isEOFError. It times out correctly when the program is run and no input has taken place ie.it is waiting for input. However once a character has been entered it no longer times out. No matter what the input wait time is.
-- get string from stdin
getFrmStdIn :: Int -> IO String
getFrmStdIn timeout = do
inAvail <- tryIOError $ hWaitForInput stdin timeout
let tOut = either (const False) id inAvail
if isLeft inAvail
then return []
else
if not tOut
then die "00 timed out"
else
(:) <$> hGetChar stdin <*> getFrmStdIn timeout
What OS and GHC version are you using?
AFAIK with the resent GHC versions hWaitForInput doesn't work on linux at all when timeout is not 0. On windows the whole IO subsystem is "a bit lacking" too.
However once a character has been entered it no longer times out. No matter what the input wait time is.
That's how it's supposed to work according to the docs:
Computation hWaitForInput hdl t waits until input is available on handle hdl. It returns True as soon as input is available on hdl, or False if no input is available within t milliseconds.
Here's a usage example of hWaitForInput where it will wait for five seconds and then check if there are characters in stdin. If not, it will say "Timeout reached". Otherwise it'll wait for the user to hit enter (hGetLine checks for the end of line) and print the user-entered string:
main :: IO ()
main =
hWaitForInput stdin 5000 >>= \inputIsAvailable ->
if inputIsAvailable
then hGetLine stdin >>= \input -> putStrLn $ "There is input: " ++ input
else putStrLn "Timeout reached"

Haskell: Why does this function keep asking for user input and not terminating

I'm learning some Haskell and I came across this small program
reverseLines :: String -> String
reverseLines input =
unlines (map reverse (lines input))
main :: IO ()
main = interact reverseLines
This program will keep asking the user for more input and reverse the input and print it on the screen.
Most of this is straight forward but one thing I can't wrap my head around is why does this function keeps running and ask the user for more input whereas if I just replace the reverseLines function with a function the simply returns some string it will not happen.
This program will stop after one execution:
foo input = "Stops"
main :: IO ()
main = interact foo
Why?
If you look at the source of interact you see this:
interact f = do s <- getContents
putStr (f s)
see the getContents? This is where the magic starts - it will read everything till EOF
Now in Haskell this is lazy-IO which can be bad but here is almost magical - see the string is read lazily and passed to your reverseLines - this one of course will only generate output as soon as it saw \n characters (the lines) and so it seems your program is some kind of REPL.
In the second one you don't consume any of the lazy-string at all so it stops ASAP
As I wrote in the comments you can play with this by either passing content into the program using a file (or echo) and pipes on the terminal:
echo "Hello World\nBye Bye" | runhaskell LazyIO.hs
or using CTRL-D to pass in the EOF yourself.
To get a feeling for it I would play with the functions more - what happens if you use something that needs to see the complete input first (try reverse without the maps)? What happens with words instead of lines, ...?
Have fun!

How can I make file I/O more transactional?

I'm writing CGI scripts in Haskell. When the user hits ‘submit’, a Haskell program runs on the server, updating (i.e. reading in, processing, overwriting) a status file. Reading then overwriting sometimes causes issues with lazy IO, as we may be able to generate a large output prefix before we've finished reading the input. Worse, users sometimes bounce on the submit button and two instances of the process run concurrently, fighting over the same file!
What's a good way to implement
transactionalUpdate :: FilePath -> (String -> String) -> IO ()
where the function (‘update’) computes the new file contents from the old file contents? It is not safe to presume that ‘update’ is strict, but it may be presumed that it is total (robustness to partial update functions is a bonus). Transactions may be attempted concurrently, but no transaction should be able to update if the file has been written by anyone else since it was read. It's ok for a transaction to abort in case of competition for file access. We may assume a source of systemwide-unique temporary filenames.
My current attempt writes to a temporary file, then uses a system copy command to overwrite. That seems to deal with the lazy IO problems, but it doesn't strike me as safe from races. Is there a tried and tested formula that we could just bottle?
The most idiomatic unixy way to do this is with flock:
http://hackage.haskell.org/package/flock
http://swoolley.org/man.cgi/2/flock
Here is a rough first cut that relies on the atomicity of the underlying mkdir. It seems to fulfill the specification, but I'm not sure how robust or fast it is:
import Control.DeepSeq
import Control.Exception
import System.Directory
import System.IO
transactionalUpdate :: FilePath -> (String -> String) -> IO ()
transactionalUpdate file upd = bracket acquire release update
where
acquire = do
let lockName = file ++ ".lock"
createDirectory lockName
return lockName
release = removeDirectory
update _ = nonTransactionalUpdate file upd
nonTransactionalUpdate :: FilePath -> (String -> String) -> IO ()
nonTransactionalUpdate file upd = do
h <- openFile file ReadMode
s <- upd `fmap` hGetContents h
s `deepseq` hClose h
h <- openFile file WriteMode
hPutStr h s
hClose h
I tested this by adding the following main and throwing a threadDelay in the middle of nonTransactionalUpdate:
main = do
[n] <- getArgs
transactionalUpdate "foo.txt" ((show n ++ "\n") ++)
putStrLn $ "successfully updated " ++ show n
Then I compiled and ran a bunch of instances with this script:
#!/bin/bash
rm foo.txt
touch foo.txt
for i in {1..50}
do
./SO $i &
done
A process that printed a successful update message if and only if the corresponding number was in foo.txt; all the others printed the expected SO: foo.txt.notveryunique: createDirectory: already exists (File exists).
Update: You actually do not want to use unique names here; it must be a consistent name across the competing processes. I've updated the code accordingly.

Resources