Suppose I have this code:
import qualified Data.ByteString as B
main = do
... --some code that gets an array of ByteStrings called "files"
putStrLn "Saving myFile1"
B.writeFile "myFile1.bin" $ files!!0
putStrLn "Saving myFile2"
B.writeFile "myFile2.bin" $ files!!1
putStrLn "Done!"
Instead of printing out each string before the subsequent file writing, it just seems to print out everything at once after the files have been saved. This kind of functionality is inconvenient for notifying the user of the progress, since they'll only get these messages once all the file writing is done.
My code is obviously not this bad (I'm using a list of functions with sequence) but this illustrates the problem well. Is there anything I'm doing wrong or is this there no way around this?
Like #WillemVanOnsem stated, this is a buffering issue. You can manually flush the buffer after each print or you can change the buffering mode entirely.
import System.IO
import qualified Data.ByteString as B
main = do
... --some code that gets an array of ByteStrings called "files"
putStrLn "Saving myFile1"
hFlush stdout
B.writeFile "myFile1.bin" $ files!!0
putStrLn "Saving myFile2"
hFlush stdout
B.writeFile "myFile2.bin" $ files!!1
putStrLn "Done!"
hFlush stdout
or
import System.IO
import qualified Data.ByteString as B
main = do
... --some code that gets an array of ByteStrings called "files"
hSetBuffering stdout LineBuffering
putStrLn "Saving myFile1"
B.writeFile "myFile1.bin" $ files!!0
putStrLn "Saving myFile2"
B.writeFile "myFile2.bin" $ files!!1
putStrLn "Done!"
Related
This question already has an answer here:
Wrong IO actions order using putStr and getLine
(1 answer)
Closed 1 year ago.
I was learning haskell from Learn You a Haskell For Great Good book. There was this code
import Control.Monad
import Data.Char
main = forever $ do
putStr "Give me some input: "
l <- getLine
putStrLn $ map toUpper l
when i am running this code in gitbash at first it is just asking for any input after giving the input text and hitting enter( say the input text was soham) it is showing Give me some input: SOHAM.
Then i changed the code to
import Control.Monad
import Data.Char
main = forever $ do
putStrLn "Give me some input: "
l <- getLine
putStrLn $ map toUpper l
and after running it is showing me Give me some input: and asking for an input. after giving the same input soham it is showing SOHAM
Again changing the code to
import Control.Monad
import Data.Char
main = forever $ do
putStr "Give me some input: "
l <- getLine
putStr $ map toUpper l
It is just taking input again and again and when i am pressing the end of file key(ctrl+C) it is showing all the output one after another side by side but the out puts are like the original code.
Why such variations are happening ?
This is likely due to buffering: with LineBuffering it will flush in whenever a new line is output. This thus means that if you use putStr, and the string itself does not contain a new line '\n' character, it will buffer the ouput and wait until a new line is written to actually write the output to the console
You can set it to NoBuffering to write the content immediately to the console. You can change the buffering settings for the stdout with hSetBuffering :: Handle -> BufferMode -> IO ():
import Control.Monad
import Data.Char
import System.IO
main = do
hSetBuffering stdout NoBuffering
forever $ do
putStr "Give me some input: "
l <- getLine
putStrLn $ map toUpper l
another option is to flush the buffer only for putStr with hFlush :: Handle -> IO (), and thus not change the buffering policy itself:
import Control.Monad
import Data.Char
import System.IO
main = do $ forever
putStr "Give me some input: "
hFlush stdout
l <- getLine
putStrLn $ map toUpper l
This question already has an answer here:
GHCi and compiled code seem to behave differently
(1 answer)
Closed 1 year ago.
Today I want Haskell to behave like any imperative language, look at this:
import Data.HashMap.Strict as HashMap
import Data.Text.IO
import Data.Text
import Data.Functor ((<&>))
putStr "Reading data from file ..."
ls <- lines <$> readFile myFile
putStrLn " done."
putStr "Processing data ..."
let hmap = HashMap.fromList $ ls <&> \l -> case splitOn " " l of
[k, v] -> (k, v)
_ -> error "expecting \"key value\""
putStrLn " done."
Basically, the user should know what the program is doing at the moment. The result of this code is the immediate output of
> Reading data from file ... done.
> Sorting data ... done.
... and then it starts doing the actual work, the output defeating its purpose.
I am well aware that it's a feature. Haskell is declarative and order of evaluation is determined by actual dependencies, not by line numbers in my .hs-file. Thus I try the following approach:
putStr "Reading data from file ..."
lines <- lines <$> readFile myFile
putStrLn $ lines `seq` " done."
putStr "Processing data ..."
let hmap = HashMap.fromList $ ls <&> \l -> case splitOn " " l of
[k, v] -> (k, v)
_ -> error "expecting \"key value\""
putStrLn $ hmap `seq` " done."
The idea: seq only returns once its first argument has been evaluated to Weak Head Normal Form. And it works, kind of. The output of my program is now nothing for a while and then, once the work as been done, all the IO occurs.
Is there a way out of this?
EDIT: I changed the question in reply to Ben's answer. The imports should now make more sense and the program really runs.
DanielWagner commented about this related question:
GHCi and compiled code seem to behave differently
which indeed solves my problem.
putStrLn $ hmap `seq` " done."
does exactly what it's supposed to. I am only missing flushing stdout. So this actually does what I need:
putStr "Reading data from file ..."
hFlush stdout -- from System.IO
lines <- lines <$> readFile myFile
putStrLn $ lines `seq` " done."
putStr "Processing data ..."
hFlush stdout
let hmap = HashMap.fromList $ ls <&> \l -> case splitOn " " l of
[k, v] -> (k, v)
_ -> error "expecting \"key value\""
putStrLn $ hmap `seq` " done."
You haven't given us the actual code that you say has this behaviour:
The output of my program is now nothing for a while and then, once the work as been done, all the IO occurs.
How do I know it's not the code you're running? Your code doesn't compile in order to be run at all! A few problems:
You get a type error from lines, because it's in the standard Prelude but that version works on String, and you're working with Text.
You haven't imported splitOn from anywhere
The obvious splitOn to import is from Data.Text, but that has type Text -> Text -> [Text] i.e. it returns a list of Text splitting at all occurrences of the separator. You're obviously expecting a pair, splitting only on the first separator.
So at the very minimum this is code you were running in ghci after more imports/definitions that you haven't shown us.
Changing it as little as I could and get it to run gave me this:
{-# LANGUAGE OverloadedStrings #-}
import qualified Data.HashMap.Strict as HashMap
import qualified Data.Text.IO as StrictIO
import qualified Data.Text as Text
myFile = "data.txt"
main = do
putStr "Reading data from file ..."
lines <- Text.lines <$> StrictIO.readFile myFile
putStrLn $ lines `seq` " done."
putStr "Processing data ..."
let hmap = HashMap.fromList $ Text.breakOn " " <$> lines
putStrLn $ hmap `seq` " done."
I generated a very simple data file with 5,000,000 lines and ran the program with runhaskell foo.hs, and there are in fact noticeable pauses between the appearance of the reading/processing messages and the "done" appearing on each line.
I see no reason why all of the IO would be delayed appear at once (including the result of the first putStrLn. How are you actually running this code (or rather, the full and/or different code that actually runs)? In the post you've written it as input for GHCi rather than a full program (judging by the imports and IO statements at the same level, with no do block or definition of any top level functions). The only thought I had is that perhaps your data file is much smaller such that the processing takes a barely perceptible amount of time, and the initial startup processing of the Haskell code itself by ghci or runhaskell is the only noticeable delay; then I can imagine there being a slight delay followed by the printing of all the messages seemingly at once.
While reading Learn You A Haskell, I found that you can use ReadWriteMode as the third parameter to openFile.
But is this supposed to handle writing and reading to a file?
I did some tests:
import System.IO
main = do
handle <- openFile "myfile" ReadWriteMode
contents <- hGetContents handle
putStrLn contents
hPutStr handle "Something"
hClose handle
However, this returned an error:
*** Exception: myfile: hPutStr: illegal operation (handle is closed)
So, what is the point of the ReadWriteMode? I am looking for any real world cases where this would be used.
Here is something that seems to work.
import System.IO
import qualified System.IO.Strict as SIO
import GHC.IO.Handle
slurp h = do
h' <- hDuplicate h
hSeek h' AbsoluteSeek 0
SIO.hGetContents h'
main = do
handle <- openFile "myfile" ReadWriteMode
contents <- slurp handle
putStrLn contents
offset <- hTell handle
putStrLn $ "handle position: " ++ show offset
hPutStrLn handle "Something new"
hClose handle
System.IO.Strict is from the strict package
Edit: You might not even need to use SIO.hGetContents - just plain hGetContents also seems to work.
Here is minimal complete example:
import Control.Monad
import System.IO
loop :: IO ()
loop =
do line <- getLine
putStrLn line
eof <- isEOF
unless eof loop
main = loop
This program is supposed to read a line, print it out, stop if there is 'end of file' character in stdin. It doesn't leave the loop at all.
If I put eof <- isEOF before putStrLn line the program behaves very strange (try it!). I cannot get it at all: how putStrLn can possibly affect input stream and why doesn't the program terminate when I put 'end of file' character into stream (with Ctrl+D)?
Description of program's behavior when eof <- isEOF goes before putStrLn line:
After entering of a line, program does not print the entered line, but expects more input. As it gets more input, it starts to print previously entered lines. This is log of a test:
foo
boo
output: foo
bar
output: boo
baz
output: bar
< here I press Ctrl-D >
output: baz
Source:
import Control.Monad
import System.IO
loop :: IO ()
loop =
do line <- getLine
eof <- isEOF
putStrLn $ "output: " ++ line
unless eof loop
main =
do hSetBuffering stdin LineBuffering
loop
From http://lambda.haskell.org/platform/doc/current/ghc-doc/libraries/haskell2010-1.1.1.0/System-IO.html#g:11:
NOTE: hIsEOF may block, because it has to attempt to read from the stream to determine whether there is any more data to be read.
The putStrLn doesn't affect the isEOF, but the isEOF prevents the program from getting to the putStrLn before more characters are available, or you have actually pressed ^D.
So you should never use hIsEOF/isEOF until the point in the program where you are ready to read more characters if there are any.
How can I change this program to immediately process every line of text in case of interactive input? Preferably flush buffer every newline character.
main = do
input <- T.getContents
mapM_ T.putStrLn $ T.lines input
Update: Something is still missing. Take a look (???? is after newline, stdout is printed out after reaching EOF on stdin) :
> cat Test.hs
import System.IO
import Data.Text as T
import Data.Text.IO as T
main = do
hSetBuffering stdout LineBuffering
input <- T.getContents
mapM_ T.putStrLn $ T.lines input
> runhaskell Test.hs
a
????
a
????
> runhaskell --version
runghc 7.6.3
>
You want to use hSetBuffering from System.IO:
import System.IO
main = do
hSetBuffering stdout LineBuffering
input <- T.getContents
mapM_ T.putStrLn $ T.lines input
It seems like you want to use lazy input to interleave reading lines and handling them.
getContents from Data.Text.IO is not lazy, and will read everything before returning anything at all.
Import the version from Data.Text.Lazy.IO instead.