hGetContents hangs when getting contents utf-8 file - haskell

Question

I'm parsing files from a git repository and, while planning to use the gitlib module for that, I'm getting the file contents using the git executable for now - until I find some tutorial or have time to dive into gitlib's code.
I have a function that essentially run a "git show" for a specific file on a specific commit, and return its contents. Here is a full working example.
import System.IO
import System.Process
import System.Exit
main = do
let commit = Commit { hash = "811e22679008298176d8be24eedc65f9e8c4900b", time = ""}
fileIO <- showFileIO "/path/to/the/repo" (commit, "/path/to/the/file")
putStr (show fileIO)
showFileIO :: String -> (Commit, String) -> IO (Commit, String, String)
showFileIO directory (commit, filepath) = do
(_, Just hout, Just herr, procHandle) <- createProcess $ createCommand command directory
hSetEncoding hout utf8
hSetEncoding herr utf8
exitCode <- waitForProcess procHandle
stdOut <- hGetContents hout
stdErr <- hGetContents herr
if exitCode == ExitSuccess
then return (commit, filepath, stdOut)
-- Continue in the case of an error.
else return (commit, filepath, "")
where command = "git show " ++ (hash commit) ++ ":" ++ filepath
createCommand :: String -> FilePath -> CreateProcess
createCommand command directory = (shell command){std_out = CreatePipe, std_err = CreatePipe, cwd = Just directory}
-- Where Commit is defined as:
data Commit = Commit { hash :: String
, time :: String
} deriving (Show)
I was initially getting some errors ("invalid byte sequence") when getting the contents of a php file with mime-type "text/x-php" and charset "utf-8", and that was resolved when I set the encoding of the Handles to utf8. There is another file with mime-type "text/html" that is actually a html.twig file (Twig templating engine) with charset "utf-8". Now the function hangs indefinitely when trying to get the contents of this file. It works fine for other files.
Any ideas what could be wrong? How do I even get to debug in Haskell something that does not give me an error or any info? Are there any debugging tools that could help with that?

Answer 1

I would try something like this: (untested)
showFileIO directory (commit, filepath) = do
(_, Just hout, Just herr, procHandle) <- createProcess $ createCommand command directory
hSetEncoding hout utf8
hSetEncoding herr utf8
stdOut <- hGetContents hout
evaluate (length stdOut) -- strictify the above lazy IO
stdErr <- hGetContents herr
evaluate (length stdErr)
exitCode <- waitForProcess procHandle
if exitCode == ExitSuccess
...
Alternatively, use some strict-IO variant of hGetContents.
Note that there still is, as far as I can see, some window for deadlock. If the command produces a vast amount of data on stderr, then the command & OS buffers will become full and writes to stderr will block. Since the Haskell consumer now first waits for stdout to be consumed completely, we have a deadlock. Note that this will not be an issue for "short" error messages.
If we want to make it more robust, we need to read from both stdout and stderr at the same time. E.g.
showFileIO directory (commit, filepath) = do
(_, Just hout, Just herr, procHandle) <- createProcess $ createCommand command directory
hSetEncoding hout utf8
hSetEncoding herr utf8
stdOutV <- newEmptyMVar
stdErrV <- newEmptyMVar
forkIO $ do
stdOut <- hGetContents hout
evaluate (length stdOut)
putMVar stdOutV stdOut
forkIO $ fo
stdErr <- hGetContents herr
evaluate (length stdErr)
putMVar stdErrV stdErr
stdOut <- takeMVar stdOutV
stdErr <- takeMVar stdErrV
exitCode <- waitForProcess procHandle
if exitCode == ExitSuccess
...
Update. This should also work, and is much simpler.
showFileIO directory (commit, filepath) = do
(_, Just hout, Just herr, procHandle) <- createProcess $ createCommand command directory
hSetEncoding hout utf8
hSetEncoding herr utf8
stdOut <- hGetContents hout
stdErr <- hGetContents herr
forkIO $ evaluate (length stdOut)
evaluate (length stdErr)
exitCode <- waitForProcess procHandle
if exitCode == ExitSuccess
...
I wouldn't be surprised if there were some library function doing all of this for you, but I can't remember anything at the moment.
Unrelated: I prefer proc to shell to construct the CreateProcess options. The latter requires careful escaping of filenames (spaces, special characters), while the former simply takes a list of strings parameters.