Piping an http stream through a haskell conduit - haskell

Question

I am trying to create a conduit that will stream data from HTTP through a conduit source. Here is what I have so far:
import qualified Network.HTTP.Client.Conduit as CC
getStream :: String -> IO (ConduitM () BS.ByteString IO ())
getStream url = do
req <- parseUrl url
return $ CC.withResponse req $ \res -> do
responseBody res $= (awaitForever $ \bytes -> liftIO $ do
putStrLn $ "Got " ++ show (BS.length bytes) ++ " but will ignore them")
But I am getting
No instance for (Control.Monad.Reader.Class.MonadReader env0 IO) …
arising from a use of ‘CC.withResponse’
In the expression: CC.withResponse req
In the second argument of ‘($)’, namely
‘CC.withResponse req
$ \ res
-> do { responseBody res $= (awaitForever $ \ bytes -> ...) }’
In a stmt of a 'do' block:
return
$ CC.withResponse req
$ \ res
-> do { responseBody res $= (awaitForever $ \ bytes -> ...) }
How come a MonadReader is expected? It doesn't make any sense to me.

Answer 1

How about this variation of the example in the Network.HTTP.Conduit docs:
{-# LANGUAGE OverloadedStrings #-}
module Lib2 () where
import Data.Conduit (($$+-), awaitForever)
import qualified Network.HTTP.Client.Conduit as CC
import Network.HTTP.Conduit (http, tlsManagerSettings, newManager)
import Control.Monad.IO.Class (liftIO)
import Control.Monad.Trans.Resource (runResourceT)
import Data.Conduit.Binary (sinkFile) -- Exported from the package conduit-extra
main2 :: IO ()
main2 = do
request <- CC.parseUrl "http://google.com/"
manager <- newManager tlsManagerSettings
runResourceT $ do
response <- http request manager
CC.responseBody response $$+- (awaitForever $ \x -> liftIO $ putStrLn "Chunk")
Original answer
The return type for getStream is wrong. Try removing the type signature and use FlexibleContexts, e.g.:
{-# LANGUAGE OverloadedStrings, FlexibleContexts #-}
module Lib () where
import Data.Conduit
import qualified Data.ByteString as BS
import qualified Network.HTTP.Client.Conduit as CC
import Control.Monad.IO.Class
getStream url = do
req <- CC.parseUrl url
CC.withResponse req $ \res -> do
CC.responseBody res $= (awaitForever $ \x -> liftIO $ putStrLn "Got a chunk")
And then :t getStream reports:
getStream
:: (monad-control-1.0.0.4:Control.Monad.Trans.Control.MonadBaseControl
IO (ConduitM a c m),
mtl-2.2.1:Control.Monad.Reader.Class.MonadReader env m, MonadIO m,
CC.HasHttpManager env,
exceptions-0.8.0.2:Control.Monad.Catch.MonadThrow m) =>
String -> ConduitM a c m ()
which shows that the return type has the form ConduitM ..., not IO ....
This also shows how MonadReader gets into the picture... The monad m must have access to an HTTP manager through a reader environment as expressed by the following constraints:
CC.HasHttpManager env
MonadReader env m
All this is saying is that m has a reader environment of some type env which itself has a way of accessing an HTTP manager.
In particular, m cannot be just the plain IO monad, which is what the error message is complaining about.
Answer to question in the comments
Here is an example of how to create a Producer from a HTTP response:
{-# LANGUAGE OverloadedStrings #-}
module Lib3 () where
import qualified Data.ByteString as BS
import qualified Network.HTTP.Client.Conduit as CC
import Network.HTTP.Conduit (http, tlsManagerSettings, newManager)
import qualified Network.HTTP.Client as Client (httpLbs, responseOpen, responseClose)
import Data.Conduit (Producer, addCleanup)
import Data.Conduit (awaitForever, await, ($$))
import qualified Network.HTTP.Client.Conduit as HCC
import Control.Monad.IO.Class (liftIO, MonadIO)
getStream url = do
request <- CC.parseUrl url
manager <- newManager tlsManagerSettings
response <- Client.responseOpen request manager
let producer :: Producer IO BS.ByteString
producer = HCC.bodyReaderSource $ CC.responseBody response
cleanup _ = do liftIO $ putStrLn "(cleaning up)"; Client.responseClose response
producerWithCleanup = addCleanup cleanup producer
return $ response { CC.responseBody = producerWithCleanup }
test = do
res <- getStream "http://google.com"
let producer = CC.responseBody res
consumer = awaitForever $ \_ -> liftIO $ putStrLn "Got a chunk"
producer $$ consumer