I am writing a Happstack server and I have a MongoDB database to connect to. For that, I made a function to create a connection pool
type MongoPool = Pool IOError Pipe
withMongo :: (MongoPool -> IO a) -> IO ()
withMongo f = do
pool <- dbPool
f pool
killAll pool
And then a function to run an Action with a created pool:
runDB :: (MonadIO m) => MongoPool -> Action IO a -> m (Either Failure a)
runDB pool f = liftIO $ do
pipe <- runIOE $ aResource pool
access pipe master dbName f
It's obvious this requires to carry the pool in all the routes as a parameter. I would like to wrap it into a ReaderT, so that runDB can have a type like Action IO a -> ServerPart (Either Failure a) or even better, Action IO a -> ServerPart a in which a failure will result in an HTTP Error 500 automatically.
I have a trouble wrapping my head around how that can be achieved and I'd love for some hints from people who've more experience with Haskell monads and happstack.
Thanks.
Through this question I found another with a very good hint, and I have built this. It seems to work fine and I thought I'd share it:
type MongoPool = Pool IOError Pipe
type DBServerPart a = ReaderT MongoPool (ServerPartT IO) a
hostName = "127.0.0.1"
dbName = "test"
defaultPoolSize = 10
runDB :: Action IO a -> DBServerPart (Either Failure a)
runDB f = do
pool <- ask
liftIO $ do
pipe <- runIOE $ aResource pool
access pipe master dbName f
withMongo :: DBServerPart a -> ServerPart a
withMongo f = do
pool <- liftIO $ dbPool
a <- runReaderT f pool
liftIO $ killAll pool
return a
dbPool = newPool fac defaultPoolSize
where fac = Factory {
newResource = connect $ host hostName,
killResource = close,
isExpired = isClosed
}
Related
I'm trying to get yesod-websockets working with my bootstrapped postgresql based yesod app. The websocket should act as a real-time update of a postgres table, thus I need to maintain a state containing a list of connected clients and their userId. How can I run a function as soon as the websocket disconnected? In a bigger picture, this function should be used to remove the websocket connection from the state list.
What I have so far:
type LobbyState = [(UserId, WS.Connection)]
addClient :: (UserId, WS.Connection) -> LobbyState -> LobbyState
addClient (userId, conn) state = (userId, conn) : state
chatApp :: UserId -> WebSocketsT Handler ()
chatApp userId = do
conn <- ask
-- connections is a MVar [(UserId, WS.Connection)]
connections <- lobbyConnections <$> getYesod
modifyMVar_ connections $ \s -> do
let s' = addClient (userId, conn) s
return s'
-- how to wait for disconnect and execute a function?
getGameR :: Handler TypedContent
getGameR = do
-- TODO: correct usage??
userId <- requireAuthId
webSockets $ chatApp userId
-- some more normal HTML handler code
In the example they use the following snippet:
race_
(forever $ atomically (readTChan readChan) >>= sendTextData)
(sourceWS $$ mapM_C (\msg ->
atomically $ writeTChan writeChan $ name <> ": " <> msg))
I understand how I could utilize a TChan to forever send updates, but how can I react to the actual disconnect event to clean up some state?
Thanks to Chris Stryczynski's comment, I was able to solve it via a catch handler.
A sample echo server with a cleanup after a client disconnects could look like this:
chatApp :: WebSocketsT Handler ()
chatApp =
( forever $ do
msg :: Text <- receiveData
sendTextData msg
)
`catch` ( \(e :: ConnectionException) -> do
let eshow = fromString $ show e
putStrLn $ eshow
-- clean up specific state here
case e of
CloseRequest code msg -> -- Handle specific close codes/msg
return ()
)
I was using servant-generic-0.1.0.3 and servant-server-0.13.0.1 to do the following:
data Site route = Site
{ page :: route :-
"page" :> Capture "x" Int :> Capture "y" Int :> Get '[JSON] [Int]
, home :: route :-
Raw
} deriving (Generic)
type API = ToServant (Site AsApi)
siteServer :: Pool Connection -> Site AsServer
siteServer pool = Site
{ page = \x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
api :: Proxy API
api = Proxy
app :: Pool Connection -> Application
app pool = serve api (toServant $ siteServer pool)
That worked fine, then I tried to use ReaderT to avoid passing Pool Connection to siteServer, so I added AppM and replaced siteServer like this:
type AppM = ReaderT (Pool Connection) IO
siteServer :: ServerT API AppM
siteServer = Site
{ page = do
pool <- ask
\x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
but I got a bunch of errors when I tried to compile it.
I followed the same steps shown in the servant cookbook, but I couldn't make this work with generic routes, although it works when using regular routes.
Am I missing something that could make this work?
At least for the record-style routes supported by servant-* >= 0.14 (see here), if you want to work with another monad than Handler, you will want to look at AsServerT and genericServerT.
Applied to your example, this means siteServer should be defined as follows (not typechecked, but should be very close to correct).
siteServer :: Site (AsServerT AppM)
siteServer = Site
{ page = ... something in AppM ...
, home = ... something in AppM ...
}
-- turning that into a usual chain of :<|>-separated handlers
oldStyleServer :: ServerT API AppM
oldStyleServer = genericServerT siteServer
-- bringing it all back in Handler
oldStyleServerInHandler :: Pool Connection -> Server API -- same as ServerT API Handler
oldStyleServerInHandler conns = hoistServer (Proxy #API) appToHandler oldStyleServer
where appToHandler = liftIO . flip runReaderT conns
-- or something along those lines
-- serving it
app :: Pool Connection -> Application
app conns = serve (Proxy #API) (oldStyleServerInHandler conns)
Edit: Since you're using servant-* < 0.14 with servant-generic, you should replace genericServerT with toServant.
There are trillions of monad tutorial including the reader and it seems all clear when you read about it. But when you actually need to write, it becomes a different matter.
I'v never used the Reader, just never got to it in practice. So I don't know how to go about it although I read about it.
I need to implement a simple database connection pool in Scotty so every action can use the pool. The pool must be "global" and accessible by all action functions. I read that the way to do it is the Reader monad. If there are any other ways please let me know.
Can you please help me and show how to do this with the Reader correctly?
I'll probably learn faster if I see how it is done with my own examples.
{-# LANGUAGE OverloadedStrings #-}
module DB where
import Data.Pool
import Database.MongoDB
-- Get data from config
ip = "127.0.0.1"
db = "index"
--Create the connection pool
pool :: IO (Pool Pipe)
pool = createPool (runIOE $ connect $ host ip) close 1 300 5
-- Run a database action with connection pool
run :: Action IO a -> IO (Either Failure a)
run act = flip withResource (\x -> access x master db act) =<< pool
So the above is simple. and I want to use the 'run' function in every Scotty action to access the database connection pool. Now, the question is how to wrap it in the Reader monad to make it accessible by all functions? I understand that the 'pool' variable must be 'like global' to all the Scotty action functions.
Thank you.
UPDATE
I am updating the question with the full code snippet. Where I pass the 'pool' variable down the function chain. If someone can show how to change it to utilize the monad Reader please.
I don't understand how to do it.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Network.HTTP.Types
import Web.Scotty
import qualified Data.Text as T
import qualified Data.Text.Lazy as LT
import Data.Text.Lazy.Internal
import Data.Monoid (mconcat)
import Data.Aeson (object, (.=), encode)
import Network.Wai.Middleware.Static
import Data.Pool
import Database.MongoDB
import Control.Monad.Trans (liftIO,lift)
main = do
-- Create connection pool to be accessible by all action functions
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 (basal pool)
basal :: Pool Pipe -> ScottyM ()
basal pool = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" (showJson pool)
showJson :: Pool Pipe -> ActionM ()
showJson pool = do
let run act = withResource pool (\pipe -> access pipe master "index" act)
d <- lift $ run $ fetch (select [] "tables")
let r = either (const []) id d
text $ LT.pack $ show r
Thanks.
UPDATE 2
I tried to do it the way it was suggested below but it does not work.
If anyone has any ideas, please. The list of compile errors is so long that I don't even know where to begin ....
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 $ runReaderT basal pool
basal :: ScottyT LT.Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" $ showJson
showJson :: ActionT LT.Text (ReaderT (Pool Pipe) IO) ()
showJson = do
p <- lift ask
let rdb a = withResource p (\pipe -> access pipe master "index" a)
j <- liftIO $ rdb $ fetch (select [] "tables")
text $ LT.pack $ show j
UPDATE 3
Thanks to cdk for giving the idea and thanks to Ivan Meredith for giving the scottyT suggestion. This question also helped: How do I add the Reader monad to Scotty's monad
This is the version that compiles. I hope it helps someone and saves some time.
import qualified Data.Text.Lazy as T
import qualified Data.Text.Lazy.Encoding as T
import Data.Text.Lazy (Text)
import Control.Monad.Reader
import Web.Scotty.Trans
import Data.Pool
import Database.MongoDB
type ScottyD = ScottyT Text (ReaderT (Pool Pipe) IO)
type ActionD = ActionT Text (ReaderT (Pool Pipe) IO)
-- Get data from config
ip = "127.0.0.1"
db = "basal"
main = do
pool <- createPool (runIOE $ connect $ host ip) close 1 300 5
let read = \r -> runReaderT r pool
scottyT 3000 read read basal
-- Application, meaddleware and routes
basal :: ScottyD ()
basal = do
get "/" shoot
-- Route action handlers
shoot :: ActionD ()
shoot = do
r <- rundb $ fetch $ select [] "computers"
html $ T.pack $ show r
-- Database access shortcut
rundb :: Action IO a -> ActionD (Either Failure a)
rundb a = do
pool <- lift ask
liftIO $ withResource pool (\pipe -> access pipe master db a)
I've been trying to figure out this exact problem myself. Thanks to hints on this SO question, and other research I've come up with the following which works for me. The key bit you were missing was to use scottyT
No doubt there is a prettier way to write runDB but I don't have much experience in Haskell, so please post it if you can do better.
type MCScottyM = ScottyT TL.Text (ReaderT (Pool Pipe) IO)
type MCActionM = ActionT TL.Text (ReaderT (Pool Pipe) IO)
main :: IO ()
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scottyT 3000 (f pool) (f pool) $ app
where
f = \p -> \r -> runReaderT r p
app :: MCScottyM ()
app = do
middleware $ staticPolicy (noDots >-> addBase "public")
get "/" $ do
p <- runDB dataSources
html $ TL.pack $ show p
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = (lift ask) >>= (\p -> liftIO $ withResource p (\pipe -> access pipe master "botland" a))
dataSources :: Action IO [Document]
dataSources = rest =<< find (select [] "datasources")
Update
I guess this a bit more pretty.
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = do
p <- lift ask
liftIO $ withResource p db
where
db pipe = access pipe master "botland" a
As you've alluded, the way to make it accessable is to wrap your computations in the Reader monad or more likely the ReaderT transformer. So your run function (changed slightly)
run :: Pool Pipe -> Action IO a -> IO (Either Failure a)
run pool act =
flip withResource (\x -> access x master db act) =<< pool
becomes
run :: Action IO a -> ReaderT (Pool Pipe) IO (Either Failure a)
run act = do
pool <- ask
withResource pool (\x -> access x master db act)
Computations inside a ReaderT r m a environment can access the r using ask and ReaderT seemingly conjures it out of thin air! In reality, the ReaderT monad is just plumbing the Env throughout the computation without you having to worry about it.
To run a ReaderT action, you use runReaderT :: ReaderT r m a -> r -> m a. So you call runReaderT on your top level scotty function to provide the Pool and runReaderT will unwrap the ReaderT environment and return you a value in the base monad.
For example, to evaluate your run function
-- remember: run act :: ReaderT (Pool Pipe) IO (Either Failure a)
runReaderT (run act) pool
but you wouldn't want to use runReaderT on run, as it is probably part of a larger computation that should also share the ReaderT environment. Try to avoid using runReaderT on "leaf" computations, you should generally call it as high up in the program logic as possible.
EDIT: The difference between Reader and ReaderT is that Reader is a monad while ReaderT is a monad transformer. That is, ReaderT adds the Reader behaviour to another monad (or monad transformer stack). If you're not familiar with monad transformers I'd recommend real world haskell - transformers.
You have showJson pool ~ ActionM () and you want to add a Reader environment with access to a Pool Pipe. In this case, you actually need ActionT and ScottyT transformers rather than ReaderT in order to work with functions from the scotty package.
Note that ActionM is defined type ActionM = ActionT Text IO, similarly for ScottyM.
I don't have all the necessary libraries installed, so this might not typecheck, but it should give you the right idea.
basal :: ScottyT Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (...)
get "/json" showJson
showJson :: ActionT Text (ReaderT (Pool Pipe) IO) ()
showJson = do
pool <- lift ask
let run act = withResource pool (\p -> access p master "index act)
d <- liftIO $ run $ fetch $ select [] "tables"
text . TL.pack $ either (const "") show d
I'm trying to write a convenient abstraction to thottle api calls. The API calls will be some kind of HTTP request. I want to be able to write all the app logic as if I were just in the IO Monad but then have the abstraction throttle the calls for me so I don't surpass some predefined limit. Having these calls run asynchronously would be nice. Right now I have this.
data APICallStats = APICallStats
{
startTime :: !UTCTime
, requestCount :: !Int
, tps :: !Int
} deriving Show
newtype APICall a = APICall (IO a)
initStats :: Int -> IO APICallStats
initStats limit = APICallStats <$> getCurrentTime <*> pure 0 <*> pure limit
makeCall :: MVar APICallStats -> APICall a -> IO a
makeCall mv (APICall f) = do
(APICallStats st c t) <- readMVar mv
now <- getCurrentTime
let inSeconds = realToFrac (diffUTCTime now st) :: Double
cp1 = fromIntegral (c+1)
td = fromIntegral t
when (cp1 / inSeconds > td)
(threadDelay (round $ (cp1 / td - inSeconds)*1000000))
modifyMVar_ mv
(\(APICallStats start req ts) -> return $ APICallStats start (req+1) ts)
f
And I can test it like this. And it works fine in a single or multiple threads because of the MVar.
testCall :: String -> APICall ()
testCall id = APICall (getCurrentTime >>= (putStrLn . ((++) (id ++ " ")) . show))
test :: IO ()
test = do
mv <- initStats 1 >>= newMVar
forever $ makeCall mv (testCall "")
threadedTest :: IO ()
threadedTest = do
mv <- initStats 1 >>= newMVar
threadId <- forkIO $ forever $ makeCall mv (testCall "thread0")
forever $ makeCall mv (testCall "main thread")
killThread threadId
This is nice and all but it is not as abstract as I want it to be. Since there will only be a couple specific API calls I can have them return the type APICall a. And then make this a monad that is an instance of MonadIO so I can just write my logic and use liftIO when I need to. But I'm unsure if I would be able to make this a monad that didn't break the laws and how exactly to go about it.
EDIT
I think I've gotten pretty close to what I want it to do.
withThrottle :: Int -> StateT (MVar APICallStats) IO a -> IO a
withThrottle limit f = do
mv <- initStats limit >>= newMVar
evalStateT f mv
process :: APICall a -> StateT (MVar APICallStats) IO a
process a = do
mv <- get
liftIO $ makeCall mv a
With this I can write something like this.
stateTest = do
withThrottle 2 $ do
process (testCall "")
process (testCall "")
process (testCall "")
liftIO $ threadDelay 10000000 -- Some long computation
process (testCall "")
process (testCall "")
process (testCall "")
process (testCall "")
With a long running app the entire thing can run is this monad. This will guarantee that It doesn't make too many outside API calls over the lifetime of the service. I just don't want the state monad. Just something that is MonadIO. So I think I just need to hide that and I'll be done. But the solution isn't very pretty so other suggestions welcome.
I am practicing at getting consistent with my error handling, and I keep hoping to see the code that I've written start shrinking. But I built up a domain-meaningful persistence function, and the amount of code I had to write just to do monad handling and custom error handling is astounding.
For "programming errors", I just call error "assertion blown"
For really mundane things, I return Nothing (the requested object doesn't exist)
For errors that should be handled, I'm returning Either E V or its equivalent by creating a Control.Monad.Error instance to handle it.
I have in my application multiple functions which I would call primitives, but they can catch certain errors and will raise them my throwing a value of the DBError type. So, I've defined them like so:
data DBError = ConversionError ConvertError
| SaveError String
| OtherError String
deriving (Show, Eq)
instance Error DBError where
noMsg = OtherError "No message found"
strMsg s = OtherError s
type DBMonad = ErrorT DBError IO
selectWorkoutByID :: IConnection a => UUID -> a -> DBMonad (Maybe SetRepWorkout)
insertWorkout :: IConnection a => SetRepWorkout -> a -> DBMonad ()
At the level of the calling application, a Workout is a unique object persisted to the database, so the application only ever calls saveWorkout, which itself uses selectWorkoutByID, insertWorkout, and updateWorkout in the ways you would expect:
saveWorkout :: IConnection a => SetRepWorkout -> a -> DBMonad ()
saveWorkout workout conn =
r <- liftIO $ withTransaction conn $ \conn -> runErrorT $ do
w_res <- selectWorkoutByID (uuid workout) conn
case w_res of
Just w -> updateWorkout workout conn >> return ()
Nothing -> insertWorkout workout conn >> return ()
case r of
Right _ -> return ()
Left err -> throwError err
This is ugly. I have to run and unwrap a DBMonad, run that in the IO monad, lift the IO back up into the DBMonad, and then check the results and re-wrap the results in the DBMonad.
How can I do this with less, and easier to read, code?
I'm expecting that using my custom application monad to handle recoverable errors would help me to reduce the amount of code I have to write, but this is doing the opposite!
Here are some additional questions:
Is there a better way to build up application-semantic errors?
Should I be using Control.Exception instead?
After reviewing http://en.wikibooks.org/wiki/Haskell/Monad_transformers, which is the first document on Monad Transformers that really helped me understand them, I figured out a decent solution.
A new version of the saveWorkout function would look like this:
saveWorkout :: IConnection a => SetRepWorkout -> a -> DBMonad ()
saveWorkout workout conn =
ErrorT $ liftIO $ withTransaction conn $ \conn -> runErrorT $ do
w_res <- selectWorkoutByID (uuid workout) conn
case w_res of
Just w -> updateWorkout workout conn >> return ()
Nothing -> insertWorkout workout conn >> return ()
The deal is this:
withTransaction is returning IO Either DBError (). liftIO has the type MonadIO m => IO a -> m a. ErrorT is the standard constructor for everything of the ErrorT monad, and I defined DBMonad to be of that monad. So, I am working with these types:
withTransaction conn $ <bunch of code> :: IO (Either DBError ())
liftIO :: MonadIO m => IO (Either DBError ()) -> m (Either DBError ())
ErrorT :: IO (Either DBError ()) -> ErrorT IO DBError ()
Ideally, since ErrorT/DBMonad are part of the MonadTrans class, I would use simply lift in order to lift IO (Either DBError ()) back up into the ErrorT monad, but at this time I cannot get it to actually type check correctly. This solution, however, still makes the code better by removing the redundent re-wrapping that I had before.