Haskell: Keeping track and modifying state with Scotty HTTP API - haskell

Can I modify my state using IO via the Scotty API? Currently, I have a state transformer inside IO monad to modify state with user input. But I want to achieve this via the Scotty API.
This is my state transformer types I currently have that I use to keep modifying the state type and allow for IO actions.
-- State Transformer type inside IO monad
type STIO st a = STM st IO a
-- State transformer inside a monad
newtype STM st m a = S (st -> m (a, st))
... and with this lift functions to lift an action into the IO monad:
lift :: Monad m => m a -> STM st m a
lift mx = S (\s -> do
x <- mx
return (x, s))
This is my current bare bones Scotty server:
port = 8080
main = server
server :: IO ()
server = do
print ("Starting Server at port " ++ show port)
scotty port $ do
get "/start" $ do
json ("{starting: "++"True"++"}")
In my head I am wanting something along these lines, but unsure how to implement it:
type State = Int
server :: STIO State ()
server = do
print ("Starting Server at port " ++ show port)
lift $ scotty port $ do
get "/start" $ do
updateCounterByOneInState
counter <- getCounterFromState
json $ "{count: " ++ counter ++ "}"
Is something like this even possible, or am I just getting confused?

Scotty, like all WAI applications, needs to be prepared to handle multiple concurrent requests in separate threads. This poses a bit of a problem for your STIO monad, since it doesn't directly support concurrent access to the state. You'd need to arrange to load the state, run the handlers, and save the state in a way that's concurrency-safe.
Technically, you can do this with help from the functions in Web.Scotty.Trans, but it's not a very good idea. For example, the following self-contained example stores the state in a concurrency-safe MVar:
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE BangPatterns #-}
import Web.Scotty.Trans as W
import Control.Monad.State as S
import Control.Concurrent.MVar
import Network.Wai.Handler.Warp (Port)
import Data.Text.Lazy (Text)
type STIO s = StateT s IO
scottySTIO :: MVar s -> Port -> ScottyT Text (STIO s) () -> (STIO s) ()
scottySTIO sref p = scottyT p $ \act -> do
s <- takeMVar sref
(r, !s') <- runStateT act s
putMVar sref s'
return r
server :: STIO Int ()
server = do
let port = 8080
liftIO $ print ("Starting Server at port " ++ show port)
s <- S.get
sref <- liftIO $ newMVar s
scottySTIO sref port $ do
W.get "/start" $ do
modify (+1)
counter <- S.get
json $ "{count: " ++ show counter ++ "}"
main :: IO ()
main = evalStateT server 0
This will "work", but it will exhibit horrible concurrency performance because it ends up serializing all requests in their entirety, instead of just protecting critical code sections. It really only makes sense if you already have a huge amount of code running in the STIO monad, you don't want to modify any of it, and you're willing to take the performance hit.
Most of the time, it will be much easier to refactor your design to store the state in a concurrency-safe manner and access it directly from IO. For example, the counter can be stored in a single MVar and safely accessed in a short critical section:
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE BangPatterns #-}
import Web.Scotty
import Control.Concurrent.MVar
import Control.Monad.IO.Class
main :: IO ()
main = do
let port = 8080
print ("Starting Server at port " ++ show port)
sref <- newMVar (0 :: Int)
scotty port $ do
get "/start" $ do
-- start of critical section
counter <- liftIO $ takeMVar sref
let !counter' = counter + 1
liftIO $ putMVar sref counter'
-- end of critical section
json $ "{count: " ++ show counter' ++ "}"

Related

How do I QuickCheck a Servant Application that is constructed from an IO?

I am writing an API server using Servant. The server includes persistent state. I would like to use QuickCheck to write tests for the server.
The implementation of various endpoints that make up the Servant Application require a database value. Unsurprisingly, creation of the database value is in the IO monad.
I don't understand how to combine the pieces from Hspec, Wai, QuickCheck, and Servant in a way that satisfies them all.
I see that I can perform an IO as part of creating the Hspec Spec itself and I see that I can specify that an IO be performed before each item in the Hspec Spec. Neither of these capabilities seems helpful in this case. The IO needs to be performed for each QuickCheck iteration of the property. Without this, the database accumulates state from each iteration which invalidates the definition of the property (or at least makes it greatly more complicated).
Below is my attempt to create a minimal, self-contained example of this scenario.
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE FlexibleContexts #-}
module Main where
import Data.IORef
import Test.QuickCheck
import Test.QuickCheck.Monadic
import qualified Test.Hspec.Wai.QuickCheck as QuickWai
import Test.Hspec
import Test.Hspec.Wai
import Text.Printf
import Servant
import Servant.API
import Data.Aeson
import Data.Text.Encoding
import Data.ByteString.UTF8
( fromString
)
data Backend = Backend (IORef Integer)
openBackend :: Integer -> IO Backend
openBackend n = Backend <$> newIORef n
data Acknowledgement = Ok Integer
instance ToJSON Acknowledgement where
toJSON (Ok n) = object [ "value" .= n ]
serveSomeNumber :: Backend -> Integer -> IO Acknowledgement
serveSomeNumber (Backend a) b = do
a' <- readIORef a
modifyIORef a (\n -> n + 1)
return $ Ok (a' + b)
type TheAPI = Capture "SomeNumber" Integer :> Post '[JSON] Acknowledgement
theServer :: Backend -> Server TheAPI
theServer backend = liftIO . serveSomeNumber backend
theAPI :: Proxy TheAPI
theAPI = Proxy
app :: Backend -> Application
app backend = serve theAPI (theServer backend)
post' n =
let
url = printf "/%d" (n :: Integer)
encoded = fromString url
in
post encoded ""
spec_g :: Backend -> Spec
spec_g (Backend expectedResult) =
describe "foo" $
it "bar" $ property $ \genN -> monadicIO $ do
n <- run genN
m <- run $ readIORef expectedResult
post' n `shouldRespondWith` ResponseMatcher { matchStatus = fromInteger (n + m) }
main :: IO ()
main = do
spec_g' <- spec_g `fmap` openBackend 16
hspec spec_g'
This doesn't type check:
/home/exarkun/Scratch/QuickCheckIOApplication/test/Spec.hs:119:3: error:
* Couldn't match type `WaiSession' with `PropertyM IO'
Expected type: PropertyM IO ()
Actual type: WaiExpectation
* In a stmt of a 'do' block:
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}
In the second argument of `($)', namely
`do n <- run genN
m <- run $ readIORef expectedResult
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}'
In the expression:
monadicIO
$ do n <- run genN
m <- run $ readIORef expectedResult
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}
|
119 | post' n `shouldRespondWith` ResponseMatcher { matchStatus = fromInteger (n + m) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I don't know if there is a way to fit a WaiExpectation into a PropertyM IO () at all. I don't even know if monadicIO is helpful here at all.
How can I fit these pieces together?
Define spec_g :: Background -> Spec, then take advantage of IO's Functor and Monad instances.
main = do
spec <- fmap spec_g (openBackend 16) -- fmap spec_g :: IO Background -> IO Spec
hspec spec
or more concisely,
main = spec_g <$> openBackend 16 >>= hspec
IIRC, you're supposed to run each spec or property with the with function. Here's a few properties I wrote some time ago:
with app $ describe "/reservations/" $ do
it "responds with 404 when no reservation exists" $ WQC.property $ \rid ->
get ("/reservations/" <> toASCIIBytes rid) `shouldRespondWith` 404
it "responds with 200 after reservation is added" $ WQC.property $ \
(ValidReservation r) -> do
_ <- postJSON "/reservations" $ encode r
let actual = get $ "/reservations/" <> toASCIIBytes (reservationId r)
actual `shouldRespondWith` 200
The app value serves the service, and as far as I recall, it runs the IO action for each test. I did it with an in-memory database using an IORef, and that seems to be working just fine:
app :: IO Application
app = do
ref <- newIORef Map.empty
return $
serve api $
hoistServer api (Handler . runInFakeDBAndIn2019 ref) $
server 150 []
The WQC.property function is from a qualified import:
import qualified Test.Hspec.Wai.QuickCheck as WQC
I wasn't too happy, however, with the way I had to structure my tests and properties with HSpec, so I ultimately rewrote all the tests to be driven by HUnit. I've an upcoming blog post that describes this, but I haven't published it yet.

Servant always give me a initial value in ReaderT Monad

I'm learning Servant and write a simple service. Here's source code:
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE RankNTypes #-}
module BigMama where
import Control.Concurrent
import Control.Concurrent.STM
import Control.Monad
import Control.Monad.Reader
import Data.Aeson
import Data.Aeson.Types
import qualified Data.ByteString.Lazy.Char8 as C
import Data.Char
import qualified Data.Map as M
import Debug.Trace
import GHC.Generics
import Prelude hiding (id)
import Servant
data MicroService = MicroService
{ name :: String
, port :: Int
, id :: Maybe String
} deriving (Generic)
instance ToJSON MicroService
instance FromJSON MicroService
instance Show MicroService where
show = C.unpack . encode
type ServiceSet = STM (TVar (M.Map String MicroService))
type LocalHandler = ReaderT ServiceSet IO
defaultServices :: ServiceSet
defaultServices = newTVar $ M.fromList []
type Api =
"bigmama" :> Get '[JSON] (Maybe MicroService)
:<|> "bigmama" :> ReqBody '[JSON] MicroService :> Post '[JSON] MicroService
api :: Proxy Api
api = Proxy
serverT :: ServerT Api LocalHandler
serverT = getService
:<|> registerService
getService :: LocalHandler (Maybe MicroService)
getService = do
stm <- ask
liftIO . atomically $ do
tvar <- stm
mss <- readTVar tvar
return $ M.lookup "file" mss
registerService :: MicroService -> LocalHandler MicroService
registerService ms = do
stm <- ask
liftIO . atomically $ do
tvar <- stm
mss <- readTVar tvar
let mss' = M.insert (name ms) ms mss
writeTVar tvar mss'
return ms
readerToHandler' :: forall a. ServiceSet -> LocalHandler a -> Handler a
readerToHandler' ss r = liftIO $ runReaderT r ss
readerToHandler :: ServiceSet -> (:~>) LocalHandler Handler
readerToHandler ss = Nat (readerToHandler' ss)
server :: Server Api
server = enter (readerToHandler defaultServices) serverT
It seems like servant providing a new defaultServices for every request. I send POST to create service (name = "file") and can't get the service back on GET request. How to share data among requests on servant?
It seems like servant providing a new defaultServices for every request.
It is, because your code as written is an STM action to do so. Following the logic—
defaultServices :: ServiceSet
defaultServices = newTVar ...
This (fragmentary) definition crucially does not run the STM action to produce a new TVar. Instead it defines a value (defaultServices) which is an STM action which can produce TVars. Following where defaultServices gets passed to, you use it in your handlers like—
getService = do
stm <- ask
liftIO . atomically $ do
tvar <- stm
...
The action stored in your Reader is unchanged from the defaultServices value itself, so this code is equivalent to—
getService = do
liftIO . atomically $ do
tvar <- defaultServices
...
And by substituting in the definition of defaultServices—
getService = do
liftIO . atomically $ do
tvar <- newTVar ...
...
This now looks obviously wrong. Instead of defaultServices being an action to produce a new TVar, it should be that TVar itself, right? So on the type level without aliases—
type ServiceSet = STM (TVar (M.Map String MicroService)) -- From this
type Services = TVar (M.Map String MicroService) -- To this
defaultServices :: Services
Now defaultServices represents an actual TVar, instead of a method of creating TVars. Writing this may seem tricky if it's your first time because you somehow have to run an STM action, but atomically just turns that into an IO action, and you probably “know” that there is no way to escape IO. This actually is incredibly common though, and a quick look at the actual stm documentation for the functions in play will point you right to the answer.
It turns out that this is one of those exciting times in your life as a Haskell developer that you get to use unsafePerformIO. The definition of atomically spells out pretty much exactly what you have to do.
Perform a series of STM actions atomically.
You cannot use atomically inside an unsafePerformIO or
unsafeInterleaveIO. Any attempt to do so will result in a runtime
error. (Reason: allowing this would effectively allow a transaction
inside a transaction, depending on exactly when the thunk is
evaluated.)
However, see newTVarIO, which can be called inside unsafePerformIO,
and which allows top-level TVars to be allocated.
Now there's one final piece of this puzzle that isn't in the documentation, which is that unless you tell GHC not to inline your top-level value produced using unsafePerformIO, you might still end up with sites where you use defaultServices having their own unique set of services. E.g., without forbidding inlining this would happen—
getService = do
liftIO . atomically $ do
mss <- readTVar defaultServices
getService = do
liftIO . atomically $ do
mss <- readTVar (unsafePerformIO $ newTVarIO ...)
...
This is a simple fix though, just add a NOINLINE pragma to your definition of defaultServices.
defaultServices :: Services
defaultServices = unsafePerformIO $ newTVar M.empty
{-# NOINLINE defaultServices #-}
Now this is a fine solution, and I've happily used it in production code, but there are some objections to it. Since you're already fine with using a ReaderT in your handler monad stack (and the above solution is mostly for people who for some reason are avoiding threading a reference around), you could just create a new TVar at program initialization and then pass that in. The briefest sketch of how that would work is below.
main :: IO ()
main = do
services <- atomically (newTVar M.empty)
run 8080 $ serve Proxy (server services)
server :: TVar Services -> Server Api
server services = enter (readerToHandler services) serverT
getService :: LocalHandler (Maybe MicroService)
getService = do
services <- ask
liftIO . atomically $ do
mss <- readTVar services
...

Memoizing and repeating IO monads

EDITED 2015-11-29: see bottom
I'm trying to write an application that has a do-last-action-again button. The command in question can ask for input, and my thought for how to accomplish this was to just rerun the resulting monad with memoized IO.
There are lots of posts on SO with similar questions, but none of the solutions seem to work here.
I lifted the memoIO code from this SO answer, and changed the implementation to run over MonadIO.
-- Memoize an IO function
memoIO :: MonadIO m => m a -> m (m a)
memoIO action = do
ref <- liftIO $ newMVar Nothing
return $ do
x <- maybe action return =<< liftIO (takeMVar ref)
liftIO . putMVar ref $ Just x
return x
I've got a small repro of my app's approach, the only real difference being my app has a big transformer stack instead of just running in IO:
-- Global variable to contain the action we want to repeat
actionToRepeat :: IORef (IO String)
actionToRepeat = unsafePerformIO . newIORef $ return ""
-- Run an action and store it as the action to repeat
repeatable :: IO String -> IO String
repeatable action = do
writeIORef actionToRepeat action
action
-- Run the last action stored by repeatable
doRepeat :: IO String
doRepeat = do
x <- readIORef actionToRepeat
x
The idea being I can store an action with memoized IO in an IORef (via repeatable) when I record what was last done, and then do it again it out with doRepeat.
I test this via:
-- IO function to memoize
getName :: IO String
getName = do
putStr "name> "
getLine
main :: IO ()
main = do
repeatable $ do
memoized <- memoIO getName
name <- memoized
putStr "hello "
putStrLn name
return name
doRepeat
return ()
with expected output:
name> isovector
hello isovector
hello isovector
but actual output:
name> isovector
hello isovector
name> wasnt memoized
hello wasnt memoized
I'm not entirely sure what the issue is, or even how to go about debugging this. Gun to my head, I'd assume lazy evaluation is biting me somewhere, but I can't figure out where.
Thanks in advance!
EDIT 2015-11-29: My intended use case for this is to implement the repeat last change operator in a vim-clone. Each action can perform an arbitrary number of arbitrary IO calls, and I would like it to be able to specify which ones should be memoized (reading a file, probably not. asking the user for input, yes).
the problem is in main you are creating a new memo each time you call the action
you need to move memoized <- memoIO getName up above the action
main :: IO ()
main = do
memoized <- memoIO getName --moved above repeatable $ do
repeatable $ do
--it was here
name <- memoized
putStr "hello "
putStrLn name
return name
doRepeat
return ()
edit: is this acceptable
import Data.IORef
import System.IO.Unsafe
{-# NOINLINE actionToRepeat #-}
actionToRepeat :: IORef (IO String)
actionToRepeat = unsafePerformIO . newIORef $ return ""
type Repeatable a = IO (IO a)
-- Run an action and store the Repeatable part of the action
repeatable :: Repeatable String -> IO String
repeatable action = do
repeatAction <- action
writeIORef actionToRepeat repeatAction
repeatAction
-- Run the last action stored by repeatable
doRepeat :: IO String
doRepeat = do
x <- readIORef actionToRepeat
x
-- everything before (return $ do) is run just once
hello :: Repeatable String
hello = do
putStr "name> "
name <- getLine
return $ do
putStr "hello "
putStrLn name
return name
main :: IO ()
main = do
repeatable hello
doRepeat
return ()
I came up with a solution. It requires wrapping the original monad in a new transformer which records the results of IO and injects them the next time the underlying monad is run.
Posting it here so my answer is complete.
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE LambdaCase #-}
import Control.Applicative (Applicative(..))
import Data.Dynamic
import Data.Maybe (fromJust)
import Control.Monad.RWS
-- | A monad transformer adding the ability to record the results
-- of IO actions and later replay them.
newtype ReplayT m a =
ReplayT { runReplayT :: RWST () [Dynamic] [Dynamic] m a }
deriving ( Functor
, Applicative
, Monad
, MonadIO
, MonadState [Dynamic]
, MonadWriter [Dynamic]
, MonadTrans
)
-- | Removes the first element from a list State and returns it.
dequeue :: MonadState [r] m
=> m (Maybe r)
dequeue = do
get >>= \case
[] -> return Nothing
(x:xs) -> do
put xs
return $ Just x
-- | Marks an IO action to be memoized after its first invocation.
sample :: ( MonadIO m
, Typeable r)
=> IO r
-> ReplayT m r
sample action = do
a <- dequeue >>= \case
Just x -> return . fromJust $ fromDynamic x
Nothing -> liftIO action
tell [toDyn a]
return a
-- | Runs an action and records all of its sampled IO. Returns a
-- action which when invoked will use the recorded IO.
record :: Monad m
=> ReplayT m a
-> m (m a)
record action = do
(a, w) <- evalRWST (runReplayT action) () []
return $ do
evalRWST (runReplayT action) () w
return a

Scotty: connection pool as monad reader

There are trillions of monad tutorial including the reader and it seems all clear when you read about it. But when you actually need to write, it becomes a different matter.
I'v never used the Reader, just never got to it in practice. So I don't know how to go about it although I read about it.
I need to implement a simple database connection pool in Scotty so every action can use the pool. The pool must be "global" and accessible by all action functions. I read that the way to do it is the Reader monad. If there are any other ways please let me know.
Can you please help me and show how to do this with the Reader correctly?
I'll probably learn faster if I see how it is done with my own examples.
{-# LANGUAGE OverloadedStrings #-}
module DB where
import Data.Pool
import Database.MongoDB
-- Get data from config
ip = "127.0.0.1"
db = "index"
--Create the connection pool
pool :: IO (Pool Pipe)
pool = createPool (runIOE $ connect $ host ip) close 1 300 5
-- Run a database action with connection pool
run :: Action IO a -> IO (Either Failure a)
run act = flip withResource (\x -> access x master db act) =<< pool
So the above is simple. and I want to use the 'run' function in every Scotty action to access the database connection pool. Now, the question is how to wrap it in the Reader monad to make it accessible by all functions? I understand that the 'pool' variable must be 'like global' to all the Scotty action functions.
Thank you.
UPDATE
I am updating the question with the full code snippet. Where I pass the 'pool' variable down the function chain. If someone can show how to change it to utilize the monad Reader please.
I don't understand how to do it.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Network.HTTP.Types
import Web.Scotty
import qualified Data.Text as T
import qualified Data.Text.Lazy as LT
import Data.Text.Lazy.Internal
import Data.Monoid (mconcat)
import Data.Aeson (object, (.=), encode)
import Network.Wai.Middleware.Static
import Data.Pool
import Database.MongoDB
import Control.Monad.Trans (liftIO,lift)
main = do
-- Create connection pool to be accessible by all action functions
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 (basal pool)
basal :: Pool Pipe -> ScottyM ()
basal pool = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" (showJson pool)
showJson :: Pool Pipe -> ActionM ()
showJson pool = do
let run act = withResource pool (\pipe -> access pipe master "index" act)
d <- lift $ run $ fetch (select [] "tables")
let r = either (const []) id d
text $ LT.pack $ show r
Thanks.
UPDATE 2
I tried to do it the way it was suggested below but it does not work.
If anyone has any ideas, please. The list of compile errors is so long that I don't even know where to begin ....
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 $ runReaderT basal pool
basal :: ScottyT LT.Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" $ showJson
showJson :: ActionT LT.Text (ReaderT (Pool Pipe) IO) ()
showJson = do
p <- lift ask
let rdb a = withResource p (\pipe -> access pipe master "index" a)
j <- liftIO $ rdb $ fetch (select [] "tables")
text $ LT.pack $ show j
UPDATE 3
Thanks to cdk for giving the idea and thanks to Ivan Meredith for giving the scottyT suggestion. This question also helped: How do I add the Reader monad to Scotty's monad
This is the version that compiles. I hope it helps someone and saves some time.
import qualified Data.Text.Lazy as T
import qualified Data.Text.Lazy.Encoding as T
import Data.Text.Lazy (Text)
import Control.Monad.Reader
import Web.Scotty.Trans
import Data.Pool
import Database.MongoDB
type ScottyD = ScottyT Text (ReaderT (Pool Pipe) IO)
type ActionD = ActionT Text (ReaderT (Pool Pipe) IO)
-- Get data from config
ip = "127.0.0.1"
db = "basal"
main = do
pool <- createPool (runIOE $ connect $ host ip) close 1 300 5
let read = \r -> runReaderT r pool
scottyT 3000 read read basal
-- Application, meaddleware and routes
basal :: ScottyD ()
basal = do
get "/" shoot
-- Route action handlers
shoot :: ActionD ()
shoot = do
r <- rundb $ fetch $ select [] "computers"
html $ T.pack $ show r
-- Database access shortcut
rundb :: Action IO a -> ActionD (Either Failure a)
rundb a = do
pool <- lift ask
liftIO $ withResource pool (\pipe -> access pipe master db a)
I've been trying to figure out this exact problem myself. Thanks to hints on this SO question, and other research I've come up with the following which works for me. The key bit you were missing was to use scottyT
No doubt there is a prettier way to write runDB but I don't have much experience in Haskell, so please post it if you can do better.
type MCScottyM = ScottyT TL.Text (ReaderT (Pool Pipe) IO)
type MCActionM = ActionT TL.Text (ReaderT (Pool Pipe) IO)
main :: IO ()
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scottyT 3000 (f pool) (f pool) $ app
where
f = \p -> \r -> runReaderT r p
app :: MCScottyM ()
app = do
middleware $ staticPolicy (noDots >-> addBase "public")
get "/" $ do
p <- runDB dataSources
html $ TL.pack $ show p
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = (lift ask) >>= (\p -> liftIO $ withResource p (\pipe -> access pipe master "botland" a))
dataSources :: Action IO [Document]
dataSources = rest =<< find (select [] "datasources")
Update
I guess this a bit more pretty.
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = do
p <- lift ask
liftIO $ withResource p db
where
db pipe = access pipe master "botland" a
As you've alluded, the way to make it accessable is to wrap your computations in the Reader monad or more likely the ReaderT transformer. So your run function (changed slightly)
run :: Pool Pipe -> Action IO a -> IO (Either Failure a)
run pool act =
flip withResource (\x -> access x master db act) =<< pool
becomes
run :: Action IO a -> ReaderT (Pool Pipe) IO (Either Failure a)
run act = do
pool <- ask
withResource pool (\x -> access x master db act)
Computations inside a ReaderT r m a environment can access the r using ask and ReaderT seemingly conjures it out of thin air! In reality, the ReaderT monad is just plumbing the Env throughout the computation without you having to worry about it.
To run a ReaderT action, you use runReaderT :: ReaderT r m a -> r -> m a. So you call runReaderT on your top level scotty function to provide the Pool and runReaderT will unwrap the ReaderT environment and return you a value in the base monad.
For example, to evaluate your run function
-- remember: run act :: ReaderT (Pool Pipe) IO (Either Failure a)
runReaderT (run act) pool
but you wouldn't want to use runReaderT on run, as it is probably part of a larger computation that should also share the ReaderT environment. Try to avoid using runReaderT on "leaf" computations, you should generally call it as high up in the program logic as possible.
EDIT: The difference between Reader and ReaderT is that Reader is a monad while ReaderT is a monad transformer. That is, ReaderT adds the Reader behaviour to another monad (or monad transformer stack). If you're not familiar with monad transformers I'd recommend real world haskell - transformers.
You have showJson pool ~ ActionM () and you want to add a Reader environment with access to a Pool Pipe. In this case, you actually need ActionT and ScottyT transformers rather than ReaderT in order to work with functions from the scotty package.
Note that ActionM is defined type ActionM = ActionT Text IO, similarly for ScottyM.
I don't have all the necessary libraries installed, so this might not typecheck, but it should give you the right idea.
basal :: ScottyT Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (...)
get "/json" showJson
showJson :: ActionT Text (ReaderT (Pool Pipe) IO) ()
showJson = do
pool <- lift ask
let run act = withResource pool (\p -> access p master "index act)
d <- liftIO $ run $ fetch $ select [] "tables"
text . TL.pack $ either (const "") show d

Haskell - Actor based mutability

I'm working on a haskell network application and I use the actor pattern to manage multithreading. One thing I came across is how to store for example a set of client sockets/handles. Which of course must be accessible for all threads and can change when clients log on/off.
Since I'm coming from the imperative world I thought about some kind of lock-mechanism but when I noticed how ugly this is I thought about "pure" mutability, well actually it's kind of pure:
import Control.Concurrent
import Control.Monad
import Network
import System.IO
import Data.List
import Data.Maybe
import System.Environment
import Control.Exception
newStorage :: (Eq a, Show a) => IO (Chan (String, Maybe (Chan [a]), Maybe a))
newStorage = do
q <- newChan
forkIO $ storage [] q
return q
newHandleStorage :: IO (Chan (String, Maybe (Chan [Handle]), Maybe Handle))
newHandleStorage = newStorage
storage :: (Eq a, Show a) => [a] -> Chan (String, Maybe (Chan [a]), Maybe a) -> IO ()
storage s q = do
let loop = (`storage` q)
(req, reply, d) <- readChan q
print ("processing " ++ show(d))
case req of
"add" -> loop ((fromJust d) : s)
"remove" -> loop (delete (fromJust d) s)
"get" -> do
writeChan (fromJust reply) s
loop s
store s d = writeChan s ("add", Nothing, Just d)
unstore s d = writeChan s ("remove", Nothing, Just d)
request s = do
chan <- newChan
writeChan s ("get", Just chan, Nothing)
readChan chan
The point is that a thread (actor) is managing a list of items and modifies the list according to incoming requests. Since thread are really cheap I thought this could be a really nice functional alternative.
Of course this is just a prototype (a quick dirty proof of concept).
So my question is:
Is this a "good" way of managing shared mutable variables (in the actor world) ?
Is there already a library for this pattern ? (I already searched but I found nothing)
Regards,
Chris
Here is a quick and dirty example using stm and pipes-network. This will set up a simple server that allows clients to connect and increment or decrement a counter. It will display a very simple status bar showing the current tallies of all connected clients and will remove client tallies from the bar when they disconnect.
First I will begin with the server, and I've generously commented the code to explain how it works:
import Control.Concurrent.STM (STM, atomically)
import Control.Concurrent.STM.TVar
import qualified Data.HashMap.Strict as H
import Data.Foldable (forM_)
import Control.Concurrent (forkIO, threadDelay)
import Control.Monad (unless)
import Control.Monad.Trans.State.Strict
import qualified Data.ByteString.Char8 as B
import Control.Proxy
import Control.Proxy.TCP
import System.IO
main = do
hSetBuffering stdout NoBuffering
{- These are the internal data structures. They should be an implementation
detail and you should never expose these references to the
"business logic" part of the application. -}
-- I use nRef to keep track of creating fresh Ints (which identify users)
nRef <- newTVarIO 0 :: IO (TVar Int)
{- hMap associates every user (i.e. Int) with a counter
Notice how I've "striped" the hash map by storing STM references to the
values instead of storing the values directly. This means that I only
actually write the hashmap when adding or removing users, which reduces
contention for the hash map.
Since each user gets their own unique STM reference for their counter,
modifying counters does not cause contention with other counters or
contention with the hash map. -}
hMap <- newTVarIO H.empty :: IO (TVar (H.HashMap Int (TVar Int)))
{- The following code makes heavy use of Haskell's pure closures. Each
'let' binding closes over its current environment, which is safe since
Haskell is pure. -}
let {- 'getCounters' is the only server-facing command in our STM API. The
only permitted operation is retrieving the current set of user
counters.
'getCounters' closes over the 'hMap' reference currently in scope so
that the server never needs to be aware about our internal
implementation. -}
getCounters :: STM [Int]
getCounters = do
refs <- fmap H.elems (readTVar hMap)
mapM readTVar refs
{- 'init' is the only client-facing command in our STM API. It
initializes the client's entry in the hash map and returns two
commands: the first command is what the client calls to 'increment'
their counter and the second command is what the client calls to log
off and delete
'delete' command.
Notice that those two returned commands each close over the client's
unique STM reference so the client never needs to be aware of how
exactly 'init' is implemented under the hood. -}
init :: STM (STM (), STM ())
init = do
n <- readTVar nRef
writeTVar nRef $! n + 1
ref <- newTVar 0
modifyTVar' hMap (H.insert n ref)
let incrementRef :: STM ()
incrementRef = do
mRef <- fmap (H.lookup n) (readTVar hMap)
forM_ mRef $ \ref -> modifyTVar' ref (+ 1)
deleteRef :: STM ()
deleteRef = modifyTVar' hMap (H.delete n)
return (incrementRef, deleteRef)
{- Now for the actual program logic. Everything past this point only uses
the approved STM API (i.e. 'getCounters' and 'init'). If I wanted I
could factor the above approved STM API into a separate module to enforce
the encapsulation boundary, but I am lazy. -}
{- Fork a thread which polls the current state of the counters and displays
it to the console. There is a way to implement this without polling but
this gets the job done for now.
Most of what it is doing is just some simple tricks to reuse the same
console line instead of outputting a stream of lines. Otherwise it
would be just:
forkIO $ forever $ do
ns <- atomically getCounters
print ns
-}
forkIO $ (`evalStateT` 0) $ forever $ do
del <- get
lift $ do
putStr (replicate del '\b')
putStr (replicate del ' ' )
putStr (replicate del '\b')
ns <- lift $ atomically getCounters
let str = show ns
lift $ putStr str
put $! length str
lift $ threadDelay 10000
{- Fork a thread for each incoming connection, which listens to the client's
commands and translates them into 'STM' actions -}
serve HostAny "8080" $ \(socket, _) -> do
(increment, delete) <- atomically init
{- Right now, just do the dumb thing and convert all keypresses into
increment commands, with the exception of the 'q' key, which will
quit -}
let handler :: (Proxy p) => () -> Consumer p Char IO ()
handler () = runIdentityP loop
where
loop = do
c <- request ()
unless (c == 'q') $ do
lift $ atomically increment
loop
{- This uses my 'pipes' library. It basically is a high-level way to
say:
* Read binary packets from the socket no bigger than 4096 bytes
* Get the first character from each packet and discard the rest
* Handle the character using the above 'handler' function -}
runProxy $ socketReadS 4096 socket >-> mapD B.head >-> handler
{- The above pipeline finishes either when the socket closes or
'handler' stops looping because it received a 'q'. Either case means
that the client is done so we log them out using 'delete'. -}
atomically delete
Next up is the client, which simply opens a connections and forwards all key presses as single packets:
import Control.Monad
import Control.Proxy
import Control.Proxy.Safe
import Control.Proxy.TCP.Safe
import Data.ByteString.Char8 (pack)
import System.IO
main = do
hSetBuffering stdin NoBuffering
hSetEcho stdin False
{- Again, this uses my 'pipes' library. It basically says:
* Read characters from the console using 'commands'
* Pack them into a binary format
* send them to a server running at 127.0.0.1:8080
This finishes looping when the user types a 'q' or the connection is
closed for whatever reason.
-}
runSafeIO $ runProxy $ runEitherK $
try . commands
>-> mapD (\c -> pack [c])
>-> connectWriteD Nothing "127.0.0.1" "8080"
commands :: (Proxy p) => () -> Producer p Char IO ()
commands () = runIdentityP loop
where
loop = do
c <- lift getChar
respond c
unless (c == 'q') loop
It's pretty simple: commands generates a stream of Chars, which then get converted to ByteStrings and then sent as packets to the server.
If you run the server and a few clients and have them each type in a few keys, your server display will output a list showing how many keys each client typed:
[1,6,4]
... and if some of the clients disconnect they will be removed from the list:
[1,4]
Note that the pipes component of these examples will simplify greatly in the upcoming pipes-4.0.0 release, but the current pipes ecosystem still gets the job done as is.
First, I'd definitely recommend using your own specific data type for representing commands. When using (String, Maybe (Chan [a]), Maybe a) a buggy client can crash your actor simply by sending an unknown command or by sending ("add", Nothing, Nothing), etc. I'd suggest something like
data Command a = Add a | Remove a | Get (Chan [a])
Then you can pattern match on commands in storage in a save way.
Actors have their advantages, but also I feel that they have some drawbacks. For example, getting an answer from an actor requires sending it a command and then waiting for a reply. And the client can't be completely sure that it gets a reply and that the reply will be of some specific type - you can't say I want only answers of this type (and how many of them) for this particular command.
So as an example I'll give a simple, STM solution. It'd be better to use a hash table or a (balanced tree) set, but since Handle implements neither Ord nor Hashable, we can't use these data structures, so I'll keep using lists.
module ThreadSet (
TSet, add, remove, get
) where
import Control.Monad
import Control.Monad.STM
import Control.Concurrent.STM.TVar
import Data.List (delete)
newtype TSet a = TSet (TVar [a])
add :: (Eq a) => a -> TSet a -> STM ()
add x (TSet v) = readTVar v >>= writeTVar v . (x :)
remove :: (Eq a) => a -> TSet a -> STM ()
remove x (TSet v) = readTVar v >>= writeTVar v . delete x
get :: (Eq a) => TSet a -> STM [a]
get (TSet v) = readTVar v
This module implements a STM based set of arbitrary elements. You can have multiple such sets and use them together in a single STM transaction that succeeds or fails at once. For example
-- | Ensures that there is exactly one element `x` in the set.
add1 :: (Eq a) => a -> TSet a -> STM ()
add1 x v = remove x v >> add x v
This would be difficult with actors, you'd have to add it as another command for the actor, you can't compose it of existing actions and still have atomicity.
Update: There is an interesting article explaining why Clojure designers chose not to use actors. For example, using actors, even if you have many reads and only very little writes to a mutable structure, they're all serialized, which can greatly impact performance.

Resources