I am developing a game and chose Happstack for the persistence part. I find it quite easy to use, i made a quick example for myself to understand it:
getAllObjects :: MonadIO m => m [Thing]
getAllObjects = do
elems <- query GetObjects
return elems
addAnObject :: (MonadIO m) => Thing -> m ()
addAnObject thing = do update $ AddObject thing
test command = do
control <- startSystemState macidProxy
result <- command
shutdownSystem control
return result
checkpoint = do
control <- startSystemState macidProxy
createCheckpoint control
shutdownSystem control
and everytime i 'test' it, it create an event.file. then i 'checkpoint' and creates a new checkpoint file, it is ok for me, the problem is that the old events files keep growing! i manualy delete everyfile (except last checkpoint and current).
Is there some code im missing from happstack to do the 'delete old things'?
There is no built-in mechanism for purging old event files. Lemmih has talked about adding such facilities to acid-state at some point in time.
EDIT: The darcs version of acid-state now has a function 'createArchive' to archive old log files that are no longer needed to restore the current state.
Related
I have an API written using the Servant library which connects to a postgres db. The connection string for my db is stored in a configuration file. Everytime I make a request to any endpoint that interacts with the db I have to read the file to get the connection string, this is what im trying to avoid.
Step by step example of what im trying to achieve:
Application starts up.
Contents of the config file are read and bound to some type/object.
I make a request to my endpoint to create an entry in the db.
I read the connection string from the type/object that I bound it to and NOT the config file.
Every subsequent request for the lifetime of the application does not have to read the config file everytime it wants to interact with the database.
Doing this in something like java/c# you would just bind the contents of a file to some POCO which would be added to your DI container as a singleton so it can be referenced anywhere in your application and persist between each api request. If I have 100 requests that ineract with the db, none of those 100 requests would need to read config file to get the connection string as it was already loaded into memory when the app started.
I have thought about using the cache package, but is there an easier way to do something like this without a third party package?
Let's begin with this trivial Servant server:
import Servant
import Servant.Server
type FooAPI = Get '[JSON] Int
fooServer :: Server FooAPI
fooServer = pure 1
Suppose we don't want to hardcode that 1. We could turn fooServer into a function like
makeFooServer :: Int -> Server FooAPI
makeFooServer n = pure n
Then, in main, we could read the n from a file then call makeFooServer to construct the server. Something similar could be done for your database connection.
There's another approach that might be sometimes preferrable. Servant lets you define servers whose handlers live in a monad different from Handler, and then transform them into regular servers (tutorial).
We can write a server in which the handler monad is a ReaderT holding the configuration:
import Control.Monad.Trans.Reader
type RHandler env = ReaderT env Handler
fooServer' :: ServerT FooAPI (RHandler Int)
fooServer' = do
n <- ask
pure n
Where ServerT is a more general form of Server that lets you specify the handler monad in an extra type argument.
Then, we use the hoistServer function to supply the initial environment and go back to a regular server:
-- "Server FooAPI" is the same as "ServerT FooAPI Handler"
-- so the transformation we need is simply to run the `ReaderT`
-- by supplying an environment.
backToNormalServer :: Int -> Server FooAPI
backToNormalServer n = hoistServer (Proxy #FooAPI) (flip runReaderT n) fooServer'
The ServerT FooAPI (RHandler Int) approach has the advantage that you still have a server value that you can directly manipulate and pass around, instead of it being the result of a function.
Also, for some advanced use cases, the environment might reflect information derived from the structure of each endpoint.
Here is the following code :
start :: Settings -> IO ()
start settings # Settings {healthCheckLoggerId} = do
waitTillHealthy
healthCheckLoggerId
settings
Server.getDependencies
Server.healthCheck
Server.getDependencies
settings
(runServerOnWarp)
where
runServerOnWarp :: Server.Dependencies -> IO()
runServerOnWarp dependencies # Server.Dependencies {logger,port} = do
run port $ application
(proxy :: Proxy GSDMonitoringStreamingApi)
monitoringServer
dependencies
monitoringServer :: ServantServer GSDMonitoringStreamingApi Server.Dependencies
monitoringServer dependencies = streamCommand dependencies
where
streamCommandResponse :: Server.Dependencies ->
WorkspaceId ->
Handler (PipeStream (Either StreamIssue (Persisted CommandResponse)))
streamCommandResponse Server.Dependencies {eventStoreClientDependencies} =
return . toPipes . GsdMonitoring.streamCommandResponse eventStoreClientDependencies
Explanation of the issue :
the function Server.getDependencies is wrapping a bracket pattern that get a connection to a database (the client of that database recommend using one connection for all not one for each request).
First I'm running a healthCheck which is blocking till everything is fine. Each time I'm testing the health check of a dependency each time I'm acquiring a new connection. Once the environnement is healthy, I'm getting again the Server.Dependencies (db connection), and I'm running the server. Everything running into runServerOnWarp is executed in the -- ^ computation to run in-between section from
bracket
:: IO a -- ^ computation to run first (\"acquire resource\")
-> (a -> IO b) -- ^ computation to run last (\"release resource\")
-> (a -> IO c) -- ^ computation to run in-between
-> IO c -- returns the value from the in-between computation
so all the requests are executed into that section and get the same connection, so far so good... Because that section :
run port $ application
(proxy :: Proxy GSDMonitoringStreamingApi)
monitoringServer dependencies
is never ending, we are always staying in the computation to run in-between (except when I kill the app...)
Now the issue I have is when the connection get closed.... all my request will return Left StreamIssue which is fine but I have to get a new connection from Server.getDependencies. I don't know how to do it properly :-( Hacks that I have in mind but I don't like are :
1) Having one connection for one request
2) Raising an exception when the request stream a Left streamIssue, that somehow restart the server to get new dependencies... The issue is that the client of the server never get a Left value and the communication between the 2 is violently closed because the server is restarted...
Do you have a cleaner way in mind ?
P.S : I hope the details are enough to understand the issue, don't hesitate for more information if necessary...
If losing the database connection is rare, letting the server crash, and some supervisory process restart it, works quite well.
If losing the DB connection is common enough that you can't tolerate client timeouts from restarting the server, you can add reconnect logic to whatever code uses the DB connection. I usually use resource-pool for this. I believe you can create the "pool" with only one resource, if that fits your database.
Usually getting a DB connection is fast, so I'm guessing it's not worth extra effort to route incoming requests elsewhere. withResource in resource-pool blocks until a resource is ready, so active requests can easily wait while you reconnect.
There are some database queries I want to run periodically, and according to its state, send notifications to users email and change the state of their accounts. Can I do it within Yesod itself?
I moved from Yesod's issue.
Run Handler code at a specific time within yesod · Issue #1529 · yesodweb/yesod
I do not know your complete code.
So, this is proposal.
makeApplication :: App -> IO Application
makeApplication foundation = do
unsafeHandler foundation $
forkHandler (\_ -> catchError) $ forever $ do -- catchError do not exist
waitUntil10AM -- waitUntil10AM do not exist
getCheckupR
logWare <- makeLogWare foundation
-- Create the WAI application and apply middlewares
appPlain <- toWaiAppPlain foundation
return $ logWare $ (acceptOverride . autohead . gzip def) appPlain
This code point that use unsafeHandler and forkHandler.
waitUntil10AM
I do not know your timezone, environment, database structure etc, so I want you to write the details yourself.
For example, if you put threadDelay in forever and check it once every ten minutes, put the date on which you sent the mail already in the database and call it if you do not send it and it exceeds 10AM.
catchError
Please decide what kind of processing should be done at the time of error.
I would like to handle errors in a way that it would never stop
You can name the function to be passed inside forkHandler and call it again on error.
I'd like to be able to stream S3 bucket object contents via Servant as the response body.
I'm having issues getting MonadResource instance missing for Handler:
src/Servant/Streaming/Example.hs:29:3: error:
* No instance for (MonadResource Handler)
arising from a use of `runAWS'
* In a stmt of a 'do' block: runAWS env conduits
In the expression:
do env <- newEnv Discover
runAWS env conduits
In an equation for `server':
server
= do env <- newEnv Discover
runAWS env conduits
|
29 | runAWS env conduits
| ^^^^^^^^^^^^^^^^^^^
I have made a repository to reproduce: https://github.com/domenkozar/servant-streaming-amazonka
servant-streaming-server handles ResourceT for Stream (Of BS.ByteString) (ResourceT IO) () https://github.com/plow-technologies/servant-streaming/blob/master/servant-streaming-server/src/Servant/Streaming/Server/Internal.hs#L77-L79
but since I'm using Amazonka I also need to make sure MonadResource for Handler is taken take in that bracket. It's not clear to me how to do that.
My understanding is that using enter/hoistServer this won't work since resources would be cleaned up too soon (before streaming).
Notes:
This builds upon question asked in 2016: Haskell Servant and streaming
Servant part is implemented via https://github.com/plow-technologies/servant-streaming/pull/2/files
Edits
EDIT: I've since replaced $$ with $$+-
EDIT2: I've resolved Conduit specific errors, now fighting with MonadResource
Solved with
server :: Server API
server = do
env <- newEnv Discover
res <- runInternalState (runAWS env conduits) st
return (res >> liftIO (closeInternalState st))
https://github.com/domenkozar/servant-streaming-amazonka/commit/c5fad78dd7bf733cecb8790035105c819d5f5ae9
I am using Snap framework to prototype a web application. I am trying to use Snap.Util.FileUploads.handleMultiPart to upload a file, immediately process it using iteratee, and at the same time display the progress message on the same page.
It is possible to hook Data.Enumerator.printChunks to debug progress on the console. I could not figure out how to display the progress on the same page while processing the file upload. How can a progress message be displayed using handleMultiPart during file upload?
Also handleMultiPart takes PartInfo -> Iteratee ByteString IO a to handle the file upload. Should handleMultiPart rather take MonadIO m => PartInfo -> Iteratee ByteString m a to make it simpler?
In general I don't know if it's possible to display progress on the client side via pushing an HTML response while the file is being uploaded. To the best of my knowledge, web apps that do this are typically using some JavaScript API or Flash widget to do so. Certainly it's not possible using handleMultiPart.
Streaming to the console is another matter, however -- you can easily provide an enumeratee that logs chunk information (or updates an MVar, for an alternative) and then passes control downstream. This will be less brain-bending in snap 1.0 (nearing release), which will use io-streams which are much easier to think about.
Finally, handleMultiPart works over IO because to do otherwise would require the "run" action for the monad, i.e the inverse lift from (m a -> IO a). There are ways to do this kind of thing but snap <1.0 doesn't do them, sorry --- and in snap 1.0 the issue will be moot.