I have been using this guide in order to add authentication and authorisation to my project. I am new to Yesod and have been running into some problems:
I am using the following foundation data:
data App = App (TChan Text)
For my authentication i need this data type:
data App = App SqlBackend
How can i combine the two? Do i then also have to change existing functions?
In the end i run everything like this:
main :: IO ()
main = runNoLoggingT $ withSqliteConn "email.db3" $ \conn -> liftIO $ do
runSqlConn (runMigration migrateAll) conn
chan <- newTChanIO
warp 3000 $ App chan
-- Backend SqlBackend needs to be initilized with conn (see below)
{- OLD
main = do
chan <- newTChanIO
warp 3000 $ App chan
-}
{- NEW
main = runNoLoggingT $ withSqliteConn "email.db3" $ \conn -> liftIO $ do
runSqlConn (runMigration migrateAll) conn
warp 3000 $ App conn
-}
I know that i have to combine the two. I read this article. How do i get chan and also conn together? It might sound trivial, but there is little documentation out there.
Please help me!
Typically people make their App datatype a record with multiple fields. For example, the scaffolded Yesod project uses this code:
data App = App
{ appSettings :: AppSettings
, appStatic :: Static -- ^ Settings for static file serving.
, appConnPool :: ConnectionPool -- ^ Database connection pool.
, appHttpManager :: Manager
, appLogger :: Logger
}
The example code you linked to does that as well:
data App = App
{ homepageContent :: Html
, visitorCount :: IORef Int
}
So what about something like this:
-- Small caveat: this is code untested
data App = App
{ backend :: SqlBackend
, chan :: TChan Text
}
main :: IO ()
main = do
chan <- newTChanIO
runNoLoggingT $ withSqliteConn "email.db3" $ \conn -> liftIO $ do
runSqlConn (runMigration migrateAll) conn
warp 3000 $ App { backend=conn, chan=chan}
You would have to change old code accessing your TChan, but once your App is a record, you can add more fields without making changes to the rest of your codebase.
As a side note, have you considered using the Yesod scaffolding? You'd start off with all this handled for you, with a bunch of other niceties and best practices already implemented for you.
Related
Problem trying to unit test routes. Scotty, Persistent, and Hspec-WAI.
Unlike Yesod or Spock, Scotty doesn't have a nice place to store database handlers. I've got it working by having one massive "do" that starts up the database, keeps the database pool as a local variable, then uses that variable.
app :: IO ()
app = do
-- allocate_database $ \pool
-- scotty 8080 $do
-- handleSomeRoute pool
However, Hspec-WAI wants it in the IO Application form.
scottyApp :: ScottyM () -> IO Application
Is there a sane way to inject the DB connection pool into a scottyApp ?
Here's how you can do it. Basically you open the database before you make the hspec call:
{-# LANGUAGE OverloadedStrings #-}
import Test.Hspec
import Test.Hspec.Wai
import Network.Wai (Application)
import qualified Web.Scotty as S
allocate_db :: (Int -> IO a) -> IO a
allocate_db = undefined
handleSomeRoute :: Int -> S.ScottyM ()
handleSomeRoute = undefined
main2 :: IO ()
main2 = allocate_db $ \pool -> do
let app' = handleSomeRoute pool
hspec $ with (S.scottyApp app') $ do
describe "GET /" $ do
it "responds with 200" $ do
get "/" `shouldRespondWith` 200
There are trillions of monad tutorial including the reader and it seems all clear when you read about it. But when you actually need to write, it becomes a different matter.
I'v never used the Reader, just never got to it in practice. So I don't know how to go about it although I read about it.
I need to implement a simple database connection pool in Scotty so every action can use the pool. The pool must be "global" and accessible by all action functions. I read that the way to do it is the Reader monad. If there are any other ways please let me know.
Can you please help me and show how to do this with the Reader correctly?
I'll probably learn faster if I see how it is done with my own examples.
{-# LANGUAGE OverloadedStrings #-}
module DB where
import Data.Pool
import Database.MongoDB
-- Get data from config
ip = "127.0.0.1"
db = "index"
--Create the connection pool
pool :: IO (Pool Pipe)
pool = createPool (runIOE $ connect $ host ip) close 1 300 5
-- Run a database action with connection pool
run :: Action IO a -> IO (Either Failure a)
run act = flip withResource (\x -> access x master db act) =<< pool
So the above is simple. and I want to use the 'run' function in every Scotty action to access the database connection pool. Now, the question is how to wrap it in the Reader monad to make it accessible by all functions? I understand that the 'pool' variable must be 'like global' to all the Scotty action functions.
Thank you.
UPDATE
I am updating the question with the full code snippet. Where I pass the 'pool' variable down the function chain. If someone can show how to change it to utilize the monad Reader please.
I don't understand how to do it.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Network.HTTP.Types
import Web.Scotty
import qualified Data.Text as T
import qualified Data.Text.Lazy as LT
import Data.Text.Lazy.Internal
import Data.Monoid (mconcat)
import Data.Aeson (object, (.=), encode)
import Network.Wai.Middleware.Static
import Data.Pool
import Database.MongoDB
import Control.Monad.Trans (liftIO,lift)
main = do
-- Create connection pool to be accessible by all action functions
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 (basal pool)
basal :: Pool Pipe -> ScottyM ()
basal pool = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" (showJson pool)
showJson :: Pool Pipe -> ActionM ()
showJson pool = do
let run act = withResource pool (\pipe -> access pipe master "index" act)
d <- lift $ run $ fetch (select [] "tables")
let r = either (const []) id d
text $ LT.pack $ show r
Thanks.
UPDATE 2
I tried to do it the way it was suggested below but it does not work.
If anyone has any ideas, please. The list of compile errors is so long that I don't even know where to begin ....
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scotty 3000 $ runReaderT basal pool
basal :: ScottyT LT.Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (noDots >-> addBase "static")
get "/json" $ showJson
showJson :: ActionT LT.Text (ReaderT (Pool Pipe) IO) ()
showJson = do
p <- lift ask
let rdb a = withResource p (\pipe -> access pipe master "index" a)
j <- liftIO $ rdb $ fetch (select [] "tables")
text $ LT.pack $ show j
UPDATE 3
Thanks to cdk for giving the idea and thanks to Ivan Meredith for giving the scottyT suggestion. This question also helped: How do I add the Reader monad to Scotty's monad
This is the version that compiles. I hope it helps someone and saves some time.
import qualified Data.Text.Lazy as T
import qualified Data.Text.Lazy.Encoding as T
import Data.Text.Lazy (Text)
import Control.Monad.Reader
import Web.Scotty.Trans
import Data.Pool
import Database.MongoDB
type ScottyD = ScottyT Text (ReaderT (Pool Pipe) IO)
type ActionD = ActionT Text (ReaderT (Pool Pipe) IO)
-- Get data from config
ip = "127.0.0.1"
db = "basal"
main = do
pool <- createPool (runIOE $ connect $ host ip) close 1 300 5
let read = \r -> runReaderT r pool
scottyT 3000 read read basal
-- Application, meaddleware and routes
basal :: ScottyD ()
basal = do
get "/" shoot
-- Route action handlers
shoot :: ActionD ()
shoot = do
r <- rundb $ fetch $ select [] "computers"
html $ T.pack $ show r
-- Database access shortcut
rundb :: Action IO a -> ActionD (Either Failure a)
rundb a = do
pool <- lift ask
liftIO $ withResource pool (\pipe -> access pipe master db a)
I've been trying to figure out this exact problem myself. Thanks to hints on this SO question, and other research I've come up with the following which works for me. The key bit you were missing was to use scottyT
No doubt there is a prettier way to write runDB but I don't have much experience in Haskell, so please post it if you can do better.
type MCScottyM = ScottyT TL.Text (ReaderT (Pool Pipe) IO)
type MCActionM = ActionT TL.Text (ReaderT (Pool Pipe) IO)
main :: IO ()
main = do
pool <- createPool (runIOE $ connect $ host "127.0.0.1") close 1 300 5
scottyT 3000 (f pool) (f pool) $ app
where
f = \p -> \r -> runReaderT r p
app :: MCScottyM ()
app = do
middleware $ staticPolicy (noDots >-> addBase "public")
get "/" $ do
p <- runDB dataSources
html $ TL.pack $ show p
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = (lift ask) >>= (\p -> liftIO $ withResource p (\pipe -> access pipe master "botland" a))
dataSources :: Action IO [Document]
dataSources = rest =<< find (select [] "datasources")
Update
I guess this a bit more pretty.
runDB :: Action IO a -> MCActionM (Either Failure a)
runDB a = do
p <- lift ask
liftIO $ withResource p db
where
db pipe = access pipe master "botland" a
As you've alluded, the way to make it accessable is to wrap your computations in the Reader monad or more likely the ReaderT transformer. So your run function (changed slightly)
run :: Pool Pipe -> Action IO a -> IO (Either Failure a)
run pool act =
flip withResource (\x -> access x master db act) =<< pool
becomes
run :: Action IO a -> ReaderT (Pool Pipe) IO (Either Failure a)
run act = do
pool <- ask
withResource pool (\x -> access x master db act)
Computations inside a ReaderT r m a environment can access the r using ask and ReaderT seemingly conjures it out of thin air! In reality, the ReaderT monad is just plumbing the Env throughout the computation without you having to worry about it.
To run a ReaderT action, you use runReaderT :: ReaderT r m a -> r -> m a. So you call runReaderT on your top level scotty function to provide the Pool and runReaderT will unwrap the ReaderT environment and return you a value in the base monad.
For example, to evaluate your run function
-- remember: run act :: ReaderT (Pool Pipe) IO (Either Failure a)
runReaderT (run act) pool
but you wouldn't want to use runReaderT on run, as it is probably part of a larger computation that should also share the ReaderT environment. Try to avoid using runReaderT on "leaf" computations, you should generally call it as high up in the program logic as possible.
EDIT: The difference between Reader and ReaderT is that Reader is a monad while ReaderT is a monad transformer. That is, ReaderT adds the Reader behaviour to another monad (or monad transformer stack). If you're not familiar with monad transformers I'd recommend real world haskell - transformers.
You have showJson pool ~ ActionM () and you want to add a Reader environment with access to a Pool Pipe. In this case, you actually need ActionT and ScottyT transformers rather than ReaderT in order to work with functions from the scotty package.
Note that ActionM is defined type ActionM = ActionT Text IO, similarly for ScottyM.
I don't have all the necessary libraries installed, so this might not typecheck, but it should give you the right idea.
basal :: ScottyT Text (ReaderT (Pool Pipe) IO) ()
basal = do
middleware $ staticPolicy (...)
get "/json" showJson
showJson :: ActionT Text (ReaderT (Pool Pipe) IO) ()
showJson = do
pool <- lift ask
let run act = withResource pool (\p -> access p master "index act)
d <- liftIO $ run $ fetch $ select [] "tables"
text . TL.pack $ either (const "") show d
I want to share some data across requests in Yesod. In my case that data is a MVar (Data.Map Text ReadWriteLock), but I don't think the format of the data being shared matters too much here.
In Foundation.hs, there is a comment that says I can add fields to App, and every handler will have access to the data there. This seems like an approach I could use to share data between different handlers. I have been looking through the Yesod book, but I could not find any examples of getting data from App.
How would I access the newly created field from within a handler?
I think this might be a good use case for STM. I could share a TVar (Data.Map Text ReadWriteLock). But creating a TVar wraps the TVar in the STM monad. I might be mistaken, but to me that seems like the entire Yesod "main loop" would need to be run in the STM monad.
Is using STM a viable option here? Could anyone elaborate on how this might be achieved?
This tutorial for building a file server with Yesod shows quite nicely how you can use STM to access shared data. The relevant part starts from part 2.
To elaborate on pxqr's comment, you want to do something like this.
In your Foundation.hs file (assuming you started your project with yesod init).
data App = App
{ ... other fields
, shared :: TVar Int -- New shared TVar field
}
Then in your Application.hs file where you create the App instance.
makeFoundation conf = do
.... snip .....
tv <- newTVarIO 0 -- Initialize your TVar
let logger = Yesod.Core.Types.Logger loggerSet' getter
foundation = App conf s manager logger tv -- Add TVar here
return foundation
Then in your Handler use the TVar
getHomeR :: Handler Html
getHomeR = do
app <- getYesod -- Get the instance of App
x <- liftIO $ atomically $ do -- Read and update the TVar value.
val <- readTVar (shared app)
writeTVar (shared app) (val + 1)
return val
(formWidget, formEnctype) <- generateFormPost sampleForm
let submission = Nothing :: Maybe (FileInfo, Text)
handlerName = "getHomeR" :: Text
defaultLayout $ do
aDomId <- newIdent
-- Use the TVar value (increments on each page refresh).
setTitle $ fromString (show x)
$(widgetFile "homepage")
How to, for example, insert a new User into a database using Yesod application's models? Or is there a better way?
I am dealing with scaffolded application. Now I created App instance and dont know how to perform requests using it.
:i Extra
data Extra
= Extra {extraCopyright :: Data.Text.Internal.Text,
extraAnalytics :: Maybe Data.Text.Internal.Text}
-- Defined in `Settings
let e = Extra "asdf" Nothing
let c = AppConfig {appEnv = Development, appPort = 3000, appRoot = "/", appHost = "localhost", appExtra = e}
f <- makeFoundation c
:t f
f :: App
:i App
data App
= App {settings :: AppConfig DefaultEnv Extra,
getStatic :: Yesod.Static.Static,
connPool :: persistent-1.2.3.0:Database.Persist.Class.PersistConfig.PersistConfigPool
PersistConf,
httpManager :: http-client-0.2.0.1:Network.HTTP.Client.Types.Manager,
persistConfig :: PersistConf,
appLogger :: Yesod.Core.Types.Logger}
-- Defined in `Foundation'
What next?
If you just want to do Persistent queries in ghci you can do this without creating a Yesod application. Unfortunately doing this is differs quite a bit depending on the specific back end you want to use.
For SQLite:
> import Database.Persist.Sqlite
> import Model
> pool <- createSqlitePool "yesod-test.sqlite3" 2
> runSqlite "yesod-test.sqlite3" (runMigration migrateAll)
> userId <- runSqlite "yesod-test.sqlite3" (insert (User "foo#bar.com" Nothing))
For Postgresql:
-- In Shell: $ createdb yesod-test
> import Database.PostgreSQL.Simple
> con <- connectPostgreSQL "dbname=yesod-test"
> import Database.Persist.Postgresql
> pcon <- openSimpleConn con
> import Model
> runSqlPersistM (runMigration migrateAll) pcon
> userId <- runSqlPersistM (insert (User "foo#bar.com" Nothing)) pcon
> Just user <- runSqlPersistM (get userId) pcon
> userIdent user
The scaffolding provides (at least with yesod-bin 1.4.5) a function db in Application.hs which you can use:
$ cabal repl
...
*Application> db $ insert $ User "foo#bar.com" Nothing
It looks like the accepted answer is pretty old. Here is an update for running persistent queries in IO with the postgresql backend.
import Control.Monad.Reader (ReaderT)
import Control.Monad.Logger (LoggingT, runStdoutLoggingT)
import Database.Persist.Sql (SqlBackend, runSqlConn)
import Database.Persist.Postgresql (withPostgresqlConn)
runDBIO :: ReaderT SqlBackend (LoggingT IO) a -> IO a
runDBIO = runStdoutLoggingT . withPostgresqlConn "dbname=test-db" . runSqlConn
I want to create a Happstack application with lots of access to a database. I think that a Monad Stack with IO at the bottom and a Database Write-like monad on top (with log writer in the middle) will work to have a clear functions in each access, example:
itemsRequest :: ServerConfig -> ServerPart Response
itemsRequest cf = dir "items" $ do
methodM [GET,HEAD]
liftIO $ noticeM (scLogger cf) "sended job list"
items <- runDBMonad (scDBConnString cf) $ getItemLists
case items of
(Right xs) -> ok $ toResponse $ show xs
(Left err) -> internalServerError $ toResponse $ show err
With:
getItemList :: MyDBMonad (Error [Item])
getItemList = do
-- etc...
But I have little knowledge of Monad and Monad Transformers (I see this question as an exercise to learn about it), and I have no idea how to begin the creation of Database Monad, how to lift the IO from happstack to the Database Stack,...etc.
Here is some minimal working code compiled from snippets above for confused newbies like me.
You put stuff into AppConfig type and grab it with ask inside your response makers.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Happstack.Server
import Control.Monad.Reader
import qualified Data.ByteString.Char8 as C
myApp :: AppMonad Response
myApp = do
-- access app config. look mom, no lift!
test <- ask
-- try some happstack funs. no lift either.
rq <- askRq
bs <- lookBS "lol"
-- test IO please ignore
liftIO . print $ test
liftIO . print $ rq
liftIO . print $ bs
-- bye
ok $ toResponse ("Oh, hi!" :: C.ByteString)
-- Put your stuff here.
data AppConfig = AppConfig { appSpam :: C.ByteString
, appEggs :: [C.ByteString] } deriving (Eq, Show)
config = AppConfig "THIS. IS. SPAAAAAM!!1" []
type AppMonad = ReaderT AppConfig (ServerPartT IO)
main = simpleHTTP (nullConf {port=8001}) $ runReaderT myApp config {appEggs=["red", "gold", "green"]}
You likely want to use 'ReaderT':
type MyMonad a = ReaderT DbHandle ServerPart a
The Reader monad transformer makes a single value accessible using the ask function - in this case, the value we want everyone to get at is the database connection.
Here, DbHandle is some connection to your database.
Because 'ReaderT' is already an instance of all of the happstack-server type-classes all normal happstack-server functions will work in this monad.
You probably also want some sort of helper to open and close the database connection:
runMyMonad :: String -> MyMonad a -> ServerPart a
runMyMonad connectionString m = do
db <- liftIO $ connect_to_your_db connectionString
result <- runReaderT m db
liftIO $ close_your_db_connection db
(It might be better to use a function like 'bracket' here, but I don't know that there is such an operation for the ServerPart monad)
I don't know how you want to do logging - how do you plan to interact with your log-file? Something like:
type MyMonad a = ReaderT (DbHandle, LogHandle) ServerPart a
and then:
askDb :: MyMonad DbHandle
askDb = fst <$> ask
askLogger :: MyMonad LogHandle
askLogger = snd <$> ask
might be enough. You could then build on those primitives to make higher-level functions. You would also need to change runMyMonad to be passed in a LogHandle, whatever that is.
Once you get more than two things you want access to it pays to have a proper record type instead of a tuple.