Serving a Servant NoContent response with RIO - haskell

In my attempt to write an authenticated Servant API where handlers use the RIO monad instead of Servant's own Handler monad, I am stuck on authenticated routes that return no content; i.e., Servant's NoContent type. When I try to hoist the RIO server into the Handler using hoistServerWithContext, I get a type error that I don't grok.
Here is the simplified API and server setup:
import qualified Servant as SV
import qualified Servant.Auth.Server as AS
-- A login endpoint that sets authentication and XSRF cookies upon success.
-- Login is a credentials record.
type LoginEndpoint
= "login" :> SV.ReqBody '[SV.JSON] Login :> SV.Verb 'SV.POST 204 '[SV.JSON] CookieHeader
loginServer
:: AS.CookieSettings -> AS.JWTSettings -> SV.ServerT LoginEndpoint (RIO m)
loginServer = ... -- Perform credential check here.
-- A protected endpoint that requires cookie authentication
-- The no-content handler is causing the problem described below.
type ProtectedEndpoint = "api" :> SV.Get '[SV.JSON] Text :<|> SV.DeleteNoContent
protectedServer (AS.Authenticated _) =
return "Authenticated" :<|> return SV.NoContent
protectedServer _ = throwIO SV.err401 :<|> throwIO SV.err401
-- The overall API, with cookie authentication on the protected endpoint
type Api
= LoginEndpoint :<|> (AS.Auth '[AS.Cookie] User :> ProtectedEndpoint)
-- | The overall server for all endpoints.
server :: AS.CookieSettings -> AS.JWTSettings -> SV.ServerT Api (RIO m)
server cs jwt = loginServer cs jwt :<|> protectedServer
Where User is a record type that can be serialized as JWT as part of a cookie. To hoist the server, I follow the example here:
apiProxy :: Proxy Api
apiProxy = Proxy
contextProxy :: Proxy '[AS.CookieSettings, AS.JWTSettings]
contextProxy = Proxy
newtype Env = Env
{ config :: Text }
-- Helper function to hoist our RIO handler into a Servant Handler.
hoistAppServer :: AS.CookieSettings -> AS.JWTSettings -> Env -> SV.Server Api
hoistAppServer cookieSettings jwtSettings env = SV.hoistServerWithContext
apiProxy
contextProxy
(nt env)
(server cookieSettings jwtSettings)
where
-- Natural transformation to map the RIO monad stack to Servant's Handler.
nt :: Env -> RIO Env a -> SV.Handler a
nt e m = SV.Handler $ ExceptT $ try $ runRIO e m
main :: IO ()
main = do
myKey <- AS.generateKey -- Key for encrypting the JWT.
let jwtCfg = AS.defaultJWTSettings myKey
cfg = cookieConf :. jwtCfg :. SV.EmptyContext -- cookieConf sets XSRF handling
env = Env { config = "Some configuration string" }
Warp.run 8081 $ SV.serveWithContext apiProxy cfg $ hoistAppServer cookieConf jwtCfg env
The above hoisting works fine for endpoints that return some content. However, when :<|> SV.DeleteNoContent is present in the ProtectedEndpoint (and the corresponding parts in the server), I get the following type error:
No instance for (HasServer
(Servant.Auth.Server.Internal.AddSetCookie.AddSetCookieApi
(Servant.Auth.Server.Internal.AddSetCookie.AddSetCookieApi
(NoContentVerb 'DELETE)))
'[CookieSettings, JWTSettings])
arising from a use of ‘hoistServerWithContext’
The problem does not arise on an endpoint without authentication; e.g., UnprotectedEndpoint instead of (AS.Auth '[AS.Cookie] User :> ProtectedEndpoint) in the API type definition.
hoistServerWithContext is a function of the HasServer type class, but I'm not sure which instance is of concern here. If I let GHC infer the type, I get
hoistServerWithContext :: forall k (api :: k) (context :: [*]) (m :: * -> *) (n :: * -> *).
HasServer api context =>
Proxy api
-> Proxy context
-> (forall x. m x -> n x)
-> ServerT api m
-> ServerT api n
To me, the type error (plus my experiments adding and removing the no-content handler) indicate that the protectedServer derived by Servant's type machinery is not a member of the HasServer type class. But my Haskell type-level programming skills are not up to the task, it seems. Where exactly is the problem? Am I missing a type annotation? A language extension?

The type error seems to result because servant currently does not allow adding headers to a NoContentVerb because the corresponding type instance is missing. See the Servant-Auth issue here.
Even though I don't fully understand the details, the following workaround from the above issue comment avoids the type error:
type instance ASC.AddSetCookieApi (SV.NoContentVerb 'SV.DELETE)
= SV.Verb 'SV.DELETE 204 '[SV.JSON] (ASC.AddSetCookieApiVerb SV.NoContent)

Related

How to get GHC to apply my orphaned instances `HasServer` and `HasClient` for `AuthProtect`?

I use the AuthProtect combinator from servant: Servant.API.Experimental.Auth. There is not much code there, the instance HasServer (AuthProtect tag) is in servant-server and the insance HasClient (AuthProtect tag) in whatever servant client you use.
I use servant-snap instead of servant-server and a custom HasClient implementation for an obelisk project, with a project structure that consists of three cabal packages:
frontend (compiled by ghcjs)
common (compiled by ghcjs and ghc)
backend (compiled by ghc)
I used to have a custom implementation of AuthProtect along with the instances in the common package. However, common can neither depend on servant-snap nor on snap-core because of ghcjs.
Now I moved the HasServer instance to the backend ... no problem, right? Wrong. Once the HasServer instance is orphaned, ghc does not correctly resolve my api type anymore. It's just as if the orphaned instance were not there at all.
Why is that?
What is there, I can do?
Instances are global... in theory. In practice, to support separate compilation, they propagate via imports. So import the module that defines the instance from the module that uses it. Importing it transitively is good enough -- i.e. importing a module that imports a module that imports the module that defines the instance.
Either of those solves my problem:
instance HasServer api context m => HasServer (AuthProtect "jwt" :> api) context m where
type ServerT (AuthProtect "jwt" :> api) context m =
String -> ServerT api context m
hoistServerWithContext _ pc nt s = hoistServerWithContext (Proxy :: Proxy api) pc nt . s
route (Proxy :: Proxy (AuthProtect "jwt" :> api)) context subserver =
route (Proxy :: Proxy api) context (subserver `addAuthCheck` withRequest authCheck)
where
authCheck :: Request -> DelayedM m String
authCheck =
liftIO . evalSnap (pure "account info")
(\x -> pure $! (x `seq` ()))
(\f -> let !_ = f 0 in pure ())
If I don't want specialize to AuthProtect "jwt" for some reason, I have to provide the constraint KnownSymbol tag.
instance (KnownSymbol tag, HasServer api context m) => HasServer (AuthProtect tag :> api) context m where
type ServerT (AuthProtect tag :> api) context m =
String -> ServerT api context m
hoistServerWithContext _ pc nt s = hoistServerWithContext (Proxy :: Proxy api) pc nt . s
route (Proxy :: Proxy (AuthProtect tag :> api)) context subserver =
route (Proxy :: Proxy api) context (subserver `addAuthCheck` withRequest authCheck)
where
authCheck :: Request -> DelayedM m String
authCheck =
liftIO . evalSnap (pure "account info")
(\x -> pure $! (x `seq` ()))
(\f -> let !_ = f 0 in pure ())

Should I use ReaderT to pass a database connection pool around in Servant?

I am building a web API with Servant and Persistent. I plan to define some API endpoints (about 15) that use a connection pool to access the DB.
For example, one of the endpoint definitions (Handlers) is:
getUser :: ConnectionPool -> Int -> Handler User
getUser pool uid = do
user <- inPool pool $ get (toId #User uid)
user & orErr err404 {errBody = "This user does not exist."}
where inPool is just a lifted withResource function, and orErr is a lifted fromMaybe.
Then, a higher level API definition (Servers) looks like this:
type Point (s :: Symbol) (a :: *) =
s :> Capture "id" Int :>
( Get '[JSON] a
:<|> ReqBody '[JSON] a :> Post '[JSON] NoContent
)
type UserPoint = Point "users" User
userServer :: ConnectionPool -> Server UserPoint
userServer pool uid =
getUser pool uid :<|>
postUser pool uid
And I defined the main to be:
main = runStdoutLoggingT . withPostgresqlPool connectionString numConnections $ \pool -> do
withResource pool (runSqlConn $ runMigration migrateAll)
liftIO $ run appPort (userServer pool)
But I soon noticed that I would have to pass the pool down layer by layer (In the example above there are 2 layers, and in my real project there are 3), to every function (that is over 20). My intuition tells me this is bad smell, but I am not quite sure.
Then I thought of ReaderT, because I think that may abstract the pool out. But my concern is that the introduction of ReaderT may lead to unnecessary complexity:
I need to lift many things manually;
The mental model of types will become more complicated thus harder to think about;
It means I'll have to give up the Handler type, which makes using Servant harder too.
I am not sure whether I should use ReaderT in this case. Please offer some suggestions (I'll be grateful if you could also provide some guidelines about when to use ReaderT or even other monad transformers).
UPDATE: I found that I can use where-clauses to simplify this a lot, and this basically solves my problem. but I'm not sure if this is best practice, so I'm still looking forwand to an answer.
userServer :: Pooled (Server UserPoint)
userServer pool auth = c :<|> rud where
c :: UserCreation -> Handler NoContent
c = undefined
rud uid = r :<|> u :<|> d where
r :: Handler User
r = do
checkAuth pool auth
user <- inPool pool $ get (toId #User uid)
user & orErr err404 {errBody = "This user does not exist."}
u :: User -> Handler NoContent
u = undefined
d :: Handler NoContent
d = undefined
While defining your handlers along with your server will avoid you the parameter-passing, as the server grows in complexity you might want to define some handlers separately:
Perhaps some handler provides some generic functionality and could be useful in other servers.
Defining everything together means everything is aware of everything else.
Moving handlers to the top level, or even to another module, will
help make explicit which parts of the whole they really need to know.
And this can make the handler easier to understand.
Once we separate a handler, supplying it with the environment will become necessary. This can be done with plain parameters to functions, or with a ReaderT. As the number of parameters grows, the ReaderT (often in combination with auxiliary HasX typeclasses) becomes more attractive because it frees you from having to care about parameter order.
I would have to pass the pool down layer by layer (In the example
above there are 2 layers, and in my real project there are 3), to
every function
Besides the extra (possibly inevitable) burden of having to pass parameters, I think there's a potentially worse problem lurking: you are threading a low-level detail (the connection pool) through several layers of functions. This can be bad because:
You are committing your whole application to using an actual database. What happens if, during testing, you want switch it with some kind of in-memory repository?
If you need to change the way you do persistence, the refactor will reverberate through all the layers of your application, instead of remaining localized.
One possible solution for these problems: the functions at layer N+1 should not receive as parameter the connection pool, but rather the functions they use from layer N. And those functions from layer N will already have been partially applied with the connection pool.
A trivial example: if you have some high level logic transferUser :: Conn -> Handle -> IO () that includes hardwired calls to functions readUserFromDb :: Conn -> IO User and writeUserToFile :: Handle -> User -> IO (), change it into a transferUser :: IO User -> (User -> IO) -> IO ().
Notice that the auxiliary functions from level N could be stored in the ReaderT context; the functions from level N+1 could get them from there.
It means I'll have to give up the Handler type, which makes using
Servant harder too.
You can define your server using a ReaderT transformer over Handler, and then pass it to the hoistServer function which will "whittle it down" to a runnable server:
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
import Servant
import Servant.API
import Control.Monad.Trans.Reader
type UserAPI1 = "users" :> Capture "foo" Int :> Get '[JSON] Int
data Env = Env
-- also valid type
-- server1 :: Int -> ReaderT Env Handler Int
server1 :: ServerT UserAPI1 (ReaderT Env Handler)
server1 =
\ param ->
do _ <- ask
return param
-- also valid types:
-- server2 :: ServerT UserAPI1 Handler
-- server2 :: Int -> Handler Int
server2 :: Server UserAPI1
server2 = hoistServer (Proxy :: Proxy UserAPI1) (flip runReaderT Env) server1

Using Servant.Generic routes with ReaderT (Pool Connection) IO

I was using servant-generic-0.1.0.3 and servant-server-0.13.0.1 to do the following:
data Site route = Site
{ page :: route :-
"page" :> Capture "x" Int :> Capture "y" Int :> Get '[JSON] [Int]
, home :: route :-
Raw
} deriving (Generic)
type API = ToServant (Site AsApi)
siteServer :: Pool Connection -> Site AsServer
siteServer pool = Site
{ page = \x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
api :: Proxy API
api = Proxy
app :: Pool Connection -> Application
app pool = serve api (toServant $ siteServer pool)
That worked fine, then I tried to use ReaderT to avoid passing Pool Connection to siteServer, so I added AppM and replaced siteServer like this:
type AppM = ReaderT (Pool Connection) IO
siteServer :: ServerT API AppM
siteServer = Site
{ page = do
pool <- ask
\x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
but I got a bunch of errors when I tried to compile it.
I followed the same steps shown in the servant cookbook, but I couldn't make this work with generic routes, although it works when using regular routes.
Am I missing something that could make this work?
At least for the record-style routes supported by servant-* >= 0.14 (see here), if you want to work with another monad than Handler, you will want to look at AsServerT and genericServerT.
Applied to your example, this means siteServer should be defined as follows (not typechecked, but should be very close to correct).
siteServer :: Site (AsServerT AppM)
siteServer = Site
{ page = ... something in AppM ...
, home = ... something in AppM ...
}
-- turning that into a usual chain of :<|>-separated handlers
oldStyleServer :: ServerT API AppM
oldStyleServer = genericServerT siteServer
-- bringing it all back in Handler
oldStyleServerInHandler :: Pool Connection -> Server API -- same as ServerT API Handler
oldStyleServerInHandler conns = hoistServer (Proxy #API) appToHandler oldStyleServer
where appToHandler = liftIO . flip runReaderT conns
-- or something along those lines
-- serving it
app :: Pool Connection -> Application
app conns = serve (Proxy #API) (oldStyleServerInHandler conns)
Edit: Since you're using servant-* < 0.14 with servant-generic, you should replace genericServerT with toServant.

Servant client with type variable with FromJSON contraint

I'm trying to create client bindings to Web API using servant library.
I want to be able to send any JSON objects.
import Control.Monad.Trans.Except (ExceptT, runExceptT)
import Data.Proxy
import Network.HTTP.Client (Manager)
import Servant.API
import Servant.Client
import Data.Aeson
-- | This methods accepts any instance of 'ToJSON'
-- I would like to have only this method exported from the module
send :: ToJSON a => a -> Manager -> IO (Either ServantError Result)
send x manager = runExceptT $ send_ x manager baseUrl
type MyAPI a = "acceptAnyJson"
:> ReqBody '[JSON] a
:> Post '[JSON] Result
api :: ToJSON a => Proxy (MyAPI a)
api = Proxy
send_ :: ToJSON a => a -> Manager -> BaseUrl -> ExceptT ServantError IO Result
send_ = client api
Right now when I try to compile it I have error message:
Couldn't match type ‘a0’ with ‘a’
because type variable ‘a’ would escape its scope
This (rigid, skolem) type variable is bound by
the inferred type for ‘send_’:
...
How can I parameterize my MyAPI, client and Proxy to accept type variable?
You'll need to tie the type of api to the type of the thing you're sending:
{-# LANGUAGE ScopedTypeVariables #-}
send_ :: forall a. (FromJSON a) => a -> Manager -> BaseUrl -> ExceptT ServantError IO Result
send_ = client (api :: Proxy (MyAPI a))
or why even bother with api at that point:
send_ = client (Proxy :: Proxy (MyAPI a))

Serving Static Files With Servant / Wai

I am following this tutorial http://www.parsonsmatt.org/programming/2015/06/07/servant-persistent.html to create APIs through servant. I want to customize the server to serve static files as well but couldn't find a way to do it.
I am using the stack build tool.
I modified the Main.hs file's run to include static (run port $ static $ logger $ app cfg) and I imported Network.Wai.Middleware.Static (static). I also added wai-middleware-static >=0.7.0 && < 0.71 to my cabal file.
When I run stack build I get: (Update: This part is totally my error. I added the the package to the wrong cabal file.. lame. Importing Network.Wai.Middleware.Static works and serves static files. Leaving the error below in case anyone searches for it and finds it useful.)
Could not find module ‘Network.Wai.Middleware.Static’
Perhaps you meant
Network.Wai.Middleware.Gzip (from wai-extra-3.0.7.1#waiex_GpotceEdscHD6hq9p0wPOJ)
Network.Wai.Middleware.Jsonp (from wai-extra-3.0.7.1#waiex_GpotceEdscHD6hq9p0wPOJ)
Network.Wai.Middleware.Local (from wai-extra-3.0.7.1#waiex_GpotceEdscHD6hq9p0wPOJ)
Next I tried using servant's serveDirectory as follows (simplified):
type API = "users" :> Get '[JSON] [Person]
:<|> "static" :> Raw
server = createPerson :<|> serveDirectory "/static"
I get this error:
Couldn't match type ‘IO’ with ‘EitherT ServantErr IO’
arising from a functional dependency between:
constraint ‘Servant.Server.Internal.Enter.Enter
(IO Network.Wai.Internal.ResponseReceived)
(AppM :~> EitherT ServantErr IO)
(IO Network.Wai.Internal.ResponseReceived)’
arising from a use of ‘enter’
instance ‘Servant.Server.Internal.Enter.Enter
(m a) (m :~> n) (n a)’
at <no location info>
In the expression: enter (readerToEither cfg) server
In an equation for ‘readerServer’:
readerServer cfg = enter (readerToEither cfg) server
I am a Haskell beginner and I am not familiar with Wai so unsure where to even begin. What changes do I need to make the example code in the Blog post to serve static files?
Edit: Since the comments get hidden from the default view, I am pasting my last comment here:
Here is toned down version of Matt's code from his blog. I consolidated all his modules into a single file, removed all the database stuff but did not clean up the extensions/imports. When I run this code I get the above type mismatch error. Please note that this code does not use Network.Wai.Middleware.Static and I am using qualified import of Servant StaticFiles.
As described in the relevant section of servant's tutorial, the whole deal with enter is to have your request handlers use some monad m (in your case some ReaderT monad) and to provide a way to convert a computation in m to a computation in servant's standard EitherT ServantErr IO monad.
The problem here though is that you define a bunch of request handlers in ReaderT and an additional one to serve static files, and call enter on all of these. The ReaderT handlers are converted to EitherT ... handlers just fine, but enter tries to convert the serveDirectory call from ReaderT ... to EitherT .... This is of course not going to happen anytime soon, since serveDirectory isn't a computation in ReaderT ... to begin with!
servant could arguably just leave serveDirectory alone -- at this point I don't have a definite opinion on whether we should do that or not, or if it's better to just have the file-serving handler be glued separately, to the result of calling enter on all the other endpoints. Here's how this would look like (look for -- NEW to see the changes):
type PersonAPI =
"users" :> Capture "name" String :> Get '[JSON] Person
-- NEW: removed Raw from here
-- NEW
type WholeAPI = PersonAPI :<|> Raw
type AppM = ReaderT Config (EitherT ServantErr IO)
userAPI :: Proxy PersonAPI
userAPI = Proxy
-- NEW
wholeAPI :: Proxy WholeAPI
wholeAPI = Proxy
-- NEW: changed 'userAPI' to 'wholeAPI'
app :: Config -> Application
app cfg = serve wholeAPI (readerServer cfg)
readerServer :: Config -> Server WholeAPI
readerServer cfg = enter (readerToEither cfg) server
:<|> S.serveDirectory "/static" -- NEW
readerToEither :: Config -> AppM :~> EitherT ServantErr IO
readerToEither cfg = Nat $ \x -> runReaderT x cfg
server :: ServerT PersonAPI AppM
server = singlePerson
singlePerson :: String -> AppM Person
singlePerson str = do
let person = Person { name = "Joe", email = "joe#example.com" }
return person
I have brought this topic to the attention of the other servant developers anyway, thanks! We hadn't really thought about the interaction between enter and serveDirectory so far (well, I did not).

Resources