Partial reverse proxy with Haskell Servant - haskell

I'm trying to build a web server in Haskell with Servant where part of the api works as a reverse proxy to another api.
I found an example of how to achieve this. But it seems that it doesn't work:
type API
= "cat" :> Get '[JSON] Cat
newtype Cat = Cat { cat :: String }
instance ToJSON Cat where
toJSON (Cat mew) =
object [ "cat" .= mew ]
server :: Server API
server = pure (Cat { cat = "mrowl" })
api :: Proxy (API :<|> Raw)
api = Proxy
app :: Manager -> Application
app manager =
serve api $ server :<|> waiProxyTo forwardRequest defaultOnExc manager
forwardRequest :: Request -> IO WaiProxyResponse
forwardRequest _ =
pure . WPRProxyDest . ProxyDest "127.0.0.1" $ 4567
startApp :: IO ()
startApp = do
manager <- newManager defaultManagerSettings
run 8080 (app manager)
It gives the following type error (when I try it in my own code base):
• Couldn't match type ‘Request
-> (Response -> IO ResponseReceived) -> IO ResponseReceived’
with ‘Tagged Handler Application’
Expected type: Server (API :<|> Raw)
Actual type: Handler Cat :<|> Application
• In the second argument of ‘($)’, namely
‘server :<|> waiProxyTo forwardRequest defaultOnExc manager’
In the expression:
serve api
$ server :<|> waiProxyTo forwardRequest defaultOnExc manager
In an equation for ‘app’:
app manager
= serve api
$ server :<|> waiProxyTo forwardRequest defaultOnExc manager
|
32 | serve api $ server :<|> waiProxyTo forwardRequest defaultOnExc manager
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(My interpretation is that :<|> has changed to not accept combing Server and Application since the example was written.)
What can I replace waiProxyTo forwardRequest defaultOnExc manager with to make this work?

I don't fully understand why this works but I got it working by mimicking how serveDirectoryWith works:
import Servant.Server (ServerT, Tagged)
import Network.HTTP.Client (Manager)
import Network.HTTP.ReverseProxy
( WaiProxyResponse, WaiProxyResponse(WPRProxyDest)
, ProxyDest(ProxyDest), waiProxyTo, defaultOnExc)
{- ... -}
forwardServer :: Manager -> ServerT Raw m
forwardServer manager =
Tagged $ waiProxyTo forwardRequest defaultOnExc manager
forwardRequest :: Request -> IO WaiProxyResponse
forwardRequest _ =
pure . WPRProxyDest . ProxyDest "127.0.0.1" $ 4567
app :: Manager -> Application
app manager =
serve api $ server :<|> (forwardServer manager)

Related

Can one reverse-proxy a small external site as a yesod subsite

I have a yesod site serving content that includes rest WaiSubsites. all working. I have a need to serve a small REACT based site and I'd like to use the same basic infrastructure to add a simple reverse proxying subset would embed that REACT server. I'm aware that I could put nginx etc. in front but I don't want to do that.
Is there an easy solution?
thanks,
Stephen.
update. So I've built a basic solution, but I'd like to be able to map random /x/y to /y and so I need to rewrite the response content URLs of form /y to /x/y. having trouble workin out the process response handler. anyone?
makeExternalServerProxy :: Manager -> ByteString -> Int -> Application
makeExternalServerProxy manager host port =
simpleCors $ S.serveWithContext (S.Proxy :: S.Proxy S.Raw) S.EmptyContext forwardH
where -- | proxy based forwarding of non API requests to host:port
forwardH :: S.Tagged S.Handler Application
forwardH = S.Tagged $ waiProxyToSettings
forwardRequest proxySettings manager
forwardRequest :: Network.Wai.Request -> IO WaiProxyResponse
forwardRequest req =
pure $ WPRModifiedRequest
(rewriteRequestPure (\(pq,q) _ -> ( case pq of
[] -> []
(_:xs) -> xs
, q)) req)
(ProxyDest host port)
-- | Sends a simple 402 error message with the contents of the exception.
handleException :: SomeException -> Application
handleException exc _ sr = sr $ responseLBS
HT.status402
[("content-type", "text/plain")]
("We're sorry: Internal Error:\n\n" <>
TLE.encodeUtf8 (pack $ show exc))
proxySettings = defaultWaiProxySettings
{ wpsOnExc = handleException
, wpsProcessBody = processResponse
}
processResponse :: Network.Wai.Request -> Response () -> Maybe (ConduitT ByteString (Flush Builder) IO ())
processResponse = undefined

Serving a Servant NoContent response with RIO

In my attempt to write an authenticated Servant API where handlers use the RIO monad instead of Servant's own Handler monad, I am stuck on authenticated routes that return no content; i.e., Servant's NoContent type. When I try to hoist the RIO server into the Handler using hoistServerWithContext, I get a type error that I don't grok.
Here is the simplified API and server setup:
import qualified Servant as SV
import qualified Servant.Auth.Server as AS
-- A login endpoint that sets authentication and XSRF cookies upon success.
-- Login is a credentials record.
type LoginEndpoint
= "login" :> SV.ReqBody '[SV.JSON] Login :> SV.Verb 'SV.POST 204 '[SV.JSON] CookieHeader
loginServer
:: AS.CookieSettings -> AS.JWTSettings -> SV.ServerT LoginEndpoint (RIO m)
loginServer = ... -- Perform credential check here.
-- A protected endpoint that requires cookie authentication
-- The no-content handler is causing the problem described below.
type ProtectedEndpoint = "api" :> SV.Get '[SV.JSON] Text :<|> SV.DeleteNoContent
protectedServer (AS.Authenticated _) =
return "Authenticated" :<|> return SV.NoContent
protectedServer _ = throwIO SV.err401 :<|> throwIO SV.err401
-- The overall API, with cookie authentication on the protected endpoint
type Api
= LoginEndpoint :<|> (AS.Auth '[AS.Cookie] User :> ProtectedEndpoint)
-- | The overall server for all endpoints.
server :: AS.CookieSettings -> AS.JWTSettings -> SV.ServerT Api (RIO m)
server cs jwt = loginServer cs jwt :<|> protectedServer
Where User is a record type that can be serialized as JWT as part of a cookie. To hoist the server, I follow the example here:
apiProxy :: Proxy Api
apiProxy = Proxy
contextProxy :: Proxy '[AS.CookieSettings, AS.JWTSettings]
contextProxy = Proxy
newtype Env = Env
{ config :: Text }
-- Helper function to hoist our RIO handler into a Servant Handler.
hoistAppServer :: AS.CookieSettings -> AS.JWTSettings -> Env -> SV.Server Api
hoistAppServer cookieSettings jwtSettings env = SV.hoistServerWithContext
apiProxy
contextProxy
(nt env)
(server cookieSettings jwtSettings)
where
-- Natural transformation to map the RIO monad stack to Servant's Handler.
nt :: Env -> RIO Env a -> SV.Handler a
nt e m = SV.Handler $ ExceptT $ try $ runRIO e m
main :: IO ()
main = do
myKey <- AS.generateKey -- Key for encrypting the JWT.
let jwtCfg = AS.defaultJWTSettings myKey
cfg = cookieConf :. jwtCfg :. SV.EmptyContext -- cookieConf sets XSRF handling
env = Env { config = "Some configuration string" }
Warp.run 8081 $ SV.serveWithContext apiProxy cfg $ hoistAppServer cookieConf jwtCfg env
The above hoisting works fine for endpoints that return some content. However, when :<|> SV.DeleteNoContent is present in the ProtectedEndpoint (and the corresponding parts in the server), I get the following type error:
No instance for (HasServer
(Servant.Auth.Server.Internal.AddSetCookie.AddSetCookieApi
(Servant.Auth.Server.Internal.AddSetCookie.AddSetCookieApi
(NoContentVerb 'DELETE)))
'[CookieSettings, JWTSettings])
arising from a use of ‘hoistServerWithContext’
The problem does not arise on an endpoint without authentication; e.g., UnprotectedEndpoint instead of (AS.Auth '[AS.Cookie] User :> ProtectedEndpoint) in the API type definition.
hoistServerWithContext is a function of the HasServer type class, but I'm not sure which instance is of concern here. If I let GHC infer the type, I get
hoistServerWithContext :: forall k (api :: k) (context :: [*]) (m :: * -> *) (n :: * -> *).
HasServer api context =>
Proxy api
-> Proxy context
-> (forall x. m x -> n x)
-> ServerT api m
-> ServerT api n
To me, the type error (plus my experiments adding and removing the no-content handler) indicate that the protectedServer derived by Servant's type machinery is not a member of the HasServer type class. But my Haskell type-level programming skills are not up to the task, it seems. Where exactly is the problem? Am I missing a type annotation? A language extension?
The type error seems to result because servant currently does not allow adding headers to a NoContentVerb because the corresponding type instance is missing. See the Servant-Auth issue here.
Even though I don't fully understand the details, the following workaround from the above issue comment avoids the type error:
type instance ASC.AddSetCookieApi (SV.NoContentVerb 'SV.DELETE)
= SV.Verb 'SV.DELETE 204 '[SV.JSON] (ASC.AddSetCookieApiVerb SV.NoContent)

Using Servant.Generic routes with ReaderT (Pool Connection) IO

I was using servant-generic-0.1.0.3 and servant-server-0.13.0.1 to do the following:
data Site route = Site
{ page :: route :-
"page" :> Capture "x" Int :> Capture "y" Int :> Get '[JSON] [Int]
, home :: route :-
Raw
} deriving (Generic)
type API = ToServant (Site AsApi)
siteServer :: Pool Connection -> Site AsServer
siteServer pool = Site
{ page = \x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
api :: Proxy API
api = Proxy
app :: Pool Connection -> Application
app pool = serve api (toServant $ siteServer pool)
That worked fine, then I tried to use ReaderT to avoid passing Pool Connection to siteServer, so I added AppM and replaced siteServer like this:
type AppM = ReaderT (Pool Connection) IO
siteServer :: ServerT API AppM
siteServer = Site
{ page = do
pool <- ask
\x y ->
liftIO $ withResource pool $ \conn -> someDbFunction conn x y
, home = serveDirectoryWebApp "static"
}
but I got a bunch of errors when I tried to compile it.
I followed the same steps shown in the servant cookbook, but I couldn't make this work with generic routes, although it works when using regular routes.
Am I missing something that could make this work?
At least for the record-style routes supported by servant-* >= 0.14 (see here), if you want to work with another monad than Handler, you will want to look at AsServerT and genericServerT.
Applied to your example, this means siteServer should be defined as follows (not typechecked, but should be very close to correct).
siteServer :: Site (AsServerT AppM)
siteServer = Site
{ page = ... something in AppM ...
, home = ... something in AppM ...
}
-- turning that into a usual chain of :<|>-separated handlers
oldStyleServer :: ServerT API AppM
oldStyleServer = genericServerT siteServer
-- bringing it all back in Handler
oldStyleServerInHandler :: Pool Connection -> Server API -- same as ServerT API Handler
oldStyleServerInHandler conns = hoistServer (Proxy #API) appToHandler oldStyleServer
where appToHandler = liftIO . flip runReaderT conns
-- or something along those lines
-- serving it
app :: Pool Connection -> Application
app conns = serve (Proxy #API) (oldStyleServerInHandler conns)
Edit: Since you're using servant-* < 0.14 with servant-generic, you should replace genericServerT with toServant.

Catch-all or default routing

These days it's not uncommon to need to return a file (say, index.html) from the backend if the requested route doesn't match an existing API endpoint or another static asset. This is especially handy when using react-router and browserHistory.
I'm a bit stumped as to how I might approach this with Servant. I did wonder if intercepting 404's might be the way to go, but then of course sometimes the API will need to legitimately issue 404. Here's the kind of thing I've been using to experiment:
data Wombat = Wombat
{ id :: Int
, name :: String
} deriving (Eq, Show, Generic)
instance ToJSON Wombat
wombatStore :: [Wombat]
wombatStore =
[ Wombat 0 "Gertrude"
, Wombat 1 "Horace"
, Wombat 2 "Maisie"
, Wombat 3 "Julius"
]
wombats :: Handler [Wombat]
wombats = return wombatStore
wombat :: Int -> Handler Wombat
wombat wid = do
case find (\w -> Main.id w == wid) wombatStore of
Just x -> return x
Nothing -> throwE err404
type API =
"api" :> "wombats" :> Get '[JSON] [Wombat] :<|>
"api" :> "wombats" :> Capture "id" Int :> Get '[JSON] Wombat :<|>
Raw
api :: Proxy API
api = Proxy
server :: Server API
server = wombats
:<|> wombat
:<|> serveDirectory "static"
app :: Application
app = serve api server
main :: IO ()
main = run 3000 app
I'd love to see an example of how I could go about adding a 'default route' that sends an HTML response if the request doesn't match an API endpoint or anything in the static directory. Toy repo here.
You got it, basically. serveDirectory "static" can be replaced by any wai Application, so for instance, we could have:
...
{-# LANGUAGE OverloadedStrings #-}
...
import Network.Wai (responseLBS)
import Network.HTTP.Types (status200)
...
server :: Server API
server = wombats
:<|> wombat
:<|> hello
hello :: Application
hello req respond = respond $
responseLBS
status200 --
[("Content-Type", "text/plain")] -- headers
"Hello, World!" -- content
...
To a first approximation, wai applications are simply Request -> Response, but the docs tell a fuller story:
Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
So since you've got access to IO, you can check if the file exists and if so serve it, otherwise do whatever you like. In fact, wai defines type Middleware = Application -> Application, so you might think up a handy Middleware that wraps up hello (or any other Application!) in a file existence-checker-and-server.
Here is another route:
serveDirectory is defined as
serveDirectory = staticApp . defaultFileServerSettings . addTrailingPathSeparator
defaultFileServerSettings contains a field ssLookupFile which you can change to serve what you want if the file was not found. Perhaps:
import WaiAppStatic.Types
import WaiAppStatic.Storage.Filesystem
import Network.Wai.Application.Static
import System.FilePath
fileOrIndex root pieces = do
res <- ssLookupFile (defaultFileServerSettings root) pieces
case res of
LRNotFound -> undefined -- index.html here
_ -> return res
serveStatic root =
let root' = addTrailingPathSeparator root in
staticApp $ (defaultFileServerSettings root') {ssLookupFile = fileOrIndex root'}

IO monad issues when trying to implement WAI HTTP Server + Fallback Proxy

What I'm trying to do is to create a somewhat smart reverse proxy server that should process some requests on its own and forward the others to the backend of choice. To make it challenging I'm trying hard to do it in Haskell, which I am a total newbie in.
Here's the code I've come up so far:
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import Data.ByteString
import Network.HTTP.ReverseProxy
import Network.HTTP.Types
import Network.Wai
import Network.Wai.Handler.Warp
import Network.Wai.Middleware.RequestLogger
import qualified Network.HTTP.Client as HC
helloApp :: Application
helloApp req respond =
respond $ responseLBS status200 [("Content-Type", "text/plain")] "Hello"
proxyStubApp :: Application
proxyStubApp req respond =
respond $ responseLBS status200 [("Content-Type", "text/plain")] "You've hit the stub"
proxyApp :: IO Application
proxyApp = do
manager <- HC.newManager HC.defaultManagerSettings
return $ waiProxyTo (const $ return $ WPRProxyDest ProxyDest { pdHost = "localhost", pdPort = 9393 }) defaultOnExc manager
app :: Application
app req respond =
serve req respond
where serve = lookupServeFunction req
lookupServeFunction :: Request -> Application
lookupServeFunction req
| isInfixOf "sample_path" (rawPathInfo req) = proxyStubApp
| otherwise = helloApp
main = run 3011 =<< (logStdoutDev <$> return app)
It works fine, but when I exchange proxyStubApp for actual proxyApp I am forced to add IO all over the place. Particularly it gets added to app, consequently leaving me with the following compilation error message:
Couldn't match expected type ‘Request -> t5 -> t4’
with actual type ‘IO Application’
The equation(s) for ‘app’ have two arguments,
but its type ‘IO Application’ has none
I feel like I understand why it is happening, but I'm out of ideas of how to cope with it :( Or am I doing something totally wrong?
Thank you!
P.S. Here are the dependencies should you want to compile the thing on your own: wai warp http-types text bytestring wai-extra time http-reverse-proxy http-client
The IO in IO Application is kind-of redundant. Note that
type Application = Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
so, expanding proxyApp's arguments (what you already do in proxyStubApp), you get
proxyApp :: Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
proxyApp req continuation = do
manager <- HC.newManager HC.defaultManagerSettings
waiProxyTo (...) req respond
That works, because in either case
proxyApp :: IO Application
proxyApp = do
manager <- HC.newManager ...
...
and
proxyApp :: Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
proxyApp req continuation = do
manager <- HC.newManager ...
...
the IO action HC.newManager ... is "run within IO".
You may find it conceptually clearer to construct an Application in IO and hand it to some other place, and I won't argue with you. I want to note though, that you choose the Application based on the Request, so in a way you are in the hypothetical HTTP monad when choosing, so lookupServeFunction's signature Request -> Application makes more sense to me.
If you want to keep that type signature for proxyApp,
lookupServeFunction and app will have to be in IO as well and main will have to change accordingly, e.g.
myApp <- app
...
As haoformayor said, It is generally easier to work without the outer IO layer.
You might also like to simplify main.
fmap logStdoutDev (return app)
is the same as
return (logStdoutDev app)
and
run 3011 =<< return (logStdoutDev app)
is the same as
run 3011 (logStdoutDev app)
You might want to install hlint, which will help you spot these.

Resources