I use Servant. I need to set some callback function that will be executed on every request and it will forward the request processing/handling further (to other handler, as if this callback did not exist at all). I need it to change some IORef MyGlobalState in this callback. How to do it? Some example? Sure, I can call it explicitly in all handlers' bodies but maybe there is some right/canonical way to do it...
Consider this example server:
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
import Servant
import Servant.API
import Network.Wai
import Network.Wai.Handler.Warp
import Data.IORef
import qualified Data.ByteString.Char8 as C
type API = "one" :> Get '[PlainText] String :<|> "two" :> Get '[PlainText] String
api :: Proxy API
api = Proxy
server :: Server API
server = return "1\n" :<|> return "2\n"
app :: Application
app = serve api server
main = run 3000 app
The app value is a WAI Application, defined by:
type Application = Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
Conceptually, an Application accepts a Request, and invokes a callback on its Response. WAI supports the concept of Middleware which can wrap an application, allowing preprocessing of every request and post-processing of every response in the IO monad:
type Middleware = Application -> Application
So, you can write a piece of middleware to update an IORef on every received request like so:
counter :: IORef Int -> Middleware
counter cref = convert
where
convert :: Application -> Application -- AKA Middleware
convert oldapp = newapp
where
newapp :: Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived -- AKA Application
newapp req respond = do
n <- atomicModifyIORef cref (\n' -> (n'+1,n'+1))
putStrLn $ "Request #" ++ show n ++ ": " ++ showRequest req
oldapp req respond
showRequest req = C.unpack (requestMethod req) ++ " " ++ C.unpack (rawPathInfo req)
I've broken this up into separate functions to make it clear how the middleware is constructed piece by piece, but the definition of counter can be simplified to the equivalent:
counter :: IORef Int -> Middleware
counter cref oldapp req respond
= do n <- atomicModifyIORef cref (\n' -> (n'+1,n'+1))
putStrLn $ "Request #" ++ show n ++ ": " ++ showRequest req
oldapp req respond
where
showRequest req = C.unpack (requestMethod req) ++ " " ++ C.unpack (rawPathInfo req)
Now, all you need to do is wrap your app with the middleware in main:
main = do
cref <- newIORef (0 :: Int)
run 3000 $ counter cref app
Full code:
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
import Servant
import Servant.API
import Network.Wai
import Network.Wai.Handler.Warp
import Data.IORef
import qualified Data.ByteString.Char8 as C
type API = "one" :> Get '[PlainText] String :<|> "two" :> Get '[PlainText] String
api :: Proxy API
api = Proxy
server :: Server API
server = return "1\n" :<|> return "2\n"
app :: Application
app = serve api server
counter :: IORef Int -> Middleware
counter cref oldapp req respond
= do n <- atomicModifyIORef cref (\n' -> (n'+1,n'+1))
putStrLn $ "Request #" ++ show n ++ ": " ++ showRequest req
oldapp req respond
where
showRequest req = C.unpack (requestMethod req) ++ " " ++ C.unpack (rawPathInfo req)
main = do
cref <- newIORef (0 :: Int)
run 3000 $ counter cref app
Related
I am writing an API server using Servant. The server includes persistent state. I would like to use QuickCheck to write tests for the server.
The implementation of various endpoints that make up the Servant Application require a database value. Unsurprisingly, creation of the database value is in the IO monad.
I don't understand how to combine the pieces from Hspec, Wai, QuickCheck, and Servant in a way that satisfies them all.
I see that I can perform an IO as part of creating the Hspec Spec itself and I see that I can specify that an IO be performed before each item in the Hspec Spec. Neither of these capabilities seems helpful in this case. The IO needs to be performed for each QuickCheck iteration of the property. Without this, the database accumulates state from each iteration which invalidates the definition of the property (or at least makes it greatly more complicated).
Below is my attempt to create a minimal, self-contained example of this scenario.
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE FlexibleContexts #-}
module Main where
import Data.IORef
import Test.QuickCheck
import Test.QuickCheck.Monadic
import qualified Test.Hspec.Wai.QuickCheck as QuickWai
import Test.Hspec
import Test.Hspec.Wai
import Text.Printf
import Servant
import Servant.API
import Data.Aeson
import Data.Text.Encoding
import Data.ByteString.UTF8
( fromString
)
data Backend = Backend (IORef Integer)
openBackend :: Integer -> IO Backend
openBackend n = Backend <$> newIORef n
data Acknowledgement = Ok Integer
instance ToJSON Acknowledgement where
toJSON (Ok n) = object [ "value" .= n ]
serveSomeNumber :: Backend -> Integer -> IO Acknowledgement
serveSomeNumber (Backend a) b = do
a' <- readIORef a
modifyIORef a (\n -> n + 1)
return $ Ok (a' + b)
type TheAPI = Capture "SomeNumber" Integer :> Post '[JSON] Acknowledgement
theServer :: Backend -> Server TheAPI
theServer backend = liftIO . serveSomeNumber backend
theAPI :: Proxy TheAPI
theAPI = Proxy
app :: Backend -> Application
app backend = serve theAPI (theServer backend)
post' n =
let
url = printf "/%d" (n :: Integer)
encoded = fromString url
in
post encoded ""
spec_g :: Backend -> Spec
spec_g (Backend expectedResult) =
describe "foo" $
it "bar" $ property $ \genN -> monadicIO $ do
n <- run genN
m <- run $ readIORef expectedResult
post' n `shouldRespondWith` ResponseMatcher { matchStatus = fromInteger (n + m) }
main :: IO ()
main = do
spec_g' <- spec_g `fmap` openBackend 16
hspec spec_g'
This doesn't type check:
/home/exarkun/Scratch/QuickCheckIOApplication/test/Spec.hs:119:3: error:
* Couldn't match type `WaiSession' with `PropertyM IO'
Expected type: PropertyM IO ()
Actual type: WaiExpectation
* In a stmt of a 'do' block:
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}
In the second argument of `($)', namely
`do n <- run genN
m <- run $ readIORef expectedResult
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}'
In the expression:
monadicIO
$ do n <- run genN
m <- run $ readIORef expectedResult
post' n
`shouldRespondWith`
ResponseMatcher {matchStatus = fromInteger (n + m)}
|
119 | post' n `shouldRespondWith` ResponseMatcher { matchStatus = fromInteger (n + m) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I don't know if there is a way to fit a WaiExpectation into a PropertyM IO () at all. I don't even know if monadicIO is helpful here at all.
How can I fit these pieces together?
Define spec_g :: Background -> Spec, then take advantage of IO's Functor and Monad instances.
main = do
spec <- fmap spec_g (openBackend 16) -- fmap spec_g :: IO Background -> IO Spec
hspec spec
or more concisely,
main = spec_g <$> openBackend 16 >>= hspec
IIRC, you're supposed to run each spec or property with the with function. Here's a few properties I wrote some time ago:
with app $ describe "/reservations/" $ do
it "responds with 404 when no reservation exists" $ WQC.property $ \rid ->
get ("/reservations/" <> toASCIIBytes rid) `shouldRespondWith` 404
it "responds with 200 after reservation is added" $ WQC.property $ \
(ValidReservation r) -> do
_ <- postJSON "/reservations" $ encode r
let actual = get $ "/reservations/" <> toASCIIBytes (reservationId r)
actual `shouldRespondWith` 200
The app value serves the service, and as far as I recall, it runs the IO action for each test. I did it with an in-memory database using an IORef, and that seems to be working just fine:
app :: IO Application
app = do
ref <- newIORef Map.empty
return $
serve api $
hoistServer api (Handler . runInFakeDBAndIn2019 ref) $
server 150 []
The WQC.property function is from a qualified import:
import qualified Test.Hspec.Wai.QuickCheck as WQC
I wasn't too happy, however, with the way I had to structure my tests and properties with HSpec, so I ultimately rewrote all the tests to be driven by HUnit. I've an upcoming blog post that describes this, but I haven't published it yet.
I am trying to make client bindings for the News api provided by newsapi.org using the Haskell servant library. I have created the following endpoint:
type NewsAPI = "top-headlines" :> QueryParam "country" String :> QueryParam "apiKey" String :> Get '[JSON] TopHeadlines
And attempted to call it as follows:
topheadlines :: Maybe String -> Maybe String -> ClientM TopHeadlines
api :: Proxy NewsAPI
api = Proxy
topheadlines = client api
query = topheadlines (Just "us") (Just "<api key>")
run3 :: IO ()
run3 = do
manager' <- newManager defaultManagerSettings
users <- runClientM query (mkClientEnv manager' (BaseUrl Https "newsapi.org/v2" 443 ""))
print users
I keep getting a connection error that I don't entirely understand how to reason about:
Left (ConnectionError "HttpExceptionRequest Request {\n host = \"newsapi.org/v2\"\n port = 443\n secure = True\n requestHeaders = [(\"Accept\",\"application/json;charset=utf-8,application/json\")]\n path = \"/top-headlines\"\n queryString = \"?country=us&api_key=90a38fab85c440fa88521e0789248f83\"\n method = \"GET\"\n proxy = Nothing\n rawBody = False\n redirectCount = 10\n responseTimeout = ResponseTimeoutDefault\n requestVersion = HTTP/1.1\n}\n TlsNotSupported")
Not sure why there is no connection. Another set of client bindings I have is working fine.
It is a combination of two things:
api_key should be apiKey
Rely on parseBaseUrl
burl <- parseBaseUrl "http://newsapi.org/v2"
This worked for me in the sample project I set up.
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeOperators #-}
module Main where
import Control.Monad.Free
import Servant.Client.Free
import qualified Network.HTTP.Client as HTTP
import qualified Servant.Client.Internal.HttpClient as I
import Network.Wai.Handler.Warp (run)
import Servant
import System.Environment (getArgs)
type NewsAPI = "top-headlines" :> QueryParam "country" String :> QueryParam "apiKey" String :> Get '[JSON] String
topheadlines :: Maybe String -> Maybe String -> Free ClientF String
topheadlines = client api
api :: Proxy NewsAPI
api = Proxy
main :: IO ()
main = do
test
test :: IO ()
test = case topheadlines (Just "us") (Just "API_KEY") of
Pure n ->
putStrLn $ "ERROR: got pure result: " ++ show n
Free (Throw err) ->
putStrLn $ "ERROR: got error right away: " ++ show err
Free (StreamingRequest _req _k) ->
putStrLn $ "ERROR: need to do streaming request"
Free (RunRequest req k) -> do
burl <- parseBaseUrl "http://newsapi.org/v2"
mgr <- HTTP.newManager HTTP.defaultManagerSettings
let req' = I.requestToClientRequest burl req
putStrLn $ "Making request: " ++ show req'
res' <- HTTP.httpLbs req' mgr
putStrLn $ "Got response: " ++ show res'
I am following the basic dispatching section of wai application.
I am able to catch the url parameter. How can I perform IO operation using these params.
I would like to use runCommand of System.Process to execute a system command using these parameters.
:t runCommand give
runCommand :: String -> IO ProcessHandle
my Main.hs
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Monad
import Network.Wai
import Network.HTTP.Types
import Network.Wai.Handler.Warp (run)
import Data.ByteString
import Control.Monad
import System.Process
import qualified Data.ByteString.Lazy.Char8 as L8
main :: IO ()
main = do
run 8080 app
app :: Application
app request respond = respond $ case rawPathInfo request of
"/" -> indexHtml
"/wake" -> wakeMeUP request
_ -> fourNotFour
indexHtml :: Response
indexHtml = responseFile
status200
[("Content-Type","text/html")]
"index.html"
Nothing
wakeMeUP :: Request -> Response
wakeMeUP request =
let query = queryString request
hour = join $ lookup "hour" query
min = join $ lookup "min" query
--I would like to use runCommand "using hour and min variables"
in responseLBS
status200
[("Content-Type","text/plain")]
"Alarm set at...to be coded later"
fourNotFour :: Response
fourNotFour = responseLBS
status404
[("Content-Type","text/plain")]
"404 not found"
Your design prevents it, because of how you have written app,
app request respond = respond $ case rawPathInfo request of
which says that you immediately respond. Note the type of Application:
type Application = Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
Since the result type has an IO, you have the opportunity to do I/O before yielding a value. So you could:
app request respond = do
whateverResult <- runCommand "whatever"
respond $ ...
(You could also do it afterward, at least according to the types:
app request respond = do
rcvd <- respond $ ...
runCommand "whatever"
return rcvd
Though that's a bit odd to do for the continuation-passing idiom being used here (the (a -> b) -> b pattern in the Application type). It means that the command will be run after everything else, for some definition of "everything else" that we can't know without reading the wai source.)
Anyway, you probably don't want the command to be run inside app, but rather in wakeMeUp, which means you need to change some types around. In particular,
wakeMeUp :: Request -> IO Response
-- ^^
and suitably monadify the function. Then your app needs to not call respond immediately, so you can say
app request respond =
response <- case rawPathInfo request of
"/" -> return indexHtml
-- ^^^^^^
"/wake" -> wakeMeUp request
-- no change, because wakeMeUp now has an IO return type
...
respond response
If this is gibberish to you, it's time to do some monad tutorials. Or if you just want to get the damn thing working, I recommend Dan Piponi's The IO Monad For People who Simply Don't Care. Happy hacking!
I'm trying to find a way to check if a webpage exists in Haskell. The server is HTTP2 / HTTPS only and I'm trying to check if the page exists in a servant application.
Is there any Haskell packages with good documentation to just check if the status code is 200 or 404? And working with strong HTTPS and HTTP2 servers?
Here what I currently have with http-conduit but I'm receiving weird exceptions (TlsExceptionHostPort (HandshakeFailed (Error_Protocol ("expecting server hello, got alert : [(AlertLevel_Fatal,HandshakeFailure)]",True,HandshakeFailure))) "thibaud.dauce.fr" 443 and StatusCodeException).
... other imports
import qualified Network.HTTP.Conduit as HTTP
... other types
type AppM = ReaderT Config (EitherT ServantErr IO)
newComment :: String -> OneComment -> AppM Int64
newComment baseUrl oneComment = do
time <- liftIO getCurrentTime
response <- HTTP.withManager $ \manager -> do
request <- HTTP.parseUrl $ url oneComment
HTTP.httpLbs request manager
case (statusIsSuccessful $ HTTP.responseStatus response, startswith baseUrl (url oneComment)) of
(_, False) -> return 0
(True, True) -> do
theNewComment <- runDb $ insert $ Comment (url oneComment) (content oneComment) time
return $ fromSqlKey theNewComment
_ -> return 0
Some examples using wreq
{-# LANGUAGE OverloadedStrings #-}
import Network.Wreq
import Control.Lens
import Control.Exception as E
import Network.HTTP.Client (HttpException)
test1 = do
r <- get "https://httpbin.org/get"
print $ r ^. responseStatus . statusCode
-- throws an exception
test2 = do
r <- get "https://www.google123123.com"
print $ r ^. responseStatus . statusCode
testUrl url = do
r <- get url
return $ r ^. responseStatus . statusCode
-- catching the exception
test3 = do
st <- testUrl "https://www.google123123123.com" `E.catch` handler
print st
where
handler :: HttpException -> IO Int
handler _ = return 999
Continuing my exploration of conduit and aeson, how would I go about using my own data type in stead of Value in this (slightly modified) code snippet from the Yesod book.
{-# LANGUAGE OverloadedStrings, TemplateHaskell #-}
import Network.Wai (Response, responseLBS, Application, requestBody)
import Network.HTTP.Types (status200, status400)
import Network.Wai.Handler.Warp (run)
import Data.Aeson.Parser (json)
import Data.Conduit.Attoparsec (sinkParser)
import Control.Monad.IO.Class (liftIO)
import Data.Aeson (Value(..), encode, object, (.=))
import Control.Exception (SomeException)
import Data.ByteString (ByteString)
import Data.Conduit (ResourceT, ($$))
import Control.Exception.Lifted (handle)
import qualified Data.HashMap.Strict as M
import Data.Aeson.TH (deriveJSON)
-- I ADDED THIS
data JSONRequest = JSONRequest {
command :: ByteString,
params :: M.HashMap ByteString ByteString
}
deriveJSON id ''JSONRequest
-- END OF WHAT I ADDED
main :: IO ()
main = run 3000 app
app :: Application
app req = handle invalidJson $ do
value <- requestBody req $$ sinkParser json
newValue <- liftIO $ dispatch value
return $ responseLBS
status200
[("Content-Type", "application/json")]
$ encode newValue
invalidJson :: SomeException -> ResourceT IO Response
invalidJson ex = return $ responseLBS
status400
[("Content-Type", "application/json")]
$ encode $ object
[ ("message" .= show ex)
]
-- Application-specific logic would go here.
dispatch :: Value -> IO Value
dispatch = return
Basically, I want to change the type of dispatch to JSONRequest -> IO JSONRequest. How do I tell the parser to use my own derived instance of fromJSON?
I tried just adding a type declaration, praying for polymorphic return type on json, but I realised it is strictly for Value.
Just looking at the types, don't you just need to fmap your fromJSON over the result coming from json? With a suitable signature for dispatch we just need:
-- import Data.Aeson
app :: Application
app req = handle invalidJson $ do
result <- requestBody req $$ sinkParser (fmap fromJSON json)
next_result <- liftIO $ dispatch result
return $ responseLBS status200 [("Content-Type", "application/json")]
$ encode next_result
dispatch :: Result JSONRequest -> IO JSONRequest
dispatch (Error str) = undefined
dispatch (Success jsonreq) = return jsonreq
But maybe it's a little clearer written thus:
-- import Data.Aeson
-- import qualified Data.Attoparsec as Atto
toRequest :: Value -> Result JSONRequest
toRequest = fromJSON -- specialized now to your fromJSON
jsonRequestParser :: Atto.Parser (Result JSONRequest)
jsonRequestParser = fmap toRequest json
app :: Application
app req = handle invalidJson $ do
result <- requestBody req $$ sinkParser jsonRequestParser
next_result <- liftIO $ dispatch result
return $ responseLBS status200 [("Content-Type", "application/json")]
$ encode next_result
dispatch :: Result JSONRequest -> IO JSONRequest
dispatch (Error str) = undefined
dispatch (Success jsonreq) = return jsonreq
I left the parser returning a Result JSONRequest so dispatch is handling Error cases too, which might mean you need your exception handling somehow?