Visualize a bytestring body on a webserver run on Spock (localhost for instance)
My goal : create website and view a bytestring (converted to text)
Framework: Http Simple for performing request to restAPI
Spock for my server
I don't want for instance to create a JSON as I need to manipulate/inspect my response before creating a JSON structure. General idea is that I want to use the response body to construct a JSON query structure (the user will be able to compose his question) that will be sent to the restAPI website.
I manage to build a request like this:
connect = do
request' <- (parseRequest "http://localhost")
let request = setRequestMethod "POST"
$ setRequestHost (S8.pack ("xx.xxx.xxx.xxx"))
$ setRequestPath "/api/Integration/Login"
$ setRequestBodyJSON me
$ setRequestPort 1000
$ request'
response <- httpJSON request
return (getResponseBody response :: Auth)
then I used it to query the API page
getRequest :: RequestPath -> HtmlT IO L.ByteString
getRequest rpath = do
atoken <- liftIO connect
request' <- liftIO (parseRequest "http://localhost")
let request = setRequestMethod "POST"
$ setRequestHost (S8.pack ("xx.xxx.xxx.xxx"))
$ setRequestPort 1000
$ setRequestPath (S8.pack ("/api/Integration/" ++ rpath))
$ addRequestHeader hAuthorization (S8.pack (unpack (token_type (atoken)) ++ " " ++ unpack (access_token (atoken))))
$ setRequestBodyJSON r1
$ request'
response <- httpLBS request
return (getResponseBody (response))
then I follow with a short SpockM monad:
app1 = do get root $ text "root"
fct
with fct equal to
fct = do get "/further" $ lucidIO ( fmap TL.decodeUtf8 (getRequest "GetProperties"))
Everything compile fine I am even able to see the result in GHCI with invocation like : connect >>= (\ x -> print x) (same with getRequest "GetProperties" )
What I don't understand is that lucidIO should give me a ActionCtxtT ctx m b type, which perfectly fit the type of a handler (for example like the text function in the do get ... $ text -> ActionCtxT ctx m a) and should be processed by the spock function in main() ie runSpock 8080 (spock spockCfg app1)
I tried to get rid of the ByteString 'ending' type replacing it with a () in order to mimic as close as possible the Html () type which shows up and work in lot of examples I studied.
All parsing and request building is done with the HTTP.Simple (it's not very elegant I know for instance it just have to work) which pulls me from start in a monad (due to the first function 'parseRequest' -> m Request) from which I cannot escape until lucidIO - may be I am choosing the wrong Monad (ie IO : but with IO I am able to check everything in ghci). Could you give me some hints on how to get this ByteString printed in my browser?
So finally I achieve what I was looking for - woua I am really proud of me ...
Okay for those who will look for the same thing, what I've manage to do, to recap my main problem was to escape the IO monad (my choice may be not clever but still) in which I was stuck due to the use of request parsers from HTTP.simple library.
My code change a little bit but the general idea stays the same:
building a Response query:
getResponseMethod :: RequestPath -> RequestBody -> IO (Maybe Value)
from which thanks to the decode function (aeson package) a Maybe Value is obtained (wrapped in IO but that's okay)
then my little spock server:
main :: IO ()
main = do
spockCfg <- defaultSpockCfg () PCNoDatabase ()
runSpock 8080 (spock spockCfg app)
I work a lot to have the right app -> SpockM () () () ()
I started with the simplest app we could imagine:
app = do get root $ text "Hello!"
noticing that the text function is producing a MonadIO m => ActionCtxT cxt m a monad so my thought was that if I 'sprinkle' some clever LiftIO thing it should do the job.
I create a helper function:
extrct :: MonadIO m => ActionCtxT ctx m Text
extrct = liftIO $ do
a <- getResponseMethod "GetProperties" r1
return (pack $ show a)
and with a twist of hand adjust my app
app :: SpockM () () () ()
app = do get root $ do
a <- extrct
text a
and finally I was able to see the string representation of the Maybe Value :: JSON on my spock local webserver. That's what I was looking for. Now I can work on cleaning my code. From what I understand using liftIO will place the IO monad in the rigth place in the Monad Stack that's because IO is always at the bottom?
Related
I have written this ManagerSettings to log all requests and responses for my http-conduit application. (By the way, I am importing ClassyPrelude).
tracingManagerSettings :: ManagerSettings
tracingManagerSettings =
tlsManagerSettings { managerModifyRequest = \req -> do
putStr "TRACE: "
print req
putStrLn ""
pure req
, managerModifyResponse = \r -> do
responseChunks <- brConsume $ responseBody r
let fullResponse = mconcat responseChunks
putStr "TRACE: RESPONSE: "
putStrLn $ decodeUtf8 fullResponse
pure $ r { responseBody = pure fullResponse }
}
However, it's not working - when I use it, the application is hanging and trying to consume all the RAM in the machine after printing the first request and first response, which suggests some kind of infinite loop.
Also, the request is printed twice.
I made a previous attempt that was similar, but didn't modify r. That failed because after I had already read the response completely, there was no more response data to read.
If I replace this with tlsManagerSettings, http-conduit works again.
My application is using libstackexchange, which I have modified to allow the ManagerSettings to be customised. I am using http-conduit version 2.2.4.
How can I diagnose the issue? How can I fix it?
managerModifyResponse doesn't work with a Response ByteString, it works with a Response BodyReader, where type BodyReader = IO ByteString along with the contract that if it produces a non-empty ByteString there is more input that can be read.
The problem you're running into is that pure fullResponse never returns an empty ByteString unless it always does. You need to provide a somewhat more complex IO action to capture the intended behavior. Maybe something along these lines (untested):
returnOnce :: Monoid a => a -> IO (IO a)
returnOnce x = do
ref <- newIORef x
pure $ readIORef ref <* writeIORef ref mempty
As for how to debug this? Not sure about generic methods. I was just suspicious that you probably needed a solution along these lines, and the docs for BodyReader confirmed it.
I'm new to Haskell so apologies in advance for the potentially stupid question.
I'd like to build a data structure that is constructed from two http requests in my application.
My first request gets a basic list of users which I could choose to decode to Maybe [User]
r <- getWith opts "https://www.example.com/users"
let users = decode $ r ^. responseBody :: Maybe [User]
But if I'd like to enrich my user data by calling a second endpoint for each of the users that respond by doing something like
r2 <- getWth opts "https://www.example.com/users/{userid}/addresses"
let enrichedUser = decode $ r2 ^. responseBody :: Maybe EnrichedUser
I can't quite piece these parts together at the minute. I'm in a do block thats expecting an IO ()
Any help would be appreciated!
I'm assuming that the type of enrichedUser is supposed to be Maybe EnrichedUser and not Maybe [EnrichedUser], right?
If so, after extracting the [User] list from users :: Maybe [User], the problem you're facing is running a monadic action (to fetch the web page) for each User. There's a handy combinator for this in Control.Monad:
mapM :: (Monad m) => (a -> m b) -> ([a] -> m [b])
which can be specialized in your situation to:
mapM :: (User -> IO EnrichedUser) -> ([User] -> IO [EnrichedUser])
This says, if you know how to write a function that takes a User and creates an IO action that will create an EnrichedUser, you can use mapM to turn this into a function that takes a list [User] and creates an IO action to create a whole list [EnrichedUser].
In your application, I imagine the former function would look something like:
enrich :: User -> IO EnrichedUser
enrich u = do
let opts = ...
let url = "https://www.example.com/users/"
++ userToUserID u ++ "/addresses"
r2 <- getWith opts url
let Just enrichedUser = decode $ r2 ^. responseBody
return enrichedUser
where decode = ...
and then you can write (in your IO do-block):
r <- getWith opts "https://www.example.com/users"
let Just users = decode $ r ^. responseBody
enrichedUsers <- mapM enrich users
-- here, enrichedUsers :: [EnrichedUser]
...etc...
I've omitted the Maybe processing here for simplicity. If enriching fails, you probably want to somehow coerce a regular User into a default EnrichedUser anyway, so you'd modify the bottom of the enrich function to read:
let enrichedUser = case decode $ r2 ^. responseBody of
Nothing -> defaultEnrichment u
Just e -> e
return enrichedUser
and everything else would stay the same.
I am using the LevelDB library and Snap framework together. I have:
main :: IO ()
main = runResourceT $ do
db <- open "thedb" defaultOptions { createIfMissing = True }
liftIO $ serveSnaplet defaultConfig $ initWeb db
Now in my handler, I'm unsure how to get back to MonadResource IO in order to query the database:
handleWords :: Handler App App ()
handleWords = do
words <- uses thedb $ \db -> $ get db def "words"
writeBS $ pack $ show words
Which gives me a: No instance for (MonadResource IO) arising from a use of 'get'
Any ideas? I feel like I'm missing something about how to properly create a monad "stack". Thanks
MonadResource/ResourceT is one way of acquiring scarce resources in a way that guarantees resources will be freed in the case of an exception. Another approach is the bracket pattern, which is supported by Snap via the bracketSnap function. You can use this to create the ResourceT context needed by LevelDB:
import qualified Control.Monad.Trans.Resource as Res
bracketSnap Res.createInternalState Res.closeInternalState $ \resState -> do
let openAction = open "thedb" defaultOptions { createIfMissing = True }
db <- Res.runInternalState openAction resState
This could be made simpler with some changes in Snap and leveldb:
Instead of only providing the open function, which presumes a MonadResource context, there could be a function which returns a Resource value. I'm making this tweak in persistent for the 2.0 release.
Snap could provide support for either MonadResource or the Resource monad (two separate concepts with unfortunately similar names).
Snap doesn't need to support MonadResource or Resource for you to do this. You're doing the monad transformer composition in the wrong direction. A look at the types will help.
serveSnaplet :: Config Snap AppConfig -> SnapletInit b b -> IO ()
runResourceT :: MonadBaseControl IO m => ResourceT m a -> m a
So you're trying to put an IO in the place that a ResourceT is expected. You should approach this the other way around. Put your open "thedb" ... call inside your application's Initializer with a liftIO. But open is a MonadResource, so you need to use the ResourceT instance to get it into an IO. It will look something like this:
app = makeSnaplet "app" "An snaplet example application." Nothing $ do
...
db <- liftIO $ runResourceT $ open "thedb" defaultOptions
Then store the db handle in your App state and you can retrieve it later using Handler's MonadReader or MonadState instances.
this is probably a simple question and I've seen a similar one on SO, but I'm still stuck.
I'm trying to do an HTTP call to pull in the contents of another blog and display it on my page. This is more of a learning exercise than anything.
Here's my handler
blog :: App1Handler ()
blog = do
contents <- Requester.getUrl "http://someblog.com/"
heistLocal (bindString "contents" contents) . render $ "blog"
Requester.getUrl has the signature getUrl :: String -> IO T.Text
And the error I get back is
src/Main.hs:50:15:
Couldn't match expected type Handler App1 App1 t0' with actual typeIO T.Text'
In the return type of a call of `getUrl'
In a stmt of a 'do' block:
contents <- getUrl "http://someblog.com/"
In the expression:
do { contents <- getUrl "http://someblog.com/";
heistLocal (bindString "contents" contents) . render $ "blog" }
From what I gather, I'm stuck inside of the IO monad and it wants the type Handler App1 App1 t0. I've experimented with sticking liftIO in places, but I'm pretty confused on this one.
Can anyone point me in the right direction?
Thanks!
You just have to liftIO the IO action returned by getUrl, like this:
contents <- liftIO $ Requester.getUrl "http://someblog.com/"
The reasoning here is simple. You have a do-block of type App1Handler (), which means that the
right hand side of any <- statement within this do-block must have type App1Handler a.
However, getUrl returns IO Text, so you need to a function to convert from
IO a to App1Handler a which is exactly what liftIO does.
liftIO :: MonadIO m => IO a -> m a
I want to write little integration tests for my Snap web handlers but I am stuck. Here is the scenario. I have a Snap web handler that (run-of-the-mill style) CRUDs up a type and it looks something like this:
create :: AppHandler ()
create = method POST $ do
lastName <- decodeUtf8 . fromJust <$> getParam "lastName"
firstName <- decodeUtf8 . fromJust <$> getParam "firstName"
createPerson $ Person firstName lastName
modifyResponse (setResponseCode 204)
The Snap.Test module has some things to help build up a request and I use it to make a request for my handler:
createOwnerReq :: RequestBuilder IO ()
createOwnerReq = postUrlEncoded "host/person/create" $
fromList [ ("firstName", ["Greg-Shaw"])
, ("lastName", ["Snoy'Sullivan"])
]
Here's the problem, I want to make a TestUnit TestCase for this handler so I need the run the handler on the createOwnerReq request. The module Snap.Test provides:
runHandler :: MonadIO a => RequestBuilder m () -> Snap a -> m Response
so
... do
resp <- runHandler createOwnerReq ???
But wait!!! My request handler is of type AppHandler () but runHandler requires a Handler of type Snap a.
How do I lift my AppHandler type into the Snap monad? Help please, this is kind of trippin' me out.
Ibolla's return create trick probably doesn't do what you want. It compiles correctly because runHandler takes a Snap a which will work on a Snap action with any return value. return create :: Snap (AppHandler ()), which is very different from the Snap () that you were probably expecting.
We are working on a Snap.Snaplet.Test equivalent that will wrap the runHandler function provided by Snap.Test to allow you to test Handlers. This will probably be included in the 0.10 release of the snap package.
In the interim, you can solve the problem manually by using runSnaplet to convert your SnapletInit into a Snap () action that can be passed to Snap.Test.runHandler. This won't let you test an individual Handler, but it will let you test any of the routes defined in your application's initializer.
EDIT: In snap-0.10, we added test support for snaplets.