Handling HTTP Query parameters in http-conduit - haskell

I want to download the content of the URL
http://example.com/foobar?key1=value1&key2=value2
using http-conduit (GET request).
How can I do that:
a) Assuming I already know the full (i.e. encoded URL)
b) If some parameters are dynamic and therefore not URL-encoded?
Note: This question was answered Q&A-style and therefore intentionally does not show any research effort.

Regarding a):
You can use simpleHttp with an URL containing query parameters just like the example in the docs:
{-# LANGUAGE OverloadedStrings #-}
import Network.HTTP.Conduit
import qualified Data.ByteString.Lazy as LB
main :: IO ()
main =
simpleHttp "http://example.com/foobar?key1=value1&key2=value2" >>= LB.putStr
Regarding b):
You need a list of key/value tuples of type [(ByteString, Maybe ByteString)] that contains your query parameters.
{-# LANGUAGE OverloadedStrings #-}
import Network.HTTP.Conduit
import Data.ByteString (ByteString)
import qualified Data.ByteString.Lazy.Char8 as LB
queryParams :: [(ByteString, Maybe ByteString)]
queryParams = [
("key1", Just "value1"),
("key2", Just "value2")]
main :: IO ()
main = do
request <- parseUrl "http://example.com/foobar"
let request' = setQueryString queryParams request
response <- withManager $ httpLbs request'
LB.putStrLn $ responseBody response
Note: This requires at least http-conduit 2.1.
Also note that it is recommended to reuse Manager instances where applicable.

Related

LiftIO, do block, and syntax

I'm getting to grips with writing an API in Haskell using Scotty. My files are provided below. My questions are:
In the routes definition, I'm extracting from liftIO whatsTheTime in a do block. This works, but it seems verbose. Is there a nicer syntax?
In the whatsTheTime definition, I'm needing to do fromString. I'd have thought OverloadedString would take care of that, but that's not the case. I'd really appreciate it if somebody pointed out why it doesn't work without fromString.
In a stack project, if I need a directive like OverloadedStrings, do I need to include it every file that needs it, or just at the top of the main entrypoint?
Api.hs:
{-# LANGUAGE OverloadedStrings #-}
module Api
( whatsTheTime
) where
import Data.Time (getCurrentTime)
import Web.Scotty
import Data.String
whatsTheTime :: IO (ActionM ())
whatsTheTime = do
time <- getCurrentTime
return $ text $ fromString ("The time is now " ++ show time)
Main.hs:
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE DeriveGeneric #-}
module Main where
import Api
import Web.Scotty
import Control.Monad.IO.Class
routes = do
get "/" $ do
res <- liftIO whatsTheTime
res
main :: IO ()
main = do
putStrLn "Starting server..."
scotty 3000 routes
(1) This:
do
res <- liftIO whatsTheTime
res
Desugars to this:
liftIO whatsTheTime >>= \ res -> res
If you look at the type of \ m -> m >>= id:
(Monad m) => m (m a) -> m a
That’s exactly the type of join (Hoogle), so you can use:
get "/" $ join $ liftIO whatsTheTime
join is a common idiom for “execute this action which returns an action, and also execute the returned action”.
(2) OverloadedStrings is for overloading of string literals. You have an overloaded literal "The time is now ", but you constrain it to be of type String by using it as an operand of (++) with a String (the result of show time). You can pack the result of show time as a Text instead using fromString or Data.Text.pack:
import Data.Monoid ((<>))
import qualified Data.Text as Text
-- ...
return $ text $ "The time is now " <> Text.pack (show time)
(3) LANGUAGE pragmas operate per file; as #mgsloan notes, you can add OverloadedStrings to the default-extensions: field of your library or executable in your .cabal file.

How to display Response from an HTTP GET Request in front-end using Scotty?

I'm trying out Scotty for the first time and I can't seem to get past making my GET request. The Response is returned as type
IO (Response bytestring-0.10.8.1:Data.ByteString.Lazy.Internal.ByteString)
I know I need to convert it to a type that can be output by Scotty but I can't figure out how to do that.
My full code is :
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Lens
import Control.Monad.IO.Class
import Data.Aeson (FromJSON, ToJSON, Value, decode,
encode)
import Data.Map as Map
import GHC.Generics
import Lib
import Network.Wreq as Wreq
import Web.Scotty as Scotty
main :: IO ()
main =
scotty 3000 $
Scotty.get "/" $ do
-- html "Hello World!"
Wreq.get"https://www.metaweather.com/api/location/search/?query=New%20York"
I tried using LiftIO but that is still giving me a Type Error. I wanted to know how exactly I should convert my Response so that I can display it in the front-end just like I displayed my initial "Hello World" with html.
If you are just looking for a quick proof of concept and aren't worried about erroneous responses, you could use the responseBody lens and send the lazy byte string to raw instead of html:
main :: IO ()
main =
scotty 3000 $
Scotty.get "/" $ do
r <- liftIO $ Wreq.get "https://www.metaweather.com/api/location/search/?query=New%20York"
raw (r ^. responseBody)

Exception POSTing multipart form with http-conduit

I'm trying to POST a multipart form request to an internal website which should reply with an XML response. Using another simple script I have in Python with the requests library, everything works fine, however, using http-conduit I keep receiving an exception ExpectedBlankAfter100Continue.
If I replace the internal url with "https://httpbin.org/post", I also receive a reply back without issue.
Is there something I'm doing wrong? It seems like either a bug in the library or the site is not behaving as expected. If the latter is the case, is there an option for me to disable this check in http-conduit?
Sample code:
{-# LANGUAGE OverloadedStrings #-}
import Network
import Network.HTTP.Conduit
import Network.HTTP.Client.MultipartFormData
import qualified Data.ByteString.Lazy.Char8 as BL
import Data.Maybe (fromJust)
import System.Environment (getArgs)
import Control.Monad.IO.Class
main = do
[x] <- getArgs
--let url = "https://url.net/api.asp"
let url = "https://httpbin.org/post"
withSocketsDo $ withManager $ \m -> do
r <- flip httpLbs m =<< (formDataBody (request $ BL.pack x) $ fromJust $ parseUrl url)
liftIO $ BL.putStrLn $ responseBody r
request :: BL.ByteString -> [Part]
request x = <code removed>
This sounds like the server is returning a malformed 100-continue response. But there's not enough information here to properly debug this, it's probably better to handle this in a Github issue.

Haskell Aeson destructuring generic parse

I have a JSON request in the style of
{"command":"get","params":{"something":"something else"}}
and this code snippet from the Yesod book
{-# LANGUAGE OverloadedStrings #-}
import Network.Wai (Response, responseLBS, Application, requestBody)
import Network.HTTP.Types (status200, status400)
import Network.Wai.Handler.Warp (run)
import Data.Aeson.Parser (json)
import Data.Conduit.Attoparsec (sinkParser)
import Control.Monad.IO.Class (liftIO)
import Data.Aeson (Value(..), encode, object, (.=))
import Control.Exception (SomeException)
import Data.ByteString (ByteString)
import Data.Conduit (ResourceT, ($$))
import Control.Exception.Lifted (handle)
main :: IO ()
main = run 3000 app
app :: Application
app req = handle invalidJson $ do
value <- requestBody req $$ sinkParser json
newValue <- liftIO $ modValue value
return $ responseLBS
status200
[("Content-Type", "application/json")]
$ encode newValue
invalidJson :: SomeException -> ResourceT IO Response
invalidJson ex = return $ responseLBS
status400
[("Content-Type", "application/json")]
$ encode $ object
[ ("message" .= show ex)
]
-- Application-specific logic would go here.
modValue :: Value -> IO Value
modValue (Object o)
| -- key "command" corresponds to value "get"
| otherwise = fail "Invalid command"
But I can't wrap my head around how I would destructure the generated Value data structure. How do I go about getting the values of keys etc. I realise I could parse to an explicitly defined data structure, but that would bring other kinds of problems to my use case.
In modValue I've put a comment where I can't figure out what to put. I tried treating it as a Map, since that's how it is implemented inside Aeson, but that obviously doesn't type check.
EDIT:
Adding Data.HashMap to imports and using the line
| M.lookup "command" o == Just "get" = return $ object [("result" .= (String "YAY"))]
gives the following error message.
main.hs:39:26:
Couldn't match expected type `M.Map k0 a0'
with actual type `aeson-0.6.0.2:Data.Aeson.Types.Internal.Object'
In the second argument of `M.lookup', namely `o'
In the first argument of `(==)', namely `M.lookup "command" o'
In the expression: M.lookup "command" o == Just "get"
EDIT2:
On a sudden hunch, I tracked down an error message I got earlier involving "unordered-containers". This is the package that Aeson uses. But I realised that I also had the package hashmap installed, which is imported as Data.HashMap. The hashmaps from unordered-containers are imported as Data.HashMap.Strict or Lazy!
Changing the line import qualified Data.HashMap as M to import qualified Data.HashMap.Strict as M fixed it anyway. Now the given answer works!
Since an aeson JSON object is a Hashmap, you can use the Hasmap interface, in this case lookup.
import qualified Data.HashMap.Strict as M
M.lookup "command" o == Just "get"

Snap, IO and acid-state

Trying to use acid-state in Snap, and I hit a roadblock.
Here is what I got so far.
First my acid-state related objects (it's a dummy book with a isbn number):
{-# LANGUAGE DeriveDataTypeable #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE OverloadedStrings #-}
module Models where
import Prelude hiding ((.), id)
import Control.Category ((.))
import Control.Monad.Reader (asks)
import Data.ByteString (ByteString)
import Data.SafeCopy (base, deriveSafeCopy)
import qualified Data.Text as T
import Data.Typeable (Typeable)
import Data.Acid (Update, Query, makeAcidic)
import Control.Monad.Reader (ask)
import Control.Applicative ((<$>))
import Data.Data (Data)
data Book = Book { isbn :: String }
deriving (Eq, Ord, Read, Data, Show, Typeable)
$(deriveSafeCopy 0 'base ''Book)
-- Retrieve the book's isbn
queryIsbn :: Query Book String
queryIsbn = isbn <$> ask
$(makeAcidic ''Book ['queryIsbn])
And then my actual attempt at integrating it with Snap. As you can see, I am having trouble defining __ doQuery__ function, that should return a string isbn:
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE OverloadedStrings #-}
module Application where
import Control.Monad.Trans.Class (lift)
import Data.Text.Encoding (decodeUtf8)
import Text.XmlHtml (Node(TextNode),Node (Element),
getAttribute, setAttribute, nodeText)
import Data.ByteString (ByteString)
import Data.Maybe
import Snap.Core
import Snap.Snaplet
import Snap.Snaplet.Heist (Heist, HasHeist(heistLens), heistInit,
addSplices, liftHeist, render)
import Snap.Util.FileServe
import Text.Templating.Heist (HeistT, Template, getParamNode)
import Data.Lens.Template
import Models
import Data.Acid.Advanced (query')
import Data.Acid (AcidState, openLocalState, closeAcidState, IsAcidic, query)
import Data.Text (pack)
import Control.Monad.IO.Class (liftIO, MonadIO)
import Snap (snapletValue)
import Data.Lens.Common (getL, (^$), (^.), Lens)
import Control.Monad.Reader (ask, asks)
import Control.Applicative ((<$>))
import Data.Typeable (typeOf)
import Prelude hiding ((.), id)
import Control.Category ((.), id)
------------------------------------------------------------------------------
type AppHandler = Handler App App
--------------
-- Acid
---------------
-- Used for holding data for the snapplet
data Acid st = Acid { _state :: AcidState st }
-- Initializer function for the snapplet
seedBook = Book "9213-23123-2311"
acidInit :: SnapletInit b (Acid Book)
acidInit = makeSnaplet "storage" "Snaplet providing storage functionality" Nothing initializer
--The 'm' is the type variable of the MonadSnaplet type class. 'b' is the base state, and 'v' is the state of the current "view" snaplet (or simply, current state).
initializer :: Initializer b v (Acid Book)
initializer = do
st <- liftIO (openLocalState seedBook)
--onUnload (closeAcidState st)
return $ Acid st
-----------------------
-- Snap Global State
--------------------
data App = App
{ _heist :: Snaplet (Heist App),
_acid :: Snaplet (Acid Book)
}
makeLens ''App
----------------------------------------------------------------------------------
instance HasHeist App where
heistLens = subSnaplet heist
-----------------------------------------------
-- | Initialize app
-----------------------------------------------
appInit :: SnapletInit App App
appInit = makeSnaplet "app" "Website" Nothing $ do
h <- nestSnaplet "" heist $ heistInit "templates"
a <- nestSnaplet "isbn" acid (acidInit)
addRoutes routes --see below
addSplices [ ("menuEntry", liftHeist menuEntrySplice) ]
return $ App h a
------------------------------------------------
-- | The application's routes.
------------------------------------------------
routes :: [(ByteString, Handler App App ())]
routes = [ ("/books", handleBooks)
, ("/contact", render "contact")
, ("/isbn", liftIO doQuery >>= writeBS )
, ("", serveDirectory "static")
]
-- Is this Function signature possible? Or must it run inside Snap or other monad?
doQuery :: IO ByteString
doQuery = do -- ???????????
--somehow retrieve acid store from snaplet
--run queryIsbn on it
--return isbn string
return "BLAH"
handleBooks :: Handler App App ()
handleBooks = render "books"
Any help on what I am missing would be greatly appreciated. If something is not clear, please let me know and I'll update the question.
MathematicalOrchid is correct, the simplest answer to your problem is to use liftIO on the openLocalState call.
But from a broader view, what you're doing here has already been done for you by the snaplet-acid-state package, so I would recommend that you just use that. The repository also includes an example application demonstrating how to use it.
I have no idea about the packages you're using, but it looks like the problem is simply that openLocalState is an IO action, but your type signature requires it to be an Initializer action.
Fixing it might be as simple as stuffing a call to liftIO in there. I'm not really sure though... I don't know which module each of these types comes from.

Resources