What does the dot equals (.=) and dot colon (.:) syntax mean in this example taken from the Aeson JSON library?
instance ToJSON Coord where
toJSON (Coord xV yV) = object [ "x" .= xV,
"y" .= yV ]
-- A FromJSON instance allows us to decode a value from JSON. This
-- should match the format used by the ToJSON instance.
instance FromJSON Coord where
parseJSON (Object v) = Coord <$>
v .: "x" <*>
v .: "y"
parseJSON _ = empty
Full example on Github: https://github.com/bos/aeson/blob/master/examples/Simplest.hs
The comments are all correct. You can Hoogle for these function and find the correct haddock documentation, which shows these operators are functions defined in a library and not some integral part of the Haskell language. In general, a starting . is used by libraries to define infix functions because many other desirable symbols are either invalid for functions (ex :.) or the singular character is reserved syntax in Haskell (ex =, :).
(.=) is used to help create JSON objects while .: is for parsing JSON. Normally with Aeson you have a one-to-one match between some data structure and some set of JSON messages, so for example:
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson
import Data.Text
import Control.Applicative ((<$>), (<*>))
import Control.Monad (mzero)
data ServerMsg = ServerMsg.
{ msgId :: Int
, msg :: Text
} deriving (Show)
instance ToJSON ServerMsg where
toJSON d = object [ "msgId" .= msgId d
, "msg" .= msg d
]
instance FromJSON ServerMsg where
parseJSON (Object o) =
ServerMsg <$> o .: "msgId" <*> o .: "msg"
parseJSON _ = mzero
testJSON :: Value
testJSON = toJSON (ServerMsg 398242 "Hello user3526171")
testParse :: Result ServerMsg
testParse = fromJSON testJSON
Now you can load this module in GHCi and play around easily:
*Main> testJSON
Object fromList [("msg",String "Hello user3526171"),("msgId",Number 398242.0)]
*Main> testParse
Success (ServerMsg {msgId = 398242, msg = "Hello user3526171"})
Related
here is my JSON structure, there are N records that has a name as ID to represent a children
{"Kids":
{"Jack":{"age":10}
,"Jane":{"age":9}
, .......
}
}
in the data type in Haskell
data Kid = Kid { name::String, age::Int}
instance FromJSON Kid where
parseJSON (Object v) =
....
question is ,how to make the key ( name ) as part of the constructor ? the expected output signature is like:
decode "input json string" -> [Kid]
when the expect decode function was called, it will return a list of type Kid. Thanks for reading this & appreciate any help .
By using the withObject function, you get access to an Object which is actually a KeyMap which you can manipulate much like the usual Map from e.g. containers. If you're on an older aeson version, Object will instead be a HashMap, so you can use that as well.
EDIT: I remember that Map itself also has a FromJSON, so you can probably use that instead for a shorter "solution":
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson (FromJSON(..), withObject, (.:), fromJSON)
import Data.Map (Map)
import qualified Data.Map as Map
data Kid = MkKid {name :: String, age :: Int}
newtype Kids = MkKids {unKids :: [Kid]}
instance FromJSON Kids where
parseJSON = withObject "Kids" $ \o -> do
kvmap <- o .: "Kids"
pure $ MkKids $ map (uncurry MkKid) $ Map.toList kvmap
Old "solution", which manipulates they KeyMap
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson.Key (Key)
import qualified Data.Aeson.Key as Key
import qualified Data.Aeson.KeyMap as KeyMap
import Data.Aeson.Types (Parser, Object, Value)
import Data.Aeson (FromJSON(..), withObject, (.:), fromJSON)
data Kid = MkKid {name :: String, age :: Int}
newtype Kids = MkKids {unKids :: [Kid]}
instance FromJSON Kids where
parseJSON = withObject "Kids" $ \o -> do
inner <- o .: "Kids"
withObject "inner" parseKids inner
where
parseKids :: Object -> Parser Kids
parseKids obj =
fmap MkKids $ traverse toKid $ KeyMap.toList obj
toKid :: (Key, Value) -> Parser Kid
toKid (k, v) = do
age <- parseJSON v
let name = Key.toString k
pure $ MkKid {name, age}
I have an json value of:
{
"name": "xyz1",
"extra": [
{
"this_string_A": "Hello"
},
{
"this_string_B": "World"
}
]
}
And a data type of:
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
In the above case I would want it to parse with a result of Abc "xyz1" (Just "Hello") (Just "World").
I can't figure out how to conditionally parse the values within extra (which is a JSON array) within the aeson Parser context. How can I get extra[0].this_string_a for example? I
What I tried:
I thought I could create my own Parser (Maybe String) function but ran into confusing errors:
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> myParse v
<*> myParse v
myParse :: Object -> Parser (Maybe String)
myParse x = withArray "extra" myParse2 (x)
myParse2 :: Array -> Parser (Maybe String)
myParse2 = undefined
typecheck fails with:
• Couldn't match type ‘unordered-containers-0.2.10.0:Data.HashMap.Base.HashMap
text-1.2.3.1:Data.Text.Internal.Text Value’
with ‘Value’
Expected type: Value
Actual type: Object
• In the third argument of ‘withArray’, namely ‘(x)’
And if I replace x with Object x then I get parse error of:
Left "Error in $: parsing extra failed, expected Array, but encountered Object"
Full example (run test function to test):
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
module Example where
import GHC.Generics
import Data.Aeson
import Data.Aeson.Types
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> (v.: "extra") -- find where object has key of this_string_a ??
<*> (v.: "extra") -- find where object has key of this_string_b ??
test :: Either String Abc
test = eitherDecode exampleJson
exampleJson = "{ \"name\": \"xyz1\", \"extra\": [ { \"this_string_A\": \"Hello\" }, { \"this_string_B\": \"World\" } ] }"
The withXXX "helpers" make everything kind of awkward, but here goes.
The Aeson Parser type is misnamed, and this causes confusion.
The idea with Aeson Parser objects is that they represent a monadic parse result. (This is different from the Parser objects you find in Parsec, etc., which represent actual monadic parsers.) So, you should think of a Parser a as isomorphic to an Either ParseError a -- a monadic result with the possibility of failure.
These parse results are usually combined applicatively. So if you have a parser like:
data Xyz = Xyz { x :: String, y :: String }
instance FromJSON Xyz where
parseJSON = withObject "Xyz" $ \v ->
Xyz <$> v .: "x" <*> v .: "y"
the parse results v .: "x" and v .: "y" have type Parser String which is really like Either ParseError a, and the last line of that instance is the usual method of combining successful and unsuccessful results in an applicative manner, along the lines of:
Xyz <$> Right "value_x" <*> Left "while parsing Xyz: key y was missing"
Now, the function parseJSON has type Value -> Parser a. This is what should properly be called a parser, but to avoid confusion, let's call it a "parse function". A parse function takes a JSON representation (a Value, or an Object or some other JSON thingy) and returns a parse result. The withXXX family of functions are used to adapt parse functions between JSON thingies. If you have a parse function that expects an Object, like:
\v -> Xyz <$> v .: "x" <*> v .: "y" :: Object -> Parser Xyz
and you want to adapt it to parseJSON :: Value -> Parser Xyz, you use withObject "str" :: (Object -> Parser Xyz) -> (Value -> Parser Xyz) to do it.
Getting back to your problem, if you'd like to write a core parser that looks like:
\v -> Abc <$> v .: "name" <*> extra .:? "this_string_A"
<*> extra .:? "this_string_B"
you want extra to be an Object, and you want to extract it monadically from the overall JSON object v :: Object, using appropriate withXXX helpers to adapt parse functions from one input JSON thingy type to another. So, let's write a monadic function (a parse function, in fact) to do that:
getExtra :: Object -> Parser Object
getExtra v = do
First, we monadically extract the optional "extra" component from v. We use the conditional form here, so mextra :: Maybe Value.
mextra <- v .:? "extra"
Second, let's monadically create our final Object out of "mextra". This will be the JSON Object whose keys are "this_string_A" and "this_string_B" with the array layer removed. Note the type of this case expression will be Parser Object, a parse result of type Object = HashMap key value. For the Just case, we have a Value that we expect to be an array, so let's use the withArray helper to ensure that. Note that the withArray "str" helper function takes our parse function of type \arr -> do ... :: Array -> Parser Object and adapts it to Value -> Parser Object so it can be applied to vv :: Value.
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
Now, arr is an Array = Vector Value. We hope it's an array of Objects. Let's pull the Values out as a list:
let vallst = toList arr
and then monadically traverse the list with the help of withObject to ensure they're all Objects as expected. Note the use of pure here, since we want to extract the Objects as-is without any additional processing:
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
Now, we have an objlst :: [Object]. They're a set of singleton hashmaps with disjoint keys, and the Object / hashmap we want is their union, so let's return that. The parenthesis here ends the withArray expression that's being applied to vv:
return $ HashMap.unions objlst)
For the Nothing case ("extra" not found), we merely return an empty hashmap:
Nothing -> return HashMap.empty
The full function looks like this:
getExtra :: Object -> Parser Object
getExtra v = do
mextra <- v .:? "extra"
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
let vallst = toList arr
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
return $ HashMap.unions objlst)
Nothing -> return HashMap.empty
and you use it in your parser instance like so:
instance FromJSON Abc where
parseJSON =
withObject "Abc" $ \v -> do
extra <- getExtra v
Abc <$> v .: "name" <*> extra .:? "this_string_A" <*> extra .:? "this_string_B"
With a test case:
example :: BL.ByteString
example = "{\"name\": \"xyz1\", \"extra\": [{\"this_string_A\": \"Hello\"}, {\"this_string_B\": \"World\"}]}"
main = print (eitherDecode example :: Either String Abc)
it works like so:
λ> main
Right (Abc {name = "xyz1", a = Just "Hello", b = Just "World"})
The full code:
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson (eitherDecode, FromJSON, Object, parseJSON, withArray, withObject, (.:), (.:?))
import Data.Aeson.Types (Parser)
import GHC.Generics (Generic)
import qualified Data.ByteString.Lazy as BL (ByteString)
import qualified Data.HashMap.Strict as HashMap (empty, unions)
import Data.Function ((&))
import Data.Foldable (toList)
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
instance FromJSON Abc where
parseJSON =
withObject "Abc" $ \v -> do
extra <- getExtra v
Abc <$> v .: "name" <*> extra .:? "this_string_A" <*> extra .:? "this_string_B"
getExtra :: Object -> Parser Object
getExtra v = do
mextra <- v .:? "extra"
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
let vallst = toList arr
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
return $ HashMap.unions objlst)
Nothing -> return HashMap.empty
example :: BL.ByteString
example = "{\"name\": \"xyz1\", \"extra\": [{\"this_string_A\": \"Hello\"}, {\"this_string_B\": \"World\"}]}"
main = print (eitherDecode example :: Either String Abc)
Partial answer...
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> (v .: "extra" >>= myParse)
<*> (v .: "extra" >>= myParse)
myParse :: Array -> Parser (Maybe String)
myParse x = withArray "extra" (lookupDictArray "this_string_a") (Array x)
lookupDictArray :: Text -> Array -> Parser (Maybe String)
lookupDictArray k a = do
let v = Vector.find (maybe False (HashMap.member k) . parseMaybe parseJSON) a
case v of
Just v' -> withObject "grrrrrrrrrrr" (\v -> v .: k) v'
Nothing -> pure Nothing
Fails to typecheck with:
src/Example.hs:32:69-77: error:
• Ambiguous type variable ‘a0’ arising from a use of
‘parseJSON’
prevents the constraint ‘(FromJSON a0)’ from being
solved.
Probable fix: use a type annotation to specify
what ‘a0’ should be.
These potential instances exist:
instance FromJSON DotNetTime
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
instance FromJSON Value
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
instance (FromJSON a, FromJSON b) => FromJSON
(Either a b)
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
...plus 29 others
...plus 60 instances involving out-of-scope types
(use -fprint-potential-instances to see them all)
• In the first argument of ‘parseMaybe’, namely
‘parseJSON’
In the second argument of ‘(.)’, namely
‘parseMaybe parseJSON’
In the first argument of ‘Vector.find’, namely
‘(maybe False (member k) . parseMaybe
parseJSON)’
|
32 | let v = (Vector.find (maybe False (HashMap.member
k) . parseMaybe parseJSON) a)
Let's say there is a JSON like:
{
"bob_id" : {
"name": "bob",
"age" : 20
},
"jack_id" : {
"name": "jack",
"age" : 25
}
}
Is it possible to parse it to [Person] with Person defined like below?
data Person = Person {
id :: Text
,name :: Text
,age :: Int
}
You cannot define an instance for [Person] literally, because aeson already includes an instance for [a], however you can create a newtype, and provide an instance for that.
Aeson also includes the instance FromJSON a => FromJSON (Map Text a), which means if aeson knows how to parse something, it knows how to parse a dict of that something.
You can define a temporary datatype resembling a value in the dict, then use the Map instance to define FromJSON PersonList, where newtype PersonList = PersonList [Person]:
data PersonInfo = PersonInfo { infoName :: Text, infoAge :: Int }
instance FromJSON PersonInfo where
parseJSON (Object v) = PersonInfo <$> v .: "name" <*> v .: "age"
parseJSON _ = mzero
data Person = Person { id :: Text, name :: Text, age :: Int }
newtype PersonList = PersonList [Person]
instance FromJSON PersonList where
parseJSON v = fmap (PersonList . map (\(id, PersonInfo name age) -> Person id name age) . M.toList) $ parseJSON v
If you enable FlexibleInstances, you can make instance for [Person]. You can parse your object to Map Text Value and then parse each element in map:
{-# LANGUAGE UnicodeSyntax, OverloadedStrings, FlexibleInstances #-}
module Person (
) where
import Data.Aeson
import Data.Aeson.Types
import Data.Text.Lazy
import Data.Text.Lazy.Encoding
import Data.Map (Map)
import qualified Data.Map as M
data Person = Person {
id ∷ Text,
name ∷ Text,
age ∷ Int }
deriving (Eq, Ord, Read, Show)
instance FromJSON [Person] where
parseJSON v = do
objs ← parseJSON v ∷ Parser (Map Text Value)
sequence [withObject "person"
(\v' → Person i <$> v' .: "name" <*> v' .: "age") obj |
(i, obj) ← M.toList objs]
test ∷ Text
test = "{\"bob_id\":{\"name\":\"bob\",\"age\":20},\"jack_id\":{\"name\":\"jack\",\"age\":25}}"
res ∷ Maybe [Person]
res = decode (encodeUtf8 test)
mniip's answer converts the JSON Object to a Map, which leads to a result list sorted by ID. If you don't need the results sorted in that fashion, it's probably better to use a more direct approach to speed things up. In particular, an Object is really just a HashMap Text Value, so we can use HashMap operations to work with it.
Note that I renamed the id field to ident, because most Haskell programmers will assume that id refers to the identity function in Prelude or to the more general identity arrow in Control.Category.
module Aes where
import Control.Applicative
import Data.Aeson
import Data.Text (Text)
import qualified Data.HashMap.Strict as HMS
data PersonInfo = PersonInfo { infoName :: Text, infoAge :: Int }
instance FromJSON PersonInfo where
-- Use mniip's definition here
data Person = Person { ident :: Text, name :: Text, age :: Int }
newtype PersonList = PersonList [Person]
instance FromJSON PersonList where
parseJSON (Object v) = PersonList <$> HMS.foldrWithKey go (pure []) v
where
go i x r = (\(PersonInfo nm ag) rest -> Person i nm ag : rest) <$>
parseJSON x <*> r
parseJSON _ = empty
Note that, like Alexander VoidEx Ruchkin's answer, this sequences the conversion from PersonInfo to Person explicitly within the Parser monad. It would therefore be easy to modify it to produce a parse error if the Person fails some sort of high-level validation. Alexander's answer also demonstrates the utility of the withObject combinator, which I'd have used if I'd known it existed.
I'm using Aeson to parse json quote data from Yahoo's API. A quote might look like this:
{
"date": "2010-03-10",
"Date": "2010-03-10",
"Open": "0.37",
"High": "0.37",
"Low": "0.34",
"Close": "0.35",
"Volume": "443000",
"Adj_Close": "0.35"
}
(that's using this YQL query)
As you can see, the numbers are quoted. I can write a fromJSON implementation like this:
instance FromJSON Quote where
parseJSON (Object o) =
Quote <$> o .: "Date"
<*> o .: "Open"
<*> o .: "High"
<*> o .: "Low"
<*> o .: "Close"
<*> o .: "Volume"
parseJSON _ = mzero
which is the same as what would be derived. Unfortunately this only works if I want Open, High, Low, etc, to be a Text type. The parse fails if I try and have any of those fields as Double, say.
I can write this:
<*> (fmap read $ o .: "Open")
to get it as anything I like, but this uses read, which is a partial function. How would I get the above functionality without using a partial function?
First, find a safe read function. I had to hoogle it.
Second, you have to use more than just applicative to get choice. In the below, I used readMay for the safe read and made a helper function to extract the fields while applying the Read instance.
{-# LANGUAGE OverloadedStrings, NoMonomorphismRestriction #-}
import Safe
import Data.Aeson
import Data.Aeson.Types (Parser)
import Data.Text as T
import Data.Word
import Control.Applicative
import Control.Monad
data Quote = Quote Text Double Double Double Double Word64
instance FromJSON Quote where
parseJSON (Object o) = do
let readField :: (Read a) => T.Text -> Parser a
readField f = do
v <- o .: f
case readMay (T.unpack v) of
Nothing -> fail $ "Bad Field: " ++ T.unpack f
Just r -> return r
Quote <$> o .: "Date"
<*> readField "Open"
<*> readField "Close"
<*> readField "Low"
<*> readField "High"
<*> readField "Volume"
parseJSON _ = mzero
To make long stroy short, my code is about parse a json file using aeson
Here is my two pieces of code:
a.hs
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson
import qualified Data.ByteString.Lazy.Char8 as C
import Control.Monad
import Control.Applicative
data AuctionInfo = AuctionInfo {
realm :: Realm ,
alliance :: Auctions ,
horde :: Auctions ,
neutral :: Auctions
} deriving (Show )
instance FromJSON AuctionInfo where
parseJSON (Object o) = do
r <- o .: "realm" >>= parseJSON
a <- o .: "alliance" >>= parseJSON
h <- o .: "horde" >>= parseJSON
n <- o .: "neutral" >>= parseJSON
return $ AuctionInfo r a h n
parseJSON _ = mzero
data Realm = Realm { name2 :: String , slug:: String} deriving (Show )
instance FromJSON Realm where
parseJSON (Object o) = Realm <$>
o .: "name" <*>
o .: "slug"
parseJSON _ = mzero
data Auctions = Auctions {auctions :: [Auc]} deriving (Show)
instance FromJSON Auctions where
parseJSON (Object o ) = Auctions <$> o.: "auctions"
parseJSON _ = mzero
data Auc = Auc {
auc :: Integer,
itme :: Int,
owner :: String,
bid :: Integer,
buyout ::Integer,
quantity :: Int,
timeLeft :: String,
rand :: Integer,
seed :: Integer
} deriving (Show )
instance FromJSON Auc where
parseJSON (Object o ) = Auc <$>
o .: "auc" <*>
o .: "item" <*>
o .: "owner" <*>
o .: "bid" <*>
o .: "buyout" <*>
o .: "quantity" <*>
o .: "timeLeft" <*>
o .: "rand" <*>
o .: "seed"
parseJSON _ = mzero
main = do
au<- C.readFile "a.json"
let x = decode au :: Maybe AuctionInfo
case x of
Just a -> do
{-putStrLn.show $ a-}
putStrLn .show.length.auctions.alliance $ a
putStrLn "ok"
Nothing -> putStrLn "fail"
my json test file
And test steps:
save the code , and name it a.hs (or what you want)
save the test data ,name it a.json (do not change its name)
if you have not install aeson, $ cabal install aseon
$ ghc a.hs -o a
$ ./a
What I get from the output is "fail".
And when I run the command $ runghc a.hs for a few times ,
I even got some ok and some fail mixed together.
I have also tried this code on my linux and 64bit mac ghc, they all output ok as I expected.
One of my friends has also tried this code on his 32bit mac ghc, fail too. And he told me that he played some black magic to my code and changed one line into
let x = decode $(C.pack. C.unpack) au :: Maybe AuctionInfo
then the output is ok. But when I did the same black magic, the output is still fail.
I just want to make sure is this my bug or a bug of ghc, or how can I determine that.
I'm not sure if the behaviour is related to this, but you absolutely shouldn't use Data.ByteString.Lazy.Char8 since that's only for 8-bit ASCII data and your input is UTF-8.
Try replacing that import with
import qualified Data.ByteString.Lazy as BL
and use BL.readFile to read in the data (of course the actual name doesn't matter, but BL is the idiomatic shorthand for the lazy bytestring package).
Note that usually you would use Data.Text for handling unicode text, but in this case the aeson API expects the binary (i.e. ByteString) representation and handles decoding the unicode internally.
EDIT: Actually now that I've thought about this some more, I don't think the problem is with using Char8 after all (although the point stands about not using it for unicode text in general) as you are not doing any conversions from String or Char (expect for the C.pack . C.unpack experiment, which would break all multi-byte characters).