When trying to parse some simple JSON using Aeson I get a type error I don't understand. I have the following JSON
jsonString = "[\"a\", [\"b\", \"c\"]]" :: L.ByteString
and I have defined the following imports and code:
import Data.Aeson
import GHC.Generics
import qualified Data.ByteString.Lazy as L
data Ch = Ch {
c1 :: String,
c2 :: (String, String)
} deriving (Show, Generic)
instance FromJSON Ch
When I try to use eitherDecode on this string with my Ch type I get an error
*Aeson> eitherDecode jsonString :: Either String Ch
Left "Error in $: expected record (:*:), encountered Array"
Can someone explain me the error and tell me how I should parse this JSON?
An approach that would work is
eitherDecode jsonString :: Either String (String, (String, String))
but I'd rather go to my type directly.
If you already know of a type that parses as intended then perhaps the easiest solution is to just write your instance in terms of that type and translating:
import Data.Aeson
import GHC.Generics
import qualified Data.ByteString.Lazy as L
data Ch = Ch {
c1 :: String,
c2 :: (String, String)
} deriving (Show, Generic)
instance FromJSON Ch where
parseJSON x =
do (a,(b,c)) <- parseJSON x
pure (Ch a (b,c))
And the result is:
*Main> :set -XOverloadedStrings
*Main> eitherDecode "[\"a\", [\"b\", \"c\"]]" :: Either String Ch
Right (Ch {c1 = "a", c2 = ("b","c")})
EDIT:
A more direct use of Aeson's API can be informative or preferred:
instance FromJSON Ch where
parseJSON =
withArray "Ch" $ \arr ->
-- from Data.Aeson.Types
if V.length arr /= 2
-- ^ from Data.Vector
then typeMismatch "Length should be 2" (Array arr)
-- ^ from Data.Aeson.Types
else Ch <$> parseJSON (arr ! 0) <*> parseJSON ( arr ! 1 )
Related
I have an json value of:
{
"name": "xyz1",
"extra": [
{
"this_string_A": "Hello"
},
{
"this_string_B": "World"
}
]
}
And a data type of:
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
In the above case I would want it to parse with a result of Abc "xyz1" (Just "Hello") (Just "World").
I can't figure out how to conditionally parse the values within extra (which is a JSON array) within the aeson Parser context. How can I get extra[0].this_string_a for example? I
What I tried:
I thought I could create my own Parser (Maybe String) function but ran into confusing errors:
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> myParse v
<*> myParse v
myParse :: Object -> Parser (Maybe String)
myParse x = withArray "extra" myParse2 (x)
myParse2 :: Array -> Parser (Maybe String)
myParse2 = undefined
typecheck fails with:
• Couldn't match type ‘unordered-containers-0.2.10.0:Data.HashMap.Base.HashMap
text-1.2.3.1:Data.Text.Internal.Text Value’
with ‘Value’
Expected type: Value
Actual type: Object
• In the third argument of ‘withArray’, namely ‘(x)’
And if I replace x with Object x then I get parse error of:
Left "Error in $: parsing extra failed, expected Array, but encountered Object"
Full example (run test function to test):
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
module Example where
import GHC.Generics
import Data.Aeson
import Data.Aeson.Types
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> (v.: "extra") -- find where object has key of this_string_a ??
<*> (v.: "extra") -- find where object has key of this_string_b ??
test :: Either String Abc
test = eitherDecode exampleJson
exampleJson = "{ \"name\": \"xyz1\", \"extra\": [ { \"this_string_A\": \"Hello\" }, { \"this_string_B\": \"World\" } ] }"
The withXXX "helpers" make everything kind of awkward, but here goes.
The Aeson Parser type is misnamed, and this causes confusion.
The idea with Aeson Parser objects is that they represent a monadic parse result. (This is different from the Parser objects you find in Parsec, etc., which represent actual monadic parsers.) So, you should think of a Parser a as isomorphic to an Either ParseError a -- a monadic result with the possibility of failure.
These parse results are usually combined applicatively. So if you have a parser like:
data Xyz = Xyz { x :: String, y :: String }
instance FromJSON Xyz where
parseJSON = withObject "Xyz" $ \v ->
Xyz <$> v .: "x" <*> v .: "y"
the parse results v .: "x" and v .: "y" have type Parser String which is really like Either ParseError a, and the last line of that instance is the usual method of combining successful and unsuccessful results in an applicative manner, along the lines of:
Xyz <$> Right "value_x" <*> Left "while parsing Xyz: key y was missing"
Now, the function parseJSON has type Value -> Parser a. This is what should properly be called a parser, but to avoid confusion, let's call it a "parse function". A parse function takes a JSON representation (a Value, or an Object or some other JSON thingy) and returns a parse result. The withXXX family of functions are used to adapt parse functions between JSON thingies. If you have a parse function that expects an Object, like:
\v -> Xyz <$> v .: "x" <*> v .: "y" :: Object -> Parser Xyz
and you want to adapt it to parseJSON :: Value -> Parser Xyz, you use withObject "str" :: (Object -> Parser Xyz) -> (Value -> Parser Xyz) to do it.
Getting back to your problem, if you'd like to write a core parser that looks like:
\v -> Abc <$> v .: "name" <*> extra .:? "this_string_A"
<*> extra .:? "this_string_B"
you want extra to be an Object, and you want to extract it monadically from the overall JSON object v :: Object, using appropriate withXXX helpers to adapt parse functions from one input JSON thingy type to another. So, let's write a monadic function (a parse function, in fact) to do that:
getExtra :: Object -> Parser Object
getExtra v = do
First, we monadically extract the optional "extra" component from v. We use the conditional form here, so mextra :: Maybe Value.
mextra <- v .:? "extra"
Second, let's monadically create our final Object out of "mextra". This will be the JSON Object whose keys are "this_string_A" and "this_string_B" with the array layer removed. Note the type of this case expression will be Parser Object, a parse result of type Object = HashMap key value. For the Just case, we have a Value that we expect to be an array, so let's use the withArray helper to ensure that. Note that the withArray "str" helper function takes our parse function of type \arr -> do ... :: Array -> Parser Object and adapts it to Value -> Parser Object so it can be applied to vv :: Value.
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
Now, arr is an Array = Vector Value. We hope it's an array of Objects. Let's pull the Values out as a list:
let vallst = toList arr
and then monadically traverse the list with the help of withObject to ensure they're all Objects as expected. Note the use of pure here, since we want to extract the Objects as-is without any additional processing:
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
Now, we have an objlst :: [Object]. They're a set of singleton hashmaps with disjoint keys, and the Object / hashmap we want is their union, so let's return that. The parenthesis here ends the withArray expression that's being applied to vv:
return $ HashMap.unions objlst)
For the Nothing case ("extra" not found), we merely return an empty hashmap:
Nothing -> return HashMap.empty
The full function looks like this:
getExtra :: Object -> Parser Object
getExtra v = do
mextra <- v .:? "extra"
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
let vallst = toList arr
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
return $ HashMap.unions objlst)
Nothing -> return HashMap.empty
and you use it in your parser instance like so:
instance FromJSON Abc where
parseJSON =
withObject "Abc" $ \v -> do
extra <- getExtra v
Abc <$> v .: "name" <*> extra .:? "this_string_A" <*> extra .:? "this_string_B"
With a test case:
example :: BL.ByteString
example = "{\"name\": \"xyz1\", \"extra\": [{\"this_string_A\": \"Hello\"}, {\"this_string_B\": \"World\"}]}"
main = print (eitherDecode example :: Either String Abc)
it works like so:
λ> main
Right (Abc {name = "xyz1", a = Just "Hello", b = Just "World"})
The full code:
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson (eitherDecode, FromJSON, Object, parseJSON, withArray, withObject, (.:), (.:?))
import Data.Aeson.Types (Parser)
import GHC.Generics (Generic)
import qualified Data.ByteString.Lazy as BL (ByteString)
import qualified Data.HashMap.Strict as HashMap (empty, unions)
import Data.Function ((&))
import Data.Foldable (toList)
data Abc = Abc
{ name :: String
, a :: Maybe String
, b :: Maybe String
} deriving (Generic, Show)
instance FromJSON Abc where
parseJSON =
withObject "Abc" $ \v -> do
extra <- getExtra v
Abc <$> v .: "name" <*> extra .:? "this_string_A" <*> extra .:? "this_string_B"
getExtra :: Object -> Parser Object
getExtra v = do
mextra <- v .:? "extra"
case mextra of
Just vv -> vv & withArray "Abc.extra" (\arr -> do
let vallst = toList arr
objlst <- traverse (withObject "Abc.extra[..]" pure) vallst
return $ HashMap.unions objlst)
Nothing -> return HashMap.empty
example :: BL.ByteString
example = "{\"name\": \"xyz1\", \"extra\": [{\"this_string_A\": \"Hello\"}, {\"this_string_B\": \"World\"}]}"
main = print (eitherDecode example :: Either String Abc)
Partial answer...
instance FromJSON Abc where
parseJSON = withObject "Abc" $ \v -> Abc
<$> v .: "name"
<*> (v .: "extra" >>= myParse)
<*> (v .: "extra" >>= myParse)
myParse :: Array -> Parser (Maybe String)
myParse x = withArray "extra" (lookupDictArray "this_string_a") (Array x)
lookupDictArray :: Text -> Array -> Parser (Maybe String)
lookupDictArray k a = do
let v = Vector.find (maybe False (HashMap.member k) . parseMaybe parseJSON) a
case v of
Just v' -> withObject "grrrrrrrrrrr" (\v -> v .: k) v'
Nothing -> pure Nothing
Fails to typecheck with:
src/Example.hs:32:69-77: error:
• Ambiguous type variable ‘a0’ arising from a use of
‘parseJSON’
prevents the constraint ‘(FromJSON a0)’ from being
solved.
Probable fix: use a type annotation to specify
what ‘a0’ should be.
These potential instances exist:
instance FromJSON DotNetTime
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
instance FromJSON Value
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
instance (FromJSON a, FromJSON b) => FromJSON
(Either a b)
-- Defined in ‘aeson-1.4.4.0:Data.Aeson.Types.FromJSON’
...plus 29 others
...plus 60 instances involving out-of-scope types
(use -fprint-potential-instances to see them all)
• In the first argument of ‘parseMaybe’, namely
‘parseJSON’
In the second argument of ‘(.)’, namely
‘parseMaybe parseJSON’
In the first argument of ‘Vector.find’, namely
‘(maybe False (member k) . parseMaybe
parseJSON)’
|
32 | let v = (Vector.find (maybe False (HashMap.member
k) . parseMaybe parseJSON) a)
What is the best way
Data.Aeson FromJSON instance
Convert Array to Object:
import Data.Aeson
data MixArray = MixArray {
vStr :: String,
vNum :: Int,
vBool :: Bool
} deriving Show
main = do
jsonStr = ["a",1,true]
mix <- eitherDecode $ jsonStr :: IO (Either String [MixArray])
show mix
to:
MixArray { vStr = "a", vNum = 1, vBool= true}
You can reuse the FromJSON instance for lists and the Value type, and then construct your type after pattern matching. If the pattern match fails, the parser will fail.
import Data.Text (unpack)
instance FromJSON MixArray where
parseJSON jsn = do
[String s, Number n, Bool b] <- parseJSON jsn
return MixArray { vStr = unpack s, vNum = truncate n, vBool = b }
I have a data type where one of the fields is a list of one of n other data types (n is small and the types are known in advance). I would like to make a JSON parser but I can't quite figure it out. I've tried creating a Pet type class and making them both instances of it, but it seemed to be a dead end. Any help would be appreciated!
As a simplified example - I have a Person data type who can have a list of pets, either dogs or cats - but not a mix of both.
Here's the example:
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import Data.Aeson
import Data.ByteString.Lazy as L
import Data.Aeson.Types (Parser)
import Control.Monad (mzero)
data Person = Person {
name :: String,
species :: String,
pets :: [?] -- a list of dogs OR cats
} deriving Show
instance FromJSON (Person a) where
parseJSON (Object v) = ???
data Dog = Dog {
dogField :: String
} deriving Show
instance FromJSON Dog where
parseJSON (Object v) = Dog <$>
v .: "dogField"
data Cat = Cat {
catField :: String
} deriving Show
instance FromJSON Cat where
parseJSON (Object v) = Cat <$>
v .: "catField"
A standard way of representing either one type or another is to use the Either type, e.g.:
data Person { ..., pets :: Either [Dog] [Cat] }
Also, you might be interested in use GHC Generics to auto-derive the To/FromJSON instances.
An example with a data structure that uses Either:
{-# LANGUAGE DeriveGeneric #-}
import Data.Aeson
import GHC.Generics
data Person = Person {
name :: String,
species :: String,
pets :: Either [String] [Int]
} deriving (Show,Generic)
instance ToJSON Person -- instances are auto-derived
instance FromJSON Person
doit = do
let me = Person "user5402" "Human" (Right [42])
print $ encode me
If you have more than two alternatives you can easily create your own sum type like this:
-- assume the possible pet types are: Dog, Cat, Rodent, Reptile
data Pets = Dogs [Dog] | Cats [Cat] | Rodents [Rodent] | Reptiles [Reptile]
deriving (Show, Generic)
data Person { ..., pets :: Pets }
deriving (Show, Generic)
doit = do
let me = Person "user5402" "Human" (Rodents [agerbil, amouse])
print $ encode me
where agerbil and amouse are Rodent values.
I'm modifying #user5402's answer because I don't like the "tag" and "contents" fields that Generics added. Also accepting his answer since he gave me the key insight of how to structure the sum type
instance FromJSON Pets where
parseJSON (Object o) = (parsePets o "pets")
parseJSON _ = mzero
parsePets :: Object -> T.Text -> Parser Pets
parsePets o key = case H.lookup key o of
Nothing -> fail $ "key " ++ show key ++ " not present"
Just v -> parseToCatsOrDogs (o .: "species") v
{-# INLINE parsePets #-}
parseToCatsOrDogs :: Parser String -> Value -> Parser Pets
parseToCatsOrDogs speciesParser (Array v) = speciesParser >>= \species -> case species of
"dog" -> (V.mapM (\x -> parseJSON $ x) v) >>= \ dogVector -> return $ Dogs (V.toList dogVector)
"cat" -> (V.mapM (\x -> parseJSON $ x) v) >>= \ catVector -> return $ Cats (V.toList catVector)
_ -> mzero
parseToCatsOrDogs _ _ = mzero
I have the following code:
{-# LANGUAGE DeriveGeneric, OverloadedStrings #-}
import Data.Aeson
import GHC.Generics
data CharClass = Fighter | Rogue | Wizard deriving (Generic, Show)
instance FromJSON CharClass
instance ToJSON CharClass
I can encode values of this type:
*Main> encode Fighter
"\"Fighter\""
But round-tripping doesn't work:
*Main> eitherDecode $ encode Fighter
Left "Failed reading: satisfy"
*Main> :t eitherDecode $ encode Fighter
eitherDecode $ encode Fighter :: FromJSON a => Either String a
It looks a lot like this answered question, but adding the expected type doesn't work:
*Main> eitherDecode $ encode Fighter :: Either String CharClass
Left "Failed reading: satisfy"
Interestingly, it does work for fromJSON/toJSON:
*Main> fromJSON $ toJSON Fighter :: Result CharClass
Success Fighter
Making at least one of the constructors non-nullary also makes things work, like if I do this:
data CharClass = Fighter Int | Rogue | Wizard deriving (Generic, Show)
And then try to round-trip again:
*Main> decode $ encode (Fighter 0) :: Maybe CharClass
Just (Fighter 0)
I'm sure I'm missing something simple, but attempting to trace this through the generic code made my head spin.
JSON is fundamentally a collection of key-value pairs, where values can be a few primitive types or another collection of key-value pairs. Nullary types just don't fit in very well with the whole idea of being JSON entities by themselves. However, they work fine when placed within other types that mesh well with the JSON concept.
data F = F { a :: CharClass, b :: CharClass }
deriving (Generic, Show)
instance FromJSON F
instance ToJSON F
This looks more like the sort of key-value collection JSON was designed for.
*Main> let x = F Fighter Rogue
*Main> x
F {a = Fighter, b = Rogue}
*Main> decode $ encode x :: Maybe F
Just (F {a = Fighter, b = Rogue})
The version of aeson that stack installed on my machine was from the 0.8 series, and in aeson 0.8 or earlier, only objects and arrays were parsed at the root level.
In aeson 0.9, decode uses value parser. So nullable object (represented as string) at top-level will work.
For 0.8 the below example works. I don't know why decodeWith isn't exposed.
{-# LANGUAGE DeriveGeneric #-}
import Control.Applicative
import GHC.Generics
import Data.Aeson
import Data.Aeson.Parser
import Data.ByteString.Lazy as L
import Data.Attoparsec.ByteString.Char8 (endOfInput, skipSpace)
import qualified Data.Attoparsec.Lazy as L
data CharClass = Fighter | Rogue | Wizard deriving (Generic, Show)
instance ToJSON CharClass
instance FromJSON CharClass
decodeWith p to s =
case L.parse p s of
L.Done _ v -> case to v of
Success a -> Just a
_ -> Nothing
_ -> Nothing
{-# INLINE decodeWith #-}
valueEOF = value <* skipSpace <* endOfInput
decodeValue :: (FromJSON a) => L.ByteString -> Maybe a
decodeValue = decodeWith valueEOF fromJSON
main :: IO ()
main = print (decodeValue (encode Fighter) :: Maybe CharClass)
I'm using this code:
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson
import Data.Maybe
import Data.ByteString.Lazy
import Control.Applicative
import Debug.Trace
import Control.Monad
import qualified Data.Aeson.Types as T
main = do
res <- liftA show (liftA decodeOriginal (Data.ByteString.Lazy.readFile "./a.json"))
Prelude.putStrLn res
interpretResult :: Maybe String -> String
interpretResult Nothing = "Error."
interpretResult x = fromJust x
data TotalLine1 = TotalLine1 {
timestamp :: Integer,
value :: Integer
} deriving (Eq, Show)
data Original = Original {
totals :: [TotalLine1]
} deriving (Eq, Show)
instance FromJSON Original where
parseJSON (Object v) = traceStack "Original" (Original <$> (parseJSON =<< (v .: "visitors.total")))
parseJSON _ = mzero
instance FromJSON TotalLine1 where
parseJSON (Object v) = TotalLine1 <$>
v .: "timestamp" <*>
v .: "value"
decodeOriginal :: ByteString -> Maybe Original
decodeOriginal b = traceStack "decoding" (do
a <- decode b :: Maybe Original
return a)
to try and parse JSON like this:
{
visitors.total: [
{
timestamp: 1365548400,
value: 1
},
{
timestamp: 1365548700,
value: 2
},
{
timestamp: 1365549000,
value: 5
},
]
}
But main just returns Nothing every time. What have I done wrong? It seems that even parseJSON isn't being called for Original.
Your JSON file is not valid.
On the one hand, the names of the fields have to be quoted,
"timestamp"
etc. and on the other, you have a trailing comma in the list of TotalLine1s, which causes the decoding of the ByteString to a Value to fail. Quote the field names and remove the trailing comma, and it works.