I'm looking to have my Haskell program read settings from an external file, to avoid recompiling for minor changes. Being familiar with YAML, I thought it would be a good choice. Now I have to put the two pieces together. Google hasn't been very helpful so far.
A little example code dealing with reading and deconstructing YAML from a file would be very much appreciated.
If I'm interested in what packages are available, I go to hackage, look at the complete package list, and then just search-in-page for the keyword. Doing that brings up these choices (along with a few other less compelling ones):
yaml: http://hackage.haskell.org/package/yaml
HsSyck: http://hackage.haskell.org/package/HsSyck
and a wrapper around HsSyck called yaml-light: http://hackage.haskell.org/package/yaml-light
Both yaml and HsSyck look updated relatively recently, and appear to be used by other packages in widespread use. You can see this by checking the reverse deps:
http://packdeps.haskellers.com/reverse/HsSyck
http://packdeps.haskellers.com/reverse/yaml
Of the two, yaml has more deps, but that is because it is part of the yesod ecosystem. One library that depends on HsSyck is yst, which I happen to know is actively maintained, so that indicates to me that HsSyck is fine too.
The next step in making my choice would be to browse through the documentation of both libraries and see which had the more appealing api for my purposes.
Of the two, it looks like HsSyck exposes more structure but not much else, while yaml goes via the json encodings provided by aeson. This indicates to me that the former is probably more powerful while the latter is more convenient.
A simple example:
First you need a test.yml file:
db: /db.sql
limit: 100
Reading YAML in Haskell
{-# LANGUAGE DeriveGeneric #-}
import GHC.Generics
import Data.Yaml
data Config = Config { db :: String
, limit :: Int
} deriving (Show, Generic)
instance FromJSON Config
main :: IO ()
main = do
file <- decodeFile "test.yml" :: IO (Maybe Config)
putStrLn (maybe "Error" show file)
With yamlparse-applicative you can describe your YAML parser in a static analysis-friendly way, so you can get a description of the YAML format from a parser for free. I'm going to use Matthias Braun's example format for this one:
{-# LANGUAGE ApplicativeDo, RecordWildCards, OverloadedStrings #-}
import Data.Yaml
import Data.Aeson.Types (parse)
import YamlParse.Applicative
import Data.Map (Map)
import qualified Data.Text.IO as T
data MyType = MyType
{ stringsToStrings :: Map String String
, mapOfLists :: Map String [String]
} deriving Show
parseMyType :: YamlParser MyType
parseMyType = unnamedObjectParser $ do
stringsToStrings <- requiredField' "strings_to_strings"
mapOfLists <- requiredField' "map_of_lists"
pure MyType{..}
main :: IO ()
main = do
T.putStrLn $ prettyParserDoc parseMyType
yaml <- decodeFileThrow "config/example.yaml"
print $ parse (implementParser parseMyType) yaml
Note that main is able to print out a schema before even seeing an instance:
strings_to_strings: # required
<key>: <string>
map_of_lists: # required
<key>: - <string>
Success
(MyType
{ stringsToStrings = fromList
[ ("key_one","val_one")
, ("key_two","val_two")
]
, mapOfLists = fromList
[ ("key_one",["val_one","val_two","val_three"])
, ("key_two",["val_four","val_five"])
]
})
Here's how to parse specific objects from your YAML file using the yaml library.
Let's parse parts of this file, config/example.yaml:
# A map from strings to strings
strings_to_strings:
key_one: val_one
key_two: val_two
# A map from strings to list of strings
map_of_lists:
key_one:
- val_one
- val_two
- val_three
key_two:
- val_four
- val_five
# We won't parse this
not_for: us
This module parses strings_to_strings and map_of_lists individually and puts them into a custom record, MyType:
{-# Language OverloadedStrings, LambdaCase #-}
module YamlTests where
import Data.Yaml ( decodeFileEither
, (.:)
, parseEither
, prettyPrintParseException
)
import Data.Map ( Map )
import Control.Applicative ( (<$>) )
import System.FilePath ( FilePath
, (</>)
)
data MyType = MyType {stringsToStrings :: Map String String,
mapOfLists :: Map String [String]} deriving Show
type ErrorMsg = String
readMyType :: FilePath -> IO (Either ErrorMsg MyType)
readMyType file =
(\case
(Right yamlObj) -> do
stringsToStrings <- parseEither (.: "strings_to_strings") yamlObj
mapOfLists <- parseEither (.: "map_of_lists") yamlObj
return $ MyType stringsToStrings mapOfLists
(Left exception) -> Left $ prettyPrintParseException exception
)
<$> decodeFileEither file
yamlTest = do
parsedValue <- readMyType $ "config" </> "example.yaml"
print parsedValue
Run yamlTest to see the parsing result.
Related
I'm using aeson-schemas-1.0.3 and I want to construct values of Object Example without round-tripping through an external serialized representation. It seems like a hack and I'm worried about the performance impact.
I have this schema defined:
type Example = [schema|
{
example: Text,
}
|]
I want to be able to write something like this:
coerceJson $ object [ "example" .= ("Example" :: Text) ]
I have a workaround which does allow that, but it involves encoding to a ByteString and decoding to the Object of the desired SchemaType, which seems expensive and inelegant:
coerceJson :: FromJSON a => Value -> a
coerceJson = fromJust . decode . encode
This seems terribly inefficient.
Here's an SSCCE (Short, Self Contained, Correct (Compilable), Example) with my hack workaround employed. It works, but I'm convinced there's a better solution.
#!/usr/bin/env stack
{- stack
runghc
--resolver lts-14.15
--package aeson-schemas-1.0.3
--package aeson
--package text
-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes #-}
{-# LANGUAGE TypeFamilies #-}
import Data.Aeson (decode, encode, object, (.=), FromJSON, Value)
import Data.Aeson.Schema
import Data.Aeson.Text (encodeToLazyText)
import Data.Maybe (fromJust)
import qualified Data.Text.IO as T
import Data.Text(Text)
import Data.Text.Lazy (toStrict)
main :: IO ()
main = do
let example = coerceJson $ object [ "example" .= ("Example" :: Text) ]
useExample example
useExample :: Object Example -> IO ()
useExample example = T.putStrLn $ toStrict $ encodeToLazyText $ object [
"example" .= [get| example.example|]
]
coerceJson :: FromJSON a => Value -> a
coerceJson = fromJust . decode . encode
type Example = [schema|
{
example: Text,
}
|]
In aeson-schemas how do you construct an Object of a SchemaType without encoding to text and decoding back?
I'm the author of aeson-schemas. There is currently no way to make a literal Object. The issue with what you're trying to do is, how do you know that the literal Object matches the schema? It's possible I could make an unsafeObject quasiquoter that would assume the object matches the schema you type it as.
I know this is old, but if you're still having problems with this, what exactly is your use-case? Often times, you'll be loading JSON data from an external source, like an API or a file.
I'm getting to grips with writing an API in Haskell using Scotty. My files are provided below. My questions are:
In the routes definition, I'm extracting from liftIO whatsTheTime in a do block. This works, but it seems verbose. Is there a nicer syntax?
In the whatsTheTime definition, I'm needing to do fromString. I'd have thought OverloadedString would take care of that, but that's not the case. I'd really appreciate it if somebody pointed out why it doesn't work without fromString.
In a stack project, if I need a directive like OverloadedStrings, do I need to include it every file that needs it, or just at the top of the main entrypoint?
Api.hs:
{-# LANGUAGE OverloadedStrings #-}
module Api
( whatsTheTime
) where
import Data.Time (getCurrentTime)
import Web.Scotty
import Data.String
whatsTheTime :: IO (ActionM ())
whatsTheTime = do
time <- getCurrentTime
return $ text $ fromString ("The time is now " ++ show time)
Main.hs:
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE DeriveGeneric #-}
module Main where
import Api
import Web.Scotty
import Control.Monad.IO.Class
routes = do
get "/" $ do
res <- liftIO whatsTheTime
res
main :: IO ()
main = do
putStrLn "Starting server..."
scotty 3000 routes
(1) This:
do
res <- liftIO whatsTheTime
res
Desugars to this:
liftIO whatsTheTime >>= \ res -> res
If you look at the type of \ m -> m >>= id:
(Monad m) => m (m a) -> m a
That’s exactly the type of join (Hoogle), so you can use:
get "/" $ join $ liftIO whatsTheTime
join is a common idiom for “execute this action which returns an action, and also execute the returned action”.
(2) OverloadedStrings is for overloading of string literals. You have an overloaded literal "The time is now ", but you constrain it to be of type String by using it as an operand of (++) with a String (the result of show time). You can pack the result of show time as a Text instead using fromString or Data.Text.pack:
import Data.Monoid ((<>))
import qualified Data.Text as Text
-- ...
return $ text $ "The time is now " <> Text.pack (show time)
(3) LANGUAGE pragmas operate per file; as #mgsloan notes, you can add OverloadedStrings to the default-extensions: field of your library or executable in your .cabal file.
I need to parse iCalendar format, which is basically what is used by Google Calendar and almost all calendar apps.
I have found this package iCalendar Hackage
But I cannot figure out how to use the parseICalendar function in this package, if someone can tell me what I am doing wrong, it would be great.
Mainly I cannot figure out how to construct an argument for the Type DecodingFunctions
parseICalendar :: DecodingFunctions
-> FilePath -- ^ Used in error messages.
-> ByteString
-> Either String ([VCalendar], [String])
My effort:
module CalendarReader
( getCalendar
, getSummary
) where
{-# LANGUAGE OverloadedStrings #-}
import qualified Data.ByteString.Lazy as B -- package "bytestring"
import qualified Text.ICalendar as ICal -- package "iCalendar"
import qualified Data.Map as Map -- package "containers"
import Network.HTTP.Simple -- package "http-conduit"
import qualified Time -- local module
import Constants
getCalendar :: IO B.ByteString
getCalendar = do
request <- parseRequest $ "GET" ++ calendarURL
response <- httpLBS request
return $ getResponseBody response
getSummary :: B.ByteString -> Time.DateTime -> Int -> String
getSummary cal dateTime dayOffset = summary
where
summary = "Event Summary"
((ICal.VCalendar { ICal.vcEvents = vcEvents' }), _) = ICal.parseICalendar ?missingArgument? logFile cal
DecodingFunctions is supposed to contains a function to convert your ByteString (binary array) to a Text (representing a string of unicode characters) and one to do the same to a case-insensitive representation (for comparison purpose I suppose). If your iCalendar is "normal" and is encoded in utf-8, you can simply use the Default instance of DecodingFunctions :
parseICalendar def logFile cal
(don't forget to import def from somewhere)
If your iCalendar is not in Utf-8, you'll have to use a decode... function from Data.Text.Lazy.Encoding and mk from Data.CaseInsensitive. For Utf16 you would have :
decodings = DecodingFunctions decodeUtf16LE (mk . decodeUtf16LE)
with the right imports.
Is it possible to generate and run TemplateHaskell generated code at runtime?
Using C, at runtime, I can:
create the source code of a function,
call out to gcc to compile it to a .so (linux) (or use llvm, etc.),
load the .so and
call the function.
Is a similar thing possible with Template Haskell?
Yes, it's possible. The GHC API will compile Template Haskell. A proof-of-concept is available at https://github.com/JohnLato/meta-th, which, although not very sophisticated, shows one general technique that even provides a modicum of type safety. Template Haskell expressions are build using the Meta type, which can then be compiled and loaded into a usable function.
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TemplateHaskell #-}
{-# OPTIONS_GHC -Wall #-}
module Data.Meta.Meta (
-- * Meta type
Meta (..)
-- * Functions
, metaCompile
) where
import Language.Haskell.TH
import Data.Typeable as Typ
import Control.Exception (bracket)
import System.Plugins -- from plugins
import System.IO
import System.Directory
newtype Meta a = Meta { unMeta :: ExpQ }
-- | Super-dodgy for the moment, the Meta type should register the
-- imports it needs.
metaCompile :: forall a. Typeable a => Meta a -> IO (Either String a)
metaCompile (Meta expr) = do
expr' <- runQ expr
-- pretty-print the TH expression as source code to be compiled at
-- run-time
let interpStr = pprint expr'
typeTypeRep = Typ.typeOf (undefined :: a)
let opener = do
(tfile, h) <- openTempFile "." "fooTmpFile.hs"
hPutStr h (unlines
[ "module TempMod where"
, "import Prelude"
, "import Language.Haskell.TH"
, "import GHC.Num"
, "import GHC.Base"
, ""
, "myFunc :: " ++ show typeTypeRep
, "myFunc = " ++ interpStr] )
hFlush h
hClose h
return tfile
bracket opener removeFile $ \tfile -> do
res <- make tfile ["-O2", "-ddump-simpl"]
let ofile = case res of
MakeSuccess _ fp -> fp
MakeFailure errs -> error $ show errs
print $ "loading from: " ++ show ofile
r2 <- load (ofile) [] [] "myFunc"
print "loaded"
case r2 of
LoadFailure er -> return (Left (show er))
LoadSuccess _ (fn :: a) -> return $ Right fn
This function takes an ExpQ, and first runs it in IO to create a plain Exp. The Exp is then pretty-printed into source code, which is compiled and loaded at run-time. In practice, I've found that one of the more difficult obstacles is specifying the correct imports in the generated TH code.
From what I understand you want to create and run a code at runtime which I think you can do using GHC API but I am not very sure of the scope of what you can achieve. If you want something like hot code swapping you can look at the package hotswap.
Using Parsec 3.1, it is possible to parse several types of inputs:
[Char] with Text.Parsec.String
Data.ByteString with Text.Parsec.ByteString
Data.ByteString.Lazy with Text.Parsec.ByteString.Lazy
I don't see anything for the Data.Text module. I want to parse Unicode content without suffering from the String inefficiencies. So I've created the following module based on the Text.Parsec.ByteString module:
{-# LANGUAGE FlexibleInstances, MultiParamTypeClasses #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
module Text.Parsec.Text
( Parser, GenParser
) where
import Text.Parsec.Prim
import qualified Data.Text as T
instance (Monad m) => Stream T.Text m Char where
uncons = return . T.uncons
type Parser = Parsec T.Text ()
type GenParser t st = Parsec T.Text st
Does it make sense to do so?
It this compatible with the rest of the Parsec API?
Additional comments:
I had to add {-# LANGUAGE NoMonomorphismRestriction #-} pragma in my parse modules to make it work.
Parsing Text is one thing, building an AST with Text is another thing. I will also need to pack my String before return:
module TestText where
import Data.Text as T
import Text.Parsec
import Text.Parsec.Prim
import Text.Parsec.Text
input = T.pack "xxxxxxxxxxxxxxyyyyxxxxxxxxxp"
parser = do
x1 <- many1 (char 'x')
y <- many1 (char 'y')
x2 <- many1 (char 'x')
return (T.pack x1, T.pack y, T.pack x2)
test = runParser parser () "test" input
Since Parsec 3.1.2 support of Data.Text is built-in!
See http://hackage.haskell.org/package/parsec-3.1.2
If you are stuck with older version, the code snippets in other answers are helpful, too.
That looks like exactly what you need to do.
It should be compatible with the rest of Parsec, include the Parsec.Char parsers.
If you're using Cabal to build your program, please put an upper bound of parsec-3.1 in your package description, in case the maintainer decides to include that instance in a future version of Parsec.
I added a function parseFromUtf8File to help reading UTF-8 encoded files in an efficient fashion. Works flawlessly with umlaut characters. Function type matches parseFromFile from Text.Parsec.ByteString. This version uses strict ByteStrings.
-- A derivate work from
-- http://stackoverflow.com/questions/4064532/using-parsec-with-data-text
{-# LANGUAGE FlexibleInstances, MultiParamTypeClasses #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
module Text.Parsec.Text
( Parser, GenParser, parseFromUtf8File
) where
import Text.Parsec.Prim
import qualified Data.Text as T
import qualified Data.ByteString as B
import Data.Text.Encoding
import Text.Parsec.Error
instance (Monad m) => Stream T.Text m Char where
uncons = return . T.uncons
type Parser = Parsec T.Text ()
type GenParser t st = Parsec T.Text st
-- | #parseFromUtf8File p filePath# runs a strict bytestring parser
-- #p# on the input read from #filePath# using
-- 'ByteString.readFile'. Returns either a 'ParseError' ('Left') or a
-- value of type #a# ('Right').
--
-- > main = do{ result <- parseFromFile numbers "digits.txt"
-- > ; case result of
-- > Left err -> print err
-- > Right xs -> print (sum xs)
-- > }
parseFromUtf8File :: Parser a -> String -> IO (Either ParseError a)
parseFromUtf8File p fname = do
raw <- B.readFile fname
let input = decodeUtf8 raw
return (runP p () fname input)