I have a 'QuasiQuoter' which is useful in source code in Haskell, but also as a standalone application. So, I need to be able to run QuasiQuoter
During the compile time in Haskell - [myGrammar|someCommand|]
In runtime (runtime compilation) in shell - mygrammar 'someCommand'
The first part is easy but the second part might be a little clumsy if solved as calling the compiler with some generated code from the runtime.
I would like to solve a second part of the problem using some nice method in Haskell which doesn't accept only the source code, but accepts QuasyQuoter datatype instead so the code is less clumsy. But I can't find any compilation method like that.
Do you know any? Thanks.
Example of usage
Haskell
The function takes tuple [(a,b,c,d,e)] and returns a list of the strings with the products.
function = [lsql| {1..5}, r=[ a.* |> (*) ], "Product of a.1 * a.2 * ... * a.5 is &a.r"|]
Bash
The command reads from stdin csv with at least 5 numerical columns and returns a list of their products (one per line).
lsql-csv '-, r=[ a.* |> (*) ], "Product of a.1 * a.2 * ... * a.5 is &a.r"'
I think the question is how to parse and process a string in a uniform way between a quasiquoter and some other chunk of code. If this interpretation is right, then you just... do that. For example:
-- implementation of these is left to the reader, but can use standard Haskell
-- programming techniques and libraries, like parsec and ADTs and stuff
command :: Parser Command
interpret :: Command -> IO ()
jit :: Command -> Exp -- or Q Exp
Then, in your lsql-csv.hs, you would write something like
main = do
[s] <- getArgs
case parse command s of
Left err -> die (show err)
Right com -> interpret com
and in your LSql/CSV/QQ.hs, you would write something like
lsql = QuasiQuoter { quoteExp = \s -> case parse command s of
Left err -> qReport True (show err) >> fail ""
Right com -> return (jit com) -- or just jit com if that's already a Q Exp
}
First, I don't have an easy to reproduce example on hand, as the code is calling out to a MATLAB engine, which requires a license. It may be possible to construct a similar example, just using C, though. I have the following snippet from a test:
ei1 :: Either SomeException MAnyArray <- try $ engineGetVar eng foopi
putStrLn $ assert (isRight ei1) " Can clearVar once"
clearVar eng foopi
ei2 :: Either SomeException MAnyArray <- try $ engineGetVar eng foopi
putStrLn $ assert (isLeft ei2) $
" Can't clearVar twice: " <> (show $ lefts [ei2])
putStrLn " Finished testClearVar"
This results in the output:
Can clearVar once
Error using save
Variable 'foopi' not found.
Can't clearVar twice: []
Finished testClearVar
The confusing bit is this expression, since the assertion appears to succeed (meaning that ei2 is a Left value, but when calling lefts [ei2], no Left values are found):
putStrLn $ assert (isLeft ei2) $
" Can't clearVar twice: " <> (show $ lefts [ei2])
If you look closely at documentation of assert you will find:
Assertions can normally be turned on or off with a compiler flag (for GHC, assertions are normally on unless optimisation is turned on with -O or the -fignore-asserts option is given). When assertions are turned off, the first argument to assert is ignored, and the second argument is returned as the result.
I assume this is a regular package you are working on and not just a file you are compiling manually with ghc. By default cabal will compile a project with -O, which means your asserts are simply ignored. What you need is either -O0 or -fno-ignore-asserts flag added. But what I would recommend is just don't rely on assert at all.
For instance:
let x = 1 in putStrLn [dump|x, x+1|]
would print something like
x=1, (x+1)=2
And even if there isn't anything like this currently, would it be possible to write something similar?
TL;DR There is this package which contains a complete solution.
install it via cabal install dump
and/or
read the source code
Example usage:
{-# LANGUAGE QuasiQuotes #-}
import Debug.Dump
main = print [d|a, a+1, map (+a) [1..3]|]
where a = 2
which prints:
(a) = 2 (a+1) = 3 (map (+a) [1..3]) = [3,4,5]
by turnint this String
"a, a+1, map (+a) [1..3]"
into this expression
( "(a) = " ++ show (a) ++ "\t " ++
"(a+1) = " ++ show (a + 1) ++ "\t " ++
"(map (+a) [1..3]) = " ++ show (map (+ a) [1 .. 3])
)
Background
Basically, I found that there are two ways to solve this problem:
Exp -> String The bottleneck here is pretty-printing haskell source code from Exp and cumbersome syntax upon usage.
String -> Exp The bottleneck here is parsing haskell to Exp.
Exp -> String
I started out with what #kqr put together, and tried to write a parser to turn this
["GHC.Classes.not x_1627412787 = False","x_1627412787 = True","x_1627412787 GHC.Classes.== GHC.Types.True = True"]
into this
["not x = False","x = True","x == True = True"]
But after trying for a day, my parsec-debugging-skills have proven insufficient to date, so instead I went with a simple regular expression:
simplify :: String -> String
simplify s = subRegex (mkRegex "_[0-9]+|([a-zA-Z]+\\.)+") s ""
For most cases, the output is greatly improved.
However, I suspect this to likely mistakenly remove things it shouldn't.
For example:
$(dump [|(elem 'a' "a.b.c", True)|])
Would likely return:
["elem 'a' \"c\" = True","True = True"]
But this could be solved with proper parsing.
Here is the version that works with the regex-aided simplification: https://github.com/Wizek/kqr-stackoverflow/blob/master/Th.hs
Here is a list of downsides / unresolved issues I've found with the Exp -> String solution:
As far as I know, not using Quasi Quotation requires cumbersome syntax upon usage, like: $(d [|(a, b)|]) -- as opposed to the more succinct [d|a, b|]. If you know a way to simplify this, please do tell!
As far as I know, [||] needs to contain fully valid Haskell, which pretty much necessitates the use of a tuple inside further exacerbating the syntactic situation. There is some upside to this too, however: at least we don't need to scratch our had where to split the expressions since GHC does that for us.
For some reason, the tuple only seemed to accept Booleans. Weird, I suspect this should be possible to fix somehow.
Pretty pretty-printing Exp is not very straight-forward. A more complete solution does require a parser after all.
Printing an AST scrubs the original formatting for a more uniform looks. I hoped to preserve the expressions letter-by-letter in the output.
The deal-breaker was the syntactic over-head. I knew I could get to a simpler solution like [d|a, a+1|] because I have seen that API provided in other packages. I was trying to remember where I saw that syntax. What is the name...?
String -> Exp
Quasi Quotation is the name, I remember!
I remembered seeing packages with heredocs and interpolated strings, like:
string = [qq|The quick {"brown"} $f {"jumps " ++ o} the $num ...|]
where f = "fox"; o = "over"; num = 3
Which, as far as I knew, during compile-time, turns into
string = "The quick " ++ "brown" ++ " " ++ $f ++ "jumps " ++ o ++ " the" ++ show num ++ " ..."
where f = "fox"; o = "over"; num = 3
And I thought to myself: if they can do it, I should be able to do it too!
A bit of digging in their source code revealed the QuasiQuoter type.
data QuasiQuoter = QuasiQuoter {quoteExp :: String -> Q Exp}
Bingo, this is what I want! Give me the source code as string! Ideally, I wouldn't mind returning string either, but maybe this will work. At this point I still know quite little about Q Exp.
After all, in theory, I would just need to split the string on commas, map over it, duplicate the elements so that first part stays string and the second part becomes Haskell source code, which is passed to show.
Turning this:
[d|a+1|]
into this:
"a+1" ++ " = " ++ show (a+1)
Sounds easy, right?
Well, it turns out that even though GHC most obviously is capable to parse haskell source code, it doesn't expose that function. Or not in any way we know of.
I find it strange that we need a third-party package (which thankfully there is at least one called haskell-src-meta) to parse haskell source code for meta programming. Looks to me such an obvious duplication of logic, and potential source of mismatch -- resulting in bugs.
Reluctantly, I started looking into it. After all, if it is good enough for the interpolated-string folks (those packaged did rely on haskell-src-meta) then maybe it will work okay for me too for the time being.
And alas, it does contain the desired function:
Language.Haskell.Meta.Parse.parseExp :: String -> Either String Exp
Language.Haskell.Meta.Parse
From this point it was rather straightforward, except for splitting on commas.
Right now, I do a very simple split on all commas, but that doesn't account for this case:
[d|(1, 2), 3|]
Which fails unfortunatelly. To handle this, I begun writing a parsec parser (again) which turned out to be more difficult than anticipated (again). At this point, I am open to suggestions. Maybe you know of a simple parser that handles the different edge-cases? If so, tell me in a comment, please! I plan on resolving this issue with or without parsec.
But for the most use-cases: it works.
Update at 2015-06-20
Version 0.2.1 and later correctly parses expressions even if they contain commas inside them. Meaning [d|(1, 2), 3|] and similar expressions are now supported.
You can
install it via cabal install dump
and/or
read the source code
Conclusion
During the last week I've learnt quite a bit of Template Haskell and QuasiQuotation, cabal sandboxes, publishing a package to hackage, building haddock docs and publishing them, and some things about Haskell too.
It's been fun.
And perhaps most importantly, I now am able to use this tool for debugging and development, the absence of which has been bugging me for some time. Peace at last.
Thank you #kqr, your engagement with my original question and attempt at solving it gave me enough spark and motivation to continue writing up a full solution.
I've actually almost solved the problem now. Not exactly what you imagined, but fairly close. Maybe someone else can use this as a basis for a better version. Either way, with
{-# LANGUAGE TemplateHaskell, LambdaCase #-}
import Language.Haskell.TH
dump :: ExpQ -> ExpQ
dump tuple =
listE . map dumpExpr . getElems =<< tuple
where
getElems = \case { TupE xs -> xs; _ -> error "not a tuple in splice!" }
dumpExpr exp = [| $(litE (stringL (pprint exp))) ++ " = " ++ show $(return exp)|]
you get the ability to do something like
λ> let x = True
λ> print $(dump [|(not x, x, x == True)|])
["GHC.Classes.not x_1627412787 = False","x_1627412787 = True","x_1627412787 GHC.Classes.== GHC.Types.True = True"]
which is almost what you wanted. As you see, it's a problem that the pprint function includes module prefixes and such, which makes the result... less than ideally readable. I don't yet know of a fix for that, but other than that I think it is fairly usable.
It's a bit syntactically heavy, but that is because it's using the regular [| quote syntax in Haskell. If one wanted to write their own quasiquoter, as you suggest, I'm pretty sure one would also have to re-implement parsing Haskell, which would suck a bit.
In the code at the bottom, two monadic actions (loadModules and setImportsQ) in Hint (Language.Haskell.Interpreter) can only be executed in one order but not the other, as shown in a minimal example below. In particular, loadModules must go first, or else the following exception will be generated when executing the interpreter.
*** Exception: compile error: WontCompile [GhcError {errMsg = "Not in scope: type
constructor or class `Int'"}]
This behavior seems to have been introduced in Hint 0.4.1.0. I must have missed something obvious..
My first questions is Hint specific: why must loadModules be called before setImportsQ?
My second question is more general: it seems that these two monadic actions do NOT feed input into each other, also the to-be-loaded module is empty, why is the order or execution important at all? This almost reminds me of the imperative world, where you not only have to worry about the input/output, but also the order of certain side effects.
The code is below. Note that MyModule.hs is an empty .hs file.
module Main where
import Language.Haskell.Interpreter
import Control.Monad
custom=["MyModule"] --mymodule.hs is just an empty .hs source file in the same folder
context=[("Data.Int",Nothing)]
main = do
x <- interp ("1") (as ::Int)
return ()
interp e as_type = do
interpreterResult <- runInterpreter $ do
loadModules custom --this line must go first
setImportsQ context --this line must go second
interpret e as_type
f <- case interpreterResult of
Left e -> error $ "compile error: " ++ show e
Right result -> return result
return f
I have this error sometimes, though not all of the time, and it's driving me crazy. I don't know if it's a bug or if there is some behaviour or fix that I am not aware of.
I am entering a multiple-line command using :{ and :}, and SOMETIMES when I want to conclude the command, like below, I receive the error as shown below:
*MyModule| :}
unknown command ':}'
use :? for help.
I'd say it works properly 97 percent of the time, but 3 percent of the time I get this situation.
As far as I know, it should always work to type :} to close the multiple line entry, as described here:
http://www.haskell.org/ghc/docs/7.4.1/html/users_guide/interactive-evaluation.html
At the moment, the only way that I know to escape this situation when it happens is ctrl+D, which kills ghci unfortunately.
A. Is this a bug or is there some reason that :} would suddenly become an "unknown command"?
B. If I get to this situation, is there a way to recover without using ctrl+D? It doesn't matter how many times I try :}, it always results in "unknown command" once I have entered this situtation, though what I expect is for this command to close the multiple line entry.
Like it says in the question, this is GHCi, version 7.6.3, on Arch Linux.
As I already noted in the comments, the reason for this behaviour is the GHCi doesn't properly reset the prompt when Ctrl-C is pressed. The source of the problem probably lies in the following code (Taken from ghci-ng):
multiLineCmd q = do
st <- lift getGHCiState
let p = prompt st
lift $ setGHCiState st{ prompt = prompt2 st }
mb_cmd <- collectCommand q ""
lift $ getGHCiState >>= \st' -> setGHCiState st'{ prompt = p }
return mb_cmd
(See InteractiveUI.hs line 712)
If collectCommand throws UserInterrupt, then the line that resets the promt will never be executed. I changed this code to:
multiLineCmd q = do
st <- lift getGHCiState
let p = prompt st
lift $ setGHCiState st{ prompt = prompt2 st }
mb_cmd <- collectCommand q "" `GHC.gfinally` lift (getGHCiState >>= \st' -> setGHCiState st'{ prompt = p })
return mb_cmd
Which fixes the problem.
This is a known bug. Everything is working fine except that ghci sometimes keeps printing the multiline prompt when it should be printing the normal prompt.