Is it possible to generate and run TemplateHaskell generated code at runtime?
Using C, at runtime, I can:
create the source code of a function,
call out to gcc to compile it to a .so (linux) (or use llvm, etc.),
load the .so and
call the function.
Is a similar thing possible with Template Haskell?
Yes, it's possible. The GHC API will compile Template Haskell. A proof-of-concept is available at https://github.com/JohnLato/meta-th, which, although not very sophisticated, shows one general technique that even provides a modicum of type safety. Template Haskell expressions are build using the Meta type, which can then be compiled and loaded into a usable function.
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TemplateHaskell #-}
{-# OPTIONS_GHC -Wall #-}
module Data.Meta.Meta (
-- * Meta type
Meta (..)
-- * Functions
, metaCompile
) where
import Language.Haskell.TH
import Data.Typeable as Typ
import Control.Exception (bracket)
import System.Plugins -- from plugins
import System.IO
import System.Directory
newtype Meta a = Meta { unMeta :: ExpQ }
-- | Super-dodgy for the moment, the Meta type should register the
-- imports it needs.
metaCompile :: forall a. Typeable a => Meta a -> IO (Either String a)
metaCompile (Meta expr) = do
expr' <- runQ expr
-- pretty-print the TH expression as source code to be compiled at
-- run-time
let interpStr = pprint expr'
typeTypeRep = Typ.typeOf (undefined :: a)
let opener = do
(tfile, h) <- openTempFile "." "fooTmpFile.hs"
hPutStr h (unlines
[ "module TempMod where"
, "import Prelude"
, "import Language.Haskell.TH"
, "import GHC.Num"
, "import GHC.Base"
, ""
, "myFunc :: " ++ show typeTypeRep
, "myFunc = " ++ interpStr] )
hFlush h
hClose h
return tfile
bracket opener removeFile $ \tfile -> do
res <- make tfile ["-O2", "-ddump-simpl"]
let ofile = case res of
MakeSuccess _ fp -> fp
MakeFailure errs -> error $ show errs
print $ "loading from: " ++ show ofile
r2 <- load (ofile) [] [] "myFunc"
print "loaded"
case r2 of
LoadFailure er -> return (Left (show er))
LoadSuccess _ (fn :: a) -> return $ Right fn
This function takes an ExpQ, and first runs it in IO to create a plain Exp. The Exp is then pretty-printed into source code, which is compiled and loaded at run-time. In practice, I've found that one of the more difficult obstacles is specifying the correct imports in the generated TH code.
From what I understand you want to create and run a code at runtime which I think you can do using GHC API but I am not very sure of the scope of what you can achieve. If you want something like hot code swapping you can look at the package hotswap.
Related
I've re-posted this question to focus more tightly on the specific error, and to better enumerate what I've already tried.
I'm trying to parse some Haskell code during the runtime of a Haskell program using the hint package.
The outer program compiles, but when I run it the inner compilation step fails. I'm getting a description of what I assume is a syntax problem, and a location in the "interactive" code, but I have no idea how to view the code in question.
Here's Main.hs
module Main where
import Data.List (intercalate)
import Polysemy (runM)
import qualified Language.Haskell.Interpreter as H
import qualified Effects as E
handleFailures :: Either H.InterpreterError a -> IO a
handleFailures (Left l) = ioError $ userError $ message l
where
message (H.WontCompile es) = intercalate "\n" (header : map unbox es)
message e = show e
header = "ERROR: Won't compile:"
unbox (H.GhcError e) = e
handleFailures (Right a) = return a
interpretation :: String -> H.Interpreter E.MyEffect
interpretation s = do
H.loadModules ["Effects"]
H.setImportsQ [("Prelude", Nothing), ("Effects", Nothing)]
effect <- H.interpret s (H.as :: E.MyEffect)
return effect
extractProgram :: String -> IO E.MyEffect
extractProgram s = do
p <- H.runInterpreter $ interpretation s
success <- handleFailures p
return success
main :: IO ()
main = do
userProvided <- readFile "UserProvided.hs"
userProgram <- extractProgram userProvided
runM . E.teletypeToIO . E.teletypePlusToIO $ userProgram
Effects.hs defines and provides helpers for a Polysemey Sem monad called MyEffect.
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE LambdaCase, BlockArguments #-}
{-# LANGUAGE GADTs, FlexibleContexts, TypeOperators, DataKinds, PolyKinds, ScopedTypeVariables #-}
module Effects where
import Polysemy
data Teletype m a where
ReadTTY :: Teletype m String
WriteTTY :: String -> Teletype m ()
makeSem ''Teletype
teletypeToIO :: Member (Embed IO) r => Sem (Teletype ': r) a -> Sem r a
teletypeToIO = interpret $ \case
ReadTTY -> embed getLine
WriteTTY msg -> embed $ putStrLn msg
data TeletypePlus m a where
ReadPlus :: TeletypePlus m String
WritePlus :: String -> TeletypePlus m ()
makeSem ''TeletypePlus
teletypePlusToIO :: Member (Embed IO) r => Sem (TeletypePlus ': r) a -> Sem r a
teletypePlusToIO = interpret $ \case
ReadPlus -> embed $ ("+" <>) <$> getLine
WritePlus msg -> embed $ putStrLn $ msg <> "+"
type MyEffect = Sem [TeletypePlus, Teletype, Embed IO] ()
UserProvided.hs contains a simple do expression in MyEffect.
do
i <- readTTY
j <- readPlus
let k = i <> j
writeTTY k
writePlus k
In order to get the polysemy package available at runtime, I have to enter run it from inside a cabal sandbox.
$ cabal build
Build profile: -w ghc-8.8.1 -O1
In order, the following will be built (use -v for more details):
- Hello-Polysemy-0.1.0.0 (exe:Hello-Polysemy) (file Main.hs changed)
Preprocessing executable 'Hello-Polysemy' for Hello-Polysemy-0.1.0.0..
Building executable 'Hello-Polysemy' for Hello-Polysemy-0.1.0.0..
[2 of 2] Compiling Main ( Main.hs, /home/mako/Git/Hello-Polysemy/dist-newstyle/buil/x86_64-linux/ghc-8.8.1/Hello-Polysemy-0.1.0.0/x/Hello-Polysemy/build/Hello-Polysemy/Hello-Polysemy-tmp/Main.o )
Linking /home/mako/Git/Hello-Polysemy/dist-newstyle/build/x86_64-linux/ghc-8.8.1/Hello-Polysemy-0.1.0.0/x/Hello-Polysemy/build/Hello-Polysemy/Hello-Polysemy ...
$ cabal exec bash
... but then ...
$ cabal run
Up to date
Hello-Polysemy: user error (ERROR: Won't compile:
<interactive>:10:135: error:
Operator applied to too few arguments: :)
So far as I can tell the only place I'm using the : operator is in Effects.hs, where (a) I'm actually using the type-operator ':, and (b) compilation succeeds just fine when Effects is imported into Main.hs.
Any suggestions for what the problem might be, or how I could learn more?
I already tried using Language.Haskell.Interpreter.Unsafe.unsafeRunInterpreterWithArgs ["-v4"]. That clarifies that it's talking about ghc-prim:GHC.Types.:{(w) d 66}), but I don't know what to do with that information.
Update:
I've tried various permutations of in-lining the "userProvided" code.
declaring the exact same Effect value inline in Main works fine. Replacing the string read from the file with an inline string of an even simpler value "writePlus \"asdf\"" doesn't change the error message.
I need to communicate some information from compile scripts into Template Haskell. Currently the compile scripts keep the information in the system environment, so I just read it using System.Environment.getEnvironment wrapped in runIO. Is there a better way, such as passing some arguments to ghc (similar to -D... for the C pre-processor), or perhaps something specifically designed for this purpose in TH?
Since so many people are interested in the question, I'll add my current approach, perhaps somebody will find it useful. Probably the best way would be if TH allowed to read -D parameters on GHC's command line, but it seems nothing like this is currently implemented.
A simple module allows TH to read compile-time environment. A helper function also allows to read files; for example read the path of a configuration file from the environment and then read the file.
{-# LANGUAGE TemplateHaskell #-}
module THEnv
(
-- * Compile-time configuration
lookupCompileEnv
, lookupCompileEnvExp
, getCompileEnv
, getCompileEnvExp
, fileAsString
) where
import Control.Monad
import qualified Data.Text as T
import qualified Data.Text.IO as T
import Language.Haskell.TH
import Language.Haskell.TH.Syntax (Lift(..))
import System.Environment (getEnvironment)
-- Functions that work with compile-time configuration
-- | Looks up a compile-time environment variable.
lookupCompileEnv :: String -> Q (Maybe String)
lookupCompileEnv key = lookup key `liftM` runIO getEnvironment
-- | Looks up a compile-time environment variable. The result is a TH
-- expression of type #Maybe String#.
lookupCompileEnvExp :: String -> Q Exp
lookupCompileEnvExp = (`sigE` [t| Maybe String |]) . lift <=< lookupCompileEnv
-- We need to explicly type the result so that things like `print Nothing`
-- work.
-- | Looks up an compile-time environment variable and fail, if it's not
-- present.
getCompileEnv :: String -> Q String
getCompileEnv key =
lookupCompileEnv key >>=
maybe (fail $ "Environment variable " ++ key ++ " not defined") return
-- | Looks up an compile-time environment variable and fail, if it's not
-- present. The result is a TH expression of type #String#.
getCompileEnvExp :: String -> Q Exp
getCompileEnvExp = lift <=< getCompileEnv
-- | Loads the content of a file as a string constant expression.
-- The given path is relative to the source directory.
fileAsString :: FilePath -> Q Exp
fileAsString = do
-- addDependentFile path -- works only with template-haskell >= 2.7
stringE . T.unpack . T.strip <=< runIO . T.readFile
It can be used like this:
{-# LANGUAGE TemplateHaskell #-}
import THEnv
main = print $( lookupCompileEnvExp "DEBUG" )
Then:
runhaskell Main.hs prints Nothing;
DEBUG="yes" runhaskell Main.hs prints Just "yes".
It looks like what you are trying to do here, The -D option in ghc seems to define a compile time variable.
Here, on the same subject is a question that seems to also answer the other part of your question.
From what I can tell, to do conditional compilation, you do something like:
#ifdef MACRO_NAME
//Do stuff here
#endif
I have a Haskell source file which using Unicode syntax:
{-# LANGUAGE UnicodeSyntax #-}
succ' :: Int → Int
succ' = succ
main :: IO ()
main = print $ succ' 1
This parses and runs fine with GHC. Additionally, stylish-haskell and hlint (both based on haskell-src-exts) can read this file without any trouble. However, when I try to parse it myself using haskell-src-exts:
import Language.Haskell.Exts (parseModule)
main = do
x <- readFile "test.hs"
print $ parseModule x
I get the error message:
ParseFailed (SrcLoc {srcFilename = "<unknown>.hs", srcLine = 6, srcColumn = 1}) "TypeOperators is not enabled"
However, providing UnicodeSyntax explicitly in the extensions list or using parseFile works just fine:
import Language.Haskell.Exts
main = do
x <- readFile "test.hs"
print $ parseModuleWithMode defaultParseMode
{ extensions = [UnicodeSyntax]
} x
parseFile "test.hs" >>= print
Any idea why the first approach fails?
From a cursory glance at the source, it doesn't look like parseModule extracts language pragmas from the source before parsing (parseFile does do that by calling getExtensions). By the time parsing has begun, it is already too late to enable unicode syntax.
Is there a way to create functions with implicit parameters or let bindings with implicit parameters using template haskell?
I.e. is it possible to generate a signature like this using template haskell:
doSomething :: (?context :: Context) => m a
Or an invocation like this:
invoc = let ?context = newContext in doSomething
I could not find suitable algebraic data types nor any functions which would help me out on this topic in the API documentation for template haskell. I'm using GHC 7.4.2.
If there is no native support for this extension in template haskell, is there some other possibility to inject code during compilation (maybe something like a general “code injection function” within template haskell?).
EDIT: I tried the suggestion from the comments, this is what happens:
runQ [d| f :: (?c :: String) => Int ; f = 7 |]
<interactive>:10:17: parse error on input `c'
whereas this works:
runQ [d| f :: Int ; f = 7|]
[SigD f_0 (ConT GHC.Types.Int),ValD (VarP f_0) (NormalB (LitE (IntegerL 7))) []]
doesn't seem to be supported.
Here's one way that's pretty fragile, but sort of works. While you can't refer
to ?x in the Exp that template haskell uses, you can refer to a definition in
another module like:
reserved_prefix_x = ?x
Below is some code that generates variables like above in one run of ghc,
and in a second run of ghc the variables actually refer to implicit parameters.
{-# LANGUAGE TemplateHaskell, NoMonomorphismRestriction #-}
module GenMod (h) where
import Data.Generics
import Data.IORef
import Data.List
import Language.Haskell.Meta.Parse as P
import Language.Haskell.TH
import Language.Haskell.TH.Quote
import Language.Haskell.TH.Syntax
import qualified Data.Set as S
import qualified Language.Haskell.Exts.QQ as Q
import System.IO.Unsafe
h = Q.hs { quoteExp = \s -> do
r <- either fail (upVars . return) (P.parseExp s)
writeMod'
return r
}
pfx = "q_"
{-# NOINLINE vars #-}
vars :: IORef (S.Set String)
vars = unsafePerformIO (newIORef S.empty)
writeMod' = runIO $ writeFile "GEN.hs" . ppMod =<< readIORef vars
writeMod = -- might be needed to avoid multiple calls to writeFile?
-- in this example this is called for every use of `h'
QuasiQuoter { quoteDec = \ _ -> do
writeMod'
[d| _ = () |] }
ppMod xs = "{-# LANGUAGE NoMonomorphismRestriction, ImplicitParams #-}\n\
\module GEN where\n" ++
unlines (map (\x -> pfx ++ x ++ " = ?" ++ x) (S.toList xs))
upVars x = do
x' <- x
runIO $ modifyIORef vars (S.union (getMatchingVars x'))
runIO $ print =<< readIORef vars
return x'
getMatchingVars =
everything
S.union
(mkQ S.empty
(\ (OccName x) -> maybe S.empty S.singleton (stripPrefix pfx x)))
A Main.hs file that uses the quasiquoter GenMod.hs:
{-# LANGUAGE NoMonomorphismRestriction, ImplicitParams, QuasiQuotes, TemplateHaskell, CPP #-}
import GenMod
#ifndef stage1
import GEN
#endif
f_ = [h| q_hithere |]
You have to call ghc twice, like:
ghci -Dstage1 Main.hs
GHCi, version 7.6.1: http://www.haskell.org/ghc/ :? for help
[1 of 2] Compiling GenMod ( GenMod.hs, interpreted )
[2 of 2] Compiling Main ( Ex.hs, interpreted )
fromList ["hithere"]
Ex.hs:8:6: Not in scope: `q_hithere'
Failed, modules loaded: GenMod.
Though ghc fails, it still generates the GEN.hs which contains:
{-# LANGUAGE NoMonomorphismRestriction, ImplicitParams #-}
module GEN where
q_hithere = ?hithere
Which will be there when you load Main (leaving out the -D flag)
*Main> :t f_
f_ :: (?hithere::t) => t
This kind of trouble probably isn't worth it. Maybe other situations of calling out to other programs from TH are more motivating such as inline calls to other languages http://hpaste.org/50837 (gfortran example)
Since I used haskell-src-meta's default parser, the quasiquote gets to use variables "reserved_prefix_x" not "?x". It should be possible to accept the "?x" without too much difficulty.
I want to do something like this in Haskell, but the compiler is not letting me.
Is there any way to accomplish this task?
-- both modules export function named "hello"
-- and I want to run it in every module
import qualified MyMod as M1
import qualified MyAnotherMod as M2
runmodules = map (\m -> m.hello) [M1, M2]
Modules in Haskell are not even remotely first-class entities in the ways this would require, I'm afraid.
However, as bzn commented, Template Haskell can be used for problems like this. The result can be a bit clumsy, but if you really need some quick metaprogramming hacks it's not a bad choice. I'm not really an expert with TH, but what you want is pretty simple, with one catch: Neither "ambiguous identifiers" nor "module names" can be captured or quoted in any way, as far as I know, so you'll have to put them in strings given as arguments to the TH function.
Here's a quick and dirty, minimal example:
{-# LANGUAGE TemplateHaskell #-}
module MapModuleTH where
import Language.Haskell.TH
mapQual :: [String] -> String -> ExpQ
mapQual ms n = listE $ map (\m -> varE . mkName $ m ++ "." ++ n) ms
mapMQual :: [String] -> String -> ExpQ
mapMQual ms n = appE (varE 'sequence) $ listE $ map (\m -> varE . mkName $ m ++ "." ++ n) ms
You phrased things as "running the function" which sounds more like doing a bunch of IO actions, not just collecting a list of stuff, so I added a variant that also sequences the result.
Note that, despite the use of strings here, this is still statically typed--if the qualified names don't exist, or the types don't match up, you'll get the expected compile-time error just as if you'd written everything out by hand.
Here's a quick example of using it. Given the following:
{-# LANGUAGE TemplateHaskell #-}
module MapModule where
import MapModuleTH
import qualified Test1 as T1
import qualified Test2 as T2
tests = do xs <- $(mapMQual ["T1", "T2"] "test")
putStrLn $ "Count: " ++ show (length xs)
Assuming the other modules are there and define test, then in GHCi we can see:
> tests
Test 1
Test 2
Count: 2
I don't think you can quote a qualified name prefix like that in template haskell, and the hello identifier isn't in scope, so you might have to fall back to programming with strings.
module ModuleParamsTH where
import Language.Haskell.TH
getAll :: String -> [String] -> ExpQ
getAll valueName moduleNames =
listE $ map (varE . mkName . (++ suffix)) moduleNames
where suffix = "." ++ valueName
which can then be used like so,
{-# LANGUAGE TemplateHaskell #-}
import ModuleParamsTH
import qualified ModuleParamsM1 as M1
import qualified ModuleParamsM2 as M2
runmodules = $(getAll "hello" ["M1", "M2"])
However, I would not do all this. You could just write [M1.hello, M2.hello] or use a type class to abstract over implementations.
Modules aren't values in Haskell. Therefore that isn't possible. What do you want to achieve?