For a tool I'm writing ( http://hackage.haskell.org/package/explore ) I need a way to read haskell function definitions at run-time, apply them to values from my tool and retrieve the results of their application.
Can anyone give me a very basic example using GHC (6.10.4 or 6.12.1) API?
example function definition to be read from a file at run-time:
f x = 10**(4/1102*x - 1)
expected program output
--mapM_ print $ map f [428, 410, 389]
3.577165388142748
3.077536885227335
2.5821307011665815
!!UPDATE!!
I posted a quick answer but it creates an object file in the directory of execution, any tips to avoid this and avoid all file IO is most welcome. I want to also see a version that does everything in memory: user provides the function definition in a GUI for example and the compilation / evaluation does not create any object files.
Use hint. It's a GHCi-like wrapper around the GHC API that is not very difficult to use.
If you want an example of its use, I used it in my Yogurt project.
adapted from: http://www.bluishcoder.co.nz/2008/11/dynamic-compilation-and-loading-of.html
f.hs:
module Func (Func.f) where
f :: Double -> Double
f x = 10**(4/1102*x - 1)
main.hs:
import GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce
import Control.Monad
main :: IO ()
main =
defaultErrorHandler defaultDynFlags $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "f.hs" Nothing
addTarget target
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Func") Nothing
setContext [] [m]
value <- compileExpr ("Func.f")
do let value' = (unsafeCoerce value) :: Double -> Double
return value'
let f = func
mapM_ print $ map f [428, 410, 389]
return ()
Nice work getting the API going. I can tell you a little bit about how the code generator works.
GHC uses the system assembler to create a .o file. If there is not an option available to get GHC to clean up after itself, then you should file a feature request against the API, using the bug tracker at http://hackage.haskell.org/trac/ghc/newticket?type=feature+request. In order to file the request, you will need to register an account.
Using the standard code generator, you will not be able to avoid file I/O entirely, just because GHC delegates the work of creating relocatable object code to the assembler. There is an experimental back end based on LLVM that might be able to do everything in memory, but I would be surprised if it is available in anything earlier than 6.13. However it could be worth asking on the GHC developers' list.
Related
Similar question (Is there a way to see the list of functions in a module, in GHCI?), though not the result that I seek.
Is there a way to get a list of what is exported by a module?
Of course in GHCi you can import it then type Some.Module., hit tab for auto-completion and it will show what I seek. But I want to capture that stuff. Roughly speaking, String -> [String].
Purpose? Suppose that I have a source file with a naked import Some.Module. Question: What belongs to Some.Module in that file? A simple way would be to output the list of what the module exports, feed that to grep and return the contenders, without the need to load that source file in GHCi (might be complicated or not possible). And everything becomes a lot clearer.
If there's a smarter approach to that, I'm listening. I heard of solutions involving GOA and lambdabot. No idea if applicable or how to make use of this.
As #HTNW mentioned in a comment, if you can run ghc on your actual file, you can use -ddump-minimal-imports. Otherwise, if you want to actually get the list of exports from another module, assuming that you're using GHC, the easiest way to do this is probably to look at the .hi interface files. ghc has some built-in support for printing human-readable representations of interface files once you know the path to one, but as the wiki page notes "This textual format is not particularly designed for machine parsing". You can also access the information you might want via the GHC API. A small example of doing something like that follows.
We start with a bunch of random imports for doing IO & from the GHC api:
import Control.Monad.IO.Class
import System.IO
import System.Environment
import GHC
import GHC.Paths (libdir)
import DynFlags
import Outputable
import Name
import Pretty (Mode(..))
With that bureaucracy out of the way, main starts by firing up the GHC Monad:
main :: IO ()
main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
runGhc (Just libdir) $ do
We're not actually generating any code so we can set hscTarget = HscNothing during the DynFlags setup boilerplate:
dflags <- getSessionDynFlags
let dflags' = dflags { hscTarget = HscNothing }
setSessionDynFlags dflags'
With that out of the way we can find the module we want from the package database (using the first command-line argument as the name):
mn <- head <$> (liftIO $ getArgs)
m <- lookupModule (mkModuleName mn) Nothing
We can use getModuleInfo to get a module info structure:
mmi <- getModuleInfo m
case mmi of
Nothing -> liftIO $ putStrLn "Could not find module interface"
If we did find the interface, everything we need for this is in the modInfoExports. If we needed more, we could also get the actual ModIface:
Just mi -> mapM_ (printExport dflags') (modInfoExports mi)
Actually printing out an exported is a bit tedious, as it requires working with Names; a simple example printExport might just use the pretty-printing functions, but these are more intended for printing human-readable output than machine-readable:
printExport :: DynFlags -> Name -> Ghc ()
printExport dflags n =
liftIO $ printSDocLn PageMode dflags stdout (defaultUserStyle dflags)
$ pprNameUnqualified n
A particularly simple way for interactive use is :browse. Load up a ghci that has access to the appropriate package, then
> :browse Some.Module
class Some.Module.Foo a where
Some.Module.foo :: a -> a
{-# MINIMAL foo #-}
Some.Module.bar :: Int
All the qualification can get a bit much, especially if there are many functions that operate on types defined in the same module. To reduce the clutter, you can bring Some.Module into scope first:
> :m + Some.Module
> :browse Some.Module
class Foo a where
foo :: a -> a
{-# MINIMAL foo #-}
bar :: Int
I'm trying to follow along with the LLVM bindings tutorial here, and running into a segfault. The following code works in the sense that it prints a module header to output.ll, but it also segfaults somewhere.
module Main where
import Control.Monad.Error
import LLVM.General.Module
import LLVM.General.Context
import qualified LLVM.General.AST as AST
--Create and write out an empty LLVM module
main :: IO ()
main = writeModule (AST.defaultModule { AST.moduleName = "myModule" })
outputFile :: File
outputFile = File "output.ll"
writeModule :: AST.Module -> IO ()
writeModule mod = withContext $ (\context ->
liftError $ withModuleFromAST context mod (\m ->
liftError $ writeLLVMAssemblyToFile outputFile m))
--perform the action, or fail on an error
liftError :: ErrorT String IO a -> IO a
liftError = runErrorT >=> either fail return
I suspect this is related to the following hint from the linked tutorial:
It is very important to remember not to pass or attempt to use resources outside of the bracket as this will lead to undefined behavior and/or segfaults.
I think in this context the "bracket" is implemented by the withContext function, which makes it seem like everything should be handled.
If I change the definition of writeModule to
writeModule mod = do assembly <- (withContext $ (\context ->
liftError $ withModuleFromAST context mod moduleLLVMAssembly))
putStrLn assembly
that is, instead of writing to a file I just print out the string representation of the LLVM assembly, no segfault is thrown.
Does anyone have experience with these bindings? I'm also interested to know about the failure cases for the warning I quoted. That is, how would one "forget" not to use resources outside the bracket? All of the functions that seem to require a Context, well, require one. Isn't this kind of resource scoping issue exactly what Haskell is good at handling for you?
Version information:
llvm-general-3.4.3.0
LLVM version 3.4
Default target: x86_64-apple-darwin13.2.0
It would help if you shared your LLVM and cabal environment, LLVM is notorious for being backwards incompatible with itself so there might be an issue with using the latest versions of the bindings.
Behind the scenes writeLLVMAssemblyToFile is using a C++ call to do the file IO operation and I speculate that it's holding a reference to the LLVM module as a result of finalizing the file resource.
Try rendering the module to a String using moduleString and then only lifting into the IO monad to call writeFile from Haskell instead of going through C++ to the write.
import LLVM.General.Context
import LLVM.General.Module as Mod
import qualified LLVM.General.AST as AST
import Control.Monad.Error
main :: IO ()
main = do
writeModule (AST.defaultModule { AST.moduleName = "myModule" })
return ()
writeModule :: AST.Module -> IO (Either String ())
writeModule ast =
withContext $ \ctx ->
runErrorT $ withModuleFromAST ctx ast $ \m -> do
asm <- moduleString m
liftIO $ writeFile "output.ll" asm
The bindings can still rather brittle in my experience, you should ask on the issue tracker if the problem persists.
EDIT: This is a workaround for an old version that has been subsequently fixed. See: https://github.com/bscarlet/llvm-general/issues/109
As a small part of a larger University project, I need to write what is essentially an extremely crude IDE. The idea is to to take input from a gtk text box, treat that string as if it is in a .hs file, and evaluate a function within it.
My main approach has been to use the GHC API to compile and evaluate a test function. I had already managed to get a toy example working for compiling from a .hs file. GHC's Target data type had an optional constructor for getting a target from a StringBuffer, so I decided to try and alter my code to get it to work from a String Buffer:
compileText :: SourceView -> IO ()
compileText tview = do
txtBuff <- textViewGetBuffer tview
startIt <- textBufferGetStartIter txtBuff
endIt <- textBufferGetEndIter txtBuff
compTime <- getClockTime
srcString <- textBufferGetText txtBuff startIt endIt False
defaultErrorHandler defaultLogAction $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
addTarget $ haskellFileFromText srcString compTime
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Test") Nothing
setContext [IIModule m]
value <- compileExpr ("Test.print")
do let value' = (unsafeCoerce value) :: String -> IO ()
return value'
func "Hello"
return ()
haskellFileFromText :: String -> ClockTime -> GHC.Target
haskellFileFromText codeStr cTime = GHC.Target (TargetModule (mkModuleName "Test")) False (Just ((stringToStringBuffer codeStr), cTime))
The following code being in the text box at the time:
module Test (Test.print) where
print :: String -> IO ()
print x = putStrLn x
However, this does not seem to work. I get the error:
textEdit: panic! (the 'impossible' happened)
(GHC version 7.4.1 for x86_64-unknown-linux):
Could not find module `Test'
Use -v to see a list of the files searched for.
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
What am I doing wrong? I feel I am crucially misunderstanding something about the way this code works.
An alternative to this method which has been suggested to me is to use something like hint or mueval to evaluate the text in the textbox. This would appear to work fine if i simply want to evaluate a single function in isolation, but would this scale if I wanted to evaluate a function which depended on the context of running 4 other functions defined within the same source file?
As C.A. McCann notes, hint does lots of this work for you. It's a wrapper around the GHC api, not just a standalone evaluator like mueval.
Even if it is missing something you need, it will be far easier to learn from it and extend it than start from scratch.
Following the GHC tutorial posted here and alterations to this code following the advice in a previous stack overflow question I asked, I have created a program which is able to compile and run a module in Test.hs with a function print to print a string to the screen:
import GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce
main :: IO ()
main =
defaultErrorHandler defaultLogAction $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "Test.hs" Nothing
addTarget target
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Test") Nothing
setContext [IIModule m]
value <- compileExpr ("Test.print")
do let value' = (unsafeCoerce value) :: String -> IO ()
return value'
func "Hello"
return ()
The problem with this code, as noted in the comments, is that it only seems to work the first time you run it (When Test.hs has not yet been complied). If you attempt to run the code a second time, the following error appears:
mkTopLevEnv: not interpreted main:Test
I believe this has something to do with the fact that the code has already been compiled. If I delete the .hi and .o files and run the program again, the program runs correctly with the correct output. What am I missing? I am currently using ghc version 7.4.1
(Note: I have tried looking through the GHC API but could not find any references to mkTopLevEnv)
Simon Marlow suggests here that replacing
guessTarget "Test.hs" Nothing
with
guessTarget "*Test.hs" Nothing
should avoid the error you're getting, on the grounds that it tells GHC not to load the .o file.
See the whole thread on a page via nabble
Of course, you could delete the .hi and .o files each time, but that's an ugly workaround.
I have a module Target, with a function Target.accessMe inside it. I compile this module in some way, then get rid of the source code.
Now, what series of arcane incantations must I do to make a different program dynamically import Target.accessMe? This program knows accessMe's type in advance. Also, consider the fact that the source code of Target is not available.
The plugins package manages to accomplish this, but seems to have serious issues with working on Windows. I've checked out plugins's source, but am having trouble understanding it.
I've tried using Hint, but can only find out how to evaluate code that I have the source for.
Thanks for any help!
The answer to this question has been given to me elsewhere. The GHC API is capable of doing this. Here are two functions, one of which compiles Target.hs, while the other accesses Target.accessMe (and doesn't require the source code of the Target module to be there anymore).
import GHC
import DynFlags
compile :: String -> IO SuccessFlag
compile name = defaultRunGhc $ do
dynflags <- getSessionDynFlags
let dynflags' = dynflags -- You can change various options here.
setSessionDynFlags dynflags'
-- (name) can be "Target.hs", "Target", etc.
target <- guessTarget name Nothing
addTarget target
load LoadAllTargets -- Runs something like "ghc --make".
That's a function that compiles a given module and returns whether compilation succeeded or not. It uses a defaultRunGhc helper function that is defined as:
import GHC.Paths (libdir)
defaultRunGhc :: Ghc a -> IO a
defaultRunGhc = defaultErrorHandler defaultDynFlags . runGhc (Just libdir)
And now a function for fetching a value from the compiled module. The module's source code need not be present at this point.
import Unsafe.Coerce (unsafeCoerce)
fetch :: String -> String -> IO Int -- Assumes we are fetching an Int value.
fetch name value = defaultRunGhc $ do
-- Again, you can change various options in dynflags here, as above.
dynflags <- getSessionDynFlags
let m = mkModule (thisPackage dynflags) (mkModuleName name)
setContext [] [(m, Nothing)] -- Use setContext [] [m] for GHC<7.
fetched <- compileExpr (name ++ "." ++ value) -- Fetching "Target.accessMe".
return (unsafeCoerce fetched :: Int)
And that's it!
The plugins package is problematic anyway. You might want to look at Hint instead.