Following the GHC tutorial posted here and alterations to this code following the advice in a previous stack overflow question I asked, I have created a program which is able to compile and run a module in Test.hs with a function print to print a string to the screen:
import GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce
main :: IO ()
main =
defaultErrorHandler defaultLogAction $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "Test.hs" Nothing
addTarget target
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Test") Nothing
setContext [IIModule m]
value <- compileExpr ("Test.print")
do let value' = (unsafeCoerce value) :: String -> IO ()
return value'
func "Hello"
return ()
The problem with this code, as noted in the comments, is that it only seems to work the first time you run it (When Test.hs has not yet been complied). If you attempt to run the code a second time, the following error appears:
mkTopLevEnv: not interpreted main:Test
I believe this has something to do with the fact that the code has already been compiled. If I delete the .hi and .o files and run the program again, the program runs correctly with the correct output. What am I missing? I am currently using ghc version 7.4.1
(Note: I have tried looking through the GHC API but could not find any references to mkTopLevEnv)
Simon Marlow suggests here that replacing
guessTarget "Test.hs" Nothing
with
guessTarget "*Test.hs" Nothing
should avoid the error you're getting, on the grounds that it tells GHC not to load the .o file.
See the whole thread on a page via nabble
Of course, you could delete the .hi and .o files each time, but that's an ugly workaround.
Related
I need to have something like
-- Main.hs
module Main where
main :: IO ()
main = do
<import Plugin>
print Plugin.computation
With a Plugin like
-- Plugin.hs
module Plugin where
computation :: Int
computation = 4
However, I need the plugin to be compiled alongside the main application. They need to be deployed together. Only the import (not the compilation) of the module should happen dynamically.
I found Dynamically loading compiled Haskell module - GHC 7.6 along the way and it works just fine with GHC 8.0.2 except for the fact that it requires the source file of the plugin to be in the current working directory when executing the application.
Edit (07.12.2017)
Is it possible to load a module from a String instead of a file using the GHC API? http://hackage.haskell.org/package/ghc-8.2.1/docs/GHC.html#t:Target suggests that it's possible, but the documentation has many holes and I can't find a way to actually do this. If this can be accomplished, I can use file-embed to include the plugin source file into the compiled binary.
Example:
module Main where
-- Dynamic loading of modules
import GHC
import GHC.Paths ( libdir )
import DynFlags
import Unsafe.Coerce
import Data.Time.Clock (getCurrentTime)
import StringBuffer
pluginModuleNameStr :: String
pluginModuleNameStr = "MyPlugin"
pluginSourceStr :: String
pluginSourceStr = unlines
[ "module MyPlugin where"
, "computation :: Int"
, "computation = 4"
]
pluginModuleName :: ModuleName
pluginModuleName = mkModuleName pluginModuleNameStr
pluginSource :: StringBuffer
pluginSource = stringToStringBuffer pluginSourceStr
main :: IO ()
main = do
currentTime <- getCurrentTime
defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
result <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
let target = Target { targetId = TargetModule $ pluginModuleName
, targetAllowObjCode = True
, targetContents = Just ( pluginSource
, currentTime
)
}
setTargets [target]
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
setContext [IIDecl $ simpleImportDecl pluginModuleName]
result <- compileExpr ("MyPlugin.computation")
let result' = unsafeCoerce result :: Int
return result'
print result
This, however, results in
<command-line>: panic! (the 'impossible' happened)
(GHC version 8.0.2 for x86_64-apple-darwin):
module ‘MyPlugin’ is a package module
Edit (08.12.2017)
I can compile the "plugin" directly into the final binary by writing the source to a temp file and then loading it like in the linked post (Dynamically loading compiled Haskell module - GHC 7.6). However, this does not play well if the plugin imports packages from Hackage:
module Main where
import Control.Monad.IO.Class (liftIO)
import DynFlags
import GHC
import GHC.Paths (libdir)
import System.Directory (getTemporaryDirectory, removePathForcibly)
import Unsafe.Coerce (unsafeCoerce)
pluginModuleNameStr :: String
pluginModuleNameStr = "MyPlugin"
pluginSourceStr :: String
pluginSourceStr = unlines
[ "module MyPlugin where"
, "import Data.Aeson"
, "computation :: Int"
, "computation = 4"
]
writeTempFile :: IO FilePath
writeTempFile = do
dir <- getTemporaryDirectory
let file = dir ++ "/" ++ pluginModuleNameStr ++ ".hs"
writeFile file pluginSourceStr
return file
main :: IO ()
main = do
moduleFile <- writeTempFile
defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
result <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget moduleFile Nothing
setTargets [target]
r <- load LoadAllTargets
liftIO $ removePathForcibly moduleFile
case r of
Failed -> error "Compilation failed"
Succeeded -> do
setContext [IIDecl $ simpleImportDecl $ mkModuleName pluginModuleNameStr]
result <- compileExpr "MyPlugin.computation"
let result' = unsafeCoerce result :: Int
return result'
print result
Is there a way to load packages when, for instance, MyPlugin contains the statement import Data.Aeson? If I add it to the plugin string, it fails with
/var/folders/t2/hp9y8x6s6rs7zg21hdzvhbf40000gn/T/MyPlugin.hs:2:1: error:
Failed to load interface for ‘Data.Aeson’
Perhaps you meant Data.Version (from base-4.9.1.0)
Use -v to see a list of the files searched for.
haskell-loader-exe: panic! (the 'impossible' happened)
(GHC version 8.0.2 for x86_64-apple-darwin):
Compilation failed
CallStack (from HasCallStack):
error, called at app/Main.hs:40:19 in main:Main
The reason for my request is database support: We use Persistent to access a database and the dynamic import is needed to support multiple databases (MySQL, PostgreSQL and SQLite) while still allowing the end user to only install one of the three database servers (with other words: not requiring the user to install all of them if they only use, for instance, PostgreSQL). The module that is database-specific should only be loaded when the user actually configures the main application to use that module.
If I don't import Database.Persist.MySQL, then the application does not require MySQL to be installed. Otherwise, the application fails with, for instance,
dyld: Library not loaded:
/usr/local/opt/mysql/lib/libmysqlclient.20.dylib
on macOS.
A file with a matching module name must exist by the looks of it - it doesn't seem to matter what the file's content is.
On Linux I can even make it be a symlink to /dev/null and things work but a symlink to itself doesn't.
I'm trying to dynamically compile and load Haskell modules using GHC API. I understand the API fluctuates quite a bit from on one version to another so I'm specifically talking about GHC 7.6.*.
I have tried running the same code on MacOS and Linux. In both cases the Plugin module compiles fine but gives the following error on load: Cannot add module Plugin to context: not interpreted
The problem is similar to the one in this where the module would only load if it was compiled in the same run of the host program.
-- Host.hs: compile with ghc-7.6.*
-- $ ghc -package ghc -package ghc-paths Host.hs
-- Needs Plugin.hs in the same directory.
module Main where
import GHC
import GHC.Paths ( libdir )
import DynFlags
import Unsafe.Coerce
main :: IO ()
main =
defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
result <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "Plugin.hs" Nothing
setTargets [target]
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
setContext [IIModule (mkModuleName "Plugin")]
result <- compileExpr ("Plugin.getInt")
let result' = unsafeCoerce result :: Int
return result'
print result
And the plugin:
-- Plugin.hs
module Plugin where
getInt :: Int
getInt = 33
The problem is that you're using IIModule. This indicates that you want to bring the module and everything in it, including non-exported stuff into the context. It's essentially the same as :load with an asterisk in GHCi. And as you've noticed, this only works with interpreted code since it let's you "look inside" the module.
But that's not what you need here. What you want is to load it as if you used :module or an import declaration, which works with compiled modules. For that, you use IIDecl which takes an import declaration which you can make with simpleImportDecl:
setContext [IIDecl $ simpleImportDecl (mkModuleName "Plugin")]
As a small part of a larger University project, I need to write what is essentially an extremely crude IDE. The idea is to to take input from a gtk text box, treat that string as if it is in a .hs file, and evaluate a function within it.
My main approach has been to use the GHC API to compile and evaluate a test function. I had already managed to get a toy example working for compiling from a .hs file. GHC's Target data type had an optional constructor for getting a target from a StringBuffer, so I decided to try and alter my code to get it to work from a String Buffer:
compileText :: SourceView -> IO ()
compileText tview = do
txtBuff <- textViewGetBuffer tview
startIt <- textBufferGetStartIter txtBuff
endIt <- textBufferGetEndIter txtBuff
compTime <- getClockTime
srcString <- textBufferGetText txtBuff startIt endIt False
defaultErrorHandler defaultLogAction $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
addTarget $ haskellFileFromText srcString compTime
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Test") Nothing
setContext [IIModule m]
value <- compileExpr ("Test.print")
do let value' = (unsafeCoerce value) :: String -> IO ()
return value'
func "Hello"
return ()
haskellFileFromText :: String -> ClockTime -> GHC.Target
haskellFileFromText codeStr cTime = GHC.Target (TargetModule (mkModuleName "Test")) False (Just ((stringToStringBuffer codeStr), cTime))
The following code being in the text box at the time:
module Test (Test.print) where
print :: String -> IO ()
print x = putStrLn x
However, this does not seem to work. I get the error:
textEdit: panic! (the 'impossible' happened)
(GHC version 7.4.1 for x86_64-unknown-linux):
Could not find module `Test'
Use -v to see a list of the files searched for.
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
What am I doing wrong? I feel I am crucially misunderstanding something about the way this code works.
An alternative to this method which has been suggested to me is to use something like hint or mueval to evaluate the text in the textbox. This would appear to work fine if i simply want to evaluate a single function in isolation, but would this scale if I wanted to evaluate a function which depended on the context of running 4 other functions defined within the same source file?
As C.A. McCann notes, hint does lots of this work for you. It's a wrapper around the GHC api, not just a standalone evaluator like mueval.
Even if it is missing something you need, it will be far easier to learn from it and extend it than start from scratch.
I have a module Target, with a function Target.accessMe inside it. I compile this module in some way, then get rid of the source code.
Now, what series of arcane incantations must I do to make a different program dynamically import Target.accessMe? This program knows accessMe's type in advance. Also, consider the fact that the source code of Target is not available.
The plugins package manages to accomplish this, but seems to have serious issues with working on Windows. I've checked out plugins's source, but am having trouble understanding it.
I've tried using Hint, but can only find out how to evaluate code that I have the source for.
Thanks for any help!
The answer to this question has been given to me elsewhere. The GHC API is capable of doing this. Here are two functions, one of which compiles Target.hs, while the other accesses Target.accessMe (and doesn't require the source code of the Target module to be there anymore).
import GHC
import DynFlags
compile :: String -> IO SuccessFlag
compile name = defaultRunGhc $ do
dynflags <- getSessionDynFlags
let dynflags' = dynflags -- You can change various options here.
setSessionDynFlags dynflags'
-- (name) can be "Target.hs", "Target", etc.
target <- guessTarget name Nothing
addTarget target
load LoadAllTargets -- Runs something like "ghc --make".
That's a function that compiles a given module and returns whether compilation succeeded or not. It uses a defaultRunGhc helper function that is defined as:
import GHC.Paths (libdir)
defaultRunGhc :: Ghc a -> IO a
defaultRunGhc = defaultErrorHandler defaultDynFlags . runGhc (Just libdir)
And now a function for fetching a value from the compiled module. The module's source code need not be present at this point.
import Unsafe.Coerce (unsafeCoerce)
fetch :: String -> String -> IO Int -- Assumes we are fetching an Int value.
fetch name value = defaultRunGhc $ do
-- Again, you can change various options in dynflags here, as above.
dynflags <- getSessionDynFlags
let m = mkModule (thisPackage dynflags) (mkModuleName name)
setContext [] [(m, Nothing)] -- Use setContext [] [m] for GHC<7.
fetched <- compileExpr (name ++ "." ++ value) -- Fetching "Target.accessMe".
return (unsafeCoerce fetched :: Int)
And that's it!
The plugins package is problematic anyway. You might want to look at Hint instead.
For a tool I'm writing ( http://hackage.haskell.org/package/explore ) I need a way to read haskell function definitions at run-time, apply them to values from my tool and retrieve the results of their application.
Can anyone give me a very basic example using GHC (6.10.4 or 6.12.1) API?
example function definition to be read from a file at run-time:
f x = 10**(4/1102*x - 1)
expected program output
--mapM_ print $ map f [428, 410, 389]
3.577165388142748
3.077536885227335
2.5821307011665815
!!UPDATE!!
I posted a quick answer but it creates an object file in the directory of execution, any tips to avoid this and avoid all file IO is most welcome. I want to also see a version that does everything in memory: user provides the function definition in a GUI for example and the compilation / evaluation does not create any object files.
Use hint. It's a GHCi-like wrapper around the GHC API that is not very difficult to use.
If you want an example of its use, I used it in my Yogurt project.
adapted from: http://www.bluishcoder.co.nz/2008/11/dynamic-compilation-and-loading-of.html
f.hs:
module Func (Func.f) where
f :: Double -> Double
f x = 10**(4/1102*x - 1)
main.hs:
import GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce
import Control.Monad
main :: IO ()
main =
defaultErrorHandler defaultDynFlags $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "f.hs" Nothing
addTarget target
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Func") Nothing
setContext [] [m]
value <- compileExpr ("Func.f")
do let value' = (unsafeCoerce value) :: Double -> Double
return value'
let f = func
mapM_ print $ map f [428, 410, 389]
return ()
Nice work getting the API going. I can tell you a little bit about how the code generator works.
GHC uses the system assembler to create a .o file. If there is not an option available to get GHC to clean up after itself, then you should file a feature request against the API, using the bug tracker at http://hackage.haskell.org/trac/ghc/newticket?type=feature+request. In order to file the request, you will need to register an account.
Using the standard code generator, you will not be able to avoid file I/O entirely, just because GHC delegates the work of creating relocatable object code to the assembler. There is an experimental back end based on LLVM that might be able to do everything in memory, but I would be surprised if it is available in anything earlier than 6.13. However it could be worth asking on the GHC developers' list.