I have two LLVM.General.Module modules, A and B. My goal is to link B into A and then write the assembly to a file. I am playing for the moment with a toy program to see how to use the LLVM-General interface. I have used the LLVM-General-Pure package to create LLVM.General.AST modules for some compiler backends.
I would like to take these ASTs into the C++ world (LLVM.General.Module), link them, and then write the assembly. I found the error while linking the C++ modules, but I have a feeling that the error is in the way I produce the C++ objects. Here is what I did:
convASTToModE ast = do
a <- withContext $ \context -> do
x <- runErrorT $ withModuleFromAST context ast $ \mod -> return mod
return x
return a
Comment: I couldn't find a function of type Module -> Module (like moduleAST to make the AST from the C++ object when using, say withModuleFromLLVMAssembly) to give to withModuleFromAST, so that I get a Module back, so I just used the unit. My reading of this is that I can do something with the module I get from the ast, and I just want the module, so I just took it. I am guessing this is wrong, but figured it would do something.
When I execute this code on one of my LLVM.General.AST.Module modules, the code seems to run fine. But when I try to use it, it breaks. For example, if x is a LLVM.General.AST.Module, and I do
mod <- convASTToModE x
case mod of
Left _ -> error ""
Right mod' -> do
assemb <- moduleLLVMAssembly mod
putStrLn assemb
I get a segmentation fault.
LLVM-General-3.4.3.0
LLVM-3.4
UPDATE:
I tried the following:
writeASTToLLVMAssembly ast filename = do
withContext $ \context -> do
runErrorT $ withModuleFromAST context ast $ \mod -> do
runErrorT $ writeLLVMAssemblyToFile (File filename) mod
return ()
Gives:
*** Error in `./LLVMImportAndLinkTest': munmap_chunk(): invalid pointer:
0x00000000012db000 ***
Aborted (core dumped)
However,
showLLVMAssFromAST ast = do
str <- withContext $ \context -> do
str2 <- runErrorT $ withModuleFromAST context ast $ \mod -> do
str3 <- moduleLLVMAssembly mod
return str3
return str2
case str of
Left _ -> error "ast not convertible to string"
Right st -> return st
Gives no problems.
LLVM-General.3.4.3.0
LLVM-3.4
For example, with the LLVM.General.AST.Module found
http://lpaste.net/106407
That is the pretty printed version of the AST obtained by uploading
http://lpaste.net/106408
into LLVM-General.
That LLVM assembly was generated from Clang by compiling a C file.
Solution is to upgrade the version of llvm-general by changing the llvm-general dependency to llvm-general >= 3.4.3 which fixes a known bug in the implementation.
Related
I'm trying to follow along with the LLVM bindings tutorial here, and running into a segfault. The following code works in the sense that it prints a module header to output.ll, but it also segfaults somewhere.
module Main where
import Control.Monad.Error
import LLVM.General.Module
import LLVM.General.Context
import qualified LLVM.General.AST as AST
--Create and write out an empty LLVM module
main :: IO ()
main = writeModule (AST.defaultModule { AST.moduleName = "myModule" })
outputFile :: File
outputFile = File "output.ll"
writeModule :: AST.Module -> IO ()
writeModule mod = withContext $ (\context ->
liftError $ withModuleFromAST context mod (\m ->
liftError $ writeLLVMAssemblyToFile outputFile m))
--perform the action, or fail on an error
liftError :: ErrorT String IO a -> IO a
liftError = runErrorT >=> either fail return
I suspect this is related to the following hint from the linked tutorial:
It is very important to remember not to pass or attempt to use resources outside of the bracket as this will lead to undefined behavior and/or segfaults.
I think in this context the "bracket" is implemented by the withContext function, which makes it seem like everything should be handled.
If I change the definition of writeModule to
writeModule mod = do assembly <- (withContext $ (\context ->
liftError $ withModuleFromAST context mod moduleLLVMAssembly))
putStrLn assembly
that is, instead of writing to a file I just print out the string representation of the LLVM assembly, no segfault is thrown.
Does anyone have experience with these bindings? I'm also interested to know about the failure cases for the warning I quoted. That is, how would one "forget" not to use resources outside the bracket? All of the functions that seem to require a Context, well, require one. Isn't this kind of resource scoping issue exactly what Haskell is good at handling for you?
Version information:
llvm-general-3.4.3.0
LLVM version 3.4
Default target: x86_64-apple-darwin13.2.0
It would help if you shared your LLVM and cabal environment, LLVM is notorious for being backwards incompatible with itself so there might be an issue with using the latest versions of the bindings.
Behind the scenes writeLLVMAssemblyToFile is using a C++ call to do the file IO operation and I speculate that it's holding a reference to the LLVM module as a result of finalizing the file resource.
Try rendering the module to a String using moduleString and then only lifting into the IO monad to call writeFile from Haskell instead of going through C++ to the write.
import LLVM.General.Context
import LLVM.General.Module as Mod
import qualified LLVM.General.AST as AST
import Control.Monad.Error
main :: IO ()
main = do
writeModule (AST.defaultModule { AST.moduleName = "myModule" })
return ()
writeModule :: AST.Module -> IO (Either String ())
writeModule ast =
withContext $ \ctx ->
runErrorT $ withModuleFromAST ctx ast $ \m -> do
asm <- moduleString m
liftIO $ writeFile "output.ll" asm
The bindings can still rather brittle in my experience, you should ask on the issue tracker if the problem persists.
EDIT: This is a workaround for an old version that has been subsequently fixed. See: https://github.com/bscarlet/llvm-general/issues/109
Using the Shake Haskell build library, how can I write a rule using a program that needs to reach a fixed point? Imagine I have a program foo that takes a file input and produces an output file, which should have foo applied repeatedly until the output file does not change. How can I write that in Shake?
The typical example of this pattern is LaTeX.
Firstly, note that calling Latex repeatedly does not always produce a fixed point, so make sure you have a bound on the iterations. Also, some distributions (MikTex) provide Latex versions that automatically run as many times as they need to, so if you use those instead the problem goes away.
Write your own foo_transitive command
The easiest way to solve the problem, assuming each run of foo has the same dependencies, is to solve the problem outside the build system. Just write a foo_transitive command, either as a shell script or as a Haskell function, that when supplied an input file produces an output file by running repeatedly and checking if it has reached a fixed point. The build system can now use foo_transitive and there are no issues about dependencies.
Encode it in the build system
You need to write two rules, one which makes one step, and one which figures out which step is the right one to use:
let step i = "tempfile" <.> show i
"tempfile.*" *> \out -> do
let i = read $ takeExtension out :: Int
if i == 0 then
copyFile "input" out
else
let prev = step (i-1)
need [prev]
-- perhaps require addition dependencies, depending on prev
system' "foo" [prev,out]
"output" *> \out -> do
let f i = do
old <- readFile' $ step (i-1)
new <- readFile' $ step i
if old == new || i > 100 then copyFile (step i) out else f (i+1)
f 1
The first rule generates tempfile.2 from tempfile.1 and so on, so we can need ["tempfile.100"] to get the 100th iteration. If the dependencies change in each step we can look at the previous result to calculate the new dependencies.
The second rule loops round checking each pair of values in the sequence, and stopping when they are equal. If you are implementing this in a production build system you may wish to avoid calling readFile' on each element twice (once as i-1 and once as i).
Expanding on #Neil Mitchell's answer, below is a sample code of foo_transitive. Having said that, for this particular case I'd just use latexmk which Does The Right Thing™.
import Control.Monad.Fix (fix, mfix)
import Control.Monad.IO.Class (MonadIO(liftIO))
import Text.Printf (printf)
type SHA = Int
data TeXCompilationStage
= Init
| BibTeX
| Recompile SHA
deriving (Show, Eq)
data TeXResult
= Stable SHA
| Unstable
deriving (Show, Eq)
f retry x budgetLaTeXCalls
| budgetLaTeXCalls <= 0
= do
liftIO $ putStrLn "Budget for LaTeX depleted; result didn't converge"
return Unstable
| otherwise
= case x of
Init -> do
liftIO $ do
putStrLn "Init"
putStrLn " # latex"
retry BibTeX (budgetLaTeXCalls-1)
BibTeX -> do
liftIO $ do
putStrLn "BibTeX"
putStrLn " # bibtex"
retry (Recompile 0) budgetLaTeXCalls
Recompile previousSHA -> do
let budgetLaTeXCalls' = budgetLaTeXCalls - 1
calculcatedSHA = 3
liftIO $ do
printf "Recompile (budget: %d)\n" budgetLaTeXCalls
printf " Prevous SHA:%d\n Current SHA:%d\n" previousSHA calculcatedSHA
if calculcatedSHA == previousSHA
then do
liftIO $ putStrLn " Stabilized"
return $ Stable calculcatedSHA
else do
liftIO $ putStrLn " Unstable"
retry (Recompile (previousSHA+1)) (budgetLaTeXCalls-1)
latex :: Int -> IO TeXResult
latex = fix f Init
As a small part of a larger University project, I need to write what is essentially an extremely crude IDE. The idea is to to take input from a gtk text box, treat that string as if it is in a .hs file, and evaluate a function within it.
My main approach has been to use the GHC API to compile and evaluate a test function. I had already managed to get a toy example working for compiling from a .hs file. GHC's Target data type had an optional constructor for getting a target from a StringBuffer, so I decided to try and alter my code to get it to work from a String Buffer:
compileText :: SourceView -> IO ()
compileText tview = do
txtBuff <- textViewGetBuffer tview
startIt <- textBufferGetStartIter txtBuff
endIt <- textBufferGetEndIter txtBuff
compTime <- getClockTime
srcString <- textBufferGetText txtBuff startIt endIt False
defaultErrorHandler defaultLogAction $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
addTarget $ haskellFileFromText srcString compTime
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Test") Nothing
setContext [IIModule m]
value <- compileExpr ("Test.print")
do let value' = (unsafeCoerce value) :: String -> IO ()
return value'
func "Hello"
return ()
haskellFileFromText :: String -> ClockTime -> GHC.Target
haskellFileFromText codeStr cTime = GHC.Target (TargetModule (mkModuleName "Test")) False (Just ((stringToStringBuffer codeStr), cTime))
The following code being in the text box at the time:
module Test (Test.print) where
print :: String -> IO ()
print x = putStrLn x
However, this does not seem to work. I get the error:
textEdit: panic! (the 'impossible' happened)
(GHC version 7.4.1 for x86_64-unknown-linux):
Could not find module `Test'
Use -v to see a list of the files searched for.
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
What am I doing wrong? I feel I am crucially misunderstanding something about the way this code works.
An alternative to this method which has been suggested to me is to use something like hint or mueval to evaluate the text in the textbox. This would appear to work fine if i simply want to evaluate a single function in isolation, but would this scale if I wanted to evaluate a function which depended on the context of running 4 other functions defined within the same source file?
As C.A. McCann notes, hint does lots of this work for you. It's a wrapper around the GHC api, not just a standalone evaluator like mueval.
Even if it is missing something you need, it will be far easier to learn from it and extend it than start from scratch.
I have a module Target, with a function Target.accessMe inside it. I compile this module in some way, then get rid of the source code.
Now, what series of arcane incantations must I do to make a different program dynamically import Target.accessMe? This program knows accessMe's type in advance. Also, consider the fact that the source code of Target is not available.
The plugins package manages to accomplish this, but seems to have serious issues with working on Windows. I've checked out plugins's source, but am having trouble understanding it.
I've tried using Hint, but can only find out how to evaluate code that I have the source for.
Thanks for any help!
The answer to this question has been given to me elsewhere. The GHC API is capable of doing this. Here are two functions, one of which compiles Target.hs, while the other accesses Target.accessMe (and doesn't require the source code of the Target module to be there anymore).
import GHC
import DynFlags
compile :: String -> IO SuccessFlag
compile name = defaultRunGhc $ do
dynflags <- getSessionDynFlags
let dynflags' = dynflags -- You can change various options here.
setSessionDynFlags dynflags'
-- (name) can be "Target.hs", "Target", etc.
target <- guessTarget name Nothing
addTarget target
load LoadAllTargets -- Runs something like "ghc --make".
That's a function that compiles a given module and returns whether compilation succeeded or not. It uses a defaultRunGhc helper function that is defined as:
import GHC.Paths (libdir)
defaultRunGhc :: Ghc a -> IO a
defaultRunGhc = defaultErrorHandler defaultDynFlags . runGhc (Just libdir)
And now a function for fetching a value from the compiled module. The module's source code need not be present at this point.
import Unsafe.Coerce (unsafeCoerce)
fetch :: String -> String -> IO Int -- Assumes we are fetching an Int value.
fetch name value = defaultRunGhc $ do
-- Again, you can change various options in dynflags here, as above.
dynflags <- getSessionDynFlags
let m = mkModule (thisPackage dynflags) (mkModuleName name)
setContext [] [(m, Nothing)] -- Use setContext [] [m] for GHC<7.
fetched <- compileExpr (name ++ "." ++ value) -- Fetching "Target.accessMe".
return (unsafeCoerce fetched :: Int)
And that's it!
The plugins package is problematic anyway. You might want to look at Hint instead.
For a tool I'm writing ( http://hackage.haskell.org/package/explore ) I need a way to read haskell function definitions at run-time, apply them to values from my tool and retrieve the results of their application.
Can anyone give me a very basic example using GHC (6.10.4 or 6.12.1) API?
example function definition to be read from a file at run-time:
f x = 10**(4/1102*x - 1)
expected program output
--mapM_ print $ map f [428, 410, 389]
3.577165388142748
3.077536885227335
2.5821307011665815
!!UPDATE!!
I posted a quick answer but it creates an object file in the directory of execution, any tips to avoid this and avoid all file IO is most welcome. I want to also see a version that does everything in memory: user provides the function definition in a GUI for example and the compilation / evaluation does not create any object files.
Use hint. It's a GHCi-like wrapper around the GHC API that is not very difficult to use.
If you want an example of its use, I used it in my Yogurt project.
adapted from: http://www.bluishcoder.co.nz/2008/11/dynamic-compilation-and-loading-of.html
f.hs:
module Func (Func.f) where
f :: Double -> Double
f x = 10**(4/1102*x - 1)
main.hs:
import GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce
import Control.Monad
main :: IO ()
main =
defaultErrorHandler defaultDynFlags $ do
func <- runGhc (Just libdir) $ do
dflags <- getSessionDynFlags
setSessionDynFlags dflags
target <- guessTarget "f.hs" Nothing
addTarget target
r <- load LoadAllTargets
case r of
Failed -> error "Compilation failed"
Succeeded -> do
m <- findModule (mkModuleName "Func") Nothing
setContext [] [m]
value <- compileExpr ("Func.f")
do let value' = (unsafeCoerce value) :: Double -> Double
return value'
let f = func
mapM_ print $ map f [428, 410, 389]
return ()
Nice work getting the API going. I can tell you a little bit about how the code generator works.
GHC uses the system assembler to create a .o file. If there is not an option available to get GHC to clean up after itself, then you should file a feature request against the API, using the bug tracker at http://hackage.haskell.org/trac/ghc/newticket?type=feature+request. In order to file the request, you will need to register an account.
Using the standard code generator, you will not be able to avoid file I/O entirely, just because GHC delegates the work of creating relocatable object code to the assembler. There is an experimental back end based on LLVM that might be able to do everything in memory, but I would be surprised if it is available in anything earlier than 6.13. However it could be worth asking on the GHC developers' list.