Typically in a Haskell project, I either work interactively with ghci or compile the entire project with cabal build.
However, in some use cases, I may have a computationally intensive routine along with some higher level scripting functionality, say for picking inputs to an analysis algorithm.
Is it possible to use GHCi + GHC such that I compile the computationally intensive module, load the compiled code to re-run with different inputs from within GHCi?
Yes, you can load compiled modules in ghci; if there is an appropriately named .hi and .o file, ghci will use those instead of interpreting the code in the corresponding .hs file. You will then only have access to the operations that are exported from that module.
In case you find yourself using a compiled loaded module when you wanted the interpreted one, you can :load *foo.hs to instruct ghci to ignore the compiled version and interpret foo.hs.
Related
Here is my desired use case:
I have a package with a single module that reads HDF5 files and writes some of their data to Haskell records. To do the work, the library uses the bindings-hdf5 package. Here is my cabal's build-depends. reader-types is a module I wrote that defines the types of the Haskell records that contain the read-in data.
build-depends: base >=4.7 && <4.8
, text
, vector
, containers
, bindings-hdf5
, reader-types
Note that my cabal file does not currently use extra-libraries or ghc-options. I can load my module, src/Mabel.hs in ghci as long as I specify the required hdf5_hl library:
ghci src/Mabel.hs -lhdf5_hl -L/long/nixos/path/lib
and within ghci, I can run my function perfectly fine.
Now, what I want to do is compile this library/module into a single, compiled file that I can later load with the GHC API in a different Haskell program. By single file, I mean that it needs to run even if the hdf5_hl library does not exist on the system. Preferably, it would also run even if text, vector, and/or containers are missing, but this is not essential because reader-types requires those types anyway. When loading the module with the GHC API, I want it to load in already compiled form, and not run interpreted.
My purpose for doing this is that I want the self-contained file to act as a single, pre-compiled plugin file that is later loaded and executed by a different Haskell executable. Other plugins might not use hdf5 at all, and the only package they are guaranteed to use is reader-types, which essentially defines the plugin interface types.
The hdf5 library on my system contains the following files: libhdf5_la.la, libhdf5_hl.so, libhdf5.la, libhdf5.so, and similar files that have the version number in the file name.
I have done a lot of googling, but am getting confused by all the edge cases I am finding. Here are some examples that I'm either sure don't fit my case, or I can't tell.
I do not want to compile a Haskell library to use from C or Python, only a Haskell program using GHC API.
I do not want to compile C wrappers for a C++ library into a Haskell module because the bindings already exist and the library is already a C library.
I do not to want compile a library that is entirely self-contained because, since I am loading it with the GHC API, I don't need the GHC runtime included in the library. (My understanding is that the plugins must be compiled with the same ghc version they will be loaded with in the GHC API).
I do not want to compile C bindings and the C library at the same time because the C library is already compiled and the bindings are specified in separate package (bindings-hdf5).
The closest resource for what I want to do is this exchange on the mailing list from 2009. However, I added extra-libraries: hdf5_hl or extra-libraries: hdf5 to my cabal file, and in both cases the resulting .a, .so, .dyn_hi, .dyn_o, .hi, and .o files in dist/build are all the exact same size as without using extra-libraries, so I'm confident it is not working correctly.
What changes to my cabal file do I need to make to create a self-contained, standalone file that I can later load with the GHC API? If this is not possible, what are the alternatives?
Instead of using the GHC API, I am also open to using the plugins library to load the plugin, but the self-contained requirements are still the same.
EDIT: I do not care what form the compiled "plugin" must take (I assume object file is the right way), but I want to load it dynamically from an separate executable at run time and execute functions it defines with known names and known types. The reason I want a single file is that there will eventually be other different plugins, and I want them all to behave the same way without having to worry about lib paths and dependencies for each one. A compiled, single file is a simpler interface for doing this than zipping/unzipping archives that include Haskell object code and their dependencies.
Is it possible to verify that given .hs source is portable to any architecture assuming there is Haskell compiler available for it? How to verify that imported modules are also portable?
Is there options, except -O0, that can speed up compilation time?
It's not matter if resulting programs will be not optimised. Actually I want to just type-check large haskell package often and fast.
Flag -fno-code dramatically speeds up compilation but it's not possible to use it because TemplateHaskell is used by this program.
Looks like a task for hdevtools! Hdevtools is used to as a backend for vim-plugin of the same name and it provides speedy syntax and type checking, directly from the editor. It is about as fast as ghci when reloading modules. I assume that it can be used from the command line.
Another alternative would be to keep a ghci instance running and use that to type check your modules.
I've found splitting up large files can speed up compilation.
In JVM based programs, you can compile a file to a .class file and be able to run the binary again, without compiling necessarily all the files.
Is it possible to do it in haskell? Is it imperative to compile and link all the files in the project? If yes, why?
What if there is no binary, you are only installing a library?
For GHC, you can change and recompile a single module without having to recompile modules depending on that, provided the exposed interface doesn't change. GHC's --make mode (default as of ghc-7.*) checks whether recompilation is necessary and recompiles only those modules where it can't determine that it's not necessary.
If you have a cabal package and you cabal build after changing one module, you can see from the compiler output that it doesn't recompile all modules in the package in general, only the changed module and [maybe] the ones depending on it.
If you build an executable, that of course has to be relinked, but many of the old object files can be reused.
If you build a library, the library archive of course has to be rebuilt, but many of the old object files can be reused.
When i import a (big) module into a Main module in one of the following ways:
import Mymodule
import qualified Mymodule as M
import Mymodule (MyDatatype)
the compiled binary grows the same huge amount compared to when i don't import that module. This happens regardless of whether i use anything inside that module or not in the Main module. Shouldn't the compiler (i am using GHC on Debian Testing) only add into the binary what is needed to run it?
In my specific case i have a huge Map in Mymodule which i don't use in the Main module. Selectively importing what i really need, did not change the growth of the compiled binary.
As far as GHC is concerned, import lists are only there for readability and avoiding name clashes; they don't affect what's linked in at all.
Also, even if you did only import a few functions from a library, they might still depend on the bulk of the library internally, so you shouldn't necessarily expect to see a size decrease from only using some of an available interface in general.
By default, GHC links in entire libraries, rather than only the pieces you use; you could avoid this by building libraries with the -split-objs option to GHC (or put split-objs: True in your cabal-install configuration file (~/.cabal/config on Unix)), but it slows down compilation, and is seemingly not recommended by the GHC developers:
-split-objs
Tell the linker to split the single object file that would normally be generated into multiple object files, one per top-level Haskell function or type in the module. This only makes sense for libraries, where it means that executables linked against the library are smaller as they only link against the object files that they need. However, assembling all the sections separately is expensive, so this is slower than compiling normally. Additionally, the size of the library itself (the .a file) can be a factor of 2 to 2.5 larger. We use this feature for building GHC’s libraries.
— The GHC manual
This will omit unused parts of libraries you use, regardless of what you import.
You might also be interested in using shared Haskell libraries.