I'm currently using Stack to build a project, which totals about 80 modules and 13,000 lines of code. Currently, compilation takes about 3 minutes for a minor change in my types files, which is unacceptable -- it prevents me from rapidly testing my code via stack ghci. It also significantly slows down a number of things: for instance, pair programming with peers or consulting with experts.
I would like to be able to figure out why compilation is taking so long, and what I can do to speed it up. Is there some way to profile compilation times?
Additionally, how can I tell whether it is stack or ghc that is slow?
I know about setting -O0, though this does not seem to help compilation times. I also know that TemplateHaskell and Typeclass Resolution can take a while -- is there any way I can get better resolution on how long these are taking?
Note: i have seen Speed up compilation in GHC, but that was from 2013, and hdevtools currently does not work on my setup.
In case this question is still relevant, you could try this:
stack ghci --fast --no-build --ghc-options="-fbyte-code"
IIRC using this got me the quickest results with stack ghci.
If this doesn't help enough, you could look in the direction of --ghc-options="-dshow-passes". I myself am also looking in this direction currently to try to speed up a build/ghci-reload.
Related
Busy discovering Haskell libs. Found FLTKHS for GUI building. Its example fltks-hello-world works great as a skeleton for discovering the all the lib has to offer. However: when changing anything in the source file (fltkhs-hello-world.hs), does one need to stack build it all over? GHCi doesn't seem to grasp the FLTKHS imports, so the REPL is IMO no viable option. Interested in your way of working to learn....
Found out that future builds are way quicker. Not as fast as GHCi though...
Does for anyone else stack compile & link the whole project when only stack ghci is asked for?
Shouldn't it skip the former and jump into ghci right away to load sources? I've observed it to do both for me on most of my projects and therefore it's doing seemingly superfluous work. Is this by design, or might have I misconfigured some of my stack projects?
Perhaps is this done by cabal underneath without stack asking for it?
And at any rate, is there something I could do to skip the compilation + linking when I just want to enter stack ghci?
From the Haskell Tool Stack Documentation:
Speeding up initial load
There are two ways to speed up the initial startup of ghci:
--no-build, to skip an initial build step. This only works if the dependencies have already been built.
You can also have stack ghci skip loading sources (if you know you only want one file loaded in, and can do it by hand) by using:
--no-load, to skip loading all defined modules into ghci. You can then directly use :load MyModule to load a specific module in your project.
I've just realized that -fobject-code may also play a part in this. I'm unsure where it is being set (not on the command line, not in the .cabal/package.yaml file and not in ~/.ghci either).
Nevertheless when I negate it with --ghci-options=-fbyte-code the recompilation doesn't seem to be taking place.
I still hope to hear others' responses as I'd like to understand more thoroughly why the -fobject-code may require minutes for full recompilation on initial load, since :reloading only takes about a second after a file has changed. (E.g. with ghcid too.)
I'm not sure what's causing this issue, but in a project, I'm building, the compiler is taking hours just to compile a module. The total size of my codebase is 352KB, but none of the modules are over 10KB large. I am using a Native port, but it's very trivial; I'm just fetching Date.now() with it.
Is there anything well-known that would cause the elm compiler to take forever to compile? I don't have many dependencies, but I'm using Html a lot. I would really appreciate any hints as to what would cause this.
Edit
So it turns out large case expressions will cause the optimizer to take a long time, as of 0.16. Here's the discussion on Elm-Discuss bringing up the issue, and a gist of the nasty case match.
I guess to be verbose and to keep a carrot out there, why would elm's compiler take this route for case-matching? What's the underlying machinery going on here? Why would the compiler take longer than an hour for optimizing 60+ pattern matches on a case statement?
Large case expressions will cause the optimizer to take a long time, as of 0.16. Here's the discussion on Elm-Discuss bringing up the issue, and a gist of the nasty case match.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
Would it be possible to make Shake reactive, using inotify (or whatever git-annex and Yesod use) so that if ever the filesystem changes in such a way to imply that rule should execute, it does so at the earliest opportunity?
The author of Shake, Neil Mitchell, answered this by saying:
There are a few ways to approach this:
You could just rerun Shake every time it detects something has
changed. Shake is highly optimised for fast rebuilds, if the change
requires doing a compile, then the time for Shake to figure out what
to rebuild is probably minimal. Requires no changes to Shake.
There are certain things Shake does on startup, like reading the
Shake database. If there is demand, and that turns out to be
noticeable in time, I would happily provide a rerun Shake cheaply API
of some sort - it's not that difficult to do.
When Shake does do a rebuild-check, the most expensive thing it
does is checking file modification times. If the inotify layer gave a
list of the files that had changed I could only recheck things that
had actually changed. For a massive project you're likely to see ~1s
checking modification times, so it probably buys you a little, and
isn't too hard to implement.
If Shake is actively building, and then something changes, you
could throw an exception, kill whatever is being built, and restart
Shake. Shake has been thoroughly tested with exceptions being thrown
at it, and does the right thing. I know at least one person uses Shake
in this way.
Finally, if Shake is actively building, you could dynamically
terminate just those rules whose inputs have changed and go again.
Shake could support this model, but it would be a reasonable amount of
work, and require re-engineering some pieces. That would be the full
reactive model, but I suspect it only starts to be a benefit when you
have a massive number of files and a few files are changing almost
continuously but most files aren't.
We also determined that combining Shake with a utility like Hobbes (also on Hackage) can make it possible to do reactive builds.
If GHC takes a long time to compile something, is there a way to find out what it's doing?
Firstly, it would be nice to know if I've actually crashed the compiler (i.e., put it into some sort of infinite loop somehow), or whether it's actually making progress, but just very slowly.
Secondly, it would be nice to know exactly what part of the compilation process GHC is having trouble with. Is it the parsing, or desugaring, or type-checking, or Core optimisation, or code generation, or...?
Is there some way to monitor what's going on? (Bearing in mind that if GHC is taking a long time, that probably means it's doing a lot of work, so if you ask for too much output it's going to be huge!)
GHC already tells you which modules it's trying to (re)compile. In my case, the problem is a single self-contained module. I'd like to know where GHC is getting stuck.
Following Daniel Fischer's comment, I tried running GHC with different verbosity options.
-v1: Produced a bit more output, but nothing during the main compilation step.
-v2: Tells you what step GHC is currently doing (parser, desugar, type check, simplifier, etc). This is pretty much what I actually wanted.
-v3: Appears to make the simplifier actually dump what it's doing to the console - bad idea while compiling 8MB of source code!
So it seems that -v2 is the place to start.
(In the specific case of the program that prompted this question, it seems GHC is spending forever in the type checking phase.)