I cannot change GHC's compilation times, but I can control the environment in which things are compiled. What can I do with git and stack to minimize downtime due to (re-)building?
Haskell stack caches compiled dependencies in the .stack-work directory, and it great for working on one thing at a time. However, I find myself switching back and forth between branches. I've done git checkout in place, and perhaps some of the .stack-work cache is usable. But I greatly prefer git worktree to get a copy of a branch in a different directory altogether. Worktree is especially hard on build times as each worktree directory will have its own .stack-work, and therefore must download/configure/compile/install its own dependencies instead of using existing ones.
I've been doing some reading about Nix and Haskell, but since stack creates its own cache, it ignores whatever Nix has compiled and installed. From what I've read, I really want Nix to handle dependencies and stack to handle my personal code. I get great benefit from stackage and stack solver, so I do not want to give them up. But when I see a screenful of "unregistering" messages when running stack build --only-dependencies && stack ghci, I know that I've got at least 10-15 minutes before I get a prompt.
Many thanks!
Related
I don't get the point about Stack.
I used to write my Haskell code in my favourite environment, ran or compiled using GHC(i), and if necessary, installed packages using Cabal. Now, that apparently is not the way to go any more, but I don't understand how to work with Stack. So far, I have only understood that I need to write stack exec ghci instead ghci to start a repl.
Apart from that, the docs always talk about 'projects' for which I have to write some yaml files. But I probably don't have any project -- I just want to launch a GHCi repl and experiment a bit with my ideas. At the moment, this fails with the unability to get the packages that I want to work with installed.
How is working with Stack meant? Is there any explanation of their use cases? Where do I find my use case in there?
Edit. My confusion comes from the fact that I want to work with some software (IHaskell) whose installation guide explains the installation via stack. Assuming I already have a GHCi installed whose package base I maintain e.g. using Cabal. How would I have to set up stack.yaml to make stack use my global GHCi for that project?
First, notice that stack will use its own package base independent from cabal. AFAIK they can't be shared... hence, if you run stack build it'll download packages (including the compiler) on its own package database.
Nevertheless stack allows to use a system compiler (but not other libraries). To do so, in the stack.yaml you must have the following two lines
resolver: lts-XX.XX -- keep reading below
system-ghc: True
The version of the stackage snapshot can be found in: https://www.stackage.org/. Each snapshot works with a different version of the compiler. Be sure to use a snapshot with the same compiler version you have in you system. If it happens your system ghc is greater than any lts, then you can set allow-newer: true in stack.yaml.
Now, if getting a different database from stack feels wrong to you, notice that you can build the project with cabal too, since stack at the end of the day spits out a cabal file. Probably, It wont work out of the box if you build with cabal. You can modify the cabal file to match exactly the version of the packages of the snapshot you are using
In summary:
You can use your system-wide ghc
you can not share libraries installed with cabal.
you can use cabal to build the project, probably modifying the ihaskell.cabal file to match versions of the stackage's snapshot.
I have two Haskell libraries lib-a and lib-b, both hosted on private git repos. lib-b depends on lib-a, both build with no problem.
Now i want to import lib-b into another project and thus add it to the stack configuration with the git directive, like this:
- git: git#github.com:dataO1/lib-b.git
commit: deadbeef102958393127912734
Stack still seems to need a specific version for lib-a:
In the dependencies for application-0.1.0.0:
lib-a needed, but the stack configuration has no specified version (no package with that name found,
perhaps there is a typo in a package's build-depends or an omission from the stack.yaml packages
list?)
needed due to application-0.1.0.0 -> lib-b-0.1.0.0
The question now is, can stack somehow figure out specific versions for nested git dependencies without explicitely specifying them? If the project grows i dont want to manually adjust this every time i update lib-a.
Sidenote: I'm using nixOS and the nix directive for all three stack projects.
Stack follows the snapshot-based model of package management, meaning that it has some "globally" specified set of packages (of fixed version) that you can use. In Stack's case this set of packages is called Stackage. The core idea is to have this clearly specified set of packages that you're working with.
So the short answer is no it cannot figure it out by itself, you have to add them by hand.
But! you need to specify only packages that are not in the snapshot. e.g. package lib-a is likely to depend on mostly packages that are commonly used in Haskell (e.g. base, aeson, ...) and those will already be in Stackage. so even if the project grows you will be adding just "a few" git refs.
So this doesn't generally tend to be a problem.
Problem: I'm working on a Haskell project that uses stack (+ nix). We have a dependency that takes 10+ minutes to compile. Every time we clean our .stack-work, we have to wait for this huge package to compile, and it's really hurting our project's efficiency. The package name is godot-haskell, and here is how the package is depended upon in our stack.yaml:
extra-deps:
- godot-haskell-0.1.0.0#sha256:9d92ff27c7b6c6d2155286f04ba2c432f96460f448fd976654ef26a84f0e35a6,26290
Question: Is there a way for us to somehow cache this package (in stack, or even in nix) so that it locally never has to get compiled (or has to get compiled at most once, even if the .stack-work directory is deleted)?
For the currently released Stack, the best way to make this happen is to put the extra-dep into a custom snapshot file instead of the extra-deps in the stack.yaml file. (The upcoming Stack release has a feature referred to as "implicit snapshots" which sidesteps this.) You can see an example of this in the Stack repo itself:
https://github.com/commercialhaskell/stack/blob/master/stack.yaml#L1
https://github.com/commercialhaskell/stack/blob/master/snapshot.yaml
I'm working with the Lean theorem prover and I would like to build features like those of stack and start building library caches. One thing that I'm unsure about is when to delete older packages in the cache. I could think that, if I have two versions of Lean installed, say 3.3.0 and 3.3.1, I could argue for keeping all the versions of all the packages that work on those. Since the Lean syntax moves fast, it is quite restrictive.
On the other hand, I could see keeping only 10 versions of any given package. This could be the 10 most recent versions or the 10 most recently requested versions.
I'd love to know some of the ideas considered in building stack and related tools.
stack keeps packages in cache forever. Or until you manually delete those packages. There's no snapshot garbage collection implemented in stack. So what you can only do is to manually call
rm -rf ~/.stack
to remove whole stack cache. It's in general good practice to do it periodically to remove outdated packages which were accumulated for some period of time. But make sure to backup .stack/global-project/stack.yaml if you changed it manually.
stack command which handles cache removal for you is under development:
https://github.com/commercialhaskell/stack/issues/133
I took the habit of managing my post-install setup things in the Setup.hs. The advantages are clear: it's coded in haskell, and it has access to the app data directory, where the binary will get installed.
I'm using it now for instance, to copy the QML data files and icons that a haskell QML application needs to the installation folder.
This all worked well until recently, but now stack started optimizing the build a lot, and if it detects that no haskell files changed, it won't re-trigger the Setup.hs. That obviously doesn't work for me, because I'd like my QML files and icons to get copied every time I run stack install during development.
I can't find a flag for stack that would trigger just the Setup.hs and nothing else (or little else -- certainly clean and --reconfigure force rebuilding too many things). How are people doing that? Should I write a shell script? But it'll have trouble finding the target directory for the installation, plus it's less portable that the haskell in my Setup.hs?
I feel stack are probably right in their reasoning and if my current way was really correct they would have enabled this scenario by now. So either there's a stack flag that I don't see, either I'm doing it wrong I think.
NOTE: stack is so smart that not even touch Setup.hs or of another .hs file helps to re-trigger its execution. I must physically change the contents of one file.
NOTE2: Ah, I re-discovered stack path to find out target path information. I could use that from a shell script to install the files. Still feels like a shame to write that in shell when I have haskell...
NOTE3: posted the question on the stack mailing list now...
NOTE4: actually opened a stack bug now