It takes several hours to recompile about 100 dependencies if I accidentally let stack unregister them when switching between --profile on and off.
I don't mean the initial building of the packages with both, but subsequent builds.
Currently I try to work around by having different copies of .stack-work. I'm trying to use hardlinks too via cp -al. But I hope stack could also just not throw away hours worth of compiled dependencies so willingly without so much as a warning. Or even better, if it kept both without unregistering.
I have Version 1.4.1, Git revision 3d29b8c68a x86_64 hpack-0.17.0
Related
1 Context
I am involved in a Haskell project that involves lots of C-bits and FFI. So I find myself frequently running and re-running commands like
$ stack build
$ stack build --force-dirty
$ stack clean
$ rm ./.stack-work
over and over in order for the C-bits to be linked properly to the Haskell bits. Put differently, sometimes things just work when running stack build, and sometimes they don't (in which case I'm forced to cycle through the above commands over and over until my project builds properly).
This means I don't have a proper understanding of how stack (through ghc) assembles the C-bits before assembling the Haskell bits. So here is one question to help me start clearing up my confusion:
2 Question
Are there any noteworthy difference between running stack clean and deleting the contents of the .stack-work directory? Are there cases where deleting the .stack-work directory is needed as a good precaution to ensure that you are actually running a clean build?
As you can see by reading the source here:
https://github.com/commercialhaskell/stack/blob/master/src/Stack/Clean.hs
There are two levels, full and shallow. I think shallow seems to be the default. It seems to be able to clean specific packages, or if you don't provide no options at all, it'll clean everything but extra-deps in local packages.
I recently downloaded the Haskell Platform from the Haskell website. Under the suggestion of the newer answers in this, I blindly ran brew install ghc cabal-install and cabal install cabal cabal-install. Did I install two versions of Haskell on my machine? What should I do to fix any problems?
It doesn't necessarily lead to problems to have multiple versions (I think I have three different versions installed). If you need the disk space uninstall one of the two (instructions for the brew one, for the packaged platform it seems you should be able to use the command sudo uninstall-hs but check it yourself first). If you don't mind the lost disk space, you only have to make sure you have your PATH set up correctly, with the directory containing the ghc binary you want to use in your PATH, before the directory of the other one.
Also, cabal install cabal-install (which you might need to run to update cabal) tends to install cabal in a different place than the platform/brew do, so there, again, you need to make sure your PATH is appropriately set. Normally cabal installs executables in ~/.cabal/bin (local installs) or /usr/local/bin (global installs). The directory containing cabal should go before the others, because an old version of cabal might stick around and you want the new one to be found first.
You probably know this but you can use which ghc and which cabal to check the location of the executable actually being used.
To make things even more complicated, lately it's popular to use Stack, which can also install ghc for you (I find this very convenient, everything is kept in a very controlled environment). So depending on your experience/use case this might be worth looking at as well (but if you just want to try Haskell I recommend you stick with the platform or the brew installation).
Currently, I am using GHC 7.10.1 on my ubuntu 14.04 system, just now I got noticed from mailing list that a new version containing a lot of bug fixes has been released. So I am considering updating my GHC.
I know how to install GHC, however, I have little knowledge about how to deal with packages I have already installed through cabal. It seems that those packages were installed into ~/.cabal, for example, libs were installed into ~/.cabal/lib/x86_64-linux-ghc-7.10.1 and registered in ~/.ghc/x86_64-linux-7.10.1.
The path name(...x86_64-linux-7.10.1...) seems to be suggesting that they could only be used by GHC 7.10.1, so if I want to use a new version GHC and remove the old ghc 7.10.1 from my system, should I clean them up and reinstall those packages?
for example, should I rm -rf ~/.cabal && rm -rf ~/.ghc and reinstall both cabal and GHC? or should I just leave those packages there? if so, would those installed packages get reused by the new GHC?
Yes, you need to reinstall the packages when you update the compiler.
However, if you delete ~/.ghc then that removes the packages from GHC's point of view. You can also delete ~/.cabal and reinstall everything, but binaries in ~/.cabal/bin (including cabal) will usually still work, so often it's easier to keep these and save some time (have a look to see what's there). There might be problems with some - I think some binaries may have the GHC path hard coded internally, so will break, but you can always reinstall them.
~/.cabal/packages contains cached downloads so it's up to you whether you delete that. The data will just be pulled down again next time your run cabal. If in doubt just delete it.
You should delete ~/.cabal/lib which contains the compiled packages and won't be compatible with a different GHC. Likewise for ~/.cabal/share.
Unless you've edited your ~/.cabal/config file by hand and want to keep the changes you can delete that too and a new one will be created.
You don't need to remove anything. If you install GHC 7.10.2 in the same location that you installed GHC 7.10.1, then all the symlinks ghc, ghci, ghc-pkg etc. will be overwritten to point to 7.10.2, but you can still run the versioned programs ghc-7.10.1, ghci-7.10.1, ghc-pkg-7.10.1, etc., or tell cabal to build with a specific compiler with the -w flag. All libraries and library registration information (whether they came with GHC, are installed in the user package database, or are in a sandbox) are (by default) stored under GHC version-specific subdirectories, so different versions of GHC will not interfere with each other.
If you're not running low on disk space I would suggest not removing GHC 7.10.1. After all 7.10.2 was released just yesterday, and if you encounter a strange issue with 7.10.2, it might be worth checking whether 7.10.1 was also affected.
When I have a sandbox, it seems cabal install ignores packages in $HOME/.ghc/x86_64-linux-7.8.4/package.conf.d.
How can I configure the sandbox such that these packages become visible?
I am seeing a vague reference to --package-db=db in https://www.haskell.org/cabal/users-guide/installing-packages.html#sandboxes-advanced-usage
but I understand neither how nor when to use it. (with sandbox init? configure? install? none seems to work - none gives any error message either.)
I know about add-source but my question refers to installed packages.
The whole point of the sandbox is that it ignores your local package database.
If you want to share installations across many sandboxes, you may install to the global database; but then you should be very careful, as fixing the badness of a broken package is much more difficult. Keep it to really core packages that you expect to be widely shared across many, many projects -- not just the half dozen you're stressing out about right now for your job.
Alternately, you may share one sandbox between the builds of many packages; simply set the CABAL_SANDBOX_CONFIG variable to an absolute path pointing to the appropriate cabal.sandbox.config file. This is significantly safer, and much more flexible, as you can choose how widely your installed packages are shared (and in bad cases, simply nuke the sandbox and start over).
Here is something you can try - copy (or symlink) the files from ~/.ghc/{arch-os-ghc-version}/package.conf.d to the sandbox's {arch-os-ghc-version}-packages.conf.d directory.
There is a question about the package.cache file. The following procedure seems to be a safe way to proceed:
Start with an empty sandbox
Copy the package.conf.d files from ~/.ghc to the sandbox (including package.cache)
Add packages to the sandbox via cabal install --only-dependencies
I don't know if the package.cache file is required or if there is a way to rebuild it.
One disadvantage is that cabal install --only-deps seems to reinstall broken packages in the sandbox even if they are not required by your application. Maybe there is work-around for this.
It's rather nice that ghc-pkg check will list broken packages, and why they are broken. But as far as I know, there is no automated way to take care of those broken packages. What is the recommended way to deal with broken packages? (Preferably not reinstall GHC)
Hopefully, you have been wise enough to not break any in your global package database. Breakage there can easily mean a reinstallation of GHC is necessary. So, let us assume that the breakage is restricted to the user package db (except possibly a package or two in the global shadowed by user packages). If only few packages are broken, you can fix your setup by unregistering the offending packages,
$ ghc-pkg unregister --user borken
that will often complain that unregistering borken will break other packages. Whether you try to unregister those first or unregister borken immediately with --force and deal with the newly broken afterwards is mostly a matter of choice. Make sure that you only unregister packages from the user db. If things aren't too grim, after unregistering a handful of packages, ghc-pkg check will report no more broken packages.
If, on the other hand, a large proportion of packages is broken, it will probably be easier to completely wipe the user db, $ rm -rf ~/.ghc/ghc-version/package.conf.d or the equivalent on other OSs.
Either way, you will have lost packages you still want to use, so you will try to reinstall them without breaking anything anew. Run
$ cabal install world --dry-run
that will try to produce a consistent install plan for all the packages you installed with cabal-install. If it fails to do so, it will print out the reasons, you may then be able to fix the issues by adding constraints to the packages listed in the world file (~/.cabal/world) - for example, although I have no broken packages (according to ghc/ghc-pkg), cabal install world --dry-run told me it could not configure vector-algorithms-0.5.2, which depends on vector >= 0.6 && < 0.8 (I have vector-0.7.1 installed). The reason is that hmatrix-0.12.0.1 requires vector >= 0.8. Replacing the -any "constraint" on hmatrix by a "< 0.12" in the world file produced a clean install-plan.
So, after a bit of fiddling with constraints in the world file, you will get an install plan from cabal. Check whether that would reinstall any packages you already have (installing a newer version is probably okay, reinstalling the same version means trouble). If you're happy with cabal's install-plan, cabal install world and brew a nice pot of tea while GHC is busy. Run ghc-pkg check once more, to verify all is in order.
A piece of generally good advice: If you don't know what installing a package entails, always use --dry-run first.
If you broke your global package database by doing global installs with cabal, the strategy of unregistering offenders may work, but it may also irrevocably break your ghc, that depends on what is broken in which way. If you broke your global db by installing packages from your OS distro, install a fresh GHC, curse the distro-packagers, and try to help them prevent further such events.
A cabal repair command would be very nice, but for the time being, repairing a broken setup is unfortunately much more work.
For some time I've relied on this ghc-pkg-clean script. It removes all broken packages and I reinstall them as needed. For more serious breakage, I use the ghc-pkg-reset script.
Today, though, I found ghc-pkg-autofix, which automates this further - broken packages become unbroken. I don't know what it does, YMMV.