I'm developing a Haskell program using Cabal in a nix-shell. I would like to have as many dependencies of the build installed by Nix as possible.
The simple.cabal file seems fairly standard (it was initially produced by stack). The full contents are here: https://pastebin.com/3wd8j0pp The only place alex and happy appear in the .cabal file are in the build-tool-depends sections.
I tried to keep the shell.nix file as simple as possible (but I'm also inexperienced in Nix):
{ pkgs ? import <nixpkgs> {}}:
let
ghc = pkgs.haskellPackages.ghcWithPackages (p:[
p.array
p.base
p.bound
p.containers
p.deriving-compat
p.haskeline
p.logict
p.mtl
p.text
p.unification-fd
p.alex
p.happy
p.BNFC
p.cabal-install
]);
alex = pkgs.haskellPackages.alex;
happy = pkgs.haskellPackages.happy;
in
pkgs.mkShell {
buildInputs = [ ghc pkgs.haskellPackages.alex pkgs.haskellPackages.happy pkgs.pkg-config];
buildTools = [ pkgs.haskellPackages.alex pkgs.haskellPackages.happy];
buildToolDepends = [pkgs.haskellPackages.alex pkgs.haskellPackages.happy];
ALEX="${alex}/bin/alex";
HAPPY="${happy}/bin/happy";
}
I saved environment variables to reference the locations of alex and happy.
Finally, I tried to tell cabal where to find alex and happy by specifying them in the extra-prog-path section of a cabal.project file. However, that didn't work, so I tried hard-coding in their location
packages: simple.cabal
extra-prog-path:
/nix/store/PATH-TO-ALEX/bin/alex
/nix/store/PATH-TO-HAPPY/bin/happy
Finally, on building to enter the nix-shell, nix-shell --pure shell.nix and then cabal build
The cabal tool build finds all the ghc library packages installed with ghcWithPackages correctly -- it requires none of them to be built. However, it seems to not know about alex nor happy. The output looks like this.
In order, the following will be built (use -v for more details):
- alex-3.2.6 (exe:alex) (requires build)
- happy-1.20.0 (exe:happy) (requires build)
I can also confirm that the version of alex and happy in the nix store are alex-3.2.6 and happy-1.20.0.
On the other hand, cabal v1-build does pick up alex and happy as installed. cabal v1-build
Resolving dependencies...
Configuring simple-0.1.0.0...
Preprocessing library for simple-0.1.0.0..
Building library for simple-0.1.0.0..
and the build completes successfully only compiling the source files in the local package.
It seems like one should favor v2-build or new-build. How can I get cabal to know where to find and also to use the alex and happy versions installed by Nix, or alternatively how to tell Nix what to do so that v2-build or new-build can find alex and happy?
Related
I installed the timezone-series Haskell module using cabal install timezone-series-0.1.5.1.
I then defined a module named Main.hs that starts with:
import Data.Time.LocalTime.TimeZone.Series -- from timezone-series-0.1.5.1
when I run ghc Main.hs, GHC throws the following error:
/home/ubuntu/Main.hs:2:1: error:
Failed to load interface for ‘Data.Time.LocalTime.TimeZone.Olson’
I tried explicitly including the cabal directory in GHC's search path using:
ghc -i/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/ Main.hs
This resulted in the correct path being searched, but GHC is only looking for files ending in the suffix ".hs":
Locations searched:
...
/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/Data/Time/LocalTime/TimeZone/Series.hs
/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/Data/Time/LocalTime/TimeZone/Series.lhs
/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/Data/Time/LocalTime/TimeZone/Series.hsig
/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/Data/Time/LocalTime/TimeZone/Series.lhsig
Cabal installed interface files instead however:
/home/ubuntu/.cabal/lib/x86_64-linux-ghc-8.0.2/timezone-olson-0.2.0-KqRNJj3zomR7zz2Yx6P5Oq/Data/Time/LocalTime/TimeZone/Olson.hi
From line 318 of GHC's source code it looks like GHC ignores "*.hi" files unless it is called in single-shot mode (with the -c flag). Is this correct? (See: https://github.com/ghc/ghc/blob/67a5a91ef5e61f3b3c84481d8a396ed48cd5d96e/compiler/GHC/Unit/Finder.hs)
How can I get GHC to import this module?
An help will be greatly appreciated!
My suggested ways of installing packages in order of my preference:
Make a cabal package and add timezone-series you want to install to the build-depends field as described in the cabal manual.
Use the experimental cabal-env tool to basically automate the process of point 3 below, but then with the global environment. This makes a new build-plan every time you install a new package, so it is like removing the package environment and building it again with all the old packages and the new package added to it. You can add specific constraints like this: cabal-env "timezone-series == 0.1.5.1".
Install a package into local package environment with cabal --package-env . --lib timezone-series. You can add as many packages as you want after the --lib option to install more than one package. If you later want to use a different set of packages simply remove the .ghc.environment.* file that is generated and rerun the installation with a new set of packages. GHC will automatically use these package environment files that are in the current or parent directories. You can specify specific constraints with the --constraint option like this: --constraint "timezone-series == 0.1.5.1".
Use cabal install --lib timezone-series to install it directly into the global environment (~/.ghc/x86_64-linux-8.0.2/environments/default), this will fail if a conflicting package was installed earlier. When you run into errors you can remove that package environment and try again.
Finally, I want to note that GHC 8.0.2 is quite old, so I would advise you to upgrade if you don't have a specific reason for using that version.
I recently upgraded to Cabal 3.2 (and GHC 8.10) and I am running into some major issues that make some of my project non-buildable anymore...
Thorough description of the problem
Here is a minimal (not) working configuration that fails every time:
I start off with a clean Cabal configuration (by deleting ~/.cabal); the reason for that will appear later in the post. I run cabal update to recreate the .cabal directory and to ensure Cabal is working.
I create a project (let's call it test1) using cabal init. This is a library project with one exposed module (conveniently named Test1) that exports some dummy function foo. I run cabal build, then cabal install --lib; everything is running smooth, so far so good.
Just to be sure, I leave the project directory and fire up GHCi. I type in :m Test1 to load the module I created earlier, and it works! I can type in foo ... and see my function executed. Also, I list the content of ~/.cabal/store/ghc-8.10.xxx and see that the test1-xxx directory is there.
I then create a new project, test2, still using cabal init. This time, I configure it to be an executable, and I add test1 as a dependency (using the build-depends field). But this time when I run cabal build, I run into some issue:
~/projects/haskell/test2> cabal build
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: test2-0.1.0.0 (user goal)
[__1] unknown package: test1 (dependency of test2)
[__1] fail (backjumping, conflict set: test1, test2)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test2, test1
It seems to me like package test1 cannot be found, however I can access it from GHCi (and GHC for that matters) and it is present in ~/.cabal/store...
But unfortunately there is more.
I create a third project, test3. This is a library, and it depends on nothing else than base (so in particular it does not depend on test1). The lib exposes one module, Test3, with one function exported, bar. I run cabal build, no problem here. But when I want to install test3 with cabal install --lib I run into some errors:
~/projects/haskell/test3> cabal install --lib
Wrote tarball sdist to
/home/<user>/projects/haskell/test3/dist-newstyle/sdist/test3-0.1.0.0.tar.gz
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] unknown package: test1 (user goal)
[__0] fail (backjumping, conflict set: test1)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test1
It seems that it cannot find test1, although it has been installed correctly; may be this is a remnant of the failed build of test2 though...
Just to be sure, I fire up GHCi and type in :m Test3, but GHCi tells me that it cannot find module Test3 (and even suggests this is a typo and I was meaning Test1), showing that test3 indeed did not get installed, although it got successfully built...
Okay there is one more quirk to this whole situation: I create once again a new project with cabal init, called test4, which is an executable that (again) depends on nothing else than base. I keep the default Main.hs (that just prints "Hello, Haskell!"). I run cabal build: no problem. Then I run cabal install and... No problem either? I run test4 in a random location and it fires up the executable, printing "Hello, Haskell!" in the terminal...
And there is one last thing: I go to some random location and I run cabal install xxx --lib where xxx is a library package available on Hackage (for example xml) and:
~> cabal install xml --lib
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] unknown package: test1 (user goal)
[__0] fail (backjumping, conflict set: test1)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test1
This is the reason why I need to nuke .cabal regularly... Right now I seem to be in some kind of stale state where I cannot install any library anymore.
Technical configuration and notes
I am running Cabal 3.2.0.0 and GHC 8.10.0.20200123. I installed them from the hvr/ghc PPA, and I made sure there are no other versions of those tools anywhere on my computer.
Just as a note, I am running Ubuntu 18.04.4 LTS (with XFCE so XUbuntu to be exact). Everything else (seem to be like it) is up to date.
Last thing, regarding the *.cabal files I use for building, they are pretty much the ones generated by cabal init, except I switch executable xxx for library in the case of libraries, and I simply add a exposed-modules field for exposing modules for the libraries (so Test1 for test1 and Test3 for test3 respectively). I also use build-depends in test2 to make the project depend on test1. Apart from that, they are pretty much left untouched.
Notes and thoughts
I must confess that I am new to Cabal 3; until last week I was using Cabal 1 (because I never bothered to update it; yes I know this is bad). With Cabal 1 I did not have any problem whatsoever, and I was perfectly able to install a package from local sources and depend on it in other projects...
I feel like I am doing something wrong; maybe am I not using the correct Cabal commands? I saw somewhere something about cabal new-build and cabal new-install but it does not seem to do anything more than cabal build and cabal install, at least in my case. I also wanted to investigate sandboxes but it seems that has disappeared since version 2 of Cabal.
There is also a slight possibility this is a Cabal bug, but I don't find any relevant issue on the bug tracker that may be related to my problem...
What do you think about this? What am I doing wrong? Do you see any alternative or possible fix?
Thanks a lot!
GHC environment files
A GHC installation comes with a certain number of packages out-of-the box. base is one of them but there are others, for example text. If you install GHC alone (no cabal or stack) and open ghci, it should let you import Data.Text without problems.
What if you want GHC or ghci to be aware of other compiled packages present in your filesystem? You can point GHC to additional package databases using command-line flags, but there's also the concept of package environment files.
Environments are plain text files that contain a list of package-related GHC flags. There might be a global environment at ~/.ghc/$ARCH-$OS-$GHCVER/environments/default, and there might also exist local environments which only affect GHC and ghci commands invoked inside the same folder. The exact rules for search are described in the GHC User Guide.
What does cabal install --lib actually do?
By default, it modifies the global environment file, so that GHC and ghci can now find that library. That's why point 3) worked. The actual compiled binaries of the library still reside in the cabal store though.
We can also create local environment files. For example cabal install sop-core --lib --package-env . will create the environment file .ghc.environment.xxx in the current folder, and the library will be available to ghc and ghci when they are invoked there.
Why isn't test1 available for test2?
Modern cabal makes a distinction between local packages and external packages.
local packages is the set of packages you are developing together in a project, being edited, recompiled and changed repeatedly. They are built "inplace" and not seen outside the project. They can depend on each other.
external packages are dependencies from build-depends: whose source code is downloaded from a package repository and which, when compiled, are put in the cabal store so that other Cabal projects might make use of them without re-compiling.
The list of local packages and other project-level configuration details are specified in a cabal.project file. But you don't need one if you work on a single isolated package; the default list of packages is simply ./*.cabal.
cabal wants to completely control the build environment of local packages, and will ignore the global environment file. In your case, you'll have to make test1 and test2 local packages in the same project (likely the best option) or publish test1 and treat it as an external package.
Note that "cabal project" is a concept relevant only during development. Packages are published independently, there are no "projects" in Hackage or other repositories, just packages.
What if I want to treat test1 as external without publishing it to Hackage?
You will have to set up a local package repository, basically a non-public Hackage.
You can tell Cabal about additional package repositories in the Cabal configuration file, that is, the file that configures cabal itself. Its location is given in the last line of cabal --help.
But how to set up the repository? The hackage-repo-tool can help with that.
Why did test3 fail? Why did further library installs fail?
That's weird, I have no idea why that happens. Did you by perchance delete the ~/.cabal folder between steps 3) and 5) ? What happens if you delete the global GHC environment file and try again?
I put this in a file (see bottom) and got the following error when loading it in ghci (i.e. start ghci then :l file):
7:13: error:
• Couldn't match expected type ‘network-uri-2.6.1.0#network-uri-2.6.1.0-7BN1tbB3iHQ2XgvmqLAYph:Network.URI.URI’
with actual type ‘Network.URI.URI’
NB: ‘Network.URI.URI’
is defined in ‘Network.URI’
in package ‘network-uri-2.6.1.0#network-uri-2.6.1.0-80FpvaNUTSDFCPv0sSze40’
‘network-uri-2.6.1.0#network-uri-2.6.1.0-7BN1tbB3iHQ2XgvmqLAYph:Network.URI.URI’
is defined in ‘Network.URI’
in package ‘network-uri-2.6.1.0#network-uri-2.6.1.0-7BN1tbB3iHQ2XgvmqLAYph’
Have I somehow installed two slightly different versions of network-uri, or what is this? How do I resolve it?
I've hardly used cabal but tried cabal install --reinstall network-uri, went fine but problem persists.
File contents:
import Network.HTTP
import Network.URI (parseURI)
import Data.Maybe (fromJust)
myRequestURL = "http://www.virginia.edu/cgi-local/ldapweb"
myRequest :: String -> Request_String
myRequest query = Request {
rqURI = fromJust $ parseURI myRequestURL
, rqMethod = POST
, rqHeaders = [ mkHeader HdrContentType "text/html"
, mkHeader HdrContentLength $ show $ length body ]
, rqBody = body
}
where body = "whitepages=" ++ query
main :: IO ()
main = do
response <- simpleHTTP $ myRequest "poon"
putStrLn ""
EDIT Re-installed haskell-platform. After this, ghc-pkg list reports all packages to be under (?) /usr/lib/ghc/package.conf.d, and the program I tried to copy loads just fine.
EDIT 2 After running cabal install some-pkg, ghc-pkg list reports that the newly installed stuff is in another directory. Indeed, ~/.cabal/config specifies install dirs for user, as well as install dirs for global. How is all this supposed to be managed properly?
GHC understands the concept of package databases. Cabal manages multiple package databases. There's a system package database, and a user package database. When you compile something with Cabal V1, it uses the union of these package databases. This is a bit of a time bomb, because, as you see, GHC does not like it when you have multiple of the same package. When you issued cabal install network-uri, cabal decided to install one copy of that package into your user database, though you already had one in the system database. When you tried to use ghc, it picked up on the two copies and got confused. Because cabal does not support removing packages, your Haskell installation was pretty much borked, and you reinstalled. I believe you could have done a more conservative rm ~/.ghc, or, had you been really adventurous, used the low-level ghc-pkg command to edit the package database.
Now that you've reinstalled the Platform, I would heavily recommend never using cabal install again. It's old, it's broken, cabal itself tells you not to use it. If you're using the Platform, I think it has stack in it, which is a different build manager, based on Cabal, which is not prone to this kind of breakage. Or, you can just stick to the V2 series of cabal commands, which is what I prefer.
Cabal V2 does not directly support your usecase, where you install a package globally and compile a program against it. This is because doing so is fundamentally broken: global package databases really just don't work for GHC. What Cabal V2 does is install packages into your home directory, ~/.cabal/store. You then have to explicitly tell it which packages you want, and it will construct a package database that contains each required package, without any wonkiness. You can get a REPL with a certain set of packages by issuing (this will take you to a temporary directory; you may need to :cd yourself back.)
cabal v2-repl -b network-uri -b package2 -b etc
If you want to actually compile files with ghc with a set packages, you have to actually define a package with a .cabal file. Here's a stub .cabal file:
name: temporary-pkg
version: 0.0.0.0
build-type: Simple
cabal-version: >=2.0
executable main -- if there is no Main, say "library" instead of "executable <name>"
default-language: Haskell2010
main-is: Main.hs -- only valid in executable stanzas
hs-source-dirs: src -- source files go here
build-depends: base, network-uri, package2, etc -- packages go here
Create and enter a directory, place that in temporary-pkg.cabal, place your sources in src/, and you can issue stuff like cabal v2-build and cabal v2-repl and cabal v2-exec main, etc. cabal will go off and install dependencies from Hackage into ~/.cabal/store as needed. cabal v2-install is therefore not that useful here. In general, v2-installing libraries is unnecessary, but v2-installing packages with executables is useful. Again, for hygiene reasons, you have to list all the packages you are importing from, but you don't need to list their dependencies or anything insane like that.
Cabal V2 still doesn't support uninstalling packages, though the point is that should not be necessary. However, if you find ~/.cabal/store getting too large for your liking, you can just nuke it, and should be built back up as packages are requested.
I have a package named commands. I want to install it into its own sandbox e.g. .cabal-sandbox/x86_64-osx-ghc-7.8.3-packages.conf.d/commands-0.0.0-f3f84f48f42ac74a69ee5fd73512bfd0.conf. currently, there is just one .hi interface file for one module Commands, I don't know how it got there.
I tried cabal install commands, by the logic of "that's how the other packages got there I think", but it fails with unknown package.
I also tried stuff with ghc-pkg like ghc-pkg update commands -f .cabal-sandbox/x86_64-osx-ghc-7.8.3-packages.conf.d but I'm not using them right. ideally, I'd like to do this with cabal.
the last thing I tried was ghc -idist/build/, but it complained about the package names in the interface files being different, command versus main ("... differs from name found in the interface file ..."). and if I faked the executable's package with ghc -package-name commands-0.0.0, the linker complained that it couldn't find the symbol _ZCMain_main_closure, because every executable needs the function main in the module Main in the package main.
I'm sure there's a better way of doing this.
I followed online examples for my cabal file:
$ cat commands.cabal
name: commands
library
exposed-modules: Commands.Types, Commands.Bits
...
the minimal failing code example is just:
$ cat Main.hs
import Commands.Types
main = return ()
in the root project directory.
Context: I need to build my executable with make (not cabal) because it links to foreign code (Objective-C via language-c-inline). my makefile: https://github.com/sboosali/Haskell-DragonNaturallySpeaking/blob/master/Makefile). thus, I have to compile a script explicitly. I don't know how to compile the executable with cabal, but I want cabal to build and test and manage my library.
By putting my package into the sandbox, I will be able to import its modules from the script, by compiling with cabal exec -- ghc. I will also be able to include the script with extra-source-files at least, and know it will work.
Here's what I'd try:
First unregister any previous commands library. Try
ghc-pkg --global unregister commands
Install new commands to your sandbox.
From your sandbox directory, try this:
cabal --enable-shared --disable-documentation --prefix=./ install /path/to/your/library/source
Note the prefix specification.
I nuked it (rm -fr .cabal-sandbox/), reinstalled everything (cabal install --only-dependencies), added itself as a source (cabal sandbox add-source .), installed it (cabal install command). and then make worked. idk...
I’m trying to write a project with the Hakyll library. In order to avoid messing up with my system, I’d install it in a cabal sandbox in the same folder where my Hakyll project lives.
Being more or less a beginner, I’m still struggling with getting the pest practices right. A simple approach would be to just do
$ cabal sandbox init
$ cabal install hakyll
$ cabal exec ghc -- --make site.hs
where the last line compiles my Hakyll generator using the libraries in the sandbox. The obvious disadvantage is that this is not reproducible. The main version of Hakyll could have change when I try to run it again from a clean checkout.
Another approach would be to write a proper project.cabal file (For example like this: chromaticleaves.cabal) and then do cabal install or cabal run.
However, I feel that this may be a bit too much information. As I do not intend to publish this project any more than needed, I’m not really convinced I need to put a project name and version number in there. (For example, in a Ruby Gemfile, I would also only specify the libraries and nothing else unless I wanted to publish a gem myself.)
So, eventually I figured that with a file like
$ cat project.cabal
cabal-version: >= 1.2
library
build-depends: base >=4.6
, containers
, process
, hakyll >=4.5
, pandoc
, pandoc-types
I can type
$ cabal sandbox init
$ cabal install --only-dependencies
$ cabal exec ghc -- --make site.hs
and it seems to download all dependencies and is able to compile the file.
Is this a reasonable approach or is the best practice really to give a full specification with name, version and executable sections in the cabal file?
Edit: Apparently, my approach does not let me do cabal repl. So either there exists a completely different way of doing it or it seems I have to go with a fuller specification.
I use your first approach myself for my Hakyll-based webpage. You don't need to create a .cabal file to pin the Hakyll version, you only need to add the following line to cabal.config:
constraints: hakyll == 4.5
I think that cabal repl will work with this approach, but you will need to load site.hs manually (:l site.hs). Or you can use cabal exec ghci -- site.hs.