Nix & Haskell - using default.nix from cabal2nix and a generic shell.nix - haskell

From a pretty basic cabal file
cabal2nix ./. > default.nix
and then a shell.nix of
let
pkgs = import <nixpkgs> {};
haskellPackages = pkgs.haskellPackages_ghc784.override {
extension = self: super: {
thispackage = self.callPackage ./default.nix {};
};
};
in pkgs.myEnvFun {
name = haskellPackages.thispackage.name;
buildInputs = [
(haskellPackages.ghcWithPackages (hs: ([
hs.cabalInstall
] ++ hs.thispackage.propagatedNativeBuildInputs)))
];
}
When in the nix-shell and running cabal configure it complains of missing packages such as text.
If I put the text package explicitly into the shell.nix such as
let
pkgs = import <nixpkgs> {};
haskellPackages = pkgs.haskellPackages_ghc784.override {
extension = self: super: {
thispackage = self.callPackage ./default.nix {};
};
};
in pkgs.myEnvFun {
name = haskellPackages.thispackage.name;
buildInputs = [
(haskellPackages.ghcWithPackages (hs: ([
hs.cabalInstall
hs.text
] ++ hs.thispackage.propagatedNativeBuildInputs)))
];
}
The cabal configure is fine, but I would expect hs.thispackage.propagatedNativeBuildInputs to be supplying these packages.
The very basic haskell project can be seen at
https://github.com/fatlazycat/haskell-nix-helloworld
Am I wrong in assuming you can work in this way ?
Thanks

The propagatedNativeBuildInputs attribute is used by Haskell libraries to
propagate their build inputs down to other builds that depend on them. Your
package, however, is not a library --- it's an executable ---, so there's no
need to propagate the build inputs and thus propagatedNativeBuildInputs is
empty. Instead, you'll find the information you need in
hs.thispackage.extraBuildInputs.
Generally speaking, the definition of these kind of nix-shell environments
has become a lot easier in the release-15.09 branch (or nixos-unstable),
though. Simply run cabal2nix --shell . >shell.nix and you get a shell.nix
file that you can use for building with nix-build shell.nix as well as for
entering an interactive development environment with nix-shell.
http://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure has
a lot more information about the subject.

Related

How to get use `callCabal2nix` to supply package list to 'ghcWithPackages'?

I have a list of ~46 dependencies.
I produce a shell using among other things a call to:
server = pkgs.haskellPackages.callCabal2nix "server" ./server.cabal { };
within an override of pkgs.haskellPackages that goes into pkgs.myHaskellPackages.
The shell works fine. However, I have another nix for building a docker and this requires a build input that comes from:
ghc = pkgs.myHaskellPackages.ghcWithPackages (p: [ longlistofdependencies ]);
How do I specify the long list of dependencies without having to write them out manually, defeating the purpose of using callCabal2nix in the first place.
let
bootstrap = import <nixpkgs> { };
nixpkgs = builtins.fromJSON (builtins.readFile ./nixpkgs-unstable.json);
src = bootstrap.fetchFromGitHub {
owner = "NixOS";
repo = "nixpkgs";
inherit (nixpkgs) rev sha256;
};
config = {
allowBroken = true;
packageOverrides = pkgs: rec {
servantSrc = pkgs.fetchFromGitHub {
owner = "haskell-servant";
repo = "servant";
rev = "73c87bc2bc0685649f2337b06ab4fdc66c4ce1dd";
sha256 = "0sw4mrncmfna30cyxrvinc1krqhfxn5dcc1ggzqfy39s0yl9q98r";
#sha256 = "0000000000000000000000000000000000000000000000000000";
} + "/servant";
myHaskellPackages = pkgs.haskellPackages.override {
overrides = haskellPackagesNew: haskellPackagesOld: rec {
server =
haskellPackagesNew.callCabal2nix "server" ./server.cabal { };
servant =
haskellPackagesNew.callCabal2nix "servant" servantSrc {};
# several other overridden packages
};
};
};
};
pkgs = import src { inherit config; };
devShell = pkgs.myHaskellPackages.shellFor {
packages = p: [
p.server
];
buildInputs = with pkgs.haskellPackages; [
hlint
brittany
haskell-language-server
];
};
# Every time a dependency changes in the cabal file, this has to be edited as well.
ghc = pkgs.myHaskellPackages.ghcWithPackages (p: [
p.aeson p.aeson-qq p.base p.blaze-html p.bytestring p.cache p.containers
p.contravariant-extras p.criterion p.either p.hashable p.hasql
p.hasql-migration p.hasql-pool p.hasql-th p.hasql-transaction p.hspec
p.http-client p.http-conduit p.http-types p.immortal-queue p.lens p.linear
p.monad-logger p.mtl p.pqueue p.pretty-simple p.QuickCheck p.raw-strings-qq
p.servant-blaze p.servant-docs p.servant-server
p.string-interpolate p.text p.text-format p.time p.tuple p.unordered-containers
p.utf8-string p.vector p.vector-split p.wai p.wai-cors p.warp p.xlsx p.yaml
]);
in {
inherit devShell;
inherit ghc;
inherit pkgs;
}
The ghc output is used in docker.nix as a build input:
let
inherit (import ./pinned-nixpkgs.nix) ghc pkgs;
pricing-server =
pkgs.stdenv.mkDerivation {
name = "my-server";
pname = "my-server";
version = "1.1.0";
src = ./.;
buildPhase = ''
ghc -O2 --make -outputdir ./tmp2 Main.hs
'';
installPhase = ''
mkdir -p $out/bin
cp Main $out/bin
cp -r ./sql $out/bin
'';
buildInputs = [ ghc ];
};
in
dockerTools.buildImage {
# irrelevant to the question.
}
A possible solution is pkgs.myHaskellPackages.server.getBuildInputs.haskellBuildInputs, or pkgs.myHaskellPackages.server.getCabalDeps.libraryHaskellDepends.
You can explore these attributes, or any expression, with nix repl. You may have to expose some values from your let bindings though. In this case I just browsed through haskellPackages.warp in nix repl <nixpkgs>.
I also noticed you use rec in an overlay. This will work for you until it doesn't. I'd recommend to remove rec to avoid accessing attributes in a third way and use the more standard haskellPackagesNew.servant instead.

How do I override a Haskell package in nixpkgs via Nix?

Essentially I'm using this:
default.nix
{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc864" }:
nixpkgs.pkgs.haskell.packages.${compiler}.callPackage ./gitchapter.nix { }
gitchapter.nix
{ mkDerivation, base, directory, extra, filepath, foldl, hpack
, HUnit, mtl, optparse-applicative, pandoc-include-code, parsec
, pretty-simple, process, QuickCheck, rainbow, regex-pcre
, regex-posix, safe, stdenv, string-conversions, system-filepath
, template-haskell, text, transformers, turtle, unix
, unordered-containers
}:
mkDerivation {
pname = "gitchapter";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
libraryToolDepends = [ hpack ];
executableHaskellDepends = [
base directory extra filepath foldl HUnit mtl optparse-applicative
pandoc-include-code parsec pretty-simple process QuickCheck rainbow
regex-pcre regex-posix safe string-conversions system-filepath
template-haskell text transformers turtle unix unordered-containers
];
preConfigure = "hpack";
license = stdenv.lib.licenses.bsd3;
}
However there is an issue with the pandoc-include-code failing to build, which seems to have since been fixed in the git repository. How can I override the package either to point to git repository or a local directory?
Would I follow the instructions at https://nixos.org/nixos/nix-pills/nixpkgs-overriding-packages.html or would this work differently due to using the nixpkgs.pkgs.haskell.packages.${compiler}.callPackage function?
Edit:
Thanks to #sara's answer I now have:
{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc864" } :
let
gitchapter = nixpkgs.pkgs.haskell.packages.${compiler}.callCabal2nix "gitchaper" (./.) {};
zzzzz = nixpkgs.pkgs.haskell.lib.overrideCabal gitchapter;
in
nixpkgs.pkgs.haskell.packages.${compiler}.callPackage (zzzzz) { }
So it I suppose now it's a matter of determining how to override that dependency now.
Consider using callCabal2nix from haskell.packages.${compiler}!
It will go through your .cabal file and generate a nix expression for a derivation from that (thus making gitchapter.nix unnecessary), which you then can override using the overrideCabal function in haskell.lib in a similar fashion to normal derivation overriding. You can then fetch the updated pandoc derivation from git, and add it as a buildInput in your override expression.
Example of local path referenced:
{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc864" } :
let
myHaskellPackages = nixpkgs.pkgs.haskell.packages.${compiler}.override {
overrides = self: super: rec {
pandoc-include-code = self.callCabal2nix "pandoc-include-code" (./pandoc-include-code) {};
};
};
in
myHaskellPackages.callCabal2nix "gitchaper" (./.) {}
Other alternatives:
Git repo:
pandoc-include-code = self.callCabal2nix "pandoc-include-code" (builtins.fetchGit {
url = "git#github.com:owickstrom/pandoc-include-code.git";
rev = "3afe94299b3a473fda0c62fdfd318435117751dd";
})
{};
Hackage (via tar archive) example:
prettyprinter = self.callCabal2nix "prettyprinter" (builtins.fetchTarball {
url = "https://hackage.haskell.org/package/prettyprinter-1.7.0/prettyprinter-1.7.0.tar.gz";
}) {};

How can I make this Haskell development environment in Nix?

I'm trying to make a Haskell development environment for a web project that just has the dependencies hakyll, blaze, and clay. Only, clay appears to fail to build, complaining that Setup: Encountered missing dependencies:
hspec >=2.2.0 && <2.6, hspec-discover >=2.2.0 && <2.6, and I can't seem to get past this.
I tried running cabal2nix . --shell > shell.nix on this cabal file:
name: open-editions
version: 0.1.0.0
build-type: Simple
cabal-version: >= 1.10
executable site
main-is: site.hs
build-depends: base == 4.*
, blaze-html
, clay
, hakyll == 4.12.*
ghc-options: -threaded
default-language: Haskell2010
But I'm running into the missing dependencies problem. Any ideas?
Edit: here's the shell.nix that I'm generating from the above, using cabal2nix:
{ nixpkgs ? import <nixpkgs> {}, compiler ? "default", doBenchmark ? false }:
let
inherit (nixpkgs) pkgs;
f = { mkDerivation, base, blaze-html, clay, hakyll, stdenv }:
mkDerivation {
pname = "open-editions";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
executableHaskellDepends = [ base blaze-html clay hakyll ];
license = "unknown";
hydraPlatforms = stdenv.lib.platforms.none;
};
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;
drv = variant (haskellPackages.callPackage f {});
in
if pkgs.lib.inNixShell then drv.env else drv
And here's how I've been trying to modify it, so that it overrides Clay:
{ haskellLib, super, nixpkgs ? import <nixpkgs> {}, compiler ? "default", doBenchmark ? false }:
let
inherit (nixpkgs) pkgs;
clay = haskellLib.doJailbreak super.clay;
f = { mkDerivation, base, blaze-html, clay, hakyll, stdenv }:
mkDerivation {
pname = "open-editions";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
executableHaskellDepends = [ base blaze-html clay hakyll ];
license = "unknown";
hydraPlatforms = stdenv.lib.platforms.none;
};
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;
drv = variant (haskellPackages.callPackage f {});
in
if pkgs.lib.inNixShell then drv.env else drv
But I'm obviously doing something wrong, since I'm getting: error: cannot auto-call a function that has an argument without a default value ('haskellLib').
This is typically caused by a package pinning an old version of dependency, not available in nixpkgs. Peter is correct that clay needs to bump the hspec version.
However, you can try to temporarily fix it by jailbreaking the package (removing the version bounds) in nixpkgs, see:
https://github.com/NixOS/nixpkgs/blob/cc98350d55522ebb2b7d35db32bc7c2fc5b8b273/pkgs/development/haskell-modules/configuration-common.nix#L1088-L1089
In case the package does not build with newer version of hspec, you could try to add the old version to the package set and override the package to use it:
https://github.com/NixOS/nixpkgs/blob/cc98350d55522ebb2b7d35db32bc7c2fc5b8b273/pkgs/development/haskell-modules/configuration-common.nix#L1170
But that would require require re-generating hackage-packages.nix so I do not recommend it.
It is better idea to patch the package to support newer version of the dependency and add the patch to overrides:
https://github.com/NixOS/nixpkgs/blob/cc98350d55522ebb2b7d35db32bc7c2fc5b8b273/pkgs/development/haskell-modules/configuration-common.nix#L1187-L1190
Also do not forget to open an issue upstream – I see you already did – or a pull request.
It's probably not really relevant anymore, but I encountered the same problem with clay.
I use an overlay to modify the (Haskell) package set. It looks like this (called overlays.nix):
let
customHaskellPackages = self: super: {
haskellPackages = super.haskellPackages.override {
overrides = hself: hsuper: {
# Since clay is not really maintained any longer, the upper
# version bounds have gone stale.
# Everything works fine; we just need to free it from its
# too-tight chains.
clay =
super.haskell.lib.doJailbreak
hsuper.clay;
# Other packages would go here, too
some-foo-package =
super.haskell.lib.dontCheck
(hself.callPackage ./some-foo-package.nix {});
};
};
};
in
[ customHaskellPackages ]
My default.nix is created by cabal2nix . > default.nix from my cabal file, which in turn just specifies clay as a dependency.
In a (very simplified) release.nix file you can use the overlay as follows:
let
overlays = import ./nix/overlays.nix;
# I might have needed this for clay, but it might have been sth else
config = { allowBroken = true; };
pkgs = import <nixpkgs> { inherit config; inherit overlays; };
in
pkgs.haskellPackage.myApplication
# Or use haskellPackages.callPackage ./something.nix {} here if you didn't
# add your project to haskellPackages in the overlay
After that, nix-build release.nix works just fine.
I don't use the --shell option to create a shell.nix file, but create it myself and import the release.nix file. Then I enrich my shell environment by adding in a couple of build tools. This works as expected as well.

How to write a shell.nix file which combines other nix-shell environments?

I want to write a shell.nix file that will provide me a development environment which includes developer tools + haskell dependencies. I would like to keep my nix expressions separate - for now that means one file containing information about my editor, and one file containing information about my haskell project.
I have tried to set up a really basic environment, but can't get it to work. I have two files which I can create a nix-shell from already:
# my-haskell-shell.nix - generated by cabal2nix --shell
{ nixpkgs ? import <nixpkgs> {}, compiler ? "default", doBenchmark ? false }:
let
inherit (nixpkgs) pkgs;
f = { mkDerivation, base, cabal-install, hpack, stdenv }:
mkDerivation {
pname = "hello-world";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
libraryToolDepends = [ hpack ];
executableHaskellDepends = [ base ];
executableToolDepends = [ cabal-install hpack ];
preConfigure = "hpack";
license = stdenv.lib.licenses.mit;
};
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;
drv = variant (haskellPackages.callPackage f {});
in
if pkgs.lib.inNixShell then drv.env else drv
# my-neovim-shell.nix
with import <nixpkgs> {};
let
my-neovim = neovim.override { vimAlias = false; };
in
stdenv.mkDerivation {
name = "neovim-dev-env";
buildInputs = [
my-neovim
];
}
Both of these produce useful shell environments! (That is, nix-shell my-haskell-env.nix and nix-shell my-neovim-env.nix give me cabal and neovim respectively, but not both).
My attempt at producing a shell.nix which will provide an environment with both neovim and cabal available is this:
# shell.nix
with import <nixpkgs> {};
let
my-neovim-shell-env = import ./my-neovim-shell.nix;
my-haskell-shell-env = import ./my-haskell-shell.nix {};
in
stdenv.mkDerivation {
name = "my-new-env";
buildInputs = [
my-neovim-shell-env
my-haskell-shell-env
];
}
This does not work however. It seems to try to build the neovim environment:
$ nix-shell
these derivations will be built:
/nix/store/k9ygid1wl75vf2nq7jzfh32mv5f8i956-neovim-dev-env.drv
building '/nix/store/k9ygid1wl75vf2nq7jzfh32mv5f8i956-neovim-dev-env.drv'...
unpacking sources
variable $src or $srcs should point to the source
builder for '/nix/store/k9ygid1wl75vf2nq7jzfh32mv5f8i956-neovim-dev-env.drv' failed with exit code 1
error: build of '/nix/store/k9ygid1wl75vf2nq7jzfh32mv5f8i956-neovim-dev-env.drv' failed
I don't know how to fix this; what am I doing wrong? I suspect maybe I should not be using mkDerivation?
Managed to find a solution. Maybe there is a better way, but this works:
Suppose you have a package.yaml, then
$ cabal2nix . > app.nix
# shell.nix
let
p = import <nixpkgs> {};
app = p.haskellPackages.callPackage (./app.nix) {};
lib = p.haskell.lib;
in
lib.overrideCabal app (old: { buildTools = [p.my-nvim] ++ (old.buildTools or []); })
(where I've added my-nvim to my ~/.config/nixpkgs/overlays/)
Now I have access to the my-nvim package and ghc, etc:
$ nix-shell
$ nvim # works
$ ghc # works
Inspiration from https://github.com/p-implies-q/nix-util

Building haskell package for n different version

I'm trying to build bytestring and it fails with an error related to quickcheck. After a quick check (no pun intended), I found out that bytestring has a dependency on quickcheck with version >= 2.4 && < 2.10 and by default nix build quickcheck version 2.10.1.
After reading this now I know I can switch the quickcheck's version.
shell.nix:
{ compiler ? "default", doBenchmark ? false }:
let
config = { packageOverrides = pkgs: rec {
haskellPackages = pkgs.haskellPackages.override {
overrides = haskellPackagesNew: haskellPackagesOld: rec {
myproject = haskellPackagesNew.callPackage ./default.nix { };
QuickCheck = haskellPackagesNew.callPackage ./quickcheck.nix { };
};
};
};
};
pkgs = (import <nixpkgs> { inherit config; }).pkgs;
f = import ./default.nix;
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;
drv = variant (haskellPackages.callPackage f { development = true;});
in
{myproject = if pkgs.lib.inNixShell then drv.env else drv;}
default.nix and quickcheck.nix is generated with cabal2nix.
Now it throws a different error that some of the package are missing its dependencies which is quickcheck with version >=2.10.
How to tell that only bytestring will use the custom version?
Edit: quickcheck's version switched to 2.9.2
Overriding a dependency on a single package
The problem is that the package set was created with the original quickcheck version. Globally overriding a much-used package like that is bound to cause problems.
Instead, you can override specific edges in the dependency graph like this:
myproject.overrideScope (self: super: {
QuickCheck = haskellPackagesNew.callPackage ./quickcheck.nix { };
});
Composability
Your example can be improved further by using haskellPackages.extend instead of overrides. This will give freedom to further extend your package set after you have extended it before.
Another change you can make is to avoid rec recursive attribute sets in the definition of overrides and overlays. It is often best to use self instead. See this blog post on flyingcircus.io.
It's also a good idea to switch from packageOverrides to overlays.
These things may not cause issues right now, but will lead to surprises in larger projects, so it's better to avoid them in the first place.
End result
I hope I got this right because I don't have the complete example.
{ compiler ? "default", doBenchmark ? false }:
let
overlays = [(pkgs: super: {
haskellPackages = super.haskellPackages.extend ( hself: hsuper: {
myproject = (hself.callPackage ./default.nix {}).overrideScope (pself: psuper: {
QuickCheck = pself.QuickCheck_2_9;
});
QuickCheck_2_9 = hself.callPackage ./quickcheck.nix { };
};
);
};
)];
pkgs = (import <nixpkgs> { inherit overlays; }).pkgs;
f = import ./default.nix;
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;
drv = variant (haskellPackages.callPackage f { development = true;});
in
{myproject = if pkgs.lib.inNixShell then drv.env else drv;}

Resources