Testing multiple Haskell dependency versions with Hydra - haskell

I would like to test my Haskell library with different versions of its dependencies using continuous integration. Is there an easy way to accomplish this with Hydra (http://nixos.org/hydra)?
One solution would be to override the versions of the dependencies of interest and use Nixpkgs versions for the other dependencies. However, I can't figure out how to override one Haskell package version while ensuring that all other Haskell packages are called with the overridden package.
Here is a simplification of my attempt to override a Haskell dependency in release.nix. The referenced Nix expressions were created with cabal2nix. The build depends on json-assertions, which depends on aeson, which depends on mtl. Since aeson is not called with the overridden mtl, the expression specifies two versions of mtl. The two versions of mtl cause the build to fail.
let pkgs = import <nixpkgs> {};
in {
example = pkgs.haskellPackages.cabal.mkDerivation (self: {
pname = "example";
version = "0.1.0.0";
isExecutable = true;
src = ./.;
buildDepends = [ (pkgs.haskellPackages.ghcWithPackages (self:
[ self.cabalInstall_1_18_0_3
(self.callPackage (import ./mtl_2_2.nix) {
transformers = self.callPackage (import ./transformers_0_4_2_0.nix) {};
})
(self.callPackage (import ./jsonAssertions_1_0_4.nix) {})
]))
];
});
}
Another solution would be to use publicly available Nix expressions for compatible sets of Haskell packages other than the ones in Nixpkgs. I have not found any, though.
Are there any projects online that use Hydra to test a Haskell package against multiple versions of its dependencies that I could use as an example?

I found out how to use Nixpkgs's new Haskell infrastructure to rewrite my example release.nix. This expression overrides mtl so that all packages that depended on the default version of mtl are called with the overridden version. However, other version inconsistencies prevent the example from building.
with (import <nixpkgs> {}).pkgs;
let haskellPackages =
pkgs.haskell-ng.packages.ghc784.override {
overrides = self: super: {
mtl = self.callPackage ./mtl_2_2.nix {};
transformers = self.callPackage ./transformers_0_4_2_0.nix {};
json-assertions = self.callPackage ./json-assertions_1_0_4.nix {};
};
};
in {
example = haskellPackages.mkDerivation {
pname = "example";
version = "0.1.0.0";
isExecutable = true;
src = ./.;
buildDepends = with haskellPackages; [ mtl json-assertions ];
license = pkgs.stdenv.lib.licenses.bsd3;
};
}

Related

How can I disable testing for a Haskell package in Nix?

I'm trying to get a development environment going for Haskell, using Nix. I have a default.nix that just refers to my .cabal file for the list of packages. But one package I want to use, numhask-space, won't build, because the tests are failing. So I'm trying to override it and skip the tests.
Here's my default.nix:
# default.nix
let
pkgs = import <nixpkgs> { };
in
pkgs.haskellPackages.developPackage {
root = ./.;
modifier = drv:
pkgs.haskell.lib.addBuildTools drv (with pkgs.haskellPackages;
[ cabal-install
ghcid
]);
source-overrides = {
numhask-space = pkgs.haskell.lib.dontCheck pkgs.haskellPackages.numhask-space;
};
}
But I must not be doing this right, because I get the error:
cabal2nix: user error (Failed to fetch source. Does this source exist? Source {sourceUrl = "/nix/store/w3pcvlj7b0k44v629k00kw2b0k86fcyj-numhask-space-0.3.0", sourceRevision = "", sourceHash = Guess "", sourceCabalDir = ""})
In short, how can I override this haskell package so that it doesn't run the tests, and I can install it in my development environment?
source-overrides overrides sources of the packages, that is, literally, the src attributes, rather than packages as a whole. You should use the overrides argument instead:
overrides = self: super: {
numhask-space = pkgs.haskell.lib.dontCheck super.numhask-space;
};

How to add a Nix project as dependency in a Haskell Stack+Nix project

I'm trying to write a haskell project using stack + nix. My current stack.yaml and shell.nix are as follows:
resolver: lts-14.6
packages:
- .
nix:
enable: true
pure: true
shell-file: shell.nix
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myproject";
buildInputs = [ pkg-config libmysqlclient postgresql_10 pcre libsodium secp256k1 zlib ];
PGPASSWORD = builtins.getEnv "PGPASSWORD";
}
Now, in my haskell project, I want to execute tezos-client and other tools provided by tezos-baking-platform. I can build it by running
nix-build -A tezos.babylonnet.kit
The problem is, after building, I can find all the executable files in, for example, /nix/store/bgqva3wgi3knivdk9pf7gdd0384hj2qf-tezos-0.0.0/bin/. But they are not exposed to nix-env and I can't find any symlink for tezos in ~/.nix-profile/bin/.
So, (1) how can I fix this? and (2) how can I set that tezos-baking-platform as a dependency of my haskell project? (i.e. through stack to ask nix for building tezos for me right before build my haskell project)
I figured out a working (but maybe not perfect) solution.
Because the tezos-baking-platform is not providing nix derivation, so the simplest way is to put the entire repository into the working repository and import its nix file. This will make tezos.babylonnet.kit available in nix so we can set it as a dependency directly.
{ghc}:
with (import <nixpkgs> {});
with import ./tezos-baking-platform/default.nix {};
haskell.lib.buildStackProject {
inherit ghc;
name = "qq";
buildInputs = [ tezos.babylonnet.kit ];
}
PS. Ideally, if the tezos-baking-platform provided a nix derivation, then we can do something like
{ghc}:
with (import <nixpkgs> {});
stdenv.mkDerivation {
name = "tezos-baking-platform";
version = "0.0.0";
src = fetchurl {
url = "https://gitlab.com/obsidian.systems/tezos-baking-platform/";
rev = "2f37c78a4b0ac26ee5e428711ab3c7ebeb9869fb";
sha256 = "0h71ivsva7hfqy0zy1pbx68a4i8lbqln3k9zkw0j2rgain39844r";
};
}
haskell.lib.buildStackProject {
inherit ghc;
name = "myproject";
buildInputs = [ tezos-baking-platform ];
}
To enter a shell with the executables in the PATH: nix shell (by default it references shell.nix).
To add tezos-baking-platform as a dependency I think you just add it in buildInputs. If this is from a channel named tezos you will have to import this:
tezos = import <tezos> {};
...
tezos.tezos-baking-platform

How can I build a haskell dependency from a GitHub source .nix file, using nix?

Ok, so I have a .nix file for my project that looks like this:
{ mkDerivation, base, blaze-html, clay, hakyll, hspec, stdenv }:
mkDerivation {
pname = "open-editions";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
executableHaskellDepends = [ base blaze-html clay hakyll ];
testHaskellDepends = [ hspec ];
license = "unknown";
hydraPlatforms = stdenv.lib.platforms.none;
}
(It's just a web project using hakyll.) The problem is, clay is broken in nixpkgs. But I'm assuming clay works on its master branch on GitHub. So how can I replace this haskell dependency clay with something that downloads and calls the GitHub package? The GitHub package has a bunch of .nix files there, so I'm guess it'd be relatively straightforward, but I don't really know where to start on this.
You can extend your haskell package set, similar to this example.
The nix file for clay can be created with cabal2nix
cabal2nix git://github.com/sebastiaanvisser/clay.git > clay.nix
Assuming your .nix file is called open-editions.nix, your default.nix may look somewhat like:
let
pkgs = import <nixpkgs> {}; # or similar, I like to pin it with niv
hs = pkgs.haskellPackages.extend(self: super: { # (1) extend the package set
open-editions = self.callPackage ./open-editions.nix {};
clay = self.callPackage ./clay.nix {}; # (2) update clay
});
in {
inherit (hs) open-editions; # (3) make nix-build pick up my-project;
}

How to add package to cabal2nix generated `env`?

I use this default.nix to build my package with nix-build and get the env with nix-shell
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
with haskellPackages;
let
myPackage = callPackage ./myPackage.nix {};
in
if lib.inNixShell then myPackage.env else myPackage
myPackage.nix generated using cabal2nix . > myPackage.nix
{ mkDerivation, base, split, stdenv }:
mkDerivation {
pname = "myPackage";
version = "0.1.0.0";
src = ./.;
isLibrary = false;
isExecutable = true;
executableHaskellDepends = [ base split ];
license = stdenv.lib.licenses.bsd3;
}
This working fine for building but I want to add development helper tools while I working on it. I don't want to edit myPackage.nix. I want to re-run cabal2nix when I edit myPackage.cabal.
I try to use buildInputs of mkDerivation but did not seem to work.
let
myPackage = callPackage ./myPackage.nix {};
in
stdenv.mkDerivation {
name = myPackage.name;
buildInputs = [ myPackage hlint hasktags ];
}
Beside nix-build stop working, it also drop me in the shell with executable of myPackage but without myPackage's env.
I know this because ghc is not avaliable while it exists in myPackage's env when using default.nix above.
How can I add those tools to env generated from cabal2nix ?
The nix-shell commands builds all dependencies of a project, sets all environment variables to their respective derivation attribute values and sources (bash) $stdenv/setup. For more details, see the Nix manual about nix-shell.
So in your last example, if you run echo $buildInputs you'll see your built package as a build input. So that works, but it's not what you want.
Instead, you need to reuse the Haskell-specific environment derivation, myPackage.env. This dummy derivation for nix-shell has a GHC that is set up to discover only your dependencies etc.
pkgs.lib.overrideDerivation myPackage.env (old: {
buildInputs = old.buildInputs ++ [ pkgs.haskellPackages.hlint ];
})
Unsolicited advice ;)
In my projects, I use a shell.nix file for this. This also lets me avoid the lib.inNixShell value that breaks referential transparency.
If your project consists of more than a Haskell package, I recommend writing an overlay. It will make your project much more coherent.
shell.nix
# This imports the project + overlay. The overlay takes care of
# adding `myPackage` to `haskellPackages`, to make it available
# throughout the project.
attrs#{...}:
let pkgs = (import ../nix attrs);
# Adapt myPackage.env
in pkgs.lib.overrideDerivation pkgs.haskellPackages.myPackage.env (old: {
buildInputs = old.buildInputs ++ [ pkgs.haskellPackages.hlint ];
})
For an example of an overlay, see zimbatm's todomvc-nix.

How do I use the new haskell-ng infrastructure on NixOS?

How do I setup a simple Haskell development environment on NixOS using the new haskell-ng infrastructure?
Here's one way of doing it on a per project basis. I'm developing a Yesod webbapp so I'm using my current setup as an example.
In my project folder I have two files: default.nix (specifies how to build my project) and shell.nix (specifies how to setup build inputs).
This way I can just run nix-shell and it drops me into an environment which has ghc installed with the provided packages and paths setup.
The setup
default.nix
{ stdenv, haskellPackages }:
let
env = haskellPackages.ghcWithPackages (p: with p; [
aeson
bytestring
cabal-install
classy-prelude
classy-prelude-conduit
classy-prelude-yesod
conduit
containers
data-default
directory
fast-logger
file-embed
hjsmin
http-conduit
monad-control
monad-logger
persistent
persistent-postgresql
persistent-template
safe
shakespeare
template-haskell
text
time
unordered-containers
vector
wai-extra
wai-logger
warp
yaml
yesod
yesod-auth
yesod-bin
yesod-core
yesod-form
yesod-static
]);
in
stdenv.mkDerivation {
name = "project-name";
buildInputs = [env];
shellHook = ''
export NIX_GHC="${env}/bin/ghc"
export NIX_GHCPKG="${env}/bin/ghc-pkg"
export NIX_GHC_DOCDIR="${env}/share/doc/ghc/html"
export NIX_GHC_LIBDIR=$( $NIX_GHC --print-libdir )
'';
}
shell.nix
{ pkgs ? (import <nixpkgs> {}) }:
(import ./default.nix) {
stdenv = pkgs.stdenv;
haskellPackages = pkgs.haskellPackages;
}
More information
For more information you can read:
A Journey into the Haskell NG infrastructure: Part I
A Journey into the Haskell NG infrastructure: Part II
Those links explains how haskell-ng works and what will happen in the future.
If you want to dig into the source of the haskell-ng infrastructure I suggest:
The file where haskellPackages is defined
The haskell packages module
The haskell-module folder
Update: Building dependencies with profiling enabled
All packages in haskell-ng are defined in hackage-packages.nix. They are built using cabal.mkDerivation.
If you want to override how a package is built you need to override how mkDerivation is called on that package. In the Haskell Wiki you have information on how to do that.. Basically you define a new haskellPackages with the overrides you want for each package:
let haskellPackages' = haskellPackages.override {
overrides = self: super: {
fullstop = super.fullstop.override {
mkDerivation = (attrs: self.mkDerivation (attrs // { doCheck = false; }));
};
};
};
This can be quite tedious to do, therefor there are helper functions in the file lib.nix in haskell-modules that you can use.
The file haskell-ng imports lib.nix and exposes it. And haskell-ng is exposed as a top-level package so you can access the lib functions easily.
Say we have a package that want to enable profiling on the text library we can do something like this:
{ pkgs ? (import <nixpkgs> {}).pkgs }:
let
lib = pkgs.haskell-ng.lib;
env = pkgs.haskellPackages.ghcWithPackages (p: with p; [
(lib.enableLibraryProfiling text)
]);
in
pkgs.stdenv.mkDerivation {
name = "hello-world-wide-web";
buildInputs = [env];
shellHook = ''
export NIX_GHC="${env}/bin/ghc"
export NIX_GHCPKG="${env}/bin/ghc-pkg"
export NIX_GHC_DOCDIR="${env}/share/doc/ghc/html"
export NIX_GHC_LIBDIR=$( $NIX_GHC --print-libdir )
'';
}
I found this information this weekend by going through the nix source for haskell-ng and testing it. I'm still quite new to NixOS and nix so there could be a better way of doing this.

Resources