Is it possible to get Haskell stack to use Haskell packages provided by nix? - haskell

I'm developing a Haskell program that will eventually have some non-Haskell dependencies, and I heard that the nix integration in stack is one way to accomplish this. However, I'm now wondering if it's possible to use nix to install Haskell dependencies too since all the libraries I'm using are on nix, and currently I have only Haskell dependencies. I tried to make a custom "shell.nix" file that describes all the Haskell dependencies I have as follows:
with (import <nixpkgs> {});
stdenv.mkDerivation {
name = "simple";
buildInputs = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC ghc
];
buildTools = [ haskellPackages.alex haskellPackages.happy ];
libraryHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
libraryToolDepends = [ haskellPackages.alex haskellPackages.happy haskellPackages.hpack ];
executableHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
executableToolDepends = [ haskellPackages.alex haskellPackages.happy ];
testHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
testToolDepends = [ haskellPackages.alex haskellPackages.happy ];
}
And in stack.yaml, set
nix:
enable: true
shell-file: shell.nix
However, when I type "stack build," alex, etc are not found. I can add all the package requirements manually to the stack.yaml file (the dependency requirements are already set in the package.yaml/package.cabal file). However, the library versions on nixpkgs satisfy the dependency requirements in the package.yaml file, so I'm curious, is it supported to tell stack to just use the versions of (Haskell) libraries provided by nixpkgs? If not, are there any other options for building a Haskell project using dependencies provided by nixpkgs whenever they are available?

This is not a real answer to the question, but it's a start of a workaround which I'll post a separate question about.
In Nix, you can use haskellPackages.ghcWithPackages (package-name), to obtain a ghc together with packages that ghc knows about. E.g.
{ pkgs ? import <nixpkgs> {}}:
let
# get ghc with libraries bundled
ghc = pkgs.haskellPackages.ghcWithPackages (p:[
p.array
p.base
...]);
...
If you then run nix-shell shell.nix and then cabal build, it will build the package using the dependencies pulled in from Nix, and everything seems to work as I was hoping. With two exceptions: alex and happy. On v1-build, the nix-installed alex and happy are used, but on v2-build they are not (but this is out of scope for this question).

Related

How can I tell Cabal where to find alex and happy so that Cabal doesn't try to build them?

I'm developing a Haskell program using Cabal in a nix-shell. I would like to have as many dependencies of the build installed by Nix as possible.
The simple.cabal file seems fairly standard (it was initially produced by stack). The full contents are here: https://pastebin.com/3wd8j0pp The only place alex and happy appear in the .cabal file are in the build-tool-depends sections.
I tried to keep the shell.nix file as simple as possible (but I'm also inexperienced in Nix):
{ pkgs ? import <nixpkgs> {}}:
let
ghc = pkgs.haskellPackages.ghcWithPackages (p:[
p.array
p.base
p.bound
p.containers
p.deriving-compat
p.haskeline
p.logict
p.mtl
p.text
p.unification-fd
p.alex
p.happy
p.BNFC
p.cabal-install
]);
alex = pkgs.haskellPackages.alex;
happy = pkgs.haskellPackages.happy;
in
pkgs.mkShell {
buildInputs = [ ghc pkgs.haskellPackages.alex pkgs.haskellPackages.happy pkgs.pkg-config];
buildTools = [ pkgs.haskellPackages.alex pkgs.haskellPackages.happy];
buildToolDepends = [pkgs.haskellPackages.alex pkgs.haskellPackages.happy];
ALEX="${alex}/bin/alex";
HAPPY="${happy}/bin/happy";
}
I saved environment variables to reference the locations of alex and happy.
Finally, I tried to tell cabal where to find alex and happy by specifying them in the extra-prog-path section of a cabal.project file. However, that didn't work, so I tried hard-coding in their location
packages: simple.cabal
extra-prog-path:
/nix/store/PATH-TO-ALEX/bin/alex
/nix/store/PATH-TO-HAPPY/bin/happy
Finally, on building to enter the nix-shell, nix-shell --pure shell.nix and then cabal build
The cabal tool build finds all the ghc library packages installed with ghcWithPackages correctly -- it requires none of them to be built. However, it seems to not know about alex nor happy. The output looks like this.
In order, the following will be built (use -v for more details):
- alex-3.2.6 (exe:alex) (requires build)
- happy-1.20.0 (exe:happy) (requires build)
I can also confirm that the version of alex and happy in the nix store are alex-3.2.6 and happy-1.20.0.
On the other hand, cabal v1-build does pick up alex and happy as installed. cabal v1-build
Resolving dependencies...
Configuring simple-0.1.0.0...
Preprocessing library for simple-0.1.0.0..
Building library for simple-0.1.0.0..
and the build completes successfully only compiling the source files in the local package.
It seems like one should favor v2-build or new-build. How can I get cabal to know where to find and also to use the alex and happy versions installed by Nix, or alternatively how to tell Nix what to do so that v2-build or new-build can find alex and happy?

How do you build an executable from haskell packages (which use stack / cabal) in nix?

I'm looking for something similar to this:
shell.nix
{ pkgs ? import <nixpkgs> {} }:
let
threadscope = ...
speedscope = ...
in
stdenv.mkDerivation {
name = "haskell-env";
nativeBuildInputs = [
# Haskell tools
ghcid ghc
# Profiling tools
threadscope
speedscope
];
}
Then I can use nix-shell:
% nix-shell
...
nix-shell % threadscope <some-commands>
nix-shell % speedscope <some-commands>
Do you usually just build from source from github / hackage / stackage or use pkgs.haskellPackages.<my-package>?
I usually take tools from haskellPackages.
If you only want a shell with tools, you can use an expression like this.
{ pkgs ? import <nixpkgs> {} }:
let
# Select a package set. haskellPackages is recommended, but
# if you need a different ghc, use pkgs.haskell.packages.ghc8102 for example
haskellPackages = pkgs.haskellPackages;
in
pkgs.mkShell {
nativeBuildInputs = [
# Haskell tools
haskellPackages.ghc
haskellPackages.ghcid
# Profiling tools
haskellPackages.threadscope
# pkgs.speedscope (not packaged yet)
];
}
If you want a shell with specific precompiled libraries, you can use the ghcWithPackages function instead of ghc. If you want precompiled dependencies for your local Haskell package(s), you can replace pkgs.mkShell by the haskellPackages.shellFor function.
If you'd like to package speedscope, the Nixpkgs manual documents the packaging process for node packages

How do I supply a C library to stack on NixOS?

I have a stack-based project that depends on a couple C libraries. One of these C libraries, zlib, is available from a native NixOS package and I can put into the nix section of stack.yaml:
nix:
enable: true
packages:
- "zlib"
The other is not part of nixpkgs. The stack documentation suggests that the alternative to using the nix section in stack.yaml is to "write a shell.nix" without elaborating much.
So I wrote one, sticking to zlib as an example:
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
buildInputs = [
pkgs.pkgconfig
pkgs.zlib
pkgs.stack
];
}
This gives me a working pkg-config for zlib:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ pkg-config --modversion zlib
1.2.11
However, it doesn't appear to make stack able to find the library:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ stack build
zlib-0.6.2: configure
Progress 1/7
-- While building package zlib-0.6.2 using:
/home/exarkun/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1 configure --with-ghc=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc --with-ghc-pkg=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/pkgdb --libdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/lib --bindir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/bin --datadir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/share --libexecdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/libexec --sysconfdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/etc --docdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --htmldir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --haddockdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --dependency=base=base-4.12.0.0 --dependency=bytestring=bytestring-0.10.8.2 --extra-include-dirs=/nix/store/a54skdf3xksiqvcvr75bjpdl1jx8dgbk-gmp-6.1.2-dev/include --extra-include-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/include --extra-include-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/include --extra-include-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/include --extra-lib-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/lib --extra-lib-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/lib --extra-lib-dirs=/nix/store/kggcrzpa5hd41b7v60wa7xjkgjs43xsl-gmp-6.1.2/lib --extra-lib-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/lib
Process exited with code: ExitFailure 1
Logs have been written to: /home/exarkun/Work/PrivateStorage/PaymentServer/.stack-work/logs/zlib-0.6.2.log
Configuring zlib-0.6.2...
Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5: Missing dependency on a foreign
library:
* Missing (or bad) header file: zlib.h
* Missing (or bad) C library: z
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
For zlib, there's no particular need to go this route, but as far as I can tell, I cannot put my non-nixpkgs package into the nix.packages list in stack.yaml.
How do I get stack to find these libraries?
Apparently a case of end-of-the-day documentation-reading-fail. This morning, it didn't take long to find the stack documentation that explains how to use a custom shell.nix. The stack docs do elaborate on how this works, including an example which shows it's not at all like the shell I thought was expected:
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myEnv";
buildInputs = [ glpk pcre ];
}
Adapting my shell to use buildStackProject instead of mkShell:
{ ghc }:
with (import <nixpkgs> { overlays = [ (import ./overlay.nix) ]; });
haskell.lib.buildStackProject {
inherit ghc;
name = "PrivacyPass";
# extra-library made available via the overlay.
# overlay probably not strictly necessary here, either.
buildInputs = [ zlib extra-library ];
}
and then pointing stack at it in stack.yaml:
nix:
enable: true
shell-file: "stack-shell.nix"
results in a successful build.

How create a nix package with haskell stack

I need to create the nix package from a project stack with postgres template.
Basically it is the following: I have a virtual machine with NixOS and I need to portlet my project to another one with NixOS. For this I will use the nix copy command, but before I need to somehow "install" my project so that it is in /nix/store.
I tried to do the same looking at another issue posted "Generating a Nix package from a stack project".
come on. What I did was:
cd /home/ProjetoApp
stack new TesteYesod yesod-postgres
cd TestYesod
stack init --resolver lts-11.22
stack install yesod-bin --install-ghc
I added in stack.yaml:
nix
pure: true
enable: true
packages: [postgresql]
So far so good. I can compile and execute only stack. Then I was trying to generate a nix package. In stack.yaml I did:
nix
pure: true
enable: true
shell-file: default.nix
And in default.nix I did:
{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="teste-yesod-1.0.0";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
If I do:
nix build
Occurs:
builder for '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod
1.0.0.drv' failed with exit code 1; last 5 log lines:
unpacking sources
unpacking source archive /nix/store/
n62hzn4igi1b7khksa6sp3cq8gk4h344-TesteYesod
source root is TestYesod
patching sources
configuring
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod-
1.0.0.drv' failed
If. How to instruct nix that the source is spread across multiple directories?
If you want a quick fix, the following should work:
{ nixpkgs ? import (builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/d42ef371c9b1b532400b0f2820885e575f4f1617.tar.gz";
sha256 = "0irb4zb6hdgaah238244jk2xf63xfb20xy3plb1194pd4xbgdr3r";
}) {}
, ghc ? nixpkgs.ghc
}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="TesteYesod";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
What's the problem? It seems that in the current 18.09 release, buildStackProject is broken. The non-existent error message is caused by a subtly failing preConfigure script, which contains the following:
addStackArgsHook = ''
for pkg in ''${pkgsHostHost[#]} ''${pkgsHostBuild[#]} ''${pkgsHostTarget[#]}
do
[ -d "$pkg/lib" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-lib-dirs=$pkg/lib"
[ -d "$pkg/include" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-include-dirs=$pkg/include"
done
'';
In your case, stack is the last dependency to be processed in the loop (and perhaps in all buildStackProject invocations in 18.09, I'm not sure), but it contains no /lib or /include directory, so the exit code with which the preConfigure script exits is 1, and so the whole build process quits. This is just because of the shorthand &&, it would work if there was a proper if.
It is however fixed in master already (https://github.com/NixOS/nixpkgs/pull/53618), so simply using a newer nixpkgs should fix that problem. Pinning the nixpkgs is something you want to do anyway if you want to use Nix's replicability guarantees to the fullest, as you can't know you're using the same nixpkgs commit, so you may be using different versions of system packages.
(If you want to know how I debugged this - it seemed the problem was in the preConfigure step, so I took a look at the code in generic-stack-builder.nix, saw that preConfigure was overridable, copy-pasted the code from there into default.nix and added a set -x at the top. That showed me the problem above, and prompted me to go look at that file in master to see if there weren't changes since 18.09. Another workaround could be simply adding true to the preConfigure script if it wasn't already fixed in master, but that wasn't necessary.)
(Also, I've opened https://github.com/NixOS/nixpkgs/issues/55548 to backport the fix to 18.09.)

NixOS and ghcWithPackages - possibly incorrect information in the wiki

I'm referring here to the page: Haskell - Nix Wiki, both the heading Local use via Nixpkgs config and System-wide use via NixOS config.
Here are the steps to replicate the problem: I download the NixOS-14.04 virtual appliance, load it in Virtual box, and try the System-wide use, so that my configuration.nix is:
{ config, pkgs, ... }:
{
imports = [ <nixos/modules/installer/virtualbox-demo.nix> ];
environment.systemPackages = [
# other system packages
(pkgs.haskellPackages.ghcWithPackages (self : [
self.haskellPlatform
]))
];
}
This is the output of nixos-rebuild test --show-trace -v
When I try the local option setting .nixpkgs/config.nix, here is the output of $ nix-env -iA nixos.pkgs.hsEnv.
The errors seem to me very similar, and so I suspect there may be an error in ghcWithPackages .nix expression. Do you have any suggestion?
This is a known bug in ghcWithPackages. See https://github.com/NixOS/nixpkgs/issues/1438 for more information.
In short, the fix is to replace ghcWithPackages with ghcWithPackagesOld.

Resources