How create a nix package with haskell stack - haskell

I need to create the nix package from a project stack with postgres template.
Basically it is the following: I have a virtual machine with NixOS and I need to portlet my project to another one with NixOS. For this I will use the nix copy command, but before I need to somehow "install" my project so that it is in /nix/store.
I tried to do the same looking at another issue posted "Generating a Nix package from a stack project".
come on. What I did was:
cd /home/ProjetoApp
stack new TesteYesod yesod-postgres
cd TestYesod
stack init --resolver lts-11.22
stack install yesod-bin --install-ghc
I added in stack.yaml:
nix
pure: true
enable: true
packages: [postgresql]
So far so good. I can compile and execute only stack. Then I was trying to generate a nix package. In stack.yaml I did:
nix
pure: true
enable: true
shell-file: default.nix
And in default.nix I did:
{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="teste-yesod-1.0.0";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
If I do:
nix build
Occurs:
builder for '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod
1.0.0.drv' failed with exit code 1; last 5 log lines:
unpacking sources
unpacking source archive /nix/store/
n62hzn4igi1b7khksa6sp3cq8gk4h344-TesteYesod
source root is TestYesod
patching sources
configuring
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod-
1.0.0.drv' failed
If. How to instruct nix that the source is spread across multiple directories?

If you want a quick fix, the following should work:
{ nixpkgs ? import (builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/d42ef371c9b1b532400b0f2820885e575f4f1617.tar.gz";
sha256 = "0irb4zb6hdgaah238244jk2xf63xfb20xy3plb1194pd4xbgdr3r";
}) {}
, ghc ? nixpkgs.ghc
}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="TesteYesod";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
What's the problem? It seems that in the current 18.09 release, buildStackProject is broken. The non-existent error message is caused by a subtly failing preConfigure script, which contains the following:
addStackArgsHook = ''
for pkg in ''${pkgsHostHost[#]} ''${pkgsHostBuild[#]} ''${pkgsHostTarget[#]}
do
[ -d "$pkg/lib" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-lib-dirs=$pkg/lib"
[ -d "$pkg/include" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-include-dirs=$pkg/include"
done
'';
In your case, stack is the last dependency to be processed in the loop (and perhaps in all buildStackProject invocations in 18.09, I'm not sure), but it contains no /lib or /include directory, so the exit code with which the preConfigure script exits is 1, and so the whole build process quits. This is just because of the shorthand &&, it would work if there was a proper if.
It is however fixed in master already (https://github.com/NixOS/nixpkgs/pull/53618), so simply using a newer nixpkgs should fix that problem. Pinning the nixpkgs is something you want to do anyway if you want to use Nix's replicability guarantees to the fullest, as you can't know you're using the same nixpkgs commit, so you may be using different versions of system packages.
(If you want to know how I debugged this - it seemed the problem was in the preConfigure step, so I took a look at the code in generic-stack-builder.nix, saw that preConfigure was overridable, copy-pasted the code from there into default.nix and added a set -x at the top. That showed me the problem above, and prompted me to go look at that file in master to see if there weren't changes since 18.09. Another workaround could be simply adding true to the preConfigure script if it wasn't already fixed in master, but that wasn't necessary.)
(Also, I've opened https://github.com/NixOS/nixpkgs/issues/55548 to backport the fix to 18.09.)

Related

How do I supply a C library to stack on NixOS?

I have a stack-based project that depends on a couple C libraries. One of these C libraries, zlib, is available from a native NixOS package and I can put into the nix section of stack.yaml:
nix:
enable: true
packages:
- "zlib"
The other is not part of nixpkgs. The stack documentation suggests that the alternative to using the nix section in stack.yaml is to "write a shell.nix" without elaborating much.
So I wrote one, sticking to zlib as an example:
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
buildInputs = [
pkgs.pkgconfig
pkgs.zlib
pkgs.stack
];
}
This gives me a working pkg-config for zlib:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ pkg-config --modversion zlib
1.2.11
However, it doesn't appear to make stack able to find the library:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ stack build
zlib-0.6.2: configure
Progress 1/7
-- While building package zlib-0.6.2 using:
/home/exarkun/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1 configure --with-ghc=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc --with-ghc-pkg=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/pkgdb --libdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/lib --bindir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/bin --datadir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/share --libexecdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/libexec --sysconfdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/etc --docdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --htmldir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --haddockdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --dependency=base=base-4.12.0.0 --dependency=bytestring=bytestring-0.10.8.2 --extra-include-dirs=/nix/store/a54skdf3xksiqvcvr75bjpdl1jx8dgbk-gmp-6.1.2-dev/include --extra-include-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/include --extra-include-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/include --extra-include-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/include --extra-lib-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/lib --extra-lib-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/lib --extra-lib-dirs=/nix/store/kggcrzpa5hd41b7v60wa7xjkgjs43xsl-gmp-6.1.2/lib --extra-lib-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/lib
Process exited with code: ExitFailure 1
Logs have been written to: /home/exarkun/Work/PrivateStorage/PaymentServer/.stack-work/logs/zlib-0.6.2.log
Configuring zlib-0.6.2...
Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5: Missing dependency on a foreign
library:
* Missing (or bad) header file: zlib.h
* Missing (or bad) C library: z
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
For zlib, there's no particular need to go this route, but as far as I can tell, I cannot put my non-nixpkgs package into the nix.packages list in stack.yaml.
How do I get stack to find these libraries?
Apparently a case of end-of-the-day documentation-reading-fail. This morning, it didn't take long to find the stack documentation that explains how to use a custom shell.nix. The stack docs do elaborate on how this works, including an example which shows it's not at all like the shell I thought was expected:
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myEnv";
buildInputs = [ glpk pcre ];
}
Adapting my shell to use buildStackProject instead of mkShell:
{ ghc }:
with (import <nixpkgs> { overlays = [ (import ./overlay.nix) ]; });
haskell.lib.buildStackProject {
inherit ghc;
name = "PrivacyPass";
# extra-library made available via the overlay.
# overlay probably not strictly necessary here, either.
buildInputs = [ zlib extra-library ];
}
and then pointing stack at it in stack.yaml:
nix:
enable: true
shell-file: "stack-shell.nix"
results in a successful build.

How to set cabal flag from shell.nix for stack project?

By default I build my project without nix, and executable is linked statically. But when building with nix, I want to link it dynamically instead. So I added a switch to myproj.cabal:
flag dynamic
description: Build only dynamic binaries
default: False
executable RunMe
ghc-options: -O2 -Wall
if !flag(dynamic)
ghc-options: -optl-static -static
Now I can build the project with
stack --nix --nix-packages=zlib install --flag myproj:dynamic
To avoid passing command line options every time, I created shell.nix:
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myproj";
buildInputs = [ zlib ];
}
Now, I don't know how to pass flag to cabal from nix file. Based on buildStackProject definition, I tried setting buildPhase, e.g.
haskell.lib.buildStackProject {
...
buildPhase = "stack build --flag=myproj:dynamic";
}
but it doesn't seem to change anything. How can I pass the flag to cabal from the nix file?
Unfortunately, the stack nix integration is rather backwards. It doesn't use nix to build your package (despite the very misleadingbuildStackPackage), just to create an environment with GHC and native dependencies. You have to use stack itself to build the project, and since stack will run cabal, it should still work with the flag. The stack nix integration will not generate a .nix file with the dependencies resolved by stack, so you can't build your project with nix-build and there's no point in entering nix-shell. You would have to use a tool like stack2nix or stackage2nix for that.
So to recap. The shell.nix above looks fine. Add this to your stack.yaml:
nix:
shell-file: shell.nix
And build the project with stack:
stack --nix --flag myproj:dynamic
That should build the project with the flag as usual, but use GHC and zlib from nix.

stack --nix build complains about ghc version mismatch

When building threepenny-gui on NixOS with stack --nix build, I got error saying I have the wrong version of ghc. Then I tried stack --nix setup, which doesn't run because bash is on an unexpected path on NixOS (that's expected, since the stack documentation only mentions stack --nix build not setup). What am I missing?
FYI, to deal with the zlib issues I have also added a shell.nix and default.nix per https://github.com/commercialhaskell/stack/issues/2130
EDIT: was able to build with the method suggested by mkkeankylej from the above link, i.e. editing ~/.stack/config.yaml and add zlib to buildInputs in shell.nix But I'd still like to know if there's a way to do it w/o falling back to nix-shell? It sounds like stack --nix build should work as long as the nix-shell method does.
First of all, threepenny-gui seems to provide no stack.yaml, i.e. the project isn't configured to be built with stack. Thus, I wonder why you even bother using stack since that is not going to be any easier than building the project with cabal-install or even Nix directly. The easiest and fastest way is probably to configure the build by running:
$ nix-shell "<nixpkgs>" -A haskellPackages.threepenny-gui.env --run "cabal configure"
Afterwards, you can simply "cabal build" the project and work with it (inside or outside of a nix-shell) as you please; the compiler and all necessary build dependencies are provided by Nix.
If you don't want that, then you can use the normal cabal-install approach:
$ cabal sandbox init
$ cabal install --only-dependencies
$ cabal configure
$ cabal build
That build is probably going to require system libraries, like libz, so you must make sure that those are available. There's a million different ways to accomplish that, but the cleanest IMHO is the following:
$ zlibinc=$(nix-build --no-out-link "<nixpkgs>" -A zlib.dev)
$ zliblib=$(nix-build --no-out-link "<nixpkgs>" -A zlib.out)
$ cabal install --only-dependencies --extra-include-dirs=$zlibinc --extra-lib-dirs=$zliblib
Last but not least, it's not obvious to me why your stack build --nix command won't succeed, because that command will use Nix to install the proper version of GHC automatically. So if that doesn't work, then my best guess is that you're using an old version of stack where that feature doesn't work properly. I've tried that build using the stack binary that Nix provides, stack 1.3.2, and it can compile a current git checkout of threepenny-gui just fine:
$ git clone git://github.com/HeinrichApfelmus/threepenny-gui.git
Cloning into 'threepenny-gui'...
remote: Counting objects: 4102, done.
remote: Total 4102 (delta 0), reused 0 (delta 0), pack-reused 4101
Receiving objects: 100% (4102/4102), 1.88 MiB | 581.00 KiB/s, done.
Resolving deltas: 100% (2290/2290), done.
$ cd threepenny-gui
$ stack init
Looking for .cabal or package.yaml files to use to init the project.
Using cabal packages:
- threepenny-gui.cabal
Selecting the best among 9 snapshots...
* Partially matches lts-7.16
websockets-snap not found
- threepenny-gui requires >=0.8 && <0.11
Using package flags:
- threepenny-gui: buildexamples = False, network-uri = True, rebug = False
* Matches nightly-2017-01-17
Selected resolver: nightly-2017-01-17
Initialising configuration using resolver: nightly-2017-01-17
Total number of user packages considered: 1
Writing configuration to file: stack.yaml
All done.
$ stack build --nix --nix-packages zlib
threepenny-gui-0.7.1.0: configure (lib)
Configuring threepenny-gui-0.7.1.0...
threepenny-gui-0.7.1.0: build (lib)
Preprocessing library threepenny-gui-0.7.1.0...
[...]
Registering threepenny-gui-0.7.1.0...
This works without any specially edited config files for nix-shell, nor does it require special customization of stack.

Can I automatically embed pandoc's default templates in my application?

Pandoc comes with several default templates, which are distributed with the pandoc package. However, if I write an application that uses pandoc as a library, those default templates don't get included in the binary. I can still use them on my machine:
module Main where
import Text.Pandoc (getDefaultTemplate)
main = getDefaultTemplate Nothing "latex" >>= print
This will print the default.latex template. However, it's not portable, since it really refers to a file somewhere on my system:
$ cd path/to/example/project
$ stack build
$ scp path/to/binary remote:remote/path
$ ssh remote:remote/path/binary
example: Could not find data file /home/Zeta/.stack/snapshots/.../pandoc-1.16.0.2/data/templates/default.latex
Since pandoc's debian package does not include those files, it's somehow able to embed them. And indeed, there is a flag -f embed_data_files. I've tried to enable it in the local stack.yaml:
extra-deps: [pandoc-1.16]
flags:
pandoc:
embed_data_files: true
But that didn't change anything, the compiled binary still complains about missing data files.
So, is there any way to automatically include pandoc's template files?
It turns out that pandoc injects its data files during its build via hsb2hs. Somehow that step failed during stack build I missed the error message.
Neither hsb2hs nor its main dependency processing-tools are part of stack's LTS, they're only in the nightly stackage versions. The following additions to stack.yaml fixed the problem:
# part of stack.yaml:
extra-deps:
- preprocessor-tools-1.0.1
- hsb2hs-0.3.1
- pandoc-1.16
flags:
pandoc:
embed_data_files: true
For those using Cabal, this is somewhat equal to
cabal sandbox init
cabal update
cabal install hsb2hs-0.3.1 && cabal install pandoc-1.16 -f embed_data_files
cabal install --dependencies-only
cabal build
Here's how I verified that the templates are actually included:
$ stack build
$ grep "usepackage\{hyperref\}" .stack-work/install/*/bin/example -a
\usepackage[$for(geometry)$$geometry$$sep$,$endfor$]{geometry}
$endif$
\usepackage{hyperref}
$if(colorlinks)$
\PassOptionsToPackage{usenames,dvipsnames}{color} % color is loaded by hyperref
That snippet is part of default.latex, so it's really included in the binary.

NixOS and ghcWithPackages - possibly incorrect information in the wiki

I'm referring here to the page: Haskell - Nix Wiki, both the heading Local use via Nixpkgs config and System-wide use via NixOS config.
Here are the steps to replicate the problem: I download the NixOS-14.04 virtual appliance, load it in Virtual box, and try the System-wide use, so that my configuration.nix is:
{ config, pkgs, ... }:
{
imports = [ <nixos/modules/installer/virtualbox-demo.nix> ];
environment.systemPackages = [
# other system packages
(pkgs.haskellPackages.ghcWithPackages (self : [
self.haskellPlatform
]))
];
}
This is the output of nixos-rebuild test --show-trace -v
When I try the local option setting .nixpkgs/config.nix, here is the output of $ nix-env -iA nixos.pkgs.hsEnv.
The errors seem to me very similar, and so I suspect there may be an error in ghcWithPackages .nix expression. Do you have any suggestion?
This is a known bug in ghcWithPackages. See https://github.com/NixOS/nixpkgs/issues/1438 for more information.
In short, the fix is to replace ghcWithPackages with ghcWithPackagesOld.

Resources