I'm referring here to the page: Haskell - Nix Wiki, both the heading Local use via Nixpkgs config and System-wide use via NixOS config.
Here are the steps to replicate the problem: I download the NixOS-14.04 virtual appliance, load it in Virtual box, and try the System-wide use, so that my configuration.nix is:
{ config, pkgs, ... }:
{
imports = [ <nixos/modules/installer/virtualbox-demo.nix> ];
environment.systemPackages = [
# other system packages
(pkgs.haskellPackages.ghcWithPackages (self : [
self.haskellPlatform
]))
];
}
This is the output of nixos-rebuild test --show-trace -v
When I try the local option setting .nixpkgs/config.nix, here is the output of $ nix-env -iA nixos.pkgs.hsEnv.
The errors seem to me very similar, and so I suspect there may be an error in ghcWithPackages .nix expression. Do you have any suggestion?
This is a known bug in ghcWithPackages. See https://github.com/NixOS/nixpkgs/issues/1438 for more information.
In short, the fix is to replace ghcWithPackages with ghcWithPackagesOld.
Related
I'm developing a Haskell program that will eventually have some non-Haskell dependencies, and I heard that the nix integration in stack is one way to accomplish this. However, I'm now wondering if it's possible to use nix to install Haskell dependencies too since all the libraries I'm using are on nix, and currently I have only Haskell dependencies. I tried to make a custom "shell.nix" file that describes all the Haskell dependencies I have as follows:
with (import <nixpkgs> {});
stdenv.mkDerivation {
name = "simple";
buildInputs = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC ghc
];
buildTools = [ haskellPackages.alex haskellPackages.happy ];
libraryHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
libraryToolDepends = [ haskellPackages.alex haskellPackages.happy haskellPackages.hpack ];
executableHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
executableToolDepends = [ haskellPackages.alex haskellPackages.happy ];
testHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
testToolDepends = [ haskellPackages.alex haskellPackages.happy ];
}
And in stack.yaml, set
nix:
enable: true
shell-file: shell.nix
However, when I type "stack build," alex, etc are not found. I can add all the package requirements manually to the stack.yaml file (the dependency requirements are already set in the package.yaml/package.cabal file). However, the library versions on nixpkgs satisfy the dependency requirements in the package.yaml file, so I'm curious, is it supported to tell stack to just use the versions of (Haskell) libraries provided by nixpkgs? If not, are there any other options for building a Haskell project using dependencies provided by nixpkgs whenever they are available?
This is not a real answer to the question, but it's a start of a workaround which I'll post a separate question about.
In Nix, you can use haskellPackages.ghcWithPackages (package-name), to obtain a ghc together with packages that ghc knows about. E.g.
{ pkgs ? import <nixpkgs> {}}:
let
# get ghc with libraries bundled
ghc = pkgs.haskellPackages.ghcWithPackages (p:[
p.array
p.base
...]);
...
If you then run nix-shell shell.nix and then cabal build, it will build the package using the dependencies pulled in from Nix, and everything seems to work as I was hoping. With two exceptions: alex and happy. On v1-build, the nix-installed alex and happy are used, but on v2-build they are not (but this is out of scope for this question).
I have a stack-based project that depends on a couple C libraries. One of these C libraries, zlib, is available from a native NixOS package and I can put into the nix section of stack.yaml:
nix:
enable: true
packages:
- "zlib"
The other is not part of nixpkgs. The stack documentation suggests that the alternative to using the nix section in stack.yaml is to "write a shell.nix" without elaborating much.
So I wrote one, sticking to zlib as an example:
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
buildInputs = [
pkgs.pkgconfig
pkgs.zlib
pkgs.stack
];
}
This gives me a working pkg-config for zlib:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ pkg-config --modversion zlib
1.2.11
However, it doesn't appear to make stack able to find the library:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ stack build
zlib-0.6.2: configure
Progress 1/7
-- While building package zlib-0.6.2 using:
/home/exarkun/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1 configure --with-ghc=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc --with-ghc-pkg=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/pkgdb --libdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/lib --bindir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/bin --datadir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/share --libexecdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/libexec --sysconfdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/etc --docdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --htmldir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --haddockdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --dependency=base=base-4.12.0.0 --dependency=bytestring=bytestring-0.10.8.2 --extra-include-dirs=/nix/store/a54skdf3xksiqvcvr75bjpdl1jx8dgbk-gmp-6.1.2-dev/include --extra-include-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/include --extra-include-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/include --extra-include-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/include --extra-lib-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/lib --extra-lib-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/lib --extra-lib-dirs=/nix/store/kggcrzpa5hd41b7v60wa7xjkgjs43xsl-gmp-6.1.2/lib --extra-lib-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/lib
Process exited with code: ExitFailure 1
Logs have been written to: /home/exarkun/Work/PrivateStorage/PaymentServer/.stack-work/logs/zlib-0.6.2.log
Configuring zlib-0.6.2...
Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5: Missing dependency on a foreign
library:
* Missing (or bad) header file: zlib.h
* Missing (or bad) C library: z
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
For zlib, there's no particular need to go this route, but as far as I can tell, I cannot put my non-nixpkgs package into the nix.packages list in stack.yaml.
How do I get stack to find these libraries?
Apparently a case of end-of-the-day documentation-reading-fail. This morning, it didn't take long to find the stack documentation that explains how to use a custom shell.nix. The stack docs do elaborate on how this works, including an example which shows it's not at all like the shell I thought was expected:
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myEnv";
buildInputs = [ glpk pcre ];
}
Adapting my shell to use buildStackProject instead of mkShell:
{ ghc }:
with (import <nixpkgs> { overlays = [ (import ./overlay.nix) ]; });
haskell.lib.buildStackProject {
inherit ghc;
name = "PrivacyPass";
# extra-library made available via the overlay.
# overlay probably not strictly necessary here, either.
buildInputs = [ zlib extra-library ];
}
and then pointing stack at it in stack.yaml:
nix:
enable: true
shell-file: "stack-shell.nix"
results in a successful build.
I need to create the nix package from a project stack with postgres template.
Basically it is the following: I have a virtual machine with NixOS and I need to portlet my project to another one with NixOS. For this I will use the nix copy command, but before I need to somehow "install" my project so that it is in /nix/store.
I tried to do the same looking at another issue posted "Generating a Nix package from a stack project".
come on. What I did was:
cd /home/ProjetoApp
stack new TesteYesod yesod-postgres
cd TestYesod
stack init --resolver lts-11.22
stack install yesod-bin --install-ghc
I added in stack.yaml:
nix
pure: true
enable: true
packages: [postgresql]
So far so good. I can compile and execute only stack. Then I was trying to generate a nix package. In stack.yaml I did:
nix
pure: true
enable: true
shell-file: default.nix
And in default.nix I did:
{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="teste-yesod-1.0.0";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
If I do:
nix build
Occurs:
builder for '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod
1.0.0.drv' failed with exit code 1; last 5 log lines:
unpacking sources
unpacking source archive /nix/store/
n62hzn4igi1b7khksa6sp3cq8gk4h344-TesteYesod
source root is TestYesod
patching sources
configuring
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod-
1.0.0.drv' failed
If. How to instruct nix that the source is spread across multiple directories?
If you want a quick fix, the following should work:
{ nixpkgs ? import (builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/d42ef371c9b1b532400b0f2820885e575f4f1617.tar.gz";
sha256 = "0irb4zb6hdgaah238244jk2xf63xfb20xy3plb1194pd4xbgdr3r";
}) {}
, ghc ? nixpkgs.ghc
}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="TesteYesod";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
What's the problem? It seems that in the current 18.09 release, buildStackProject is broken. The non-existent error message is caused by a subtly failing preConfigure script, which contains the following:
addStackArgsHook = ''
for pkg in ''${pkgsHostHost[#]} ''${pkgsHostBuild[#]} ''${pkgsHostTarget[#]}
do
[ -d "$pkg/lib" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-lib-dirs=$pkg/lib"
[ -d "$pkg/include" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-include-dirs=$pkg/include"
done
'';
In your case, stack is the last dependency to be processed in the loop (and perhaps in all buildStackProject invocations in 18.09, I'm not sure), but it contains no /lib or /include directory, so the exit code with which the preConfigure script exits is 1, and so the whole build process quits. This is just because of the shorthand &&, it would work if there was a proper if.
It is however fixed in master already (https://github.com/NixOS/nixpkgs/pull/53618), so simply using a newer nixpkgs should fix that problem. Pinning the nixpkgs is something you want to do anyway if you want to use Nix's replicability guarantees to the fullest, as you can't know you're using the same nixpkgs commit, so you may be using different versions of system packages.
(If you want to know how I debugged this - it seemed the problem was in the preConfigure step, so I took a look at the code in generic-stack-builder.nix, saw that preConfigure was overridable, copy-pasted the code from there into default.nix and added a set -x at the top. That showed me the problem above, and prompted me to go look at that file in master to see if there weren't changes since 18.09. Another workaround could be simply adding true to the preConfigure script if it wasn't already fixed in master, but that wasn't necessary.)
(Also, I've opened https://github.com/NixOS/nixpkgs/issues/55548 to backport the fix to 18.09.)
I am trying to achieve a simple declarative package management solution.
I have a clean nixos installation on virtualbox (thanks to this vagrant solution )
I am following the following subsection in the nixos manual
I do
mkdir -p ~/.config/nixpkgs/
vi ~/.config/nixpkgs/config.nix
such that cat ~/.config/nixpkgs/config.nix would now give:
{
packageOverrides = pkgs: with pkgs; {
myPackages = pkgs.buildEnv {
name = "my-packages";
paths = [ aspell bc coreutils gdb ffmpeg nixUnstable emscripten jq nox silver-searcher ];
};
};
}
And now,
nix-env -iA nixpkgs.myPackages outputs:
error: attribute ‘nixpkgs’ in selection path ‘nixpkgs.myPackages’ not found
I have also tried
nix-env -f. -iA nixpkgs.myPackages
nix-env -iA nixos.myPackages
nix-env -f. -iA nixos.myPackages
And all fail similarly.
What is a simple example of a package declaration in nixos? In this case, it seems like it tries to achieve a certain set of packages installed for a given user . I would not mind a declaration that applies to all users, but would prefer something for a given user (for future use).
EDIT: Here is the outputs when trying to get nix-info
[vagrant#nixbox:~]$ nix-shell -p nix-info --run nix-info
error: undefined variable ‘nix-info’ at (string):1:66
(use ‘--show-trace’ to show detailed location information)
[vagrant#nixbox:~]$ nix-shell -p nix-info --run nix-info --show-trace
error: while evaluating the attribute ‘nativeBuildInputs’ of the derivation ‘shell’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/build-support/trivial-builders.nix:10:14:
while evaluating ‘optionals’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/lib/lists.nix:185:21, called from /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/stdenv/generic/default.nix:204:16:
while evaluating ‘chooseDevOutputs’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/lib/attrsets.nix:463:22, called from /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/stdenv/generic/default.nix:161:22:
undefined variable ‘nix-info’ at (string):1:66
[vagrant#nixbox:~]$
I was trying like this initially:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof
Then GHC told me
Main.hs:4:1: error:
Could not find module ‘Control.Lens’
Perhaps you haven't installed the profiling libraries for package ‘lens-4.15.4’?
Use -v to see a list of the files searched for.
Searching around the web I found this: https://github.com/NixOS/nixpkgs/issues/22340
So it seems I won't be able to download from the cache. But that's okay, if at least I can build the profiled variants locally.
Can I do that somehow simply by modifying the nix expression given to -p slightly?
Then at this point in writing this question, I remembered this resource: https://github.com/NixOS/nixpkgs/blob/bd6ba7/pkgs/development/haskell-modules/lib.nix
Where I found enableLibraryProfiling. So I tried:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec (haskell.lib.enableLibraryProfiling lens)])" -j4 --run 'ghc Main.hs -prof'
Which got me to a new error:
src/Control/Lens/Internal/Getter.hs:26:1: error:
Could not find module ‘Data.Functor.Contravariant’
Perhaps you haven't installed the profiling libraries for package ‘contravariant-1.4’?
Use -v to see a list of the files searched for.
|
26 | import Data.Functor.Contravariant
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
So if I could map over all packages to enableLibraryProfiling on them, then I guess this could work. But my nix knowledge doesn't quite extend that far at the moment. How could I do that? And is this even the correct path to pursue?
With further snooping around in nix-repl and some helpful pointers from ElvishJerricco on #reflex-frp at FreeNode, I was able to construct this, which seems to work:
$ nix-shell -p "(haskell.packages.ghc821.extend (self: super: {mkDerivation = expr: super.mkDerivation (expr // { enableLibraryProfiling = true; });})).ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof'
I figured out a simple approach that I'm working into a largest blog post on Haskell development using Nix. For now, here's the text of just the profiling section:
Nix makes this fairly easy. First, we add the following to a ~/.config/nixpkgs/config.nix:
{
packageOverrides = super: let self = super.pkgs; in
{
profiledHaskellPackages = self.haskellPackages.override {
overrides = self: super: {
mkDerivation = args: super.mkDerivation (args // {
enableLibraryProfiling = true;
});
};
};
};
}
Now in the project we want to profile, we create a new profiling-shell.nix:
let nixpkgs = import <nixpkgs> {};
orig = nixpkgs.pkgs.profiledHaskellPackages.callPackage ./default.nix {};
in (nixpkgs.pkgs.haskell.lib.doBenchmark orig).env
Almost identical to our normal shell.nix, except for the usage of profiledHaskellPackages, which we just defined globally. Now, an invocation of nix-shell profiling-shell.nix will rebuild every dependency in our project with profiling enabled. The first time this is done it will take quite a long time. Luckily this doesn't corrupt our Nix store - a vanilla nix-shell does seem to present us with our regular dependencies again, without redownloading or rebuilding.
WARNING: A nix-collect-garbage -d will wipe away all the custom-built libs from our Nix Store, and we'd have to build them again if they're needed.
If we're writing a library, the closest executable on hand that we could profile would be our benchmark suite. To do that:
Add -prof and -fprof-auto to our benchmark's GHC options
Regenerate default.nix
Enter our profiling shell: nix-shell profiling-shell.nix
cabal configure --enable-library-profiling --enable-benchmarks
cabal build
dist/build/projname/projname-bench +RTS -p
Look at the produced projname-bench.prof file
Based on the results, we can make code changes, remove the profiling options, regenerate default.nix, and benchmark as usual in our normal Nix Shell.