I'm looking for something similar to this:
shell.nix
{ pkgs ? import <nixpkgs> {} }:
let
threadscope = ...
speedscope = ...
in
stdenv.mkDerivation {
name = "haskell-env";
nativeBuildInputs = [
# Haskell tools
ghcid ghc
# Profiling tools
threadscope
speedscope
];
}
Then I can use nix-shell:
% nix-shell
...
nix-shell % threadscope <some-commands>
nix-shell % speedscope <some-commands>
Do you usually just build from source from github / hackage / stackage or use pkgs.haskellPackages.<my-package>?
I usually take tools from haskellPackages.
If you only want a shell with tools, you can use an expression like this.
{ pkgs ? import <nixpkgs> {} }:
let
# Select a package set. haskellPackages is recommended, but
# if you need a different ghc, use pkgs.haskell.packages.ghc8102 for example
haskellPackages = pkgs.haskellPackages;
in
pkgs.mkShell {
nativeBuildInputs = [
# Haskell tools
haskellPackages.ghc
haskellPackages.ghcid
# Profiling tools
haskellPackages.threadscope
# pkgs.speedscope (not packaged yet)
];
}
If you want a shell with specific precompiled libraries, you can use the ghcWithPackages function instead of ghc. If you want precompiled dependencies for your local Haskell package(s), you can replace pkgs.mkShell by the haskellPackages.shellFor function.
If you'd like to package speedscope, the Nixpkgs manual documents the packaging process for node packages
Related
I'm developing a Haskell program that will eventually have some non-Haskell dependencies, and I heard that the nix integration in stack is one way to accomplish this. However, I'm now wondering if it's possible to use nix to install Haskell dependencies too since all the libraries I'm using are on nix, and currently I have only Haskell dependencies. I tried to make a custom "shell.nix" file that describes all the Haskell dependencies I have as follows:
with (import <nixpkgs> {});
stdenv.mkDerivation {
name = "simple";
buildInputs = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC ghc
];
buildTools = [ haskellPackages.alex haskellPackages.happy ];
libraryHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
libraryToolDepends = [ haskellPackages.alex haskellPackages.happy haskellPackages.hpack ];
executableHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
executableToolDepends = [ haskellPackages.alex haskellPackages.happy ];
testHaskellDepends = [
haskellPackages.array haskellPackages.base haskellPackages.bound
haskellPackages.containers haskellPackages.deriving-compat
haskellPackages.haskeline haskellPackages.logict haskellPackages.mtl
haskellPackages.text haskellPackages.unification-fd haskellPackages.alex
haskellPackages.happy haskellPackages.BNFC
];
testToolDepends = [ haskellPackages.alex haskellPackages.happy ];
}
And in stack.yaml, set
nix:
enable: true
shell-file: shell.nix
However, when I type "stack build," alex, etc are not found. I can add all the package requirements manually to the stack.yaml file (the dependency requirements are already set in the package.yaml/package.cabal file). However, the library versions on nixpkgs satisfy the dependency requirements in the package.yaml file, so I'm curious, is it supported to tell stack to just use the versions of (Haskell) libraries provided by nixpkgs? If not, are there any other options for building a Haskell project using dependencies provided by nixpkgs whenever they are available?
This is not a real answer to the question, but it's a start of a workaround which I'll post a separate question about.
In Nix, you can use haskellPackages.ghcWithPackages (package-name), to obtain a ghc together with packages that ghc knows about. E.g.
{ pkgs ? import <nixpkgs> {}}:
let
# get ghc with libraries bundled
ghc = pkgs.haskellPackages.ghcWithPackages (p:[
p.array
p.base
...]);
...
If you then run nix-shell shell.nix and then cabal build, it will build the package using the dependencies pulled in from Nix, and everything seems to work as I was hoping. With two exceptions: alex and happy. On v1-build, the nix-installed alex and happy are used, but on v2-build they are not (but this is out of scope for this question).
I have a stack-based project that depends on a couple C libraries. One of these C libraries, zlib, is available from a native NixOS package and I can put into the nix section of stack.yaml:
nix:
enable: true
packages:
- "zlib"
The other is not part of nixpkgs. The stack documentation suggests that the alternative to using the nix section in stack.yaml is to "write a shell.nix" without elaborating much.
So I wrote one, sticking to zlib as an example:
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
buildInputs = [
pkgs.pkgconfig
pkgs.zlib
pkgs.stack
];
}
This gives me a working pkg-config for zlib:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ pkg-config --modversion zlib
1.2.11
However, it doesn't appear to make stack able to find the library:
[nix-shell:~/Work/PrivateStorage/PaymentServer]$ stack build
zlib-0.6.2: configure
Progress 1/7
-- While building package zlib-0.6.2 using:
/home/exarkun/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1 configure --with-ghc=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc --with-ghc-pkg=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/pkgdb --libdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/lib --bindir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/bin --datadir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/share --libexecdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/libexec --sysconfdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/etc --docdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --htmldir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --haddockdir=/home/exarkun/.stack/snapshots/x86_64-linux-nix/lts-14.1/8.6.5/doc/zlib-0.6.2 --dependency=base=base-4.12.0.0 --dependency=bytestring=bytestring-0.10.8.2 --extra-include-dirs=/nix/store/a54skdf3xksiqvcvr75bjpdl1jx8dgbk-gmp-6.1.2-dev/include --extra-include-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/include --extra-include-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/include --extra-include-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/include --extra-lib-dirs=/nix/store/br7kq0kvbn73rhzl17js0w3pprphhzv1-git-2.19.2/lib --extra-lib-dirs=/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/lib --extra-lib-dirs=/nix/store/kggcrzpa5hd41b7v60wa7xjkgjs43xsl-gmp-6.1.2/lib --extra-lib-dirs=/nix/store/zfpm9bai9gj8vs09s2i2gkhvgsjkx13z-ghc-8.6.5/lib
Process exited with code: ExitFailure 1
Logs have been written to: /home/exarkun/Work/PrivateStorage/PaymentServer/.stack-work/logs/zlib-0.6.2.log
Configuring zlib-0.6.2...
Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5: Missing dependency on a foreign
library:
* Missing (or bad) header file: zlib.h
* Missing (or bad) C library: z
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
For zlib, there's no particular need to go this route, but as far as I can tell, I cannot put my non-nixpkgs package into the nix.packages list in stack.yaml.
How do I get stack to find these libraries?
Apparently a case of end-of-the-day documentation-reading-fail. This morning, it didn't take long to find the stack documentation that explains how to use a custom shell.nix. The stack docs do elaborate on how this works, including an example which shows it's not at all like the shell I thought was expected:
{ghc}:
with (import <nixpkgs> {});
haskell.lib.buildStackProject {
inherit ghc;
name = "myEnv";
buildInputs = [ glpk pcre ];
}
Adapting my shell to use buildStackProject instead of mkShell:
{ ghc }:
with (import <nixpkgs> { overlays = [ (import ./overlay.nix) ]; });
haskell.lib.buildStackProject {
inherit ghc;
name = "PrivacyPass";
# extra-library made available via the overlay.
# overlay probably not strictly necessary here, either.
buildInputs = [ zlib extra-library ];
}
and then pointing stack at it in stack.yaml:
nix:
enable: true
shell-file: "stack-shell.nix"
results in a successful build.
I need to create the nix package from a project stack with postgres template.
Basically it is the following: I have a virtual machine with NixOS and I need to portlet my project to another one with NixOS. For this I will use the nix copy command, but before I need to somehow "install" my project so that it is in /nix/store.
I tried to do the same looking at another issue posted "Generating a Nix package from a stack project".
come on. What I did was:
cd /home/ProjetoApp
stack new TesteYesod yesod-postgres
cd TestYesod
stack init --resolver lts-11.22
stack install yesod-bin --install-ghc
I added in stack.yaml:
nix
pure: true
enable: true
packages: [postgresql]
So far so good. I can compile and execute only stack. Then I was trying to generate a nix package. In stack.yaml I did:
nix
pure: true
enable: true
shell-file: default.nix
And in default.nix I did:
{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="teste-yesod-1.0.0";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
If I do:
nix build
Occurs:
builder for '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod
1.0.0.drv' failed with exit code 1; last 5 log lines:
unpacking sources
unpacking source archive /nix/store/
n62hzn4igi1b7khksa6sp3cq8gk4h344-TesteYesod
source root is TestYesod
patching sources
configuring
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/rckhmkgrkb6nzn7dkqqldfdm8cilpya2-teste-yesod-
1.0.0.drv' failed
If. How to instruct nix that the source is spread across multiple directories?
If you want a quick fix, the following should work:
{ nixpkgs ? import (builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/d42ef371c9b1b532400b0f2820885e575f4f1617.tar.gz";
sha256 = "0irb4zb6hdgaah238244jk2xf63xfb20xy3plb1194pd4xbgdr3r";
}) {}
, ghc ? nixpkgs.ghc
}:
with nixpkgs;
haskell.lib.buildStackProject {
name ="TesteYesod";
src = ./.;
buildInputs = [ postgresql ];
inherit ghc;
}
What's the problem? It seems that in the current 18.09 release, buildStackProject is broken. The non-existent error message is caused by a subtly failing preConfigure script, which contains the following:
addStackArgsHook = ''
for pkg in ''${pkgsHostHost[#]} ''${pkgsHostBuild[#]} ''${pkgsHostTarget[#]}
do
[ -d "$pkg/lib" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-lib-dirs=$pkg/lib"
[ -d "$pkg/include" ] && \
export STACK_IN_NIX_EXTRA_ARGS+=" --extra-include-dirs=$pkg/include"
done
'';
In your case, stack is the last dependency to be processed in the loop (and perhaps in all buildStackProject invocations in 18.09, I'm not sure), but it contains no /lib or /include directory, so the exit code with which the preConfigure script exits is 1, and so the whole build process quits. This is just because of the shorthand &&, it would work if there was a proper if.
It is however fixed in master already (https://github.com/NixOS/nixpkgs/pull/53618), so simply using a newer nixpkgs should fix that problem. Pinning the nixpkgs is something you want to do anyway if you want to use Nix's replicability guarantees to the fullest, as you can't know you're using the same nixpkgs commit, so you may be using different versions of system packages.
(If you want to know how I debugged this - it seemed the problem was in the preConfigure step, so I took a look at the code in generic-stack-builder.nix, saw that preConfigure was overridable, copy-pasted the code from there into default.nix and added a set -x at the top. That showed me the problem above, and prompted me to go look at that file in master to see if there weren't changes since 18.09. Another workaround could be simply adding true to the preConfigure script if it wasn't already fixed in master, but that wasn't necessary.)
(Also, I've opened https://github.com/NixOS/nixpkgs/issues/55548 to backport the fix to 18.09.)
I was trying like this initially:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof
Then GHC told me
Main.hs:4:1: error:
Could not find module ‘Control.Lens’
Perhaps you haven't installed the profiling libraries for package ‘lens-4.15.4’?
Use -v to see a list of the files searched for.
Searching around the web I found this: https://github.com/NixOS/nixpkgs/issues/22340
So it seems I won't be able to download from the cache. But that's okay, if at least I can build the profiled variants locally.
Can I do that somehow simply by modifying the nix expression given to -p slightly?
Then at this point in writing this question, I remembered this resource: https://github.com/NixOS/nixpkgs/blob/bd6ba7/pkgs/development/haskell-modules/lib.nix
Where I found enableLibraryProfiling. So I tried:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec (haskell.lib.enableLibraryProfiling lens)])" -j4 --run 'ghc Main.hs -prof'
Which got me to a new error:
src/Control/Lens/Internal/Getter.hs:26:1: error:
Could not find module ‘Data.Functor.Contravariant’
Perhaps you haven't installed the profiling libraries for package ‘contravariant-1.4’?
Use -v to see a list of the files searched for.
|
26 | import Data.Functor.Contravariant
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
So if I could map over all packages to enableLibraryProfiling on them, then I guess this could work. But my nix knowledge doesn't quite extend that far at the moment. How could I do that? And is this even the correct path to pursue?
With further snooping around in nix-repl and some helpful pointers from ElvishJerricco on #reflex-frp at FreeNode, I was able to construct this, which seems to work:
$ nix-shell -p "(haskell.packages.ghc821.extend (self: super: {mkDerivation = expr: super.mkDerivation (expr // { enableLibraryProfiling = true; });})).ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof'
I figured out a simple approach that I'm working into a largest blog post on Haskell development using Nix. For now, here's the text of just the profiling section:
Nix makes this fairly easy. First, we add the following to a ~/.config/nixpkgs/config.nix:
{
packageOverrides = super: let self = super.pkgs; in
{
profiledHaskellPackages = self.haskellPackages.override {
overrides = self: super: {
mkDerivation = args: super.mkDerivation (args // {
enableLibraryProfiling = true;
});
};
};
};
}
Now in the project we want to profile, we create a new profiling-shell.nix:
let nixpkgs = import <nixpkgs> {};
orig = nixpkgs.pkgs.profiledHaskellPackages.callPackage ./default.nix {};
in (nixpkgs.pkgs.haskell.lib.doBenchmark orig).env
Almost identical to our normal shell.nix, except for the usage of profiledHaskellPackages, which we just defined globally. Now, an invocation of nix-shell profiling-shell.nix will rebuild every dependency in our project with profiling enabled. The first time this is done it will take quite a long time. Luckily this doesn't corrupt our Nix store - a vanilla nix-shell does seem to present us with our regular dependencies again, without redownloading or rebuilding.
WARNING: A nix-collect-garbage -d will wipe away all the custom-built libs from our Nix Store, and we'd have to build them again if they're needed.
If we're writing a library, the closest executable on hand that we could profile would be our benchmark suite. To do that:
Add -prof and -fprof-auto to our benchmark's GHC options
Regenerate default.nix
Enter our profiling shell: nix-shell profiling-shell.nix
cabal configure --enable-library-profiling --enable-benchmarks
cabal build
dist/build/projname/projname-bench +RTS -p
Look at the produced projname-bench.prof file
Based on the results, we can make code changes, remove the profiling options, regenerate default.nix, and benchmark as usual in our normal Nix Shell.
I am trying to use nix on ubuntu, with XMonad as my window manager.
I have this working well on one host using nixOS, but I have a second device that isn't yet ready for nixOS. nix on top of Ubuntu is mostly working well there, but xmonad cannot find contributory libraries.
The relevant packages are installed:
$ nix-env -q | grep xmonad
xmonad-0.13
xmonad-contrib-0.13
xmonad-extras-0.12.1
But recompiling my xmonad.hs, it cannot find the contrib libs:
$ xmonad --recompile
Error detected while loading xmonad configuration file: /home/martyn/.xmonad/xmonad.hs
xmonad.hs:32:1: error:
Failed to load interface for ‘XMonad.Layout.NoBorders’
Use -v to see a list of the files searched for.
...
Please check the file for errors.
The relevant files are installed:
$ ls /nix/store/*xmonad-contrib*/lib/**/NoBorders*
/nix/store/4xrrwsm6362xkn9jn1b17kd891kv9z3a-xmonad-contrib-0.13/lib/ghc-8.0.2/xmonad-contrib-0.13/XMonad/Actions/NoBorders.dyn_hi
/nix/store/4xrrwsm6362xkn9jn1b17kd891kv9z3a-xmonad-contrib-0.13/lib/ghc-8.0.2/xmonad-contrib-0.13/XMonad/Actions/NoBorders.hi
/nix/store/4xrrwsm6362xkn9jn1b17kd891kv9z3a-xmonad-contrib-0.13/lib/ghc-8.0.2/xmonad-contrib-0.13/XMonad/Layout/NoBorders.dyn_hi
/nix/store/4xrrwsm6362xkn9jn1b17kd891kv9z3a-xmonad-contrib-0.13/lib/ghc-8.0.2/xmonad-contrib-0.13/XMonad/Layout/NoBorders.hi
By adding xmonad-contrib to my nixpkgs config.nix, I have gotten these libs added to the ghc package registry:
$ cat ~/.config/nixpkgs/config.nix
with (import <nixpkgs> {});
{
packageOverrides = pkgs: with pkgs; {
myHaskellEnv = pkgs.haskellPackages.ghcWithPackages (haskellPackages: with haskellPackages; [ xmonad-contrib ]);
};
}
$ nix-env -iA nixpkgs.myHaskellEnv
$ ghc-pkg list | grep xmonad
xmonad-0.13
xmonad-contrib-0.13
$
with this, this ghc(i) works well:
$ /nix/store/7mkxsq7ydqcgnjbs59v1v47wfxpwrav5-ghc-8.0.2-with-packages/bin/ghc ~/.xmonad/xmonad.hs
[1 of 1] Compiling Main ( /home/martyn/.xmonad/xmonad.hs, /home/martyn/.xmonad/xmonad.o ) [flags changed]
Linking /home/martyn/.xmonad/xmonad ...
But even the version of xmonad in that dir cannot find the libs:
$ /nix/store/7mkxsq7ydqcgnjbs59v1v47wfxpwrav5-ghc-8.0.2-with-packages/bin/xmonad --recompile
Error detected while loading xmonad configuration file: /home/martyn/.xmonad/xmonad.hs
xmonad.hs:32:1: error:
Failed to load interface for ‘XMonad.Layout.NoBorders’
Use -v to see a list of the files searched for.
I can work around this by compiling using the ghc as above, and moving the output by hand to ~/.xmonad/xmonad-x86_64-linux, and running that. But this is a wee bit hacky, and surely shouldn't be necessary?
A friend solved this for me offline, I reproduce this here for others with the same issue.
Essentially, we need to use xmonad-with-packages, and list the packages, rather than ghc-with-packages.
To achieve this, we provide our own xmonad, referenced from within ~/.nixpkgs/config.nix:
{
packageOverrides = pkgs_: with pkgs_; {
xmonad = import ./xmonad { nixpkgs = pkgs_; };
};
}
And fill out ~/.nixpkgs/xmonad/default.nix thus:
{ nixpkgs ? import <nixpkgs> {} }:
nixpkgs.xmonad-with-packages.override {
packages = hPkgs: with hPkgs; [ xmonad-contrib ];
}
This installs an xmonad that knows where to find its libraries, and everything's good!