I am trying to achieve a simple declarative package management solution.
I have a clean nixos installation on virtualbox (thanks to this vagrant solution )
I am following the following subsection in the nixos manual
I do
mkdir -p ~/.config/nixpkgs/
vi ~/.config/nixpkgs/config.nix
such that cat ~/.config/nixpkgs/config.nix would now give:
{
packageOverrides = pkgs: with pkgs; {
myPackages = pkgs.buildEnv {
name = "my-packages";
paths = [ aspell bc coreutils gdb ffmpeg nixUnstable emscripten jq nox silver-searcher ];
};
};
}
And now,
nix-env -iA nixpkgs.myPackages outputs:
error: attribute ‘nixpkgs’ in selection path ‘nixpkgs.myPackages’ not found
I have also tried
nix-env -f. -iA nixpkgs.myPackages
nix-env -iA nixos.myPackages
nix-env -f. -iA nixos.myPackages
And all fail similarly.
What is a simple example of a package declaration in nixos? In this case, it seems like it tries to achieve a certain set of packages installed for a given user . I would not mind a declaration that applies to all users, but would prefer something for a given user (for future use).
EDIT: Here is the outputs when trying to get nix-info
[vagrant#nixbox:~]$ nix-shell -p nix-info --run nix-info
error: undefined variable ‘nix-info’ at (string):1:66
(use ‘--show-trace’ to show detailed location information)
[vagrant#nixbox:~]$ nix-shell -p nix-info --run nix-info --show-trace
error: while evaluating the attribute ‘nativeBuildInputs’ of the derivation ‘shell’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/build-support/trivial-builders.nix:10:14:
while evaluating ‘optionals’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/lib/lists.nix:185:21, called from /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/stdenv/generic/default.nix:204:16:
while evaluating ‘chooseDevOutputs’ at /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/lib/attrsets.nix:463:22, called from /nix/store/5cd5ac78p0yy47yi5ml12s690bl5lsh8-nixos-16.09.1389.d7a3160/nixos/pkgs/stdenv/generic/default.nix:161:22:
undefined variable ‘nix-info’ at (string):1:66
[vagrant#nixbox:~]$
Related
I want to build https://github.com/wallix/redemption - and for the first time ever, I see bjam as a tool. This project has a tools/bjam/user-config.jam file.
The problem is, I'm trying to build this with a "custom" (that is, not the system version of) g++, which I have here:
$ which arm-linux-gnueabihf-g++-10
/home/pi/opt/gcc-10.1.0/bin/arm-linux-gnueabihf-g++-10
$ arm-linux-gnueabihf-g++-10 --version | head -n1
arm-linux-gnueabihf-g++-10 (pi-raspberry) 10.1.0
$ /home/pi/opt/gcc-10.1.0/bin/arm-linux-gnueabihf-g++-10 --v
ersion | head -n1
arm-linux-gnueabihf-g++-10 (pi-raspberry) 10.1.0
I guess, this qualifies at least as the compiler existing, right?
Anyways - I tried first, without knowing any better:
$ bjam --version
Boost.Build 2015.07-git
$ bjam toolset=arm-linux-gnueabihf-gcc-10 linkflags=-static-libstdc++ exe libs
arm.jam: No such file or directory
/usr/share/boost-build/src/build/toolset.jam:43: in toolset.using
ERROR: rule "arm.init" unknown in module "toolset".
/usr/share/boost-build/src/build-system.jam:461: in process-explicit-toolset-requests
/usr/share/boost-build/src/build-system.jam:527: in load
/usr/share/boost-build/src/kernel/modules.jam:295: in import
/usr/share/boost-build/src/kernel/bootstrap.jam:139: in boost-build
/usr/share/boost-build/boost-build.jam:8: in module scope
Then I found Building boost with different gcc version which mentions:
I cross built Boost for an ARM toolchain using something like this:
echo "using gcc : arm-unknown-linux-gnueabi : /usr/local/arm/bin/g++ ; " >> tools/build/v2/user-config.jam
Ok, so by that logic, I try:
echo "using gcc : arm-unknown-linux-gnueabi : /home/pi/opt/gcc-10.1.0/bin/arm-linux-gnueabihf-g++-10 ; " >> tools/bjam/user-config.jam
... and then:
$ bjam toolset=gcc-arm-unknown-linux-gnueabi linkflags=-static-libstdc++ exe libs
/usr/share/boost-build/src/tools/gcc.jam:123: in gcc.init from module gcc
error: toolset gcc initialization:
error: version 'arm-unknown-linux-gnueabi' requested but 'g++-arm-unknown-linux-gnueabi' not found and version '6.3.0' of default 'g++' does not match
error: initialized from
/usr/share/boost-build/src/build/toolset.jam:43: in toolset.using from module toolset
/usr/share/boost-build/src/build-system.jam:461: in process-explicit-toolset-requests from module build-system
/usr/share/boost-build/src/build-system.jam:527: in load from module build-system
/usr/share/boost-build/src/kernel/modules.jam:295: in import from module modules
/usr/share/boost-build/src/kernel/bootstrap.jam:139: in boost-build from module
/usr/share/boost-build/boost-build.jam:8: in module scope from module
Well, I agree that "version '6.3.0' of default 'g++' does not match" -> but how on earth is "'g++-arm-unknown-linux-gnueabi' not found"? What is that absolute path /home/pi/opt/gcc-10.1.0/bin/arm-linux-gnueabihf-g++-10 doing in that entry in user-config.jam otherwise?
So - can I get a more verbose printout of what actually bjam does in finding my compiler? Or even better, how can I format my "custom gcc" entry in user-config.jam, so I can get bjam to compile whatever it has to, and I can happily forget that bjam exists?
EDIT: even the official documentation for successor to bjam states:
When using gcc, you first need to specify your cross compiler in user-config.jam (see the section called “Configuration”), for example:
using gcc : arm : arm-none-linux-gnueabi-g++ ;
After that, if the host and target os are the same, for example Linux, you can just request that this compiler version to be used:
b2 toolset=gcc-arm
Isn't that exactly what I'm doing? Why doesn't it work then?
Well, I found a bit of documentation in /usr/share/boost-build/src/tools/gcc.jam:
# Initializes the gcc toolset for the given version. If necessary, command may
# be used to specify where the compiler is located. The parameter 'options' is a
# space-delimited list of options, each one specified as
# <option-name>option-value. Valid option names are: cxxflags, linkflags and
# linker-type. Accepted linker-type values are aix, darwin, gnu, hpux, osf or
# sun and the default value will be selected based on the current OS.
# Example:
# using gcc : 3.4 : : <cxxflags>foo <linkflags>bar <linker-type>sun ;
Ok, so here I have a string, delimited with colon, the seconf field says "3.4", the third field is empty - so WHERE does the "command may be used to specify where the compiler is located" go - in second or third field?
Well, I managed to get it running, quite hacky - I added these statements to /usr/share/boost-build/src/tools/gcc.jam:
...
rule init ( version ? : command * : options * )
{
#1): use user-provided command
local tool-command = ;
ECHO notice: 1) user-provided command '$(command)' version '$(version)' options '$(options)' ;
if $(version) = "arm"
{
command = arm-linux-gnueabihf-g++-10 ;
}
if $(command)
{
tool-command = [ common.get-invocation-command-nodefault gcc : g++ :
$(command) ] ;
ECHO notice: tool-command 1) user-provided '$(command)' '$(tool-command)' ;
...
The printouts were like:
notice: 1) user-provided command version 'arm' options
notice: tool-command 1) user-provided 'arm-linux-gnueabihf-g++-10' 'arm-linux-gnueabihf-g++-10'
...
... both 'command' and 'options' there are empty - as if the line I added to user-config.jam does not get parsed beyond the two first fields.
So, since the second field ("arm") does get parsed, I simply added a conditional on it, and forced the use of the command - and now that passes.
Well, I wish bjam just worked, and I did not have to go through this ...
I was trying like this initially:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof
Then GHC told me
Main.hs:4:1: error:
Could not find module ‘Control.Lens’
Perhaps you haven't installed the profiling libraries for package ‘lens-4.15.4’?
Use -v to see a list of the files searched for.
Searching around the web I found this: https://github.com/NixOS/nixpkgs/issues/22340
So it seems I won't be able to download from the cache. But that's okay, if at least I can build the profiled variants locally.
Can I do that somehow simply by modifying the nix expression given to -p slightly?
Then at this point in writing this question, I remembered this resource: https://github.com/NixOS/nixpkgs/blob/bd6ba7/pkgs/development/haskell-modules/lib.nix
Where I found enableLibraryProfiling. So I tried:
nix-shell -p "haskell.packages.ghc821.ghcWithPackages (p: with p; [text hspec (haskell.lib.enableLibraryProfiling lens)])" -j4 --run 'ghc Main.hs -prof'
Which got me to a new error:
src/Control/Lens/Internal/Getter.hs:26:1: error:
Could not find module ‘Data.Functor.Contravariant’
Perhaps you haven't installed the profiling libraries for package ‘contravariant-1.4’?
Use -v to see a list of the files searched for.
|
26 | import Data.Functor.Contravariant
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
So if I could map over all packages to enableLibraryProfiling on them, then I guess this could work. But my nix knowledge doesn't quite extend that far at the moment. How could I do that? And is this even the correct path to pursue?
With further snooping around in nix-repl and some helpful pointers from ElvishJerricco on #reflex-frp at FreeNode, I was able to construct this, which seems to work:
$ nix-shell -p "(haskell.packages.ghc821.extend (self: super: {mkDerivation = expr: super.mkDerivation (expr // { enableLibraryProfiling = true; });})).ghcWithPackages (p: with p; [text hspec lens])" -j4 --run 'ghc Main.hs -prof'
I figured out a simple approach that I'm working into a largest blog post on Haskell development using Nix. For now, here's the text of just the profiling section:
Nix makes this fairly easy. First, we add the following to a ~/.config/nixpkgs/config.nix:
{
packageOverrides = super: let self = super.pkgs; in
{
profiledHaskellPackages = self.haskellPackages.override {
overrides = self: super: {
mkDerivation = args: super.mkDerivation (args // {
enableLibraryProfiling = true;
});
};
};
};
}
Now in the project we want to profile, we create a new profiling-shell.nix:
let nixpkgs = import <nixpkgs> {};
orig = nixpkgs.pkgs.profiledHaskellPackages.callPackage ./default.nix {};
in (nixpkgs.pkgs.haskell.lib.doBenchmark orig).env
Almost identical to our normal shell.nix, except for the usage of profiledHaskellPackages, which we just defined globally. Now, an invocation of nix-shell profiling-shell.nix will rebuild every dependency in our project with profiling enabled. The first time this is done it will take quite a long time. Luckily this doesn't corrupt our Nix store - a vanilla nix-shell does seem to present us with our regular dependencies again, without redownloading or rebuilding.
WARNING: A nix-collect-garbage -d will wipe away all the custom-built libs from our Nix Store, and we'd have to build them again if they're needed.
If we're writing a library, the closest executable on hand that we could profile would be our benchmark suite. To do that:
Add -prof and -fprof-auto to our benchmark's GHC options
Regenerate default.nix
Enter our profiling shell: nix-shell profiling-shell.nix
cabal configure --enable-library-profiling --enable-benchmarks
cabal build
dist/build/projname/projname-bench +RTS -p
Look at the produced projname-bench.prof file
Based on the results, we can make code changes, remove the profiling options, regenerate default.nix, and benchmark as usual in our normal Nix Shell.
I have installed GHC through Nix:
$ nix-env -i ghc
Then I have installed the aeson package:
$ nix-env -f "<nixpkgs>" -iA haskellPackages.aeson
And pointed GHC to the Nix package folder:
$ export GHC_PACKAGE_PATH=~/.nix-profile/lib/ghc-8.0.1/package.conf.d/
Which seems to work:
$ ghc-pkg list
/Users/zoul/.nix-profile/lib/ghc-8.0.1/package.conf.d
Cabal-1.24.0.0
array-0.5.1.1
…
But the aeson package is missing from the list of packages above and can’t be loaded. Even though there’s clearly something there:
$ ls /Users/zoul/.nix-profile/lib/ghc-8.0.1/ | grep ^ae
aeson-0.11.2.1
What am I doing wrong?
You cannot install Haskell libraries in Nix that way because the ghc compiler you're using does not search your user's profile for libraries. Consequently, installing a library there has no effect. This topic is explained in great detail in the Nixpkgs user manual. I'm citing the relevant bit from "8.5.2.2. How to install a compiler with libraries":
GHC expects to find all installed libraries inside of its own lib directory. This approach works fine on traditional Unix systems, but it doesn’t work for Nix, because GHC’s store path is immutable once it’s built. We cannot install additional libraries into that location. As a consequence, our copies of GHC don’t know any packages except their own core libraries, like base, containers, Cabal, etc.
We can register additional libraries to GHC, however, using a special build function called ghcWithPackages. That function expects one argument: a function that maps from an attribute set of Haskell packages to a list of packages, which determines the libraries known to that particular version of GHC. For example, the Nix expression ghcWithPackages (pkgs: [pkgs.mtl]) generates a copy of GHC that has the mtl library registered in addition to its normal core packages:
$ nix-shell -p "haskellPackages.ghcWithPackages (pkgs: [pkgs.mtl])"
[nix-shell:~]$ ghc-pkg list mtl
/nix/store/zy79...-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d:
mtl-2.2.1
This function allows users to define their own development environment by means of an override. After adding the following snippet to ~/.nixpkgs/config.nix,
{
packageOverrides = super: let self = super.pkgs; in
{
myHaskellEnv = self.haskell.packages.ghc7102.ghcWithPackages
(haskellPackages: with haskellPackages; [
# libraries
arrows async cgi criterion
# tools
cabal-install haskintex
]);
};
}
it’s possible to install that compiler with nix-env -f "<nixpkgs>" -iA myHaskellEnv.
I can find haskell packages via nix-env:
nix-enf -f "<nixpkgs>" -qaP -A haskellPackages
...
yet when I try and install them using environment.systemPackages in /etc/nixos/configuration.nix:
environment.systemPackages = with pkgs; [
haskellPackages.haskellPlatform
];
I get the error:
error: attribute 'haskellPlatform' missing, at /etc/nixos/configuration.nix:54:5
Any ideas? I am intending on using xmonad and want to do haskell dev'.
haskellPlatform has been removed with the update to GHC 7.8.2, see the mailing list thread for more details.
You should be able to install any package found via nix-enf -f "<nixpkgs>" -qaP -A haskellPackages to environment.systemPackages.
Searching for haskellPackages.haskellPlatform with nix-env should result in the same kind of error that you get during the rebuild:
nix-env -f "<nixpkgs>" -qaP -A haskellPackages.haskellPlatform
error: attribute ‘haskellPlatform’ in selection path ‘haskellPackages.haskellPlatform’ not found
Any valid haskell package can be installed in the environment, for example xmobar:
environment.systemPackages = with pkgs; [
haskellPackages.xmobar
];
I'm referring here to the page: Haskell - Nix Wiki, both the heading Local use via Nixpkgs config and System-wide use via NixOS config.
Here are the steps to replicate the problem: I download the NixOS-14.04 virtual appliance, load it in Virtual box, and try the System-wide use, so that my configuration.nix is:
{ config, pkgs, ... }:
{
imports = [ <nixos/modules/installer/virtualbox-demo.nix> ];
environment.systemPackages = [
# other system packages
(pkgs.haskellPackages.ghcWithPackages (self : [
self.haskellPlatform
]))
];
}
This is the output of nixos-rebuild test --show-trace -v
When I try the local option setting .nixpkgs/config.nix, here is the output of $ nix-env -iA nixos.pkgs.hsEnv.
The errors seem to me very similar, and so I suspect there may be an error in ghcWithPackages .nix expression. Do you have any suggestion?
This is a known bug in ghcWithPackages. See https://github.com/NixOS/nixpkgs/issues/1438 for more information.
In short, the fix is to replace ghcWithPackages with ghcWithPackagesOld.