Or, the goal: How can I take a single package from Nix unstable in a declarative manner?
I'm new to NixOS and currently trying to install a newer version of Consul than the default 0.5.2 of my NixOS version (latest stable). I'm attempting this by overriding the derivative in my /etc/nix/configuration.nix.
I'd like to keep running stable, but I found unstable had the version of Consul that I wanted (0.7.0) already, and so I decided to use this package's attributes as a starting point to override https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/consul/default.nix
I copied it for most part into my configuration.nix, here are the relevant sections:
nixpkgs.config.packageOverrides = pkgs: rec {
consul = pkgs.lib.overrideDerivation pkgs.consul (attrs: rec {
version = "0.7.0";
name = "consul-${version}";
rev = "v${version}";
goPackagePath = "github.com/hashicorp/consul";
src = pkgs.fetchFromGitHub {
owner = "hashicorp";
repo = "consul";
inherit rev;
sha256 = "04h5y5vixjh9np9lsrk02ypbqwcq855h7l1jlnl1vmfq3sfqjds7";
};
# Keep consul.ui for backward compatability
passthru.ui = pkgs.consul-ui;
});
};
environment.systemPackages = with pkgs; [
vim
which
telnet
consul-ui
consul-alerts
consul-template
consul
];
I'm running nix-build (Nix) 1.11.2 which throws:
$ nixos-rebuild switch
building Nix...
building the system configuration...
error: cannot coerce a set to a string, at /etc/nixos/configuration.nix:19:7
(use ‘--show-trace’ to show detailed location information)
When I look at line 19 it's where name is set to "consul-${version}".
Why there is type-coercion going on here? Any tips will be greatly appreciated!
I'm also wondering if there is a better way to run just a single package in unstable, yet doing so declaratively from configuration.nix, rather than imperatively?
To add to what Rok said:
Which should point you that an error actually happens at passthru, line. If you comment it out it will probably build. I'm assuming some recursive calls are at play here and error occurs when it tries to evaluate consul/consul-ui packages.
If you're just starting out, you can safely ignore what follows and perhaps come back to it if/when you're curious about the nitty-gritty.
The problem here is that overrideDerivation is a kind of low-level approach to overriding things. Behind stdenv.mkDerivation, we have a much smaller primitive function called derivation. The derivation function takes some attributes and (more or less -- see the docs for the finer details) just passes those attributes as environment variables during the build. The stdenv.mkDerivation function, on the other hand, has a whole bunch of smarts layered on top that massages the attributes given to it before passing them onto derivation -- and in some cases, as is the case with passthru, it doesn't pass the attribute to derivation at all.
Back to overrideDerivation: it takes the final, tweaked attributes that stdenv.mkDerivation would pass to derivation, and just before that happens it allows you to override those attributes with the function you give it (e.g. that implies that, at that point, passthru has already been removed). When your function adds a passthru, that makes its way into derivation, which then wants to coerce the value of passthru into a string so it can make passthru an environment variable during the build; however, because passthru now points at a attribute-set, and such coercion isn't supported, Nix then complains.
So this sort of puts us in an odd situation. To illustrate, I'll copy the source for the consul package here:
{ stdenv, lib, buildGoPackage, consul-ui, fetchFromGitHub }:
buildGoPackage rec {
name = "consul-${version}";
version = "0.6.4";
rev = "v${version}";
goPackagePath = "github.com/hashicorp/consul";
src = fetchFromGitHub {
owner = "hashicorp";
repo = "consul";
inherit rev;
sha256 = "0p6m2rl0d30w418n4fzc4vymqs3vzfa468czmy4znkjmxdl5vp5a";
};
# Keep consul.ui for backward compatability
passthru.ui = consul-ui;
}
(Note that buildGoPackage is a wrapper around stdenv.mkDerivation.)
You might be familiar with e.g. consul.override, which allows you to supply different inputs (e.g. maybe a different version of consul-ui, or buildGoPackage), but it doesn't allow you to override things that aren't inputs (e.g. src, passthru, etc). Meanwhile, overrideDerivation allows you to modify the attrs given to derivation, but not the ones given to stdenv.mkDerivation. Ideally there would be something in-between, that would allow for manipulating the attrs given to stdenv.mkDerivation, and it so happens that there's a PR open to address this:
https://github.com/NixOS/nixpkgs/pull/18660
Welcome to Nix/NixOS :)
Whenever you need to know more about the error you can use --show-trace and that would give you more verbose error. In your case you would see something like
error: while evaluating the attribute ‘passthru’ of the derivation ‘consul-0.7.0’ at /home/rok/tmp/consul.nix:6:3:
cannot coerce a set to a string, at /home/rok/tmp/consul.nix:6:3
Which should point you that an error actually happens at passthru, line. If you comment it out it will probably build. I'm assuming some recursive calls are at play here and error occurs when it tries to evaluate consul/consul-ui packages.
As for overriding only one package from unstable channel something like this is needed
let
unstable_pkgs = import ./path/to/unstabe/nixpkgs {};
# or
# unstable_pkgs = import (pkgs.fetchFromGitHub {...}) {};
in {
...
nixpkgs.config.packageOverrides = pkgs: rec {
consul = unstable_pkgs.consul;
};
...
}
I haven't try the above, but I'm assuming it will work.
Related
A NixOS configuration is built using the /etc/nixos/configuration.nix file. This configuration has a nix.package property.
In an NixOS instance, I want to print the version/hash (i.e., unique identifier) of the nix.package object that has been used in building the current instance. Ideally, this should be stored inside a lockfile, but I don't believe the current version of nixos-rebuild uses those.
Should this not be possible, can I explicitly store this hash somewhere during the build process by modifying my /etc/nixos/configuration.nix?
Yes, you can access this attribute via NixOS' config parameter and use it in your configuration, or as part of a package.
For example, this module causes the version and the store path to be written to files in /etc upon activation.
{ config, lib, ... }:
{
config = {
environment.etc."x-nix-version".text =
config.nix.package.version;
environment.etc."x-nix-path".text =
"${config.nix.package}";
};
}
Alternatively, you can extract it from a potentially not yet built configuration using the nixos-option command or nix repl '<nixpkgs/nixos>'.
I post here a specific problem using a library (grim) in Nim, but the underlying concepts are still not super clear to me so I appreciate a solution coming with an explanation.
I would like to make a procedure returning a node. The example below is not really useful but makes the point: I want to return node but I apparently don't know what type it is.
import grim
import sequtils
proc get_a_node_with_label(graph: Graph, label: string): Node =
for node in graph.nodes:
if node.label == label:
return node
var g = newGraph("graph")
let n1 = g.addNode("n1", %(Name: "first"))
let n2 = g.addNode("n2", %(Name: "second"))
var aNode = get_a_node_with_label(g, "n2")
i get an Error: undeclared identifier: 'Node', but the type of "node" in the loop is "Node", if I echo node.type.
How should I deal with types on this occasion? What output should I declare in the procedure?
Thanks
Andrea
PS: I apologize if the question is not well asked, and I'm happy to improve it with your guidance.
You probably installed the grim library through nimble install grim. That gave you the grim-0.2.0, released early this year. The point is that Node was private in that release, so your code cannot access it.
You can opt to install the latest code, which at some point this year made Node and others public, with:
$ nimble uninstall grim
$ nimble install grim##devel
Or you can make the object public in your computer, editing (probably) ~/.nimble/pkgs/grim-0.2.0/grim/graph.nim:
30 Node = ref object
to
30 Node* = ref object
The former includes the latest code, and includes 40ish commits. On the downside, your build will be hard to reproduce, because you cannot pin the grim version.
The later should allow you to compile locally, but you will run into problems if you intend to distribute your code (i.e. forcing you or your users to patch the grim source).
You could also open an issue at the github repo, asking the author to tag a new version.
You can get objects which class is Node (aka Node objects), but you cannot write "Node" in your code, and the only way of creating a Node object is through code that has access to the Node private class (i.e. in the same file). It is usually some kind of newNode or getNode.
So you could get a Node inside your code, and pass it around, but cannot write "Node". E.g.
import grim
var g = newGraph("graph")
let n1 = g.addNode("n1", %(Name: "first"))
# This works happily
let node = g.node(n1) # This assigns a Node object to "node"
echo node # This passes the Node object to a $ proc.
# This fails to compile, albeit being functionally the same code,
# because your program doesn't know what "Node" is.
let node1: Node = g.node(n1)
Let's say I have my NixOS configuration.nix set up as follows:
{config, pkgs, ...}:
{
services.openssh.enable = true;
}
I now want to have a second file called networking.nix which sets my hostname based on an argument.
{config, pkgs, hostname, ...}:
{
networking.hostName = hostname
}
Is this possible? How can I include the file. I already tried doing it by using imports = [ ./networking.nix { hostname = "helloworld"; } ]; but that didn't work.
Thanks.
A 'NixOS configuration file' is simply a module that doesn't define options, so there is really no distinction. A configuration.nix file is just a module, and typically it does not define any options, so it can be written in the abbreviated form.
Defining options is the normal way for NixOS modules to pass information around, so that's the most idiomatic way to go about.
However, if you really must, for some very special reason, because you're doing very unusual things with NixOS, you can put arbitrary functions in imports. But you shouldn't, because it doesn't work well with the module system's custom error messages and potentially other aspects that rely on knowing where a module is defined. If you do so, do make sure it is an actual function. In your case, that would imply modifying the first line of networking.nix to make it a curried function:
hostname: {config, pkgs, ...}:
Not very pretty in my opinion. Although it is very explicit about what is going on, it deviates from what is to be expected of a NixOS module.
I encountered this problem today and came up with a fairly simple solution recommended in the manual.
foobar.nix
{ lib, withFoo ? "bar", ... }:
# simple error checking to ensure garbage isn't passed in
assert lib.asserts.assertOneOf "withFoo" withFoo [
"bar"
"baz"
# other valid choices ...
];
{
# ...
}
configuration.nix
args#{ ... }:
{
imports = [
# ...
(
import ./foobar.nix (
args
// { withFoo = "baz"; }
)
)
# ...
];
}
This is ideal for 'one off' options in your configurations.
You should be able to use the _module.argsoption [1] to do that. So your configuration.nix would be like:
{config, pkgs, ...}:
{
_module.args.hostname = "ahostname";
services.openssh.enable = true;
}
However where the values are very simple it will probably be much easier to just set them directly, e.g. just define networking.hostname in configuration.nix. This section of the manual re. merging and priorities may be helpful also [2].
Further discussion:
The value of _module.args is indeed applied to all imported configurations (though the value will only be used in modules that directly refer to it, such as the pkgs value, the ... represents all the values that aren't referenced).
For passing arguments to modules it seems a good approach to me, but from your comments perhaps a different approach might be more suitable.
Another solution could be to flip the relationship in the imports: rather than a single common config that passes multiple different arguments instead multiple different configs import the common configuration. E.g.
$cat ./common.nix
{ services.openssh.enable = true; }
$cat ./ahostname.nix
{ imports = [ ./common.nix ]; networking.hostname = "ahostname"; }
The NixOS config in this Reddit comment looks like it uses this approach. There are quite a few other NixOS configurations that people have shared publicly online so you might find some useful ideas there. The points in the answer from Robert Hensing are very useful to bear in mind as well.
However it's hard to say what might be a better solution in your case without knowing a bit more about the context in which you want to use it. You could create a new SO question with some more information on that which might make it easier to see a more appropriate solution.
I currently have a project in a loose ES6 module format and my database connection is hard coded. I am wanting to turn this into an npm module and am now facing the issue of how to best allow the end user to configure the code. My first attempt was to rewrite it as classes to be instantiated but it is making the use of the code more convoluted than before so am looking at alternatives. I am exploring my configuration options. It looks like writing to the process env would be the way but I am pondering potential issues, no-nos and other options I have not considered.
Is having the user write config to process env an acceptable method of configuring an npm module? It's a bit like a global write so am dealing with namespace considerations for one. I have also considered using package.json but that's not going to work for things like credentials. Likewise using an rc file is cumbersome. I have not found any docs on the proper methodology if any.
process.env['MY_COOL_MODULE_DB'] = ...
There are basically 5ish options as I see it:
hardcode - not an option
create a configured scope such as classes - what I have now and bleh
use a config such as node-config - not really a user friendly option for npm
store as globals/env. As suggested in comment I can wrap that process in an exported function and thereby ensure that I have a complex non collisive namespace while abstracting that from end user
Ask user to create some .rc file - I would if I was big time like AWS but not in this case.
I mention this npm use case but this really applies to the general challenge of configuring code that is exported as functions. I have use cases for classes but when the only need is creating a configured scope at the expense (in my case) of more complex code I am not sure its worth it.
Update I realize this is a bit of a discussion question but it's helped me wrap my brain around options. I think something like this:
// options.js
let options = {}
export function setOptions(o) { options = o }
export function getOptions(o) { return options }
Then have the user call setOptions() and call this getOptions internally. I realize that since Node requires the module just once that my options object will be kept configured as I pass it around.
NPM modules should IMO be agnostic as to where configuration is stored. That should be left up to the developer, and they may pick their favorite method (env vars, rc files, JSON files, whatever).
The configuration can be passed to your module in various ways. A common way is to export a function that takes an options object:
export default options => {
let db = database.connect(options.database);
...
}
From there, it really depends on what exactly your module provides. If it's just a bunch of loosely coupled functions, you can just return an object:
export default options => {
let db = database.connect(options.database);
return {
getUsers() { return db.getUsers() }
}
}
If you want to allow multiple versions of that object to exist simultaneously, you can use classes:
class MyClass {
constructor(options) {
...
}
...
}
export default options => {
return new MyClass(options)
}
Or export the entire class itself.
If the number of configuration options is limited (say 3 or less), you can also allow them to be passed as separate arguments, instead of passing an object.
I need to recursively insert entries in a hash based on some logic. The state of the hash gets updated inside the defined type loop, but not in the outer scope. The following should clarify:
class Test {
$config = {}
define my_loop()
{
$config['a'] = 'b'
notify { "1) config = $config": } # shows that $config has a=>b
}
my_loop { 'loop' : }
notify { "2) config = $config":
require => My_loop['loop'] # shows that $config is empty
}
}
So, the problem is that $config inside the loop() contains a=>b, but outside the loop() it doesn't. I must be bumping against some scoping rules here.
Thoughts?
The values of Puppet variables are set once, and they do not change thereafter. In those few places that present the appearance of behaving differently, what actually happens is either that a modified local copy or an entirely independent variable is created.
Additionally, do not nest classes or defined types inside classes. Puppet allows it for historical reasons, but it doesn't have the semantics you probably expect, and it makes the nested class / type hard to find.
Consider writing a custom function to perform your computation and return the needed hash.
Also consider whether upgrading to a supported version of Puppet would be feasible. Version 2.7 is very old.