Deploing a single bash script with nixops - nixos

I'm just starting to learn nix / nixos / nixops. I needed to install a simple bash script to remote host with nixops. And I can not realize how to do it. I have two files:
just-deploy-bash-script.nix
{
resources.sshKeyPairs.ssh-key = {};
test-host = { config, lib, pkgs, ... }: {
deployment.targetEnv = "digitalOcean";
deployment.digitalOcean.region = "sgp1";
deployment.digitalOcean.size = "s-2vcpu-4gb";
environment.systemPackages =
let
my-package = pkgs.callPackage ./my-package.nix { inherit pkgs; };
in [
pkgs.tmux
my-package
];
};
}
my-package.nix
{ pkgs ? import <nixpkgs> {}, ... }:
let
pname = "my-package";
version = "1.0.0";
stdenv = pkgs.stdenv;
in
stdenv.mkDerivation {
inherit pname version;
src = ./.;
installPhase =
let
script = pkgs.writeShellScriptBin "my-test" ''
echo This is my test script
'';
in
''
mkdir $out;
cp -r ${script} $out/
'';
}
I deploy as follows. I go to the directory in which these two files are located and then sequentially execute two commands:
nixops create -d test just-deploy-bash-script.nix
nixops deploy -d test
Deployment passes without errors and completes successfully. But when I login to the newly created remote host, I find that the tmux package from the standard set is present in the system, and my-package is absent:
nixops ssh -d test test-host
[root#test-host:~]# which tmux
/run/current-system/sw/bin/tmux
[root#test-host:~]# find /nix/store/ -iname tmux
/nix/store/hd1sgvb4pcllxj69gy3qa9qsns68arda-nixpkgs-20.03pre206749.5a3c1eda46e/nixpkgs/pkgs/tools/misc/tmux
/nix/store/609zdpfi5kpz2c7mbjcqjmpb4sd2y3j4-ncurses-6.0-20170902/share/terminfo/t/tmux
/nix/store/4cxkil2r3dzcf5x2phgwzbxwyvlk6i9k-system-path/share/bash-completion/completions/tmux
/nix/store/4cxkil2r3dzcf5x2phgwzbxwyvlk6i9k-system-path/bin/tmux
/nix/store/606ni2d9614sxkhnnnhr71zqphdam6jc-system-path/share/bash-completion/completions/tmux
/nix/store/606ni2d9614sxkhnnnhr71zqphdam6jc-system-path/bin/tmux
/nix/store/ddlx3x8xhaaj78xr0zasxhiy2m564m2s-nixos-17.09.3269.14f9ee66e63/nixos/pkgs/tools/misc/tmux
/nix/store/kvia4rwy9y4wis4v2kb9y758gj071p5v-ncurses-6.1-20190112/share/terminfo/t/tmux
/nix/store/c3m8qvmn2yxkgpfajjxbcnsgfrcinppl-tmux-2.9a/share/bash-completion/completions/tmux
/nix/store/c3m8qvmn2yxkgpfajjxbcnsgfrcinppl-tmux-2.9a/bin/tmux
[root#test-host:~]# which my-test
which: no my-test in (/root/bin:/run/wrappers/bin:/root/.nix-profile/bin:/etc/profiles/per-user/root/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin)
[root#test-host:~]# find /nix/store/ -iname *my-test*
[root#test-host:~]#
Help me figure out what's wrong with my scripts. Any links to documentation or examples of the implementation of such a task are welcome.

The shell can not find your script because it is copied into the wrong directory.
This becomes apparent after building my-package.nix:
$ nix-build my-package.nix
$ ls result/
zh5bxljvpmda4mi4x0fviyavsa3r12cx-my-test
Here you see the basename of a storepath inside a store path. This is caused by the line:
cp -r ${script} $out/
Changing it to something like this should fix that problem:
cp -r ${script}/* $out/

Related

How to modify this flake.nix so that I don't have to load the nix package every time I load the environment

I've taken this post as example for my question :https://johns.codes/blog/building-typescript-node-apps-with-nix
I try to make development environment with a npm package that has not a nix equivalent.
I think that I'm very near from the end but I probably miss something important and it doesn't work.
in the folder node-gyp-build:
npm init -y
npm install node-gyp-build
mkdir nix
rm -fr node_module
node2nix -16 --development --input package.json --lock package-lock.json --node-env ./nix/node-env.nix --composition ./nix/default.nix --output ./nix/node-package.nix
nix-build nix/default.nix
Now my question is how to use the inout in in a flake.nix in order to have developement environment with this package
I've tried with that flake.nix
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
npm_pack = import (./nix/default.nix);
in with pkgs;{
devShell = mkShell { buildInputs = [ npm_pack];};
]; };
});
}
and this command:
nix develop --extra-experimental-features nix-command --extra-experimental-features flakes --ignore-environment
error: Dependency is not of a valid type: element 1 of buildInputs for nix-shell
The npm_pack var in your example has the following attributes (many omitted)
package - The nix output of the package you are building (which can be added as a buildInput in your dev shell if its some executable)
shell - a devShell which gives you the npm deps to run the package you are trying to build, this would be equilvant to you cloning the package and runing npm install
so if you just want to use the package for another project your flake would look like
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
npm_pack = import ./nix { inherit pkgs system;};
in with pkgs;{
devShell = mkShell { buildInputs = [ npm_pack.package ];};
});
If you want to just make it so you auto have npm deps "installed" you can use the shell, so the flake would look like
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
npm_pack = import ./nix { inherit pkgs system;};
in with pkgs;{
devShell = npm_pack.shell
});

nixos: my custom package derivation fails on `cargo build`

I'm writing a package derivation to install kryptco/kr and it fails on cargo build phase.
kryptco/kr has a submodule kryptco/sigchain which is written by rust.
In the Makefile of kryptco/sigchain, they use the cargo web deploy command to build its submodule. My package deriation fails on this cargo web deploy phase.
These two errors were given.
error: warning: replace for the non root package will be ignored, specify replace at the workspace root:
package: /build/src/github.com/kryptco/kr/sigchain/sigchain_client/Cargo.toml
workspace: /build/src/github.com/kryptco/kr/sigchain/Cargo.toml
error: failed to fetch `https://github.com/rust-lang/crates.io-index`
Caused by:
[6] Couldn't resolve host name; class=Net (12)
The former error says to edit Cargo.toml and add [replace] section, but I want to avoid edit source code if I can.
What is the cause of these error?
This is my package derivation:
# Original: https://github.com/bogsen/nixos-public/blob/a0dc497eab5de528ce3006d\
36c52bc601422cf87/pkgs/krypton/default.nix
{ config, lib, pkgs, ... }: with lib; let
cfg = config.services.krypton;
cargoWeb = with pkgs; rustPlatform.buildRustPackage rec {
...
};
dependencies = with pkgs; [
cargo
emscripten
go
makeWrapper
perl
];
kr = pkgs.stdenv.mkDerivation {
name = "kr";
src = pkgs.fetchFromGitHub {
owner = "kryptco";
repo = "kr";
rev = "2.4.10";
sha256 = "1xxhlkcw2d52q1y4h40iyvq25804w7hzv0lflqnib68aps491xnj";
fetchSubmodules = true;
};
buildInputs = dependencies ++ [cargoWeb];
dontBuild = true;
postUnpack = ''
# prevent referring /homeless-shelter
export HOME=$(pwd)
# https://github.com/kryptco/kr/issues/254#issuecomment-464890476
sed -i.bak -e '8,11d' source/sigchain/Cargo.toml
export GOPATH=$(pwd)
export GOCACHE=$GOPATH/.cache/go-build
mkdir -p src/github.com/kryptco
mv source src/github.com/kryptco/kr
ln -s src/github.com/kryptco/kr source
'';
postPatch = ''
substituteInPlace Makefile --replace sudo ""
'';
makeFlags = [ "PREFIX=$(out)" ];
};
in {
config = {
nixpkgs.config.packageOverrides = pkgs: { kr = kr; };
};
}
And whole error message is here.
# use rsync so that file modifed times are only updated when the contents change
cd dashboard_yew && cargo web deploy --release --target=wasm32-unknown-emscripten && rsync --checksum --delete -r ../target/deploy/* ../target/deploy-final
error: warning: replace for the non root package will be ignored, specify replace at the workspace root:
package: /build/src/github.com/kryptco/kr/sigchain/sigchain_client/Cargo.toml
workspace: /build/src/github.com/kryptco/kr/sigchain/Cargo.toml
Updating crates.io index
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name; class=Net (12)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name; class=Net (12)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`
Caused by:
[6] Couldn't resolve host name; class=Net (12)
make[1]: *** [Makefile:25: libsigchain-with-dashboard] Error 101
make[1]: Leaving directory '/build/src/github.com/kryptco/kr/sigchain'
make: *** [Makefile:71: all] Error 2
builder for '/nix/store/78r0kh34ljzgfx658f9n99f8lxydjfxy-kr.drv' failed with exit code 2
I don't think you'll get around the [replace] edit for the config files. The build failing at cargo web deploy is due to an outdated version of openssl-sys (which is discontinued). Here's how Parity fixed it. Looks like they edited the Cargo.lock file to depend on newer versions of this lib.
You need to make it a fixed-output path, for this take a look at nix-pills.

nix-env and nix-shell have different versions of snap

I was following directions on reflex-platfrom project development,trying to test servant-reflex as a submodule.
My project is here.
In my backend.cabal, I have a built-depend:
snap >= 1.1.1.0 && < 1.2
When I nix-shell -A shells.ghc --run "cabal new-build all", it tries to install heist-1.0.1.0 and snap-1.0.0.2, then failed at
Configuring heist-1.0.1.0...
Setup: Encountered missing dependencies:
aeson >=0.6 && <1.2
To see what in my nixos-unstable, I:
`nix-channel --list`
nixos https://nixos.org/channels/nixos-unstable
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.aeson`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.aeson aeson-1.2.4.0
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.snap`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.snap snap-1.1.0.0
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.heist`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.heist heist-1.0.1.2
Q: Why does nix-shell install heist-1.0.1.0 and snap-1.0.0.2, instead of heist-1.0.1.2 and snap-1.1.0.0, which then can dependent on aeson-1.2.4.0?
Got an answer from elvishjerricco on IRC #nixos.
To doJailbreak heist, you'd use the overrides argument to
project
packages is for just declaring directories that you want to turn
into haskell packages; it'll run cabal2nix for you. overrides is for
doing derivation changes to the haskell package set.
default.nix
(import ./reflex-platform {}).project ({ pkgs, ... }: {
overrides = self: super: {
heist = pkgs.haskell.lib.doJailbreak super.heist;
map-syntax = pkgs.haskell.lib.doJailbreak super.map-syntax;
};
packages = {
common = ./common;
backend = ./backend;
frontend = ./frontend;
google-maps-reflex = ./google-maps-reflex;
};
shells = {
ghc = ["common" "backend" "frontend" "heist"]; # "backend" "frontend"];
ghcjs = ["common" "frontend"];
};
})

Yarn install production dependencies of a single package in workspace

I'm trying to install the production dependencies only for a single package in my workspace. Is that possible?
I've already tried this:
yarn workspace my-package-in-workspace install -- --prod
But it is installing all production dependencies of all my packages.
yarn 1 doesn't support it as far as I know.
If you are trying to install a specific package in a dockerfile, then there is a workaround:
copy the yarn.lock file and the root package.json
copy only the packages's package.json that you need: your package and which other packages that your package depends on (locally in the monorepo).
in the dockerfile, manually remove all the devDependnecies of all the package.json(s) that you copied.
run yarn install on the root package.json.
Note:
Deterministic installation - It is recommended to do so in monorepos to force deterministic install - https://stackoverflow.com/a/64503207/806963
Full dockefile example:
FROM node:12
WORKDIR /usr/project
COPY yarn.lock package.json remove-all-dev-deps-from-all-package-jsons.js change-version.js ./
ARG package_path=packages/dancer-placing-manager
COPY ${package_path}/package.json ./${package_path}/package.json
RUN node remove-all-dev-deps-from-all-package-jsons.js && rm remove-all-dev-deps-from-all-package-jsons.js
RUN yarn install --frozen-lockfile --production
COPY ${package_path}/dist/src ./${package_path}/dist/src
COPY ${package_path}/src ./${package_path}/src
CMD node --unhandled-rejections=strict ./packages/dancer-placing-manager/dist/src/index.js
remove-all-dev-deps-from-all-package-jsons.js:
const fs = require('fs')
const path = require('path')
const { execSync } = require('child_process')
async function deleteDevDeps(packageJsonPath) {
const packageJson = require(packageJsonPath)
delete packageJson.devDependencies
await new Promise((res, rej) =>
fs.writeFile(packageJsonPath, JSON.stringify(packageJson, null, 2), 'utf-8', error => (error ? rej(error) : res())),
)
}
function getSubPackagesPaths(repoPath) {
const result = execSync(`yarn workspaces --json info`).toString()
const workspacesInfo = JSON.parse(JSON.parse(result).data)
return Object.values(workspacesInfo)
.map(workspaceInfo => workspaceInfo.location)
.map(packagePath => path.join(repoPath, packagePath, 'package.json'))
}
async function main() {
const repoPath = __dirname
const packageJsonPath = path.join(repoPath, 'package.json')
await deleteDevDeps(packageJsonPath)
await Promise.all(getSubPackagesPaths(repoPath).map(packageJsonPath => deleteDevDeps(packageJsonPath)))
}
if (require.main === module) {
main()
}
It looks like this is easily possible now with Yarn 2: https://yarnpkg.com/cli/workspaces/focus
But I haven't tried myself.
Here is my solution for Yarn 1:
# Install dependencies for the whole monorepo because
# 1. The --ignore-workspaces flag is not implemented https://github.com/yarnpkg/yarn/issues/4099
# 2. The --focus flag is broken https://github.com/yarnpkg/yarn/issues/6715
# Avoid the target workspace dependencies to land in the root node_modules.
sed -i 's|"dependencies":|"workspaces": { "nohoist": ["**"] }, "dependencies":|g' apps/target-app/package.json
# Run `yarn install` twice to workaround https://github.com/yarnpkg/yarn/issues/6988
yarn || yarn
# Find all linked node_modules and dereference them so that there are no broken
# symlinks if the target-app is copied somewhere. (Don't use
# `cp -rL apps/target-app some/destination` because then it also dereferences
# node_modules/.bin/* and thus breaks them.)
cd apps/target-app/node_modules
for f in $(find . -maxdepth 1 -type l)
do
l=$(readlink -f $f) && rm $f && cp -rf $l $f
done
Now apps/target-app can be copied and used as a standalone app.
I would not recommend it for production. It is slow (because it installs dependencies for the whole monorepo) and not really reliable (because there may be additional issues with symlinks).
You may try
yarn workspace #my-monorepo/my-package-in-workspace install -- --prod

NixOS: Install a non-nix package?

Currently I started using NixOS, and it's working great except for that there are some packages I want to install that are not in the repository.
The first package I had issues with is Cryptomator (https://cryptomator.org)
What I tried to do is follow this tutorial: http://anderspapitto.com/posts/2015-02-28-deb-installation-nixos.html
But I couldn't get it to work... Here is what I tried:
First I created 3 files (as in the tutorial):
builder.sh (as-is from tutorial)
fhs-env.nix (also as-is)
full-cryptomator.nix (source listed below)
dumb-cryptomator.nix (source listed below)
full-cryptomator.nix
let nixpkgs = import <nixpkgs> {};
stdenv = nixpkgs.stdenv;
in rec {
dumb-cryptomator = stdenv.mkDerivation {
name = "dumb-cryptomator";
builder = ./builder.sh;
dpkg = nixpkgs.dpkg;
src = nixpkgs.fetchurl {
url = "https://bintray.com/cryptomator/cryptomator-deb/download_file?file_path=cryptomator-1.2.3-amd64.deb";
sha256 = "f611dfd77f68ddd4b7322b1668829add987c5f8e0fcd639211b46969f1eb8ef3";
};
};
full-cryptomator = nixpkgs.buildFHSUserEnv {
name = "full-cryptomator";
targetPkgs = pkgs: [ dumb-cryptomator ];
multiPkgs = pkgs: [ pkgs.dpkg ];
runScript = "Cryptomator";
};
}
dumb-cryptomator.nix
let nixpkgs = import <nixpkgs> {};
stdenv = nixpkgs.stdenv;
in rec {
dumb-cryptomator = stdenv.mkDerivation {
name = "dumb-cryptomator";
builder = ./builder.sh;
dpkg = nixpkgs.dpkg;
src = nixpkgs.fetchurl {
url = "https://bintray.com/cryptomator/cryptomator-deb/download_file?file_path=cryptomator-1.2.3-amd64.deb";
sha256 = "f611dfd77f68ddd4b7322b1668829add987c5f8e0fcd639211b46969f1eb8ef3";
};
};
}
I then run:
nix-build -A fhsEnv fhs-env.nix
nix-build -A dumb-cryptomator dumb-cryptomator.nix
nix-build -A full-cryptomator full-cryptomator.nix
When the last build is completed, I don't have a full cryptomator install in my directory (at least /opt/Cryptomator should be there):
[peter#peter-laptop:~/Downloads/cryptomator]$ ls /nix/store/6prdbjgidgqaqfnvmkrhnj8xp28z8dxw-full-cryptomator
bin
Is there anyone having more experience with making third-party packages work? Thanks!
At least you should change
runScript = "Cryptomator";
to
runScript = "${dumb-cryptomator}/Cryptomator/Cryptomator";
Debugging such kinds of problems is easy. When you run nix-build -A full-cryptomator full-cryptomator.nix, a symlink result is created, so you can access application by
$ ./result/bin/full-cryptomator
/nix/store/7nc1j85m67kjd1y1fnk6xg5b7w7a1jjd-full-cryptomator-init: line 11: exec: Cryptomator: not found
Whoops! Why's that? let's look into mentioned file:
$ cat /nix/store/7nc1j85m67kjd1y1fnk6xg5b7w7a1jjd-full-cryptomator-init
#! /nix/store/wsz11sx18n85if6gp50m870rypgavwpd-bash-4.4-p12/bin/bash
for i in /nix/store/ss4j5rr2ilh878m1g5c76njcmvmrszja-full-cryptomator-fhs/* /host/*; do
path="/${i##*/}"
[ -e "$path" ] || /nix/store/sxzm5kva1gb0hxm60xvr45m3c5l84xlz-coreutils-8.27/bin/ln -s "$i" "$path"
done
[ -d "$1" ] && [ -r "$1" ] && cd "$1"
shift
source /etc/profile
exec Cryptomator "$#"
Looks like it can't file Cryptomator executable in current PATH. When we inspect actual dumb-cryptomator, we see
$ tree /nix/store/m31pyd1hgpa9v8gnxrjijd1v3n8vxggj-dumb-cryptomator | grep -i crypto
/nix/store/m31pyd1hgpa9v8gnxrjijd1v3n8vxggj-dumb-cryptomator
└── Cryptomator
│   ├── cryptolib-1.0.7.jar
│   ├── Cryptomator-1.2.3.jar
│   ├── Cryptomator.cfg
│   ├── filesystem-crypto-1.2.3.jar
├── Cryptomator
├── Cryptomator.desktop
├── cryptomator.org-Cryptomator-MimeInfo.xml
├── Cryptomator.png
So, it doesn't belong to standard bin folder. As we don't want to mess with package alot, we provide a full path, by using ${dumb-cryptomator} string interpolation.
(that's one of the most used features in Nix, ${dumb-cryptomator} gets expanded to /nix/store/m31pyd1hgpa9v8gnxrjijd1v3n8vxggj-dumb-cryptomator during builds)

Resources