nixos: my custom package derivation fails on `cargo build` - rust

I'm writing a package derivation to install kryptco/kr and it fails on cargo build phase.
kryptco/kr has a submodule kryptco/sigchain which is written by rust.
In the Makefile of kryptco/sigchain, they use the cargo web deploy command to build its submodule. My package deriation fails on this cargo web deploy phase.
These two errors were given.
error: warning: replace for the non root package will be ignored, specify replace at the workspace root:
package: /build/src/github.com/kryptco/kr/sigchain/sigchain_client/Cargo.toml
workspace: /build/src/github.com/kryptco/kr/sigchain/Cargo.toml
error: failed to fetch `https://github.com/rust-lang/crates.io-index`
Caused by:
[6] Couldn't resolve host name; class=Net (12)
The former error says to edit Cargo.toml and add [replace] section, but I want to avoid edit source code if I can.
What is the cause of these error?
This is my package derivation:
# Original: https://github.com/bogsen/nixos-public/blob/a0dc497eab5de528ce3006d\
36c52bc601422cf87/pkgs/krypton/default.nix
{ config, lib, pkgs, ... }: with lib; let
cfg = config.services.krypton;
cargoWeb = with pkgs; rustPlatform.buildRustPackage rec {
...
};
dependencies = with pkgs; [
cargo
emscripten
go
makeWrapper
perl
];
kr = pkgs.stdenv.mkDerivation {
name = "kr";
src = pkgs.fetchFromGitHub {
owner = "kryptco";
repo = "kr";
rev = "2.4.10";
sha256 = "1xxhlkcw2d52q1y4h40iyvq25804w7hzv0lflqnib68aps491xnj";
fetchSubmodules = true;
};
buildInputs = dependencies ++ [cargoWeb];
dontBuild = true;
postUnpack = ''
# prevent referring /homeless-shelter
export HOME=$(pwd)
# https://github.com/kryptco/kr/issues/254#issuecomment-464890476
sed -i.bak -e '8,11d' source/sigchain/Cargo.toml
export GOPATH=$(pwd)
export GOCACHE=$GOPATH/.cache/go-build
mkdir -p src/github.com/kryptco
mv source src/github.com/kryptco/kr
ln -s src/github.com/kryptco/kr source
'';
postPatch = ''
substituteInPlace Makefile --replace sudo ""
'';
makeFlags = [ "PREFIX=$(out)" ];
};
in {
config = {
nixpkgs.config.packageOverrides = pkgs: { kr = kr; };
};
}
And whole error message is here.
# use rsync so that file modifed times are only updated when the contents change
cd dashboard_yew && cargo web deploy --release --target=wasm32-unknown-emscripten && rsync --checksum --delete -r ../target/deploy/* ../target/deploy-final
error: warning: replace for the non root package will be ignored, specify replace at the workspace root:
package: /build/src/github.com/kryptco/kr/sigchain/sigchain_client/Cargo.toml
workspace: /build/src/github.com/kryptco/kr/sigchain/Cargo.toml
Updating crates.io index
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name; class=Net (12)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name; class=Net (12)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`
Caused by:
[6] Couldn't resolve host name; class=Net (12)
make[1]: *** [Makefile:25: libsigchain-with-dashboard] Error 101
make[1]: Leaving directory '/build/src/github.com/kryptco/kr/sigchain'
make: *** [Makefile:71: all] Error 2
builder for '/nix/store/78r0kh34ljzgfx658f9n99f8lxydjfxy-kr.drv' failed with exit code 2

I don't think you'll get around the [replace] edit for the config files. The build failing at cargo web deploy is due to an outdated version of openssl-sys (which is discontinued). Here's how Parity fixed it. Looks like they edited the Cargo.lock file to depend on newer versions of this lib.

You need to make it a fixed-output path, for this take a look at nix-pills.

Related

Rust-analyzer in VSCode to substitute env variables in Cargo path imports

I have a Cargo workspace and want to use a WORKSPACE_HOME environment variable in the import path of local crates.
For instance: in $WORKSPACE_HOME/services/api/Cargo.toml
[dependencies]
...
addrbook = { path = "${WORKSPACE_HOME}/pkg/addrbook" }
...
I tried adding the environment value to VSCode settings.json (under rust-analyzer.cargo.extraEnv) and also tried creating a .cargo/config.toml as described here
[env]
WORKSPACE_HOME = { value = "", relative = true }
Unfortunately, cargo metadata keeps failing saying that it is unable to resolve the path
[ERROR rust_analyzer::lsp_utils] rust-analyzer failed to load workspace: Failed to read Cargo metadata from Cargo.toml file /Users/nickdecooman/Documents/Workspace/foobar/Cargo.toml, Some(Version { major: 1, minor: 63, patch: 0 }): Failed to run `"cargo" "metadata" "--format-version" "1" "--manifest-path" "/Users/nickdecooman/Documents/Workspace/foobar/Cargo.toml" "--filter-platform" "x86_64-apple-darwin"`: `cargo metadata` exited with an error: error: failed to load manifest for workspace member `/Users/nickdecooman/Documents/Workspace/foobar/services/api`
Caused by:
failed to load manifest for dependency `addrbook`
Caused by:
failed to read `/Users/nickdecooman/Documents/Workspace/foobar/services/api/${WORKSPACE_HOME}/pkg/addrbook/Cargo.toml`
Caused by:
No such file or directory (os error 2)

Trouble running ORC JIT llvm-hs examples

Dear StackOverflow Haskellers:
I tried asking this same question in reddit r/haskell but sadly I got no answers at all. I hope it does better here.
Maybe the answer is that llvm-hs isn't used that much. If you have experience using llvm with haskell, but don't use llvm-hs, I'd love to see how you use it.
I'm trying to run basic llvm-hs ORC JIT functionality, which has taken me to try and run the orc example of llvm-hs-examples. So far I've been unsuccessful in multiple ways (out of the four examples only the first one, basic, runs):
Trying to run examples with the shell.nix in https://github.com/llvm-hs/llvm-hs:
$ git clone https://github.com/llvm-hs/llvm-hs.git
$ git clone https://github.com/llvm-hs/llvm-hs-examples.git
$ cd llvm-hs
$ nix-shell shell.nix
$ cd ../llvm-hs-examples
$ cabal new-build
$ cabal run orc
which produces:
$ cabal run orc
Up to date
; ModuleID = 'basic'
source_filename = "<string>"
define i32 #add() {
entry:
ret i32 42
}
JITSymbolError ""
Eager JIT Result:
()
doing a default.nix a la haskell.nix and running with nix-build
llvm-hs-examples/default.nix:
let
examplesOverlays = [ (self: super: {
llvm-config = self.llvm_9;
}) ];
in
{ # Fetch the latest haskell.nix and import its default.nix
haskellNix ? import (builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/ef6ca0f431fe3830c25cb2d185367245c1cce894.tar.gz") {}
# haskellNix ? import (builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/c88f9eccc975b21ae1e6a6b8057a712b91e374f2.tar.gz") {}
# haskellNix ? import (builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz") {}
# haskell.nix provides access to the nixpkgs pins which are used by our CI,
# hence you will be more likely to get cache hits when using these.
# But you can also just use your own, e.g. '<nixpkgs>'.
, nixpkgsSrc ? haskellNix.sources.nixpkgs-2003
# haskell.nix provides some arguments to be passed to nixpkgs, including some
# patches and also the haskell.nix functionality itself as an overlay.
, nixpkgsArgs ? haskellNix.nixpkgsArgs
# import nixpkgs with overlays
, pkgs ? (import nixpkgsSrc (nixpkgsArgs // { overlays = nixpkgsArgs.overlays ++ examplesOverlays;}))
# , pkgs ? import nixpkgsSrc nixpkgsArgs
}: pkgs.haskell-nix.project {
# 'cleanGit' cleans a source directory based on the files known by git
src = pkgs.haskell-nix.haskellLib.cleanGit {
name = "examples";
src = ./.;
};
# For `cabal.project` based projects specify the GHC version to use.
# compiler-nix-name = "ghc884"; # Not used for `stack.yaml` based projects.
}
in llvm-hs-examples's directory ran:
$ nix-build -A examples.components.exes.orc
$ ./result/bin/orc
; ModuleID = 'basic'
source_filename = "<string>"
define i32 #add() {
entry:
ret i32 42
}
JITSymbolError ""
Eager JIT Result:
()
which is the same output as before.
I believe the last (2) approach uses stack to build, but also tried manually with stack:
a) first failed approach:
In a nix-shell with llvm-config (I used the one in llvm-hs and the one I created with haskell.nix, but same result):
$ llvm-config --version
9.0.1
$ stack build examples:orc
No packages found in snapshot which provide a "llvm-config" executable, which is a build-tool dependency of llvm-hs
llvm-hs > configure
llvm-hs > [1 of 2] Compiling Main ( /run/user/1000/stack-2e6b46f4d38b8260/llvm-hs-9.0.1/Setup.hs, /run/user/1000/stack-2e6b46f4d38b8260/llvm-hs-9.0.1/.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1/setup/Main.o )
llvm-hs > [2 of 2] Compiling StackSetupShim ( /home/hhefesto/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /run/user/1000/stack-2e6b46f4d38b8260/llvm-hs-9.0.1/.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1/setup/StackSetupShim.o )
llvm-hs > Linking /run/user/1000/stack-2e6b46f4d38b8260/llvm-hs-9.0.1/.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1/setup/setup ...
llvm-hs > setup: The program 'llvm-config' version ==9.0.* is required but it could not
llvm-hs > be found.
llvm-hs >
b) build-successful stack approach I also tried with stack's nix integration explicitly specifying llvm-config as part of buildInputs:
stack.yaml
# resolver: nightly-2020-01-30
resolver: lts-14.0
packages:
- '.'
extra-deps:
- llvm-hs-9.0.1
- llvm-hs-pure-9.0.0
- llvm-hs-pretty-0.9.0.0
flags:
llvm-hs:
shared-llvm: true
nix:
enable: true
shell-file: stackShell.nix
stackShell.nix:
let
examplesOverlays = [ (self: super: {
llvm-config = self.llvm_9;
}) ];
in
{ haskellNix ? import (builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/ef6ca0f431fe3830c25cb2d185367245c1cce894.tar.gz") {}
, nixpkgsSrc ? haskellNix.sources.nixpkgs-2003
, nixpkgsArgs ? haskellNix.nixpkgsArgs
, pkgs ? (import nixpkgsSrc (nixpkgsArgs // { overlays = nixpkgsArgs.overlays ++ examplesOverlays;}))
}:
with pkgs;
haskell.lib.buildStackProject {
name = "llvm-hs";
buildInputs = [ llvm-config
];
inherit ghc;
}
Which was able to build successfully, but same error:
$ stack build examples:orc ghc-shell-for-examples
examples> configure (exe)
Configuring examples-1.0.0.0...
examples> build (exe)
Preprocessing executable 'orc' for examples-1.0.0.0..
Building executable 'orc' for examples-1.0.0.0..
examples> copy/register
Installing executable orc in /home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin
Installing executable basic in /home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin
Installing executable arith in /home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin
Installing executable irbuilder in /home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin
$ /home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin/orc ghc-shell-for-examples
; ModuleID = 'basic'
source_filename = "<string>"
define i32 #add() {
entry:
ret i32 42
}
JITSymbolError ""
Eager JIT Result:
()
A weird thing worth noting is that stack was unable to find what it had just built with stack exec examples:orc:
$ stack exec examples:orc
Executable named examples:orc not found on path: ["/home/hhefesto/src/llvm-hs-examples/.stack-work/install/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin","/home/hhefesto/.stack/snapshots/x86_64-linux-nix/2bab12248a943811dbc5aa23a88b887ce7aef1939551d21b69d96e3f64bfcbd7/8.6.5/bin","/home/hhefesto/.stack/compiler-tools/x86_64-linux-nix/ghc-8.6.5/bin","/nix/store/9wvsbqr57k9n6d8vv6b10d04j51f9ims-ghc-8.6.5/bin","/nix/store/4xb9z8vvk3fk2ciwqh53hzp72d0hx1da-bash-interactive-4.4-p23/bin","/nix/store/9wvsbqr57k9n6d8vv6b10d04j51f9ims-ghc-8.6.5/bin","/nix/store/m6h7zh8w6s52clnyskffj5lbkakqgywn-gcc-wrapper-9.2.0/bin","/nix/store/b3zsk4ihlpiimv3vff86bb5bxghgdzb9-gcc-9.2.0/bin","/nix/store/0k65d30z9xsixil10yw3bwajbdk4yskv-glibc-2.30-bin/bin","/nix/store/x0jla3hpxrwz76hy9yckg1iyc9hns81k-coreutils-8.31/bin","/nix/store/n48b8n251dwwb04q7f3fwxdmirsakllz-binutils-wrapper-2.31.1/bin","/nix/store/hrkc2sf2883l16d5yq3zg0y339kfw4xv-binutils-2.31.1/bin","/nix/store/0k65d30z9xsixil10yw3bwajbdk4yskv-glibc-2.30-bin/bin","/nix/store/x0jla3hpxrwz76hy9yckg1iyc9hns81k-coreutils-8.31/bin","/nix/store/6dacwd7ldb2jazc218d11v2w2g55hba8-pkg-config-0.29.2/bin","/nix/store/lb61dshvvqy1rgjhhlzaiiv2fv157lr5-stack-2.1.3.1/bin","/nix/store/71n1xcigc00w3z7yc836jqcx9cb2dys8-patchelf-0.9/bin","/nix/store/xhhkr936b9q5sz88jp4l29wljbbcg39k-ncurses-6.1-20190112/bin","/nix/store/khqyxflp8wbq038wdyv5sr8sjsfwlr72-llvm-9.0.1/bin","/nix/store/84g84bg47xxg01ba3nv0h418v5v3969n-ncurses-6.1-20190112-dev/bin","/nix/store/xhhkr936b9q5sz88jp4l29wljbbcg39k-ncurses-6.1-20190112/bin","/nix/store/x0jla3hpxrwz76hy9yckg1iyc9hns81k-coreutils-8.31/bin","/nix/store/97vambzyvpvrd9wgrrw7i7svi0s8vny5-findutils-4.7.0/bin","/nix/store/dqq1bvpi3g0h4v05111b3i0ymqj4v5x1-diffutils-3.7/bin","/nix/store/p34p7ysy84579lndk7rbrz6zsfr03y71-gnused-4.8/bin","/nix/store/b0vjq4r4sp9z4l2gbkc5dyyw5qfgyi3r-gnugrep-3.4/bin","/nix/store/c8balm59sxfkw9ik1fqbkadsvjqhmbx4-gawk-5.0.1/bin","/nix/store/g7dr83wnkx4gxa5ykcljc5jg04416z60-gnutar-1.32/bin","/nix/store/kkvgr3avpp7yd5hzmc4syh43jqj03sgb-gzip-1.10/bin","/nix/store/rw96psqzgyqrcd12qr6ivk9yiskjm3ab-bzip2-1.0.6.0.1-bin/bin","/nix/store/dp6y0n9cba79wwc54n1brg7xbjsq5hka-gnumake-4.2.1/bin","/nix/store/hrpvwkjz04s9i4nmli843hyw9z4pwhww-bash-4.4-p23/bin","/nix/store/xac1zfclx1xxgcd84vqb6hy3apl171n8-patch-2.7.6/bin","/nix/store/mm0w8jc58rn01c4kz2n9jvwd6bibcihs-xz-5.2.4-bin/bin"]
Other things worth noting: I tried everything stack related with two resolvers: lts-14.0 which was already there and nightly-2020-01-30 which was recently added in a commit to llvm-hs, but same result.
Also tried changing llvm-hs version restrictions to include the latest 9.0.1 in llvm-hs-examples' cabal file, but same result.
I'm on NixOS channel 20.03
If you want me to share or try something else, please let me know, and thank you for your help.
Lastly, thank you very much for your time!

"Failed to parse manifest" when compiling rustc using a locally-modified copy of the libc crate

I need to build the rustc compiler using a modified libc crate. I cloned the libc directory and made the changes, now how do I include the modified libc in my build?
This is my Cargo.toml
[patch.crates-io]
# Similar to Cargo above we want the RLS to use a vendored version of `rustfmt`
# that we're shipping as well (to ensure that the rustfmt in RLS and the
# `rustfmt` executable are the same exact version).
rustfmt-nightly = { path = "src/tools/rustfmt" }
# See comments in `src/tools/rustc-workspace-hack/README.md` for what's going on
# here
rustc-workspace-hack = { path = 'src/tools/rustc-workspace-hack' }
# See comments in `tools/rustc-std-workspace-core/README.md` for what's going on
# here
rustc-std-workspace-core = { path = 'src/tools/rustc-std-workspace-core' }
rustc-std-workspace-alloc = { path = 'src/tools/rustc-std-workspace-alloc' }
rustc-std-workspace-std = { path = 'src/tools/rustc-std-workspace-std' }
libc = {path = "../libc"}
[patch."https://github.com/rust-lang/rust-clippy"]
clippy_lints = { path = "src/tools/clippy/clippy_lints" }
[dependencies]
# libc = {verion = "0.2", default-features= false, path = "../libc"}
This is the error I get:
mahto#hydlnxeng27:/local/mnt/workspace/mahto/rust$ ./x.py build --config config.toml src/libstd 2>&1 | tee build.log
Updating only changed submodules
Submodules updated in 0.04 seconds
error: failed to parse manifest at `/local/mnt/workspace/mahto/rust/Cargo.toml`
Caused by:
virtual manifests do not specify [dependencies]
failed to run: /local/mnt/workspace/mahto/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /local/mnt/workspace/mahto/rust/src/bootstrap/Cargo.toml
Build completed unsuccessfully in 0:00:00
After commenting-out the dependencies section in Cargo.toml, I get this new error:
error[E0433]: failed to resolve: unresolved import
error: aborting due to previous error
For more information about this error, try `rustc --explain E0433`.
error: could not compile `libc`.

nix-env and nix-shell have different versions of snap

I was following directions on reflex-platfrom project development,trying to test servant-reflex as a submodule.
My project is here.
In my backend.cabal, I have a built-depend:
snap >= 1.1.1.0 && < 1.2
When I nix-shell -A shells.ghc --run "cabal new-build all", it tries to install heist-1.0.1.0 and snap-1.0.0.2, then failed at
Configuring heist-1.0.1.0...
Setup: Encountered missing dependencies:
aeson >=0.6 && <1.2
To see what in my nixos-unstable, I:
`nix-channel --list`
nixos https://nixos.org/channels/nixos-unstable
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.aeson`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.aeson aeson-1.2.4.0
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.snap`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.snap snap-1.1.0.0
`nix-env -f "<nixpkgs>" -qaP -A haskellPackages.heist`
warning: Nix search path entry '/home/demo/.nix-defexpr/channels' does not exist, ignoring
haskellPackages.heist heist-1.0.1.2
Q: Why does nix-shell install heist-1.0.1.0 and snap-1.0.0.2, instead of heist-1.0.1.2 and snap-1.1.0.0, which then can dependent on aeson-1.2.4.0?
Got an answer from elvishjerricco on IRC #nixos.
To doJailbreak heist, you'd use the overrides argument to
project
packages is for just declaring directories that you want to turn
into haskell packages; it'll run cabal2nix for you. overrides is for
doing derivation changes to the haskell package set.
default.nix
(import ./reflex-platform {}).project ({ pkgs, ... }: {
overrides = self: super: {
heist = pkgs.haskell.lib.doJailbreak super.heist;
map-syntax = pkgs.haskell.lib.doJailbreak super.map-syntax;
};
packages = {
common = ./common;
backend = ./backend;
frontend = ./frontend;
google-maps-reflex = ./google-maps-reflex;
};
shells = {
ghc = ["common" "backend" "frontend" "heist"]; # "backend" "frontend"];
ghcjs = ["common" "frontend"];
};
})

Multiple resource folders

I'm trying to add one more resource folder in my Android project.
I created a new folder extra-res so my project structure looks like this:
+ src
+ main
+ res
+ layout
+ ...etc...
+ extra-res
+ layout
So I added this to build.gradle:
android {
.........
sourceSets {
main {
res.srcDirs = ['res', 'extra-res']
}
}
}
But after editing the build.gradle file the build fails.
:app:processDebugResources
C:\Users\vovasoft\AndroidStudioProjects\sdbm\app\build\intermediates\manifests\full\debug\AndroidManifest.xml
Error:(13, 23) No resource found that matches the given name (at
'icon' with value '#drawable/ic_launcher').
Error:(14, 24) No resource found that matches the given name (at 'label' with value '#string/app_name').
Error:Execution failed for task ':app:processDebugResources'.
com.android.ide.common.internal.LoggedErrorException: Failed to run
command:
aapt.exe package -f --no-crunch -I android.jar -M \AndroidStudioProjects\sdbm\app\build\intermediates\manifests\full\debug\AndroidManifest.xml -S \AndroidStudioProjects\sdbm\app\build\intermediates\res\debug
-A \AndroidStudioProjects\sdbm\app\build\intermediates\assets\debug
-m -J C:\Users\vovasoft\AndroidStudioProjects\sdbm\app\build\generated\source\r\debug -F (at 'label' with value '#string/activity_edit_field').
Before editing build.gradle the build was successful.
I gave you some wrong information when I answered your original question in https://stackoverflow.com/a/28176489/2985303. That'll teach me about not testing an answer before posting it. You need to more fully qualify the resource directory paths, like so:
android {
sourceSets {
main {
res.srcDirs = ['src/main/res', 'src/main/extra-res']
}
}
}
I was getting that error beacause I forgot to include my new resource folder in build.gradle file.

Resources