Why doesn't `nix-env -q` find my installed packages? (NixOS channels, profiles and packages) - nixos

I recently installed NixOS and I ended up with 3 profiles:
bruno (a user profile),
default (used by root), and
system (used by NixOS).
I found it convenient to use a stable channel for the system profile and an unstable channel for me (bruno profile):
~> nix-channel --list
unstable https://nixos.org/channels/nixos-unstable
~> sudo nix-channel --list
nixos https://nixos.org/channels/nixos-17.09
I have then declaratively installed a few packages through /etc/nixos/configuration.nix:
environment.systemPackages = with pkgs; [
firefox
chromium
htop
# ...
];
And some imperatively: nix-env --install firefox.
Now listing my installed packages, I'd expect to also see the ones provided by the system profile (as they are available in my user profile):
~> htop --version
htop 2.0.2 - (C) 2004-2016 Hisham Muhammad
Released under the GNU GPL.
~> nix-env -q | grep htop
~> nix-env -q | grep firefox
firefox-57.0
Only Firefox is listed. Let's try using the root profile:
~> sudo nix-env -q | grep htop
Same thing, it's actually completely empty.
Maybe using the system profile:
~> sudo nix-env -p /nix/var/nix/profiles/system -q
Still nothing.
Coming from traditional package managers (Debian, Red Hat), I find confusing that Nix being defined as "The Purely Functional Package Manager" does not seem to provide a tool to query packages universally - nix-env is mentioned throughout the manuals and feels like Debian apt's alter ego.
Is there such a tool, or is this a non-problem, that is, people are generally fine with not having a list of all packages present across profiles/environments?

nix-env -q will only report packages that are installed into imperative 'environments', like those created by nix-env -i.
nix-env is a tool for imperative package management that is a thin layer over the otherwise declarative and immutable Nix system. The profiles mechanism provides a means for mutability and nix-env creates manifest.nix in the profile to record the set of packages that are in the environment.
A NixOS system only uses the profiles but does not provide a manifest.nix file. This makes sense, because a NixOS system gets rebuilt from scratch every time, which makes it nicely declarative.
As a consequence nix-env can not query a NixOS profile.
So, nix-env -q queries your imperatively installed user packages. sudo nix-env -q shows those imperatively installed by root, which does not include your NixOS packages, because they are part of your system, declaratively. Pointing nix-env -q -p at your NixOS system gives an empty list, because it's not an environment created by nix-env.
The reason your user can use system commands is not because the system is part of the (Nix) environment in the user profile, but because your (UNIX) environment variables point to both profiles.
$ which firefox
/home/user/.nix-profile/bin/firefox
$ which cp
/run/current-system/sw/bin/cp
$ echo $PATH
[...]
To figure out what is installed on your system, you may run the following commands:
nix-env -q to figure out what is installed in an imperative user environment.
nixos-option environment.systemPackages to query the set of packages that will appear in /run/current-system/sw when your configuration is built and activated. To quote its documentation, "These packages are automatically available to all users."
nix-store -q --requisites /run/current-system ~/.nix-profile the combined closure of dependencies of the current system and your user profile
nix-store -q --references /run/current-system direct dependencies of the current system
See --query section or nix-store --help for more options.

Related

clang 3.8+ -fopenmp on linux: ld cannot find -lomp

I have installed clang 3.8 from the base repositories for both Debian Jessie and Fedora 24. When I try to compile a simple HelloWorld.cpp test program with clang++, and i pass the -fopenmp flag, in both cases i get the same error:
/usr/bin/ld: cannot find -lomp
clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
I see that if I instead pass -fopenmp=libgomp, it works. However, the Clang OpenMP website says that the OpenMP runtime is shipped with Clang 3.8. Why, then, can it not find the default libomp library? I do not see this library anywhere on my system.
There is good chances that the OpenMP development package is missing on your system.
On Ubuntu: sudo apt install libomp-dev
If you have libomp installed correctly you will need to use -fopenmp=libomp. libgomp is for gcc. You might check that clang isn't symbollically linked to gcc on your computer.
TL;DR
If you have libomp.so for llvm in somewhere like /usr/lib/llvm-12/lib make file /etc/ld.so.conf.d/libomp.conf with the line /usr/lib/llvm-12/lib in it, then run sudo ldconfig.
Intro
In my case, I had libomp-12-dev installed, but it was not in my linker's library path. See the footnote on how I found the library. There are a couple solutions in this scenario:
Add library path with ldconfig
If you want this in your default library path, consider using ldconfig [man page].
This will look for files in /etc/ld.so.conf. For me, running Ubuntu 20.04, this file only points to including files in the directory /etc/ld.so.conf.d.
$ cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
As such, I made a config llvm-libomp-12 in my /etc/ld.so.conf.d directory that looks like this:
$ cat /etc/ld.so.conf.d/libomp.conf
# libomp.so for llvm
/usr/lib/llvm-12/lib
Then I asked ldconfig to update the paths with sudo ldconfig. You can add the -v flag and it will print all libraries and paths it is aware of.
Add library to environment variable
We can also direct the linker to our library using the $LD_LIBRARY_PATH environment variable
This may be advantageous if you're on a multiuser system and don't want to impact others, or if you have temporary changes to your library paths you would like to make in your shell.
See what your current $LD_LIBRARY_PATH is with echo $LD_LIBRARY_PATH. You may not have this set by default. Add paths to this variable, each delimited by a colon.
For your current shell session, simply append or prepend to your $LD_LIBRARY_PATH like this (assuming bash, zsh, or fish >v3.0):
export "$LD_LIBRARY_PATH:/path/to/lib"
Or for a more permanent change limited to your user, add the above export to your shell's config file (e.g. ~/.bashrc).
Manually specify library path(s) in compiler flags
Nice for a one-off specific library that you don't always want in your default library paths. Specify the path to the library as a flag like this:
-L/path/to/lib
For example:
clang++ -L/usr/lib/llvm-12/lib [...]
make -L/usr/lib/llvm-12/lib
Footnotes
On searching
If you don't know where a given library you need is, you can use things like find. Personally though, I used a package called mlocate that indexes files on my machine and allows you to search them.
Installing mlocate
sudo apt install mlocate
Updating the indexes
sudo updatedb
Searching for a substring
locate libomp.so
When I searched for where my libomp libraries were, I did this:
$ locate libomp.so
/usr/lib/llvm-12/lib/libomp.so
/usr/lib/llvm-12/lib/libomp.so.5
/usr/lib/x86_64-linux-gnu/libomp.so.5
Notably it seemed like clang was using the libomp.so.5 in the linux-gnu directory, but I needed it to be using the llvm library.
Environment used in this post
$ lsb_release --all
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
$ uname -a
Linux bip 5.8.0-48-generic #54~20.04.1-Ubuntu SMP Sat Mar 20 13:40:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
TODO
Some notes that could be added to this question:
Confirm and list priority of env vars vs config files vs flags (does this vary between compilers and linkers?)
Ordering library paths when using multiple config files (can we prefix with numbers to ensure the order libraries are parsed?)

Cross-compile a Rust application from Linux to Windows

Basically I'm trying to compile the simplest code to Windows while I am developing on Linux.
fn main() {
println!("Hello, and bye.")
}
I found these commands by searching the internet:
rustc --target=i686-w64-mingw32-gcc main.rs
rustc --target=i686_pc_windows_gnu -C linker=i686-w64-mingw32-gcc main.rs
Sadly, none of them work. It gives me an error about the std crate missing
$ rustc --target=i686_pc_windows_gnu -C linker=i686-w64-mingw32-gcc main.rs
main.rs:1:1: 1:1 error: can't find crate for `std`
main.rs:1 fn main() {
^
error: aborting due to previous error
Is there a way to compile code on Linux that will run on Windows?
Other answers, while technically correct, are more difficult than they need to be. There's no need to use rustc (in fact it's discouraged, just use cargo), you only need rustup, cargo and your distribution's mingw-w64.
Add the target (you can also change this for whatever target you're cross compiling for):
rustup target add x86_64-pc-windows-gnu
You can build your crate easily with:
cargo build --target x86_64-pc-windows-gnu
No need for messing around with ~/.cargo/config or anything else.
EDIT: Just wanted to add that while you can use the above it can also sometimes be a headache. I wanted to add that the rust tools team also maintains a project called cross: https://github.com/rust-embedded/cross
This might be another solution that you want to look into
The Rust distribution only provides compiled libraries for the host system. However, according to Arch Linux's wiki page on Rust, you could copy the compiled libraries from the Windows packages in the download directory (note that there are i686 and x86-64 packages) in the appropriate place on your system (in /usr/lib/rustlib or /usr/local/lib/rustlib, depending on where Rust is installed), install mingw-w64-gcc and Wine and you should be able to cross-compile.
If you're using Cargo, you can tell Cargo where to look for ar and the linker by adding this to ~/.cargo/config (where $ARCH is the architecture you use):
[target.$ARCH-pc-windows-gnu]
linker = "/usr/bin/$ARCH-w64-mingw32-gcc"
ar = "/usr/$ARCH-w64-mingw32/bin/ar"
Note: the exact paths can vary based on your distribution. Check the list of files for the mingw-w64 package(s) (GCC and binutils) in your distribution.
Then you can use Cargo like this:
$ # Build
$ cargo build --release --target "$ARCH-pc-windows-gnu"
$ # Run unit tests under wine
$ cargo test --target "$ARCH-pc-windows-gnu"
UPDATE 2019-06-11
This fails for me with:
Running `rustc --crate-name animation examples/animation.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' -C metadata=006e668c6384c29b -C extra-filename=-006e668c6384c29b --out-dir /home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/examples --target x86_64-pc-windows-gnu -C ar=x86_64-w64-mingw32-gcc-ar -C linker=x86_64-w64-mingw32-gcc -C incremental=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/incremental -L dependency=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps -L dependency=/home/roman/projects/rust-sdl2/target/debug/deps --extern bitflags=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libbitflags-2c7b3e3d10e1e0dd.rlib --extern lazy_static=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/liblazy_static-a80335916d5ac241.rlib --extern libc=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/liblibc-387157ce7a56c1ec.rlib --extern num=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libnum-18ac2d75a7462b42.rlib --extern rand=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/librand-7cf254de4aeeab70.rlib --extern sdl2=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libsdl2-3f37ebe30a087396.rlib --extern sdl2_sys=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libsdl2_sys-3edefe52781ad7ef.rlib -L native=/home/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-x86_64-pc-windows-gnu-0.4.0/lib`
error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
Maybe this will help https://github.com/rust-lang/rust/issues/44787
Static compile sdl2
There is option to static-compile sdl but it didn't work for me.
Also mixer is not included when used with bundled.
Let's cross-compile examples from rust-sdl2 project from Ubuntu to Windows x86_64
In ~/.cargo/config
[target.x86_64-pc-windows-gnu]
linker = "x86_64-w64-mingw32-gcc"
ar = "x86_64-w64-mingw32-gcc-ar"
Then run this:
sudo apt-get install gcc-mingw-w64-x86-64 -y
# use rustup to add target https://github.com/rust-lang/rustup.rs#cross-compilation
rustup target add x86_64-pc-windows-gnu
# Based on instructions from https://github.com/AngryLawyer/rust-sdl2/
# First we need sdl2 libs
# links to packages https://www.libsdl.org/download-2.0.php
sudo apt-get install libsdl2-dev -y
curl -s https://www.libsdl.org/release/SDL2-devel-2.0.9-mingw.tar.gz | tar xvz -C /tmp
# Prepare files for building
mkdir -p ~/projects
cd ~/projects
git clone https://github.com/Rust-SDL2/rust-sdl2
cd rust-sdl2
cp -r /tmp/SDL2-2.0.9/x86_64-w64-mingw32/lib/* ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/
cp /tmp/SDL2-2.0.9/x86_64-w64-mingw32/bin/SDL2.dll .
Build examples at once
cargo build --target=x86_64-pc-windows-gnu --verbose --examples
Or stop after first fail:
echo; for i in examples/*; do [ $? -eq 0 ] && cargo build --target=x86_64-pc-windows-gnu --verbose --example $(basename $i .rs); done
Run
cargo build will put binaries in target/x86_64-pc-windows-gnu/debug/examples/
Copy needed files:
cp /tmp/SDL2-2.0.4/x86_64-w64-mingw32/bin/SDL2.dll target/x86_64-pc-windows-gnu/debug/examples/
cp assets/sine.wav target/x86_64-pc-windows-gnu/debug/examples/
Then copy directory target/x86_64-pc-windows-gnu/debug/examples/ to your Windows machine and run exe files.
Run in cmd.exe
If you want to see the console output when running exe files, you may run them from cmd.exe.
To open cmd.exe in current directory in file explorer, right click with shift on empty place in window and choose Open command window here.
Backtraces with mingw should work now - if not use msvc https://github.com/rust-lang/rust/pull/39234
There is Docker based solution called cross. All the required tools are in virtualized environment so you don't need to install additional packages for your machine. See Supported targets list.
From project's README:
Features
cross will provide all the ingredients needed for cross compilation without touching your system installation.
cross provides an environment, cross toolchain and cross compiled libraries, that produces the most portable binaries.
“cross testing”, cross can test crates for architectures other than i686 and x86_64.
The stable, beta and nightly channels are supported.
Dependencies
rustup
A Linux kernel with binfmt_misc support is required for cross testing.
One of these container engines is required. If both are installed, cross will default to docker.
Docker. Note that on Linux non-sudo users need to be in the docker group. Read the official post-installation steps. Requires version 1.24 or later.
Podman. Requires version 1.6.3 or later.
Installation
$ cargo install cross
Usage
cross has the exact same CLI as Cargo but as it relies on Docker you'll have to start the daemon before you can use it.
# (ONCE PER BOOT)
# Start the Docker daemon, if it's not already running
$ sudo systemctl start docker
# MAGIC! This Just Works
$ cross build --target aarch64-unknown-linux-gnu
# EVEN MORE MAGICAL! This also Just Works
$ cross test --target mips64-unknown-linux-gnuabi64
# Obviously, this also Just Works
$ cross rustc --target powerpc-unknown-linux-gnu --release -- -C lto
The solution that worked for me was. It is similar to one of the accepted answers but I did not require to add the toolchain.
rustup target add x86_64-pc-windows-gnu
cargo build --target x86_64-pc-windows-gnu
Refer to the documentation for more details.
I've had success on Debian (testing) without using Mingw and Wine just following the official instructions. They look scary, but in the end it didn't hurt that much.
The official instructions also contain info on how to cross-compile C/C++ code. I haven't needed that, so it's something I haven't actually tested.
A couple of remarks for individual points in the official instructions. The numbers match the numbers in the official instructions.
Debian: sudo apt-get install lld
Make a symlink named lld-link to lld somewhere in your $PATH. Example: ln -s /usr/bin/lld local_bin/lld-link
I don't cross-compile C/C++, haven't used this point personally.
This is probably the most annoying part. I installed Rust on a Windows box via rustup, and copied the libraries from the directories named in the official docs to the Linux box. Beware, there were sometimes uppercase library filenames, but lld wants them all lowercase (Windows isn't case-sensitive, Linux is). I've used the following to rename all files in current directory to lowercase:
for f in `find`; do mv -v "$f" "`echo $f | tr '[A-Z]' '[a-z]'`"; done
Personally, I've needed both Kit directories and just one of the VC dirs.
I don't cross-compile C/C++, haven't used this point personally.
Just make $LIB_ROOT in the script at the end of this post point to the lib directory from point 3.
Mandatory
I don't cross-compile C/C++, haven't used this point personally.
Depending the target architecture, either of the following:
rustup target add i686-pc-windows-msvc
rustup target add x86_64-pc-windows-msvc
For cross-building itself, I'm using the following simple script (32-bit version):
#!/bin/sh
# "cargo build" for the 32-bit Windows MSVC architecture.
# Set this to proper directory
LIB_ROOT=~/opt/rust-msvc
# The rest shouldn't need modifications
VS_LIBS="$LIB_ROOT/Microsoft Visual Studio 14.0/VC/lib/"
KIT_8_1_LIBS="$LIB_ROOT/Windows Kits/8.1/Lib/winv6.3/um/x86/"
KIT_10_LIBS="$LIB_ROOT/Windows Kits/10/Lib/10.0.10240.0/ucrt/x86/"
export LIB="$VS_LIBS;$KIT_8_1_LIBS;$KIT_10_LIBS"
cargo build --target=i686-pc-windows-msvc "$#"
I'm using the script the same way I would use cargo build
Hope that helps somebody!

Centos openmp with gcc 4.1.2

I cant tell if openmp is installed in this fresh install of centos I just put on with the developer tools.
I tried to run ldconfig -p | grep openmp and I dont see it listed.
Is it available for centos with that version of gcc?
Thanks
/sbin/ldconfig -p | grep gomp
library is call libgomp
Try:
$ ldconfig -p | grep -i "gomp"
libgomp.so.1 (libc6,x86-64) => /usr/lib64/libgomp.so.1
Basically, it's called gomp... GCC's OpenMP.
The alternative is to compile a hello world openmp program; if it doesn't work, it isn't installed.
Search "yum search gomp" this way on a Red Hat distro. To find out what the groups are use "yum grouplist | less". This pages the output so you can see the complete list. You will notice which groups are installed and available. Of course you can always use "yum help".
Not familiar with CentOS or yum, but take a stab with this:
yum list installed | grep "penmp"
Good luck!

How to determine the OS version for which was compiled rpm?

How can i determine the Linux version (distribution) for which was compiled rpm packet?
I believe this is what you are after.
$ rpm -q gnome-speech --queryformat '%10{NAME} %20{VENDOR} %20{RELEASE} %20{ARCH}\n'
gnome-speech Red Hat, Inc. 1.fc6 i386
$ rpm -q hwdata --queryformat '%10{NAME} %20{VENDOR} %20{RELEASE} %20{ARCH}\n'
hwdata Red Hat, Inc. 1.el5 noarch
There are lots of nifty bits you can go after with queryformat, see this guide for a reference.
General reference on rpm-philosophy-multi-architecture.
There is no such ways to determine the OS name and version from the content of the file.
Normally a RPM file name contains all these details. According to the RPM file naming convention the file name must be like :
name-version-release.architecture.rpm
Please check this link for details.
You can use rpm -q to get the OS data from the OS tag in the RPM, but you need to specify a queryformat as it is not in the normal -i output. You can use -p to refer to a specific RPM file for the testing.
$ rpm -q -p myfiletotest.rpm --queryformat '%10{NAME} %10{OS} %10{VERSION} %10{RELEASE} %10{ARCH}\n'
mypackage linux 2.2.10 1_14.el6 x86_64
The OS name is the second field in this output. The RPM does not distinguish between releases of the OS though; you will only see 'linux', 'aix', 'darwin', and so on - not 'centos-6'.

Using GHC, cabal with GMP installed in user-space

I have been trying to install Haskell Platform and cabal-install installed on Linux in user-space on a system that doesn't have the GNU Multi-Precision package (GMP) installed.
I managed to get GHC-6.12.1 installed and GHCi working by setting up LB_LIBRARY_PATH to point at the lib directory where I installed GMP, but then ran into problems in the next step, getting cabal-install to work. It kept trying to (statically) link to GMP.
This fails because the GMP is not installed in the system and ld hasn't a clue where to find the libraries, and there is no environment variable (that I am aware of) that can tell ld where to find the user-installed GMP, and (apparently) no way of telling configuring Cabal to supply the relevant -L flag.
After much fruitless searching and hacking attempts I hit on the absurdly simple idea of installing my own ld shell script that invokes the system ld with the appropriate -L flag.
This is shell scripting 101, of course:
#!/bin/sh
/usr/bin/ld -L$HOME/gnu/lib "$#"
With this script installed in a directory on my PATH ahead of /usr/bin all the problems seem to have gone away.
Basically, your ghc is not working yet. Yes, it can compile things, but it cannot link programs because it needs to link them to gmp.
What we can do is to edit some core package, e.g. the rts package, so that ghc will always use the right -L flag:
ghc-pkg describe rts > rts.pkg
vi rts.pkg # add the gmp dir to the `library-dirs` field
sudo ghc-pkg update rts.pkg

Resources