So I would like to build numpy against MKL BLAS on Ubuntu 20.04.
I have no trouble building and linking against system installed mkl (from e.g. the Ubuntu intel-mkl package). As documented in site.cfg.example I created a copy site.cfg next to it and filled in the relevant information
[mkl]
library_dirs = /path/to/my/local/mkl/lib
include_dirs = /path/to/my/local/mkl/include
libraries = mkl_rt
I can create a wheel with python -m build --wheel that is linked against that MKL, I can install it fine with pip install my_numpy.whl and after import numpy, numpy.show_config() shows the expected info regarding mkl.
Now what I would like to do is to be independent from
system installed mkl
absolute paths in library_dirs and include_dirs
All the necessary library and include files for building and linking against MKL are available in the PyPI packages mkl and mkl-include. I would like to configure a build process, where these two packages are downloaded, built and linked against and also added to the requirements. Now I understand running python3 -m build creates a separate venv and preinstalls any packages listed in pyproject.toml. Here my edited version with mkl and mkl-include added at the bottom
[build-system]
# Minimum requirements for the build system to execute.
requires = [
"packaging==20.5; platform_machine=='arm64'", # macos M1
"setuptools<49.2.0",
"wheel==0.36.2",
"Cython>=0.29.24,<3.0", # Note: keep in sync with tools/cythonize.py
"mkl-include<2021.2",
"mkl<2021.2"
]
I am stuck at what to put into site.cfg under library_dirs and include_dirs to use the files from the packages installed by the build venv. Is there a way to include the current venv's prefix into those paths? If yes, how? Do the numpy.distutils offer anything in that regard?
Or do I have to approach this problem differently? E.g. is there a way to build numpy and create a .whl, but without explicitly linking, s.t. linking can be delayed until actual installation time of the built wheel, where the actual MKL runtime libraries are found (in this case /path/to/my/current/venv/lib) and linked against? Is that even possible with a wheel?
Thanks for your help!
I am following the steps on for unix like systems https://elixir-lang.org/install.html#unix-and-unix-like
this is the error that I am getting
sudo apt-get install esl-erlang
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
esl-erlang : Depends: libncurses5 but it is not installable
Depends: libsctp1 but it is not installable
Recommends: erlang-mode but it is not going to be installed
E: Unable to correct problems, you have held broken packages
I tried using asdf-vm as well but still the same libcurses5 issue is there
asdf install erlang 21.1
asdf_21.1 is not a kerl-managed Erlang/OTP installation
No build named asdf_21.1
Downloading OTP-21.1.tar.gz to /home/superuser/.asdf/plugins/erlang/kerl-home/archives
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 120 100 120 0 0 241 0 --:--:-- --:--:-- --:--:-- 241
100 51.3M 0 51.3M 0 0 4747k 0 --:--:-- 0:00:11 --:--:-- 3990k
Extracting source code
Building Erlang/OTP 21.1 (asdf_21.1), please wait...
WARNING: It appears that a required development package 'libncurses5-dev' is not installed.
Configure failed.
checking whether lock counters should be enabled... no
checking whether dlopen() needs to be called before first call to dlerror()... no
checking for kstat_open in -lkstat... (cached) no
checking for tgetent in -ltinfo... no
checking for tgetent in -lncurses... no
checking for tgetent in -lcurses... no
checking for tgetent in -ltermcap... no
checking for tgetent in -ltermlib... no
configure: error: No curses library functions found
The problem was that with cosmic the repositories were not available on http://archive.ubuntu.com.
changing them in /etc/apt/sources.list i.e replacing all occurences of http://archive.ubuntu.com with http://old-releases.ubuntu.com did the trick.
I am using IntelliJ's rust plugin (version 0.2.0.2114-182) with IDEA (2018.2.3).
There is a yellow bar at the top of my editor window that says "cannot attach stdlib sources automatically without rustup". This is not surprising since gentoo does not use rustup. It compiles rust from source.
There is an option to "attach manually", but I have no idea what directory it wants me to pick; or even what I should look for to figure out what the proper directory is; and I'm not even sure the gentoo ebuild created a directory with the necessary stdlib sources.
How can I provide the stdlib sources to the rust plugin in a way that is compatible with gentoo's package management system? It seems like answers to How to provide standard library sources for IntelliJ IDEA's Rust project? bypass gentoo's ebuilds and might cause the accumulation of cruft over time.
The dev-lang/rust Gentoo package has an rls use-flag (standing for Rust Language Server), which has a side-effect of installing Rust sources to /usr/lib/rustlib/src:
$ equery files dev-lang/rust | grep lib.rs
/usr/lib/rustlib/src/rust/src/libcore/lib.rs
/usr/lib/rustlib/src/rust/src/libstd/lib.rs
(...)
The solution is therefore to enable the rls use-flag and then point Intellij IDEA to /usr/lib/rustlib/src/rust/src:
I believe this is a more idiomatic solution on Gentoo than to bypass portage and/or use rustup.
Note that dev-lang/rust-bin package currently does not have the rls use-flag and I haven't found a way to install Rust lib sources with it.
Contributions regarding dev-lang/rust use-flags (and their docs) in Gentoo would be probably appreciated.
I have an experimental technique that has different warts from the solution Sven found:
I installed rustup in addition to having the gentoo-managed rust ebuild.
$ wget -O sh.rustup.rs https://sh.rustup.rs
--2019-09-24 10:28:49-- https://sh.rustup.rs/
Resolving sh.rustup.rs... 13.249.122.62, 13.249.122.9, 13.249.122.96, ...
Connecting to sh.rustup.rs|13.249.122.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10782 (11K) [text/x-sh]
Saving to: 'sh.rustup.rs'
sh.rustup.rs 100%[===================>] 10.53K --.-KB/s in 0.001s
2019-09-24 10:28:49 (10.4 MB/s) - 'sh.rustup.rs' saved [10782/10782]
However, rustup is sad that I already have rust installed in /usr/bin
$ sh sh.rustup.rs
info: downloading installer
error: it looks like you have an existing installation of Rust at:
error: /usr/bin
error: rustup cannot be installed alongside Rust. Please uninstall first
error: if this is what you want, restart the installation with `-y'
error: cannot install while Rust is installed
Brute force "solves" this problem:
$ sh sh.rustup.rs -y
info: downloading installer
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2019-08-15, rust version 1.37.0 (eae3437df 2019-08-13)
info: downloading component 'rustc'
85.3 MiB / 85.3 MiB (100 %) 9.6 MiB/s in 9s ETA: 0s
info: downloading component 'rust-std'
61.2 MiB / 61.2 MiB (100 %) 8.2 MiB/s in 8s ETA: 0s
info: downloading component 'cargo'
info: downloading component 'rust-docs'
11.3 MiB / 11.3 MiB (100 %) 8.4 MiB/s in 1s ETA: 0s
info: installing component 'rustc'
85.3 MiB / 85.3 MiB (100 %) 18.4 MiB/s in 4s ETA: 0s
info: installing component 'rust-std'
61.2 MiB / 61.2 MiB (100 %) 20.9 MiB/s in 5s ETA: 0s
info: installing component 'cargo'
info: installing component 'rust-docs'
11.3 MiB / 11.3 MiB (100 %) 606.4 KiB/s in 9s ETA: 0s
info: default toolchain set to 'stable'
stable installed - rustc 1.37.0 (eae3437df 2019-08-13)
Rust is installed now. Great!
To get started you need Cargo's bin directory ($HOME/.cargo/bin) in your PATH
environment variable. Next time you log in this will be done automatically.
To configure your current shell run source $HOME/.cargo/env
After that you might want to rig rustup to use the same version the ebuild installed (or maybe you want to leave it tracking the latest stable, your choice)
$ equery list rust
* Searching for rust ...
[IP-] [ ] dev-lang/rust-1.34.2:stable/1.34
$ rustup default 1.34.2
info: using existing install for '1.34.2-x86_64-unknown-linux-gnu'
info: default toolchain set to '1.34.2-x86_64-unknown-linux-gnu'
1.34.2-x86_64-unknown-linux-gnu unchanged - rustc 1.34.2 (6c2484dc3 2019-05-13)
But you definitely want the rust-src component
$ rustup component add rust-src
info: downloading component 'rust-src'
info: installing component 'rust-src'
After that, the rust stdlib sources should be in $HOME/.rustup/toolchains/$VERSION-$ARCH/lib/rustlib/src/rust/src and you should be able to enter that path into the settings for the rust plugin.
You may also use that settings dialog to choose your toolchain (/usr/bin, $HOME/.cargo/bin, or $HOME/.rustup/toolchains/$VERSION-$ARCH/bin; I'm just guessing that .cargo/bin points to whatever rustup default you have set at the moment).
The drawback to this solution is that you now have 2 (or more) installations of rust. The system ebuild, plus any versions you have installed into $HOME/.cargo and $HOME/.rustup using rustup. There may be more problems, considering that I've only been using this for an hour or so.
Had the same problem. The solution I settled for is to use rustup and remove portage's rust from my system. You can then tell portage to assume rust is provided by using /etc/portage/package.provided to make emerges like firefox work with rustup's version.
Here is a short article describing the process: https://laumann.xyz/gentoo/2018/05/01/gentoo-package-provided.html
Iirc you don't need everything he puts in bash.profile at the end, but right now I can't tell you what exactly you can leave out
The paths mentioned by the other posts are correct:
"/usr/bin" for the "toolchain location".
"/usr/lib/rustlib/src/rust/library" for the "standard library".
After having set them the plugin was able to work correctly but the "autocomplete" for methods/functions that belong to Rust's standard library was still not working (same problem on all my 3 Gentoo notebooks, reproduced with CLion 2021.2 & 2021.3 + Rust plugin 0.4.164 and some other older versions + Rust 1.58.1 & 1.56.1).
For example, when typing "String::" I didn't get the list that shows its functions/methods (e.g. "from()", "as_bytes()", etc...).
To be exact, I that functionality was working the very first time that I set the correct paths, but not anymore after an explicit restart of CLion (it took me a while to realize this, made me crazy).
The root cause of the problem seems to be that the rust-plugin wants to be able to write to the stdlib-directories (I didn't analyze what it writes nor I'm going to speculate).
Example of an error message:
error: failed to write /opt/rust-bin-1.58.1/lib/rustlib/src/rust/library/unwind/Cargo.lock
Until CLion changes this behaviour the workaround would obviously be to make the location of Rust's standard library writeable by the user which is running CLion.
Doing that directly would of course trigger a lot of problems (security + pkg management), thererfore I decided to look for alternatives.
I decided to use OverlayFS to be able to still use Gentoo's original installation of Rust and at the same time to reroute the changes written by CLion to a separate directory.
In my case the details are:
1-
Created the 3 custom dirs needed by OverlayFS:
mkdir /opt/stecustom/clion/clion-rust_stdlib-overlay_upperdir
mkdir /opt/stecustom/clion/clion-rust_stdlib-overlay_workdir
mkdir /mnt/clion-rust_stdlib-overlay-result
2- Did an initial manual "mount":
mount -v -t overlay overlay -o noatime,lowerdir=/usr/lib/rustlib,upperdir=/opt/stecustom/clion/clion-rust_stdlib-overlay_upperdir,workdir=/opt/stecustom/clion/clion-rust_stdlib-overlay_workdir /mnt/clion-rust_stdlib-overlay-result
(the contents of "/usr/lib/rustlib" get on top the additional layer "/opt/stecustom/clion/clion-rust_stdlib-overlay_upperdir" to which all changes will be written - I don't know what "workdir" is supposed to host but it's a required parameter - and the merged layers are made available in " /mnt/clion-rust_stdlib-overlay-result":)
3- Changed in the overlay the ownership of all files:
cd /mnt/clion-rust_stdlib-overlay-result
chown -R MY_NON_ROOT_USER /mnt/clion-rust_stdlib-overlay-result/*
4- Made CLion's Rust plugin use the new overlayed dir:
"/usr/bin" for the "toolchain location" (same as before).
"/mnt/clion-rust_stdlib-overlay-result/src/rust/library" for the "standard library" (new!).
5- Restarted CLion and checked that everything worked
6- Made the overlayfs permanent in "/etc/fstab":
overlay /mnt/clion-rust_stdlib-overlay-result overlay noatime,lowerdir=/usr/lib/rustlib,upperdir=/opt/stecustom/clion/clion-rust_stdlib-overlay_upperdir,workdir=/opt/stecustom/clion/clion-rust_stdlib-overlay_workdir 0 2
7- Rebooted and checked again in CLion.
On a FreeBSD system, when executing "pkg upgrade", there is a message indicating the number of candidates. The pkg documentation does not clarify what these candidates are, or any way to list them.
Example output on a new system...
pkg upgrade
Checking for upgrades (1 candidates): 100%
Processing candidates (1 candidates): 100%
Checking integrity... done (0 conflicting)
Your packages are up to date.
Since the candidates remain after running the command, they cannot be an indication of upgradable packages. So what is a "candidate"? Is there any way to list the candidates?
A 'candidate' is an equivalent of 'potentially affected package'. Life example shows:
# pkg upgrade -n
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking for upgrades (251 candidates): 100%
Processing candidates (251 candidates): 100%
The following 250 package(s) will be affected (of 0 checked):
Installed packages to be REMOVED:
ImageMagick-6.9.9.28_2,1
New packages to be INSTALLED:
xkeyboard-config: 2.24_1
...
Number of packages to be removed: 1
Number of packages to be installed: 9
Number of packages to be upgraded: 195
Number of packages to be reinstalled: 45
I'm unable to install the crossbar.io WAMP router's binary distro on FreeBSD 10.1
I'm doing it step-by-step by the manual (http://crossbar.io/docs/Installation-on-FreeBSD/), but during the pkg install crossbar i get this output:
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 6 MiB 2.9MB/s 00:02
Processing entries: 100%
FreeBSD repository update completed. 25427 packages processed.
Updating crossbar repository catalogue...
crossbar repository is up-to-date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
crossbar: 0.13.2 [crossbar]
Number of packages to be installed: 1
The process will require 161 MiB more space.
Proceed with this action? [y/N]: y
[1/1] Installing crossbar-0.13.2...
[1/1] Extracting crossbar-0.13.2: 0%
pkg: Directory /opt/crossbar/lib-python/2.7/plat-mac/lib-scriptpackages/SystemEvents not specified in the manifest
[1/1] Extracting crossbar-0.13.2: 100%
And nothing's get installed only empty folders under /opt/crossbar
Am I doing something wrong?
Any input will be appreciated.