How is count_ones implemented in Rust? - rust

I tried looking for how Rust implements count_ones(). I'm curious because it seems to vastly outperform my own naive implementation (no kidding), and I would really like to see why it is so performant. My guess is that Rust is using some asm to do the work. For completeness, here was my attempt:
/*
* my attempt to implement count_ones for i32 types
* but this is much slower than the default
* implementation.
*/
fn count_ones(num: i32) -> u32 {
let mut ans: u32 = 0;
let mut _num = num;
while _num > 0 {
if _num & 0x1 == 0x1 {
ans += 1;
}
_num >>= 1;
}
ans
}
I found this on the rust repo, but I can't make sense of it (still new to Rust!) (reproduced below).
#[inline]
fn count_ones(self) -> u32 {
unsafe { $ctpop(self as $ActualT) as u32 }
}

Let's follow the code step-by-step.
First, looking at the snippet you've posted - it contains several macro variables (identifiers with a dollar sign prepended), so it is assumed that this code is, in fact, a part of macro definition. Scrolling up, we get the following:
macro_rules! uint_impl {
($T:ty = $ActualT:ty, $BITS:expr,
$ctpop:path,
$ctlz:path,
$cttz:path,
$bswap:path,
$add_with_overflow:path,
$sub_with_overflow:path,
$mul_with_overflow:path) => {
#[stable(feature = "rust1", since = "1.0.0")]
#[allow(deprecated)]
impl Int for $T {
// skipped
}
}
}
Now, to see that are the variable values here, we should find where this macro is invoked. In general, this might be hard, due to the macro scoping rules, but here we'll just search the same file, and here it is:
uint_impl! { u8 = u8, 8,
intrinsics::ctpop8,
intrinsics::ctlz8,
intrinsics::cttz8,
bswap8,
intrinsics::u8_add_with_overflow,
intrinsics::u8_sub_with_overflow,
intrinsics::u8_mul_with_overflow }
(and multiple another invocations). Comparing this with the macro definition, we see that the function we're looking for will be expanded to the following:
#[inline]
fn count_ones(self) -> u32 {
unsafe { intrinsics::ctpop8(self as u8) as u32 }
}
And, finally, intrinsics::ctpop8 is, as Stargateur mentioned in comment, an LLVM intrinsic, i.e. this call is directly converted into LLVM instruction.
However, there's a little better way to find out what is what.
Let's now look for the function we're interested in in the std documentation. Searching for count_ones brings together a bunch of functions, for each primitive number type independently; we'll take a look on the implementation for u8. Clicking the src link on the function brings us to the code:
doc_comment! {
concat!("Returns the number of ones in the binary representation of `self`.
# Examples
Basic usage:
```
", $Feature, "let n = 0b01001100", stringify!($SelfT), ";
assert_eq!(n.count_ones(), 3);", $EndFeature, "
```"),
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_stable(feature = "const_math", since = "1.32.0")]
#[inline]
pub const fn count_ones(self) -> u32 {
intrinsics::ctpop(self as $ActualT) as u32
}
}
...which just directly calls the intrinsics::ctpop function we've found before.
Now you might wonder, why these two searches yielded different pieces of code. Reason is simple: the commit you're referring to is from the fairly old version of rustc - pre-1.0, if I understand correctly; at that time, numerical operations were implemented as part of Num trait, not directly on primitive types. If you check out the implementation for version 1.44.1, which is the current one at the time of writing, you'll see the same code I've quoted above from the docs.

Related

How to define one function for two types in Rust?

I am writing a Rust application. I'd like to have a method to display the text whether it is a string or number. Furthermore, I came up with the following solution, but it is duplicating the code. Is there a better way to do it in Rust?
Notice: I am not looking for a built-in function to print variables. It's just an example. I am looking for a way to implement the same feature for two types.
trait Display {
fn display(self);
}
impl Display for String {
fn display(self) -> () {
println!("You wrote: {}", self);
}
}
impl Display for i32 {
fn display(self) -> () {
println!("You wrote: {}", self);
}
}
fn main() {
let name: String = String::from("Tom Smykowski");
name.display();
let age: i32 = 22;
age.display();
}
You came close. But there is already a trait for converting things to strings - std::fmt::Display (and the automatically-implemented ToString) - so you don't need to have your own trait:
fn display<T: std::fmt::Display>(v: T) {
println!("You wrote: {v}");
}
fn main() {
let name: String = String::from("Tom Smykowski");
display(name);
let age: i32 = 22;
display(age);
}
Even if you don't need to display the types but do something else with them, we can take the idea from Display - instead of defining the whole functionality, define only the pieces that are different. For example, you can create a trait to convert the numbers to strings (or the opposite), or just have functions for each different piece - for example, printing itself without "You wrote: ".
I came up with the following solution, but it is duplicating the code. Is there a better way to do it in Rust?
Add a simple declarative macro on top, that is very common in the stdlib and all. e.g.
macro_rules! impl_display {
($t:ty) => {
impl Display for $t {
fn display(self) {
println!("You wrote {self}");
}
}
}
}
impl_display!(String);
impl_display!(i32);
impl_display!(i64);
impl_display!(f32);
Although:
usually the implementations would be different, though not always e.g. implementing an operation on all numeric types, or all unsigned numbers, that's one of the most common context you'll see it in the stdlib: the stdlib has no numeric trait but methods are usually implemented on all numeric types, so there's a handful of macros used for all of them, and when new methods are added they're just added to the relevant macro
here you're already relying on the existence and implementation of std::fmt::Display so you should just use that, your trait is not really useful

Metaprogamming name to function and type lookup in Rust?

I am working on a system which produces and consumes large numbers of "events", they are a name with some small payload of data, and an attached function which is used as a kind of fold-left over the data, something like a reducer.
I receive from the upstream something like {t: 'fieldUpdated', p: {new: 'new field value'}}, and must in my program associate the fieldUpdated "callback" function with the incoming event and apply it. There is a confirmation command I must echo back (which follows a programatic naming convention), and each type is custome.
I tried using simple macros to do codegen for the structs, callbacks, and with the paste::paste! macro crate, and with the stringify macro I made quite good progress.
Regrettably however I did not find a good way to metaprogram these into a list or map using macros. Extending an enum through macros doesn't seem to be possible, and solutions such as the use of ctors seems extremely hacky.
My ideal case is something this:
type evPayload = {
new: String
}
let evHandler = fn(evPayload: )-> Result<(), Error> { Ok(()) }
// ...
let data = r#"{"t": 'fieldUpdated', "p": {"new": 'new field value'}}"#'
let v: Value = serde_json::from_str(data)?;
Given only knowledge of data how can use macros, specifically (boilerplate is actually 2-3 types, 3 functions, some factory and helper functions) in a way that I can do a name-to-function lookup?
It seems like Serde's adjacently, or internally tagged would get me there, if I could modify a enum in a macro https://serde.rs/enum-representations.html#internally-tagged
It almost feels like I need a macro which can either maintain an enum, or I can "cheat" and use module scoped ctors to do a quasi-static initialization of the names and types into a map.
My program would have on the order of 40-100 of these, with anything from 3-10 in a module. I don't think ctors are necessarily a problem here, but the fact that they're a little grey area handshake, and that ctors might preclude one day being able to cross-compile to wasm put me off a little.
I actually had need of something similar today; the enum macro part specifically. But beware of my method: here be dragons!
Someone more experienced than me — and less mad — should probably vet this. Please do not assume my SAFETY comments to be correct.
Also, if you don't have variant that collide with rust keywords, you might want to tear out the '_' prefix hack entirely. I used a static mut byte array for that purpose, as manipulating strings was an order of magnitude slower, but that was benchmarked in a simplified function. There are likely better ways of doing this.
Finally, I am using it where failing to parse must cause panic, so error handling somewhat limited.
With that being said, here's my current solution:
/// NOTE: It is **imperative** that the length of this array is longer that the longest variant name +1
static mut CHECK_BUFF: [u8; 32] = [b'_'; 32];
macro_rules! str_enums {
($enum:ident: $($variant:ident),* $(,)?) => {
#[allow(non_camel_case_types)]
#[derive(Debug, Default, Hash, Clone, PartialEq, Eq, PartialOrd, Ord)]
enum $enum {
#[default]
UNINIT,
$($variant),*,
UNKNOWN
}
impl FromStr for $enum {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
unsafe {
// SAFETY: Currently only single threaded
CHECK_BUFF[1..len].copy_from_slice(s.as_bytes());
let len = s.len() + 1;
assert!(CHECK_BUFF.len() >= len);
// SAFETY: Safe as long as CHECK_BUFF.len() >= s.len() + 1
match from_utf8_unchecked(&CHECK_BUFF[..len]) {
$(stringify!($variant) => Ok(Self::$variant),)*
_ => Err(format!(
"{} variant not accounted for: {s} ({},)",
stringify!($enum),
from_utf8_unchecked(&CHECK_BUFF[..len])
))
}
}
}
}
impl From<&$enum> for &'static str {
fn from(variant: &$enum) -> Self {
unsafe {
match variant {
// SAFETY: The first byte is always '_', and stripping it of should be safe.
$($enum::$variant => from_utf8_unchecked(&stringify!($variant).as_bytes()[1..]),)*
$enum::UNINIT => {
eprintln!("uninitialized {}!", stringify!($enum));
""
}
$enum::UNKNOWN => {
eprintln!("unknown {}!", stringify!($enum));
""
}
}
}
}
}
impl Display for $enum {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", Into::<&str>::into(self))
}
}
};
}
And then I call it like so:
str_enums!(
AttributeKind:
_alias,
_allowduplicate,
_altlen,
_api,
...
_enum,
_type,
_struct,
);
str_enums!(
MarkupKind:
_alias,
_apientry,
_command,
_commands,
...
);

What is the equivalent of explicit template instantiation in Rust's const generics?

In C++ I can postpone the implementation of template types and functions such that I can provide only implementations for the types I explicitly support.
For example, I can provide in C++ only the versions of doit<int32_t> and doit<int64_t>() that are covered by my unit tests. This also has the upside to shorten the compilation times since the implementation of doit<T> is not recompiled at every compilation unit it is included.
// File doit.h
template < typename T > void doit();
// File doit.cpp
template < typename T > void doit() { ...implementation... };
template void doit<int32_t>();
template void doit<uint32_t>();
How do I achieve the same with Rust's const generics? For example, for the function below, I only want to provide implementations for BITS in 0..=5.
pub fn ilog2< const BITS : u32 >( num: u64 ) -> u32 {
let maxnum : u64 = 1u64 << BITS;
if num<maxnum { return num as u32; }
let mask : u64 = maxnum - 1;
let msb : u32 = 63 - num.leading_zeros();
let off : u64 = (num >> (msb - BITS)) & mask;
return off as u32 + ((msb - BITS + 1) << BITS);
}
The Rust solution to the first case is fairly straight-forward, you can constrain the generic type to have certain properties. In this case you can define a trait that is implemented for only i32 and u32:
pub trait Is32BitInteger {}
impl Is32BitInteger for i32 {}
impl Is32BitInteger for u32 {}
pub fn doit<T>() where T: Is32BitInteger {}
However keep in mind that Rust generics only behave as they're constrained to behave. You'll need to add functions to the trait or add other constraints depending on what you want to do with T within doit.
Unfortunately, there isn't really a solution to the second case. Const generics in Rust are still taking their first steps, and as such there's a lot of limitations. I only know of two possibilities but both require incomplete nightly features:
the first uses feature(generic_const_exprs) and the Assert::<{...}>: IsTrue trick that you might find on around the github issues on const generics: playground.
the second uses feature(adt_const_params) to make a wrapper type instead of u32 whose invariant is that the value is constrained to 0..=5 that panics otherwise (causing a compiler error): playground.
I wouldn't necessarily recommend using either of these due to their unstable nature. In the future, we may get syntax akin to where BITS <= 5 more directly, maybe.

Why can associated constants not depend on type parameters?

Is this just a current limitation or is there a technical reason? As generic functions get compiled to specialized code, I don’t see what should prevent it from working. It also works fine in the main function.
Example (playground):
#![feature(associated_consts)]
trait HasNumber<T> {
const Number: usize;
}
enum One {}
enum Two {}
enum Foo {}
impl<T> HasNumber<One> for T {
const Number: usize = 1;
}
impl<T> HasNumber<Two> for T {
const Number: usize = 2;
}
fn use_number<T, H: HasNumber<T>>() {
let a: [u8; H::Number] = unsafe { ::std::mem::uninitialized() };
}
fn main() {
let a: [u8; <Foo as HasNumber<One>>::Number] = unsafe { ::std::mem::uninitialized() };
println!("{}", <Foo as HasNumber<One>>::Number);
println!("{}", <Foo as HasNumber<Two>>::Number);
}
Short answer: It's not implemented yet, since it's hard to get right. There's even an open RFC named "Constants that depend on type parameters in generic code" for it.
Long answer:
This used to be a compiler-bug that caused the compiler to crash. It was "fixed" by #quantheory in PR 25091 by making this an error instead of a crash. #quantheory commented that
I haven't been able to deal with the array size or recursion issues yet for associated consts, though my hope was that the change I made for range match patterns might help with array sizes, too.
#quantheory also notes that this will stay an error until something along the lines of RFC 1062 is merged. Comments on the RFC are always appreciated, as they might hilight forgotten use-cases.

Why are booleans copyable even though the documentation doesn't indicate that?

I am trying to understand the ownership and borrowing concept. At first I thought it was pretty simple once you understood it. But...
fn main() {
let a = 5;
let _y = double(a);
println!("{}", a);
}
fn double(x: i32) -> i32 {
x * 2
}
At first I would have expected this to not compile, because a would have been moved to _y.
I was a bit confused, but I found out that I would have been right except that i32 is an exception to the rule because it implements the copy trait.
I looked at the Copy trait and as I understand it, they list all types that implement this trait at the bottom.
So the bool type is not present and so I assumed it's default behaviour was to be "moved". But...
fn main() {
let a = true;
let _y = change_truth(a);
println!("{}", a);
}
fn change_truth(x: bool) -> bool {
!x
}
Doesn't fail either.
Now I am quite confused. I found the Clone trait that seems to be closely related to the copy trait. But unless I missed it, they don't really mention it in the learning doc.
Can someone give me some more info ?
Update:
I have filed an issue on the Rust repository.
I have also made a pull request with some change proposals.
Your understanding is pretty spot-on, this seems to be an issue with the docs. The documentation doesn't show Copy instances for any of the primitives types, even though they are definitely Copy. As an example to show that the compiler considers bool to be Copy, the following compiles just fine:
fn takes_copyable<T: Copy>(foo: T) {}
fn main() {
takes_copyable(true);
}

Resources