Can't understand rust lifetime conflict - rust

I was doing a dummy app to get a grasp on Rust concepts.
While doing an XML structure I got the error
cannot infer an appropriate lifetime for lifetime parameter in
function call due to conflicting requirements
The definition is
impl<'a> XmlFile<'a>
and
pub fn get_node<'b>(self, node: &'b [u8]) -> &'b [u8]
From what I understand, the Rust compiler does not like that the return variable can be dropped after the function ends, if the XML file drops at a different time (since they have 'a and 'b lifetimes).
But if I put the same, I get the error
lifetime 'a is already in scope
, so I don't see a way to solve the error.
Any idea what I am missing? I think I must still be lacking some Rust concept.
Edit: Misconception from my part adding the code that causes the problem
#[allow(unused_parens)]
pub struct XmlFile<'a> {
last_open_node: &'a[u8],
last_published: String,
index_pos: u64,
content: &'a[u8],
}
impl<'a> XmlFile<'a> {
pub fn new<'b: 'a>(file: &'b [u8]) -> XmlFile<'b> {
let new_xml = XmlFile {
last_open_node: &[0: u8],
last_published: "".to_string(),
index_pos: 0,
content: file,
};
return new_xml;
}
pub fn get_node<'b: 'a>(&self, node: &'b [u8]) -> &'b [u8] {
let buf_index: u64 = 0;
let has_matched: bool = false;
self.index_pos = 0;
for c in self.content {
self.index_pos += 1;
if (c == &b'<') {
buf_index = self.index_pos;
while (c != &b' ') {
for b in node {
if b == &self.content[buf_index as usize] {
has_matched = true;
buf_index += 1
} else {
has_matched = false;
continue;
}
}
if has_matched {
while(self.content[buf_index as usize] != b'>'){
buf_index+=1;
}
let r = &self.content[self.index_pos as usize..buf_index as usize];
return r;
}
}
}
}
return &[0 : u8];
}
pub fn get_rss_version<'b:'a>(&self) -> Result<u64 , &'static str>{
let found_slice = Self::get_node(&self, "rss".as_bytes());
if(found_slice != &[0:u8]){
let version_value = Self::get_value(found_slice);
if(version_value.is_ok()){
return Ok(version_value.unwrap()) ;
}
else{
return Err("Couldn't retrieve version from tag");
}
}
else{
println!("Couldn't find tag <rss");
return Err("Couldn't find tag <rss");
}
}
}

Let's look at your signature for get_node:
pub fn get_node<'b: 'a>(&mut self, node: &'b [u8]) -> &'b [u8] { ... }
and what you're actually returning within this method:
let r = &self.content[self.index_pos as usize..buf_index as usize];
return r;
The signature for get_node indicates this method will be returning a sub-slice of node, but you're actually returning a sub-slice of the XmlFile's content.
One solution to the problem is to understand that the return value isn't a part of node, but instead a part of self.content. Therefore, we can change the signature to:
pub fn get_node<'b>(&mut self, node: &'b [u8]) -> &'a [u8] { ... }
In this case we can even elide the manual specification of lifetimes entirely:
pub fn get_node(&mut self, node: &[u8]) -> &[u8] { ... }
Here's a cleaned up version of your get_node method that actually compiles:
pub fn get_node(&mut self, node: &[u8]) -> &[u8] {
let mut buf_index: u64;
let mut has_matched: bool = false;
self.index_pos = 0;
for c in self.content {
self.index_pos += 1;
if c == &b'<' {
buf_index = self.index_pos;
while c != &b' ' {
for b in node {
if b == &self.content[buf_index as usize] {
has_matched = true;
buf_index += 1
} else {
has_matched = false;
continue;
}
}
if has_matched {
while self.content[buf_index as usize] != b'>' {
buf_index += 1;
}
let r = &self.content[self.index_pos as usize..buf_index as usize];
return r;
}
}
}
}
return &[0u8];
}

Related

rust lockfree ringbuffer can't work on release mode

I implement a lockfree ringbuffer, and then i test for debug is ok, but in release mode it can't work allways.
use std::path::Display;
use std::sync::Arc;
#[derive(Debug)]
pub struct RingBuffer<T, const m_size: usize> {
idx_head: usize,
idx_tail: usize,
m_data: [T; m_size],
}
pub trait Queue<T> {
fn new_empty() -> Self;
fn push(&mut self, value: T) -> bool;
fn pop(&mut self) -> Option<&T>;
fn is_full(&self) -> bool;
fn is_empty(&self) -> bool;
}
impl<T, const Size: usize> Queue<T> for RingBuffer<T, Size>
{
fn new_empty() -> Self {
RingBuffer::<T, Size> {
idx_head: 0,
idx_tail: 0,
m_data: array_init::array_init(|_| {
unsafe {
std::mem::zeroed()
}
}),
}
}
fn push(&mut self, value: T) -> bool {
let mut head = self.idx_head + 1;
if head == Size {
head = 0;
}
if head == self.idx_tail {
return false;
}
self.m_data[self.idx_head] = value;
self.idx_head = head;
return true;
}
fn pop(&mut self) -> Option<&T> {
let mut tail = self.idx_tail;
if self.idx_head == tail {
return None;
}
let res = &self.m_data[tail];
tail += 1;
if tail == Size {
tail = 0;
}
self.idx_tail = tail;
return Some(res);
}
fn is_full(&self) -> bool {
self.idx_tail == (self.idx_head + 1) % Size
}
fn is_empty(&self) -> bool {
self.idx_head == self.idx_tail
}
}
pub struct SharedRingBuffer<T, const m_size: usize> {
pub ringbuffer: Arc<RingBuffer<T, m_size>>,
}
impl<T, const Size: usize> Clone for SharedRingBuffer<T, Size> {
fn clone(&self) -> Self {
Self {
ringbuffer: self.ringbuffer.clone(),
}
}
}
impl<T, const Size: usize, > Queue<T> for SharedRingBuffer<T, Size> {
fn new_empty() -> Self {
Self {
ringbuffer: Arc::new(RingBuffer::<T, Size>::new_empty()),
}
}
fn push(&mut self, value: T) -> bool {
unsafe {
(*Arc::get_mut_unchecked(&mut self.ringbuffer)).push(value)
}
}
fn pop(&mut self) -> Option<&T> {
unsafe {
(*Arc::get_mut_unchecked(&mut self.ringbuffer)).pop()
}
}
fn is_full(&self) -> bool {
self.ringbuffer.is_full()
}
fn is_empty(&self) -> bool {
self.ringbuffer.is_empty()
}
}
////////////////////// for test//////////////////////////
fn test_speed1() {
let mut q: SharedRingBuffer<i32, 8> = SharedRingBuffer::new_empty();
let mut t0 = std::time::SystemTime::now();
let t = {
let mut q = q.clone();
std::thread::spawn(move || {
loop {
let t = match q.pop() {
None => {
// std::thread::sleep(Duration::from_millis(10));
continue;
}
Some(res) => res
};
if *t == -1 {
break;
}
std::thread::sleep(Duration::from_millis(1));
}
let now = std::time::SystemTime::now();
println!("res: {}", now.duration_since(t0).unwrap().as_millis());
})
};
for i in 0..99 {
loop {
if q.push(i) {
// std::thread::sleep(Duration::from_millis(10));
break;
}
}
}
q.push(-1);
t.join().unwrap();
}
When i addition std::thread::sleep(Duration::from_millis(10)) for q.push and q.pop method it is work well.
rustc 1.67.0-nightly (95a3a7277 2022-10-31)
binary: rustc
commit-hash: 95a3a7277b44bbd2dd3485703d9a05f64652b60e
commit-date: 2022-10-31
host: x86_64-pc-windows-msvc
release: 1.67.0-nightly
LLVM version: 15.0.4
I expect the RingBuffer can work well.
The equivalent code is:
fn test_speed2() {
let (send, recv) = channel::<i32>();
let mut is_run = SharedValue::new(true);
let mut t0 = std::time::SystemTime::now();
let t = {
let is_run = is_run.clone();
std::thread::spawn(move || {
loop {
let t = match recv.recv() {
Err(e) => {
break;
}
Ok(res) => res
};
if t == -1 {
break;
}
std::thread::sleep(Duration::from_millis(1));
}
let now = std::time::SystemTime::now();
// println!("res: {}", now.duration_since(t0).unwrap().as_millis());
})
};
for i in 0..99 {
send.send(i).unwrap();
}
send.send(-1).unwrap();
t.join().unwrap();
}
I hope ringbuffer can replace channel to communicate between two threads,Because ringbuffer is lockfree and faster.
Your code causes undefined behavior by creating two mutable references to the same object at the same time via Arc::get_mut_unchecked(). It looks like this was even your intention, but it is blatantly violating Rust's rules. Even when using unsafe, you cannot violate the requirement that mutable references are exclusive.
Running your code with cargo miri reports this undefined behavior:
error: Undefined Behavior: Data race detected between Read on thread `<unnamed>` and Write on thread `main` at alloc1894+0x10
--> bar/src/main.rs:45:12
|
45 | if self.idx_head == tail {
| ^^^^^^^^^^^^^ Data race detected between Read on thread `<unnamed>` and Write on thread `main` at alloc1894+0x10
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: BACKTRACE:
= note: inside `<RingBuffer<i32, 8> as Queue<i32>>::pop` at bar/src/main.rs:45:12
note: inside `<SharedRingBuffer<i32, 8> as Queue<i32>>::pop` at bar/src/main.rs:89:18
--> bar/src/main.rs:89:18
|
89 | unsafe { (*Arc::get_mut_unchecked(&mut self.ringbuffer)).pop() }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at bar/src/main.rs:108:31
--> bar/src/main.rs:108:31
|
108 | let t = match q.pop() {
| ^^^^^^^
You will need to rethink your design. You'll probably need a foundation like this to make it safe to modify between threads:
use std::cell::UnsafeCell;
use std::mem::MaybeUninit;
use std::sync::atomic::AtomicUsize;
pub struct RingBuffer<T, const SIZE: usize> {
idx_head: AtomicUsize,
idx_tail: AtomicUsize,
m_data: [UnsafeCell<MaybeUninit<T>>; SIZE],
}
This is actually caused by the CPU cache,The solution is as follows:
fn push(&mut self, value: T) -> bool {
let mut head = unsafe {
std::ptr::read_volatile(&self.idx_head) + 1
};
let tail = unsafe {
std::ptr::read_volatile(&self.idx_tail)
};
if head == Size {
head = 0;
}
if head == tail {
return false;
}
self.m_data[self.idx_head] = value;
unsafe {
std::ptr::write_volatile(&mut self.idx_head, head);
}
return true;
}
fn pop(&mut self) -> Option<&T> {
let mut tail = unsafe {
std::ptr::read_volatile(&self.idx_tail)
};
let head = unsafe {
std::ptr::read_volatile(&self.idx_head)
};
if head == tail {
return None;
}
let res = &self.m_data[tail];
tail += 1;
if tail == Size {
tail = 0;
}
unsafe {
std::ptr::write_volatile(&mut self.idx_tail, tail);
}
return Some(res);
}

How to test result of core::fmt::Display trait implementation in #![no_std] env

I have following struct and want to test implementation of Display trait:
use core::fmt::{Display, Formatter, Result};
struct B {}
impl Display for B {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
write!(f, "A {} C", "B")
}
}
#[test]
fn it_works() {
assert_eq!(format!("{}", B {}), "A B C")
}
it works in general case, however in #![no_std] environment it yields an error due to the fact that format! macro is missing (it allocates, so cannot be used in no_std).
Is there idiomatic way to test core::std::Display trait in no_std scenarios?
I've tried to create my own core::std::Formatter instance with custom buffer implementation to somehow circumvent it, and test fmt method directly, but instancing this type is considered compiler internal.
The best option is to enable std for tests:
#![cfg_attr(no_std, not(test))]
If you cannot do it, and you know an upper limit for the format size, you can implement core::fmt::Write for a u8 array wrapper and using it. The arrayvec has already got you covered with ArrayString:
let mut formatted = ArrayString::<10>::new();
use core::fmt::Write;
write!(formatted, "{}", B {}).expect("failed to format, probably buffer too small");
assert_eq!(&formatted, "A B C")
Playground.
Why not just implement core::fmt::Write on your custom buffer and use core::fmt::write! to it?
format! is basically a write! to a string buffer, both really call Write::write_fmt, with format! having the conveniences that it provides its own (string) buffer and panics on error.
macro_rules! write {
($dst:expr, $($arg:tt)*) => {
$dst.write_fmt($crate::format_args!($($arg)*))
};
}
macro_rules! format {
($($arg:tt)*) => {{
let res = $crate::fmt::format($crate::__export::format_args!($($arg)*));
res
}}
}
pub fn format(args: Arguments<'_>) -> string::String {
let capacity = args.estimated_capacity();
let mut output = string::String::with_capacity(capacity);
output.write_fmt(args).expect("a formatting trait implementation returned an error");
output
}
It's possible to do it without allocation, and without any buffer, just by implementing core::fmt::Write trait, as #Masklinn mentioned.
Here is sample implementation:
#![no_std]
use core::fmt::{Display, Formatter, Result, Write};
struct B {}
impl Display for B {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
write!(f, "A {} C", "B")
}
}
struct Comparator<'a> {
valid: bool,
to_compare: &'a str
}
impl<'a> Comparator<'a> {
fn new(s: &'a str) -> Self {
Self { valid: true, to_compare: s }
}
fn is_valid(self) -> bool {
self.valid && self.to_compare.is_empty()
}
}
impl<'a> Write for Comparator<'a> {
fn write_str(&mut self, s: &str) -> Result {
if s.eq(self.to_compare) {
self.valid = self.valid && true;
self.to_compare = "";
return Ok(());
}
if self.to_compare.starts_with(s) && self.to_compare.len() >= s.len() {
self.to_compare = &self.to_compare[s.len()..];
} else {
self.valid = false
}
Ok(())
}
}
#[test]
fn it_works() {
let mut cmp = Comparator::new("A B C");
let _ = write!(&mut cmp, "{}", B{});
assert!(cmp.is_valid());
}
#[test]
fn too_short() {
let mut cmp = Comparator::new("A B");
let _ = write!(&mut cmp, "{}", B{});
assert!(!cmp.is_valid());
}
#[test]
fn too_long() {
let mut cmp = Comparator::new("A B C D");
let _ = write!(&mut cmp, "{}", B{});
assert!(!cmp.is_valid());
}
EDIT: fixed bug in the code, was:
if self.to_compare.starts_with(s) && self.to_compare.len() >= s.len() {
self.to_compare = &self.to_compare[s.len()..];
self.valid = true
} else {
self.valid = false
}
fixed:
if self.to_compare.starts_with(s) && self.to_compare.len() >= s.len() {
self.to_compare = &self.to_compare[s.len()..];
} else {
self.valid = false
}

Rust: Modifying to an object contained in a Vec from another object of the same Vec

I have a Vec<dyn MyObj> and the first implementation of MyObj contained in the Vec can contains something that refers to a second implementation of MyObj contained in the same Vec.
I'd like that the first implementation can mutate the second implementation.
Here is my first idea:
use std::{rc::{Rc, Weak}, cell::RefCell};
trait MyObj {
fn f(&mut self);
}
type CRef = Weak<RefCell<Container>>;
struct Container {
list: Vec<Box<dyn MyObj>>,
this: CRef,
}
impl Container {
fn new() -> Rc<RefCell<Self>> {
let res = Rc::new(RefCell::new(Self {
list: vec![],
this: Weak::new(),
}));
{
let this = Rc::downgrade(&res);
let mut ref_on_res = res.borrow_mut();
ref_on_res.this = this;
}
res
}
fn register(&mut self, v: impl MyObj + 'static) -> ObjRef {
let index = self.list.len();
self.list.push(Box::new(v));
ObjRef::new(self.this.clone(), index)
}
fn get(&mut self, index: usize) -> &mut (dyn MyObj + 'static) {
let elt = &mut self.list[index];
Box::as_mut(elt)
}
}
struct ObjRef {
c: CRef,
i: usize,
}
impl ObjRef {
fn new(c: CRef, i: usize) -> Self {
Self { c, i }
}
}
impl MyObj for ObjRef {
fn f(&mut self) {
let i = self.i;
self.c.upgrade().map(|c| c.borrow_mut().get(i).f());
}
}
struct A {
r: ObjRef,
}
// First implementation
impl A {
fn new(r: ObjRef) -> A {
A { r }
}
}
impl MyObj for A {
fn f(&mut self) {
self.r.f();
}
}
// Second implementation
struct B(usize);
impl MyObj for B {
fn f(&mut self) {
self.0 += 1;
println!("B({})", self.0);
}
}
fn main() {
let c = Container::new();
let mut r = {
let b = c.borrow_mut().register(B(100));
c.borrow_mut().register(A::new(b))
};
r.f(); // -> Panic: already borrowed: BorrowMutError
}
Obviously, it panics and i understand why but i have no idea to fix this problem.
Have you any idea to do this kind of modification ?

How to define mutual recursion with closures?

I can do something like this:
fn func() -> (Vec<i32>, Vec<i32>) {
let mut u = vec![0;5];
let mut v = vec![0;5];
fn foo(u: &mut [i32], v: &mut [i32], i: usize, j: usize) {
for k in i+1..u.len() {
u[k] += 1;
bar(u, v, k, j);
}
}
fn bar(u: &mut [i32], v: &mut [i32], i: usize, j: usize) {
for k in j+1..v.len() {
v[k] += 1;
foo(u, v, i, k);
}
}
foo(&mut u, &mut v, 0, 0);
(u,v)
}
fn main() {
let (u,v) = func();
println!("{:?}", u);
println!("{:?}", v);
}
but I would prefer to do something like this:
fn func() -> (Vec<i32>, Vec<i32>) {
let mut u = vec![0;5];
let mut v = vec![0;5];
let foo = |i, j| {
for k in i+1..u.len() {
u[k] += 1;
bar(k, j);
}
};
let bar = |i, j| {
for k in j+1..v.len() {
v[k] += 1;
foo(i, k);
}
};
foo(0, 0);
(u,v)
}
fn main() {
let (u,v) = func();
println!("{:?}", u);
println!("{:?}", v);
}
The second example doesn't compile with the error: unresolved name bar.
In my task I can do it through one recursion, but it will not look clear.
Does anyone have any other suggestions?
I have a solution for mutually recursive closures, but it doesn't work with multiple mutable borrows, so I couldn't extend it to your example.
There is a way to use define mutually recursive closures, using an approach similar to how this answer does single recursion. You can put the closures together into a struct, where each of them takes a borrow of that struct as an extra argument.
fn func(n: u32) -> bool {
struct EvenOdd<'a> {
even: &'a Fn(u32, &EvenOdd<'a>) -> bool,
odd: &'a Fn(u32, &EvenOdd<'a>) -> bool
}
let evenodd = EvenOdd {
even: &|n, evenodd| {
if n == 0 {
true
} else {
(evenodd.odd)(n - 1, evenodd)
}
},
odd: &|n, evenodd| {
if n == 0 {
false
} else {
(evenodd.even)(n - 1, evenodd)
}
}
};
(evenodd.even)(n, &evenodd)
}
fn main() {
println!("{}", func(5));
println!("{}", func(6));
}
While defining mutually recursive closures works in some cases, as demonstrated in the answer by Alex Knauth, I don't think that's an approach you should usually take. It is kind of opaque, has some limitations pointed out in the other answer, and it also has a performance overhead since it uses trait objects and dynamic dispatch at runtime.
Closures in Rust can be thought of as functions with associated structs storing the data you closed over. So a more general solution is to define your own struct storing the data you want to close over, and define methods on that struct instead of closures. For this case, the code could look like this:
pub struct FooBar {
pub u: Vec<i32>,
pub v: Vec<i32>,
}
impl FooBar {
fn new(u: Vec<i32>, v: Vec<i32>) -> Self {
Self { u, v }
}
fn foo(&mut self, i: usize, j: usize) {
for k in i+1..self.u.len() {
self.u[k] += 1;
self.bar(k, j);
}
}
fn bar(&mut self, i: usize, j: usize) {
for k in j+1..self.v.len() {
self.v[k] += 1;
self.foo(i, k);
}
}
}
fn main() {
let mut x = FooBar::new(vec![0;5], vec![0;5]);
x.foo(0, 0);
println!("{:?}", x.u);
println!("{:?}", x.v);
}
(Playground)
While this can get slightly more verbose than closures, and requires a few more explicit type annotations, it's more flexible and easier to read, so I would generally prefer this approach.

Cannot modify a struct field from implementation: "cannot borrow immutable borrowed content as mutable"

I'm trying to implement an iterator which will yield prime numbers. I store already found prime numbers in a Vec<u64>.
Here's my implementation:
struct Primes {
primes: Vec<u64>,
}
impl Primes {
fn new() -> Primes {
Primes { primes: vec!(2, 3) }
}
fn iter(&self) -> PrimesIterator {
PrimesIterator { primes: &self.primes, index : 0 }
}
}
struct PrimesIterator<'a> {
primes: & 'a Vec<u64>,
index: usize,
}
impl<'a> Iterator for PrimesIterator<'a> {
type Item = u64;
fn next(&mut self) -> Option<u64> {
if self.index < self.primes.len() {
let result = self.primes[self.index];
self.index += 1;
Some(result)
} else {
let mut n = *self.primes.last().unwrap();
loop {
n += 2;
if is_prime(self.primes, n) {
self.primes.push(n);
self.index += 1;
return Some(n);
}
}
}
}
}
fn is_prime(primes: &[u64], n: u64) -> bool {
for &p in primes.iter() {
if n % p == 0 {
return false;
}
if p * p > n {
return true;
}
}
return false;
}
but when I'm trying to compile it, I'm getting the following error:
main.rs: error: cannot borrow immutable borrowed content `*self.primes` as mutable
main.rs: self.primes.push(n);
I declared self as &mut so I don't really understand what's wrong here and how to fix that.
Your PrimesIterator type contains a non-mutable reference to a Vec<u64>. You need to declare it as a mutable reference:
struct PrimesIterator<'a> {
primes: &'a mut Vec<u64>,
index: usize,
}
This will of course require you to also modify the iter() function to make sure it passes a mutable reference:
impl Primes {
fn new() -> Primes {
Primes { primes: vec!(2, 3) }
}
fn iter(&mut self) -> PrimesIterator {
PrimesIterator { primes: &mut self.primes, index : 0 }
}
}

Resources