I am trying to make my own programming language in rust, and most features are done, so I thought I could add to the UnknownIdentifier error the ability to find the closest match to whatever the user wanted
However before I even got to finding the closest match I found out that cloning HashMap<String, Object> moves it into the closure
ErrorGenerator::error function:
#[allow(non_snake_case)]
mod ErrorGenerator {
pub fn error(name: &str, explanation: &str, line: usize, col: usize, file: String, after_f: Box<dyn Fn() -> ()>) -> ! {
eprintln!("\n[ERROR] {}, Line {:?}, Column {:?}", file, line, col);
eprintln!(" {}: {}", name, explanation);
after_f();
exit(1);
}
}
ErrorGenerator::error(
"UnknownIdentifier",
&format!("unknown identifier: `{}`, this identifier could not be found", tokenc.repr()),
tokenc.line,
tokenc.col,
tokenc.file,
Box::new(||{
let mut hashc: Vec<String> = hashs.clone().into_keys().collect();
hashc.sort();
}),
);
This is the error it gives:
error[E0597]: `hashs` does not live long enough
--> src/runtime/runtime.rs:960:70
|
959 | Box::new(||{
| - -- value captured here
| _____________________________________|
| |
960 | | let mut hashc: Vec<String> = hashs.clone().into_keys().collect();
| | ^^^^^ borrowed value does not live long enough
961 | | hashc.sort();
962 | | }),
| |______________________________________- cast requires that `hashs` is borrowed for `'static`
...
1203 | }
| - `hashs` dropped here while still borrowed
The problem's solution is probably either:
A way to borrow in 'static lifetime a mutable variable created in a method into a closure
or
A way to clone HashMap<String, Object> without moving it into the closure
You can find the full code in https://github.com/kaiserthe13th/tr-lang/tree/unknown-id-err-impl
What happens is that the compiler doesn't clone hashs then passes the clone to your callback; instead, it passes a reference to hashs to your callback and clones it inside the callback.
However, the callback is required to be 'static, and if it holds a reference to the containing function it is not! So the compiler is complaining.
What you want is to clone the hashmap before, then pass the clone to the callback. Like:
ErrorGenerator::error(
"UnknownIdentifier",
&format!("unknown identifier: `{}`, this identifier could not be found", tokenc.repr()),
tokenc.line,
tokenc.col,
tokenc.file,
{
let hashc = hashs.clone();
Box::new(|| {
let mut hashc: Vec<String> = hashc.into_keys().collect();
hashc.sort();
})
},
);
If you'll do that, you'll also recognize that the closure needs to be FnOnce() since you're moving out of hashc (.into_keys()). So after_f: Box<dyn FnOnce()>.
Related
As you can see in the following code, I have two traits, one is called Hittable, and the other is called Material (I have been studying the book "ray-tracing-in-one-weekend", but use Rust).
The Hittable trait implements hit function for some objects (just like Sphere in this code), and every kind of objects includes its material (just like Glass, Wood...).
In my real project, the Sphere struct and another struct (called HitRecord in this book, used as mut reference to pass result in hit function), they both include &dyn Material, so that I need add lifetime parameter for both of them. However, to accomplish that, I should add lifetime parameter in the trait declaration, so I can assign the same lifetime parameter for Sphere and hit.
But the compiler indicates that the reference still under borrowed when the main function ends, I have no idea for that...
trait Hittable<'a> {
fn hit(&self);
}
trait Material {
fn result(&self);
}
struct Glass;
impl Material for Glass {
fn result(&self) {
println!("Glass is broken!");
}
}
struct Sphere<'a> {
name: String,
mat_ptr: &'a dyn Material,
}
impl<'a> Hittable<'a> for Sphere<'a> {
fn hit(&self) {
println!("Name is {}", self.name);
self.mat_ptr.result();
}
}
struct HT<'a> {
pub objects: Vec<Box<dyn Hittable<'a>>>,
}
fn main() {
let mut list = HT { objects: vec![] };
let surface_material = Glass;
let s = Sphere {
name: String::from("球"),
mat_ptr: &surface_material,
};
list.objects.push(Box::new(s));
}
the message shows
Compiling rust_test v0.1.0 (/home/hnyls2002/rust_test)
error[E0597]: `surface_material` does not live long enough
--> src/main.rs:38:18
|
38 | mat_ptr: &surface_material,
| ^^^^^^^^^^^^^^^^^
| |
| borrowed value does not live long enough
| cast requires that `surface_material` is borrowed for `'static`
...
41 | }
| - `surface_material` dropped here while still borrowed
For more information about this error, try `rustc --explain E0597`.
This is because dyn Hittable<'a> is actually dyn Hittable<'a> + 'static, and thus s is required to live for 'static. The fix is to change HT to:
struct HT<'a> {
pub objects: Vec<Box<dyn Hittable<'a> + 'a>>,
}
Then you'll get a long but pretty self-explanatory error:
error[E0597]: `surface_material` does not live long enough
--> src/main.rs:38:18
|
38 | mat_ptr: &surface_material,
| ^^^^^^^^^^^^^^^^^ borrowed value does not live long enough
...
41 | }
| -
| |
| `surface_material` dropped here while still borrowed
| borrow might be used here, when `list` is dropped and runs the destructor for type `HT<'_>`
|
= note: values in a scope are dropped in the opposite order they are defined
Because surface_material is defined after list, it'll be dropped before and when the potential destructor for dyn Hittable will run it may access the freed surface_material. The fix is just to swap the declaration order of list and surface_material:
fn main() {
// always define before the container you are pushing into
let surface_material = Glass;
let mut list = HT { objects: vec![] };
// ...
}
I would like to return binary data in chunks of specific size. Here is a minimal example.
I made a wrapper struct for hyper::Response to hold my data like status, status text, headers and the resource to return:
pub struct Response<'a> {
pub resource: Option<&'a Resource>
}
This struct has a build method that creates the hyper::Response:
impl<'a> Response<'a> {
pub fn build(&mut self) -> Result<hyper::Response<hyper::Body>, hyper::http::Error> {
let mut response = hyper::Response::builder();
match self.resource {
Some(r) => {
let chunks = r.data
.chunks(100)
.map(Result::<_, std::convert::Infallible>::Ok);
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
},
None => response.body(hyper::Body::from("")),
}
}
}
There is also another struct holding the database content:
pub struct Resource {
pub data: Vec<u8>
}
Everything works until I try to create a chunked response. The Rust compiler gives me the following error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:14:15
|
14 | match self.resource {
| ^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime `'a` as defined on the impl at 11:6...
--> src/main.rs:11:6
|
11 | impl<'a> Response<'a> {
| ^^
note: ...so that the types are compatible
--> src/main.rs:14:15
|
14 | match self.resource {
| ^^^^^^^^^^^^^
= note: expected `Option<&Resource>`
found `Option<&'a Resource>`
= note: but, the lifetime must be valid for the static lifetime...
note: ...so that the types are compatible
--> src/main.rs:19:31
|
19 | response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
| ^^^^^^^^^^^^^^^^^^^^^^^^
= note: expected `From<&[u8]>`
found `From<&'static [u8]>`
I don't know how to fulfill these lifetime requirements. How can I do this correctly?
The problem is not in the 'a itself, but in the fact that the std::slice::chunks() function returns an iterator that borrows the original slice. You are trying to create a stream future from this Chunks<'_, u8> value, but the stream requires it to be 'static. Even if your Resource did not have the 'a lifetime, you would still have the r.data borrowed, and it would still fail.
Remember that here 'static does not mean that the value lives forever, but that it can be made to live as long as necessary. That is, the future must not hold any (non-'static) borrows.
You could clone all the data, but if it is very big, it can be costly. If so, you could try using Bytes, that is just like Vec<u8> but reference counted.
It looks like there is no Bytes::chunks() function that returns an iterator of Bytes. Fortunately it is easy to do it by hand.
Lastly, remember that iterators in Rust are lazy, so they keep the original data borrowed, even if it is a Bytes. So we need to collect them into a Vec to actually own the data (playground):
pub struct Resource {
pub data: Bytes,
}
impl<'a> Response<'a> {
pub fn build(&mut self) -> Result<hyper::Response<hyper::Body>, hyper::http::Error> {
let mut response = hyper::Response::builder();
match self.resource {
Some(r) => {
let len = r.data.len();
let chunks = (0..len)
.step_by(100)
.map(|x| {
let range = x..len.min(x + 100);
Ok(r.data.slice(range))
})
.collect::<Vec<Result<Bytes, std::convert::Infallible>>>();
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
}
None => response.body(hyper::Body::from("")),
}
}
}
UPDATE: We can avoid the call to collect() if we notice that stream::iter() takes ownership of an IntoIterator that can be evaluated lazily, as long as we make it 'static. It can be done if we do a (cheap) clone of r.data and move it into the lambda (playground):
let data = r.data.clone();
let len = data.len();
let chunks = (0..len).step_by(100)
.map(move |x| {
let range = x .. len.min(x + 100);
Result::<_, std::convert::Infallible>::Ok(data.slice(range))
});
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
I'm trying to implement an EventListener like interface in rust. I have a trait that takes a callback, the callback should mutate a variable from the scope it was defined in. But I get an error saying the borrowed value does not live long enough.
pub struct Target<T> {
funcs: Vec<Box<dyn FnMut(T) -> ()>>,
}
impl<T: Clone + 'static> Target<T> {
pub fn new() -> Target<T> {
return Target { funcs: Vec::new() };
}
pub fn add_listener(&mut self, func: Box<dyn FnMut(T) -> ()>) -> () {
self.funcs.push(Box::new(func));
}
pub fn trigger(&mut self, x: T) {
for callback in &mut self.funcs {
callback(x.clone());
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn trigger_mutation() {
let mut a = 0;
let mut t: Target<i32> = Target::new();
t.add_listener(Box::new(|x: i32| {
println!("{}", x);
a = x;
}));
t.trigger(42);
let b = a.clone();
assert_eq!(b, 42);
}
}
I run it and get this:
$ cargo test -- --nocapture
Compiling luma-connector v0.1.0 (/home/blake/git/connector)
error[E0597]: `a` does not live long enough
--> src/target.rs:32:13
|
30 | t.add_listener(Box::new(|x: i32| {
| - -------- value captured here
| ________________________|
| |
31 | | println!("{}", x);
32 | | a = x + 1;
| | ^ borrowed value does not live long enough
33 | | }));
| |__________- cast requires that `a` is borrowed for `'static`
...
37 | }
| - `a` dropped here while still borrowed
But I get an error saying the borrowed value does not live long enough.
Well yes, your typing of Target implies nothing about scoping, so as far as the Rust typesystem is concerned, the closure could just fly into space unbouded by time (e.g. add_listener could pass it to a separate thread), therefore outlive trigger_mutation, which means a does not live long enough.
There are two ways to resolve this issue:
Use Arc/Rc with interior mutability (resp. Mutex and RefCell) in order to relax the lifetime of a: Arc version[0], Rc version, this is probably the simplest, and the least restrictive on Target, though it comes at a runtime cost.
Alternatively you can "scope" Target to tell Rust that it doesn't escape, therefore everything's perfectly kosher. I'm not sure it's the best way (hopefully somebody else can contribute that information) but bounding the FnMuts on a lifetime will allow rustc to reason about this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e67a4ab0faa8cc5d01c75293623c9fb4
This is basically free at runtime, but it means Target can't really escape its function.
So the former is probably what you want, the latter seems not super-useful for an events / notification system, but YMMV.
[0] an Atomic* would also work for the Arc version and be a bit easier & cheaper than a mutex, though it probably isn't very relevant for a test case.
I basically have the following code that needs to modify a vector field (by removing some items through the retains function) and in the predicate of that retains function, I need to call a mutable function on self :
struct Task {}
struct Processor {
counter: u32,
tasks: Vec<Task>
}
impl Processor {
fn execute(&mut self, task: &Task) -> bool {
self.counter += 1;
// Do some stuff with task and return wether it was sucessfully executed
true
}
fn run(&mut self) {
self.tasks.retain(|task| !self.execute(task))
}
}
The compiler complains with two errors:
error[E0501]: cannot borrow `self.tasks` as mutable because previous closure requires unique access
--> src/main.rs:90:9
|
90 | self.tasks.retain(|task| !self.execute(task))
| ^^^^^^^^^^^------^------^^----^^^^^^^^^^^^^^^
| | | | |
| | | | first borrow occurs due to use of `self` in closure
| | | closure construction occurs here
| | first borrow later used by call
| second borrow occurs here
error[E0500]: closure requires unique access to `self` but it is already borrowed
--> src/main.rs:90:27
|
90 | self.tasks.retain(|task| !self.execute(task))
| ---------- ------ ^^^^^^ ---- second borrow occurs due to use of `self` in closure
| | | |
| | | closure construction occurs here
| | first borrow later used by call
| borrow occurs here
I do understand the issue, but how do I make this work ?
Rust doesn't yet have a way of only borrowing certain members of self. Because of this, it can't infer that you aren't mutating both self.tasks and self.counter at the same time. All it knows is that you're trying to mutate self twice at the same time.
To work around this, you'll need to move tasks out of self, then perform the operation you want, then move it back into self.
Rust has a handy operation that does exactly this in the standard library. Since Vec implements Default (yielding an empty vector, no allocation necessary), we can use std::mem::take:
fn run(&mut self) {
// swaps `self.tasks` with an empty `Vec`, and yields `tasks` to us
// this won't allocate, because `Vec::new` and therefore `Vec::default` do not
let mut tasks = std::mem::take(&mut self.tasks);
// do what we want with it
tasks.retain(|task| !self.execute(task));
// move it back into `self`
self.tasks = tasks;
}
Playground
Option 1: Split your struct into the right parts so they can be borrowed separately. The key part here is that execute doesn't take &mut Processor, only &mut Engine.
struct Task {}
struct Engine {
counter: u32,
}
struct Processor {
engine: Engine,
tasks: Vec<Task>
}
impl Engine {
fn execute(&mut self, task: &Task) -> bool {
self.counter += 1;
true
}
}
impl Processor {
fn run(&mut self) {
let engine = &mut self.engine;
self.tasks.retain(|task| !engine.execute(task))
}
}
Option 2: Move the tasks vector out with std::mem::take, so they're temporarily owned by the function instead of the Processor, then put them back:
impl Processor {
fn run(&mut self) {
let mut tasks = std::mem::take(&mut self.tasks);
tasks.retain(|task| !self.execute(task));
self.tasks = tasks;
}
}
(This does not copy the memory for all the tasks, because the Vec's heap allocated storage is undisturbed by moving the Vec value itself.)
Option 3: If in a more complex situation you can't find a split between parts or a temporary move that works for all your operations, then you could use RefCell to do run-time instead of compile-time borrow checking — you'd need one cell for tasks and another cell or cells for the parts execute needs.
I'm new to Rust and trying to understand basic directory traversal. Nearly all the examples I have found utilize the walkdir or glob library, which I've had good success with. However, I'm trying to do this now with just the std lib.
There is a primitive example in the standard lib docs listing the following function:
fn visit(path: &Path, cb: &dyn Fn(&PathBuf)) -> io::Result<()> {
for e in read_dir(path)? {
let e = e?;
let path = e.path();
if path.is_dir() {
visit(&path, cb)?;
} else if path.is_file() {
cb(&path);
}
}
Ok(())
}
The part I'm confused about is how to access the cb function in the context of a closure. I'm having a hard time finding an example.
For instance, I want to do something basic like collect the resulting paths into a Vec. Obviously, this does not work:
fn main() {
// create a new path
let path = Path::new(PATH);
let mut files = Vec::new();
visit(path, |e| {
files.push(e);
});
}
The error I'm receiving is:
expected reference `&dyn for<'r> std::ops::Fn(&'r std::path::PathBuf)`
found closure `[closure#src/main.rs:24:17: 26:6 files:_]
So my question is, how can I return a Fn and process the result in a closure context?
There are multiple issues with your code, but the first one that you are getting the error message for is because &dyn Fn(&PathBuf) expects a reference to a function. You can resolve that error by following the suggestion from the error message: help: consider borrowing here: '&|e| files.push(e)'
That turns your call into:
visit(path, &|e| files.push(e));
However, this code is still incorrect and results in yet another error:
error[E0596]: cannot borrow `files` as mutable, as it is a captured variable in a `Fn` closure
--> playground\src\main.rs:48:22
|
48 | visit(path, &|e| files.push(e));
| ^^^^^ cannot borrow as mutable
This time, it's because you're mutating files inside a Fn (immutable closure). To fix that, you need to change your function type to FnMut (see Closures As Input Parameters for more information):
fn visit(path: &Path, cb: &dyn FnMut(&PathBuf))
But you're still not done. There is now another error, but like the first, it comes with a suggestion for what needs to be changed:
error[E0596]: cannot borrow `*cb` as mutable, as it is behind a `&` reference
--> playground\src\main.rs:39:13
|
32 | fn visit(path: &Path, cb: &dyn FnMut(&PathBuf)) -> io::Result<()> {
| -------------------- help: consider changing this to be a mutable reference: `&mut dyn for<'r> std::ops::FnMut(&'r std::path::PathBuf)`
...
39 | cb(&path);
| ^^ `cb` is a `&` reference, so the data it refers to cannot be borrowed as mutable
In order for your closure to mutably borrow the data it uses, you also have to take a mutable reference to the closure itself, and you'll need to update your visit() call to match:
fn visit(path: &Path, cb: &mut dyn FnMut(&PathBuf))
...
visit(path, &mut |e| files.push(e));
Almost there, but there is one final error to resolve:
error[E0521]: borrowed data escapes outside of closure
--> playground\src\main.rs:48:26
|
47 | let mut files = Vec::new();
| --------- `files` declared here, outside of the closure body
48 | visit(path, &mut |e| files.push(e));
| - ^^^^^^^^^^^^^ `e` escapes the closure body here
| |
| `e` is a reference that is only valid in the closure body
You've defined your closure to take a reference to a PathBuf (&PathBuf), but you're trying to push those references into a Vec that is outside of the closure, which won't work because those references will be invalid once the closure goes out of scope. Instead, you should use an owned value -- simply PathBuf. You'll also need to update your usage of the closure to pass the PathBuf instead of a reference:
fn visit(path: &Path, cb: &mut dyn FnMut(PathBuf))
...
cb(path);
It finally works! Here is what the full program looks like now. Note that you should also unwrap() your call to visit() since it returns a Result. I've also added a simple for loop to print out the file names.
use std::path::{Path, PathBuf};
use std::fs::*;
use std::io;
fn visit(path: &Path, cb: &mut dyn FnMut(PathBuf)) -> io::Result<()> {
for e in read_dir(path)? {
let e = e?;
let path = e.path();
if path.is_dir() {
visit(&path, cb)?;
} else if path.is_file() {
cb(path);
}
}
Ok(())
}
fn main() {
let path = Path::new("./your/path/here");
let mut files = Vec::new();
visit(path, &mut |e| files.push(e)).unwrap();
for file in files {
println!("{:?}", file);
}
}