How do I box Arc and Mutexed variables? - multithreading

The following code gets some variables in closures and returns a struct containing that data.
I can't return the struct with that data even when I box the struct and clone the variables; they are impossible to take out of this scope. I thought about using a callback closure but I don't really want to do that. Is there any way to take those out without having a callback?
pub fn get(addr: &str) -> std::io::Result<Box<Response>> {
use std::sync::{Arc, Mutex};
let mut crl = curl::easy::Easy::new();
crl.url(format!("{}{}", API_ADDR, addr).as_str()).unwrap();
// extract headers
let headers: Vec<String> = Vec::with_capacity(10);
let headers = Arc::new(Mutex::new(headers));
{
let headers = headers.clone();
crl.header_function(move |h| {
let mut headers = headers.lock().unwrap();
(*headers).push(String::from_utf8_lossy(h).into_owned());
true
})
.unwrap();
}
// extract body
let body = Arc::new(Mutex::new(String::with_capacity(1024)));
{
let body = body.clone();
crl.write_function(move |b| {
let mut body = body.lock().unwrap();
body.push_str(std::str::from_utf8(b).unwrap());
Ok(b.len())
})
.unwrap();
}
crl.perform().unwrap();
Ok(Box::new(Response {
resp: body.lock().unwrap().clone(),
headers: headers.lock().unwrap().clone(),
}))
}

The key error seems to be this one:
error[E0597]: `body` does not live long enough
--> src/lib.rs:85:15
|
85 | resp: body.lock().unwrap().clone(),
| ^^^^ borrowed value does not live long enough
...
89 | }
| - `body` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
The same for the headers object.
I was able to get a simplified reproducer of this by stubbing out a lot of your code:
use std::sync::{Arc, Mutex};
pub struct Response {
resp: String,
headers: Vec<String>,
}
pub fn get(addr: &str) -> std::io::Result<Box<Response>> {
let headers: Vec<String> = Vec::with_capacity(10);
let headers = Arc::new(Mutex::new(headers));
let body = Arc::new(Mutex::new(String::with_capacity(1024)));
Ok(Box::new(Response {
resp: body.lock().unwrap().clone(),
headers: headers.lock().unwrap().clone(),
}))
}
I think this has to do with the lifetimes of the temporary variables constructed in the final Ok(Box::new(...)) return values.
I was able to get it to compile by pulling the lock/unwrap outside.
let body = body.lock().unwrap();
let headers = headers.lock().unwrap();
Ok(Box::new(Response {
resp: body.clone(),
headers: headers.clone(),
}))
From the fuller explaination given in Why do I get "does not live long enough" in a return value? I've found that you can write this as
return Ok(Box::new(Response {
resp: body.lock().unwrap().clone(),
headers: headers.lock().unwrap().clone(),
}));
i.e. adding an explicit return and a trailing semicolon. Though I have a feeling clippy might say that its bad style.

Related

Create vector of trait objects

I am trying to create a vector of trait objects but I'm getting a type mismatch. I also tried using .map() instead of a for loop but had the same problem. This is a minimal version of my real code, I realise this example doesn't require the use of trait objects but my real code does since there will be types other than Link in the List, which also implement Speak. How can I create a Vec with the correct type so I can pass it to trait_objects?
use std::fs;
fn main() {
// let links = fs::read_dir("my_dir").unwrap();
// let links: Vec<Box<dyn Speak>> = links
// .map(|file| {
// let myfile = file.unwrap();
// let href = myfile.path().to_str().unwrap();
// Box::new(Link { attr: href })
// })
// .collect();
let links = Vec::new();
for entry in fs::read_dir("my_dir").unwrap() {
let myfile = entry.unwrap();
let href = myfile.path().to_str().unwrap();
links.push(Box::new(Link { attr: href }));
}
let link_objects = ListOfTraitObjects {
trait_objects: links,
};
for link in link_objects.trait_objects.iter() {
println!("{}", link.speak());
}
}
trait Speak {
fn speak(&self) -> String;
}
struct ListOfTraitObjects {
trait_objects: Vec<Box<dyn Speak>>,
}
struct Link<'a> {
attr: &'a str,
}
impl Speak for Link<'_> {
fn speak(&self) -> String {
self.attr.to_string()
}
}
There are several issues with your code. The first is that you're creating a Vec<Box<Link>> instead of a Vec<Box<Speak>>. By default Box::new (foo) creates a box of the same type as foo. If you want a box of some trait implemented by foo, you need to be explicit:
links.push(Box::new(Link { attr: href }) as Box::<dyn Speak>);
Playground
However this gives two new errors:
error[E0716]: temporary value dropped while borrowed
--> src/lib.rs:7:20
|
7 | let href = myfile.path().to_str().unwrap();
| ^^^^^^^^^^^^^ - temporary value is freed at the end of this statement
| |
| creates a temporary which is freed while still in use
8 | links.push(Box::new(Link { attr: href }) as Box::<dyn Speak>);
| ----------------------------- cast requires that borrow lasts for `'static`
error[E0596]: cannot borrow `links` as mutable, as it is not declared as mutable
--> src/lib.rs:8:9
|
4 | let links = Vec::new();
| ----- help: consider changing this to be mutable: `mut links`
...
8 | links.push(Box::new(Link { attr: href }) as Box::<dyn Speak>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
The first error is because your Link struct borrows a string from entry, but entry only lives as long as the current loop iteration. Even though the error states that the cast requires a 'static lifetime, you would get a similar error if you were simply trying to build a Vec<Link> without casting (playground). You need to change the definition of your Link struct to contain an owned string:
struct Link {
attr: String,
}
impl Speak for Link {
fn speak(&self) -> String {
self.attr.clone()
}
}
The second error is easier and can be fixed simply by following the compiler suggestion and replacing let links = … with let mut links = ….
Full working example:
use std::fs;
fn main() {
let mut links = Vec::new();
for entry in fs::read_dir("my_dir").unwrap() {
let myfile = entry.unwrap();
let href = myfile.path().to_str().unwrap().to_string();
links.push(Box::new(Link { attr: href }) as Box::<dyn Speak>);
}
let link_objects = ListOfTraitObjects {
trait_objects: links,
};
for link in link_objects.trait_objects.iter() {
println!("{}", link.speak());
}
}
trait Speak {
fn speak(&self) -> String;
}
struct ListOfTraitObjects {
trait_objects: Vec<Box<dyn Speak>>,
}
struct Link {
attr: String,
}
impl Speak for Link {
fn speak(&self) -> String {
self.attr.clone()
}
}
Playground
The same fixes can be applied to your iterator-based solution:
let links = fs::read_dir("my_dir").unwrap();
let links: Vec<_> = links
.map(|file| {
let myfile = file.unwrap();
let href = myfile.path().to_str().unwrap().to_string();
Box::new(Link { attr: href }) as Box<dyn Speak>
})
.collect();
Playground

How to fix use of moved value in Rust?

I am trying to convert a yaml file to xml using Rust and I am not able to figure out how to fix this error regarding the use of moved value. I think I understand why this error is coming, but haven't got a clue about what to do next.
Here's the code:
struct Element {
element_name: String,
indentation_count: i16,
}
struct Attribute<'a> {
attribute_name: &'a str,
attribute_value: &'a str,
}
fn convert_yaml_to_xml(content: String, indentation_count: i16) -> String {
let mut xml_elements: Vec<Element> = vec![];
let mut attributes: Vec<Attribute> = vec![];
xml_elements.push(Element {element_name: "xmlRoot".to_string(), indentation_count: -1});
let mut target: Vec<u8> = Vec::new();
let mut xml_data_writer = EmitterConfig::new().perform_indent(true).create_writer(&mut target);
let mut attribute_written_flag = false;
let mut xml_event;
xml_event = XmlEvent::start_element("xmlRoot");
for line in content.lines() {
let current_line = line.trim();
let caps = indentation_count_regex.captures(current_line).unwrap();
let current_indentation_count = caps.get(1).unwrap().as_str().to_string().len() as i16;
if ELEMENT_REGEX.is_match(current_line) {
loop {
let current_attribute_option = attributes.pop();
match current_attribute_option {
Some(current_attribute_option) => {
xml_event.attr(current_attribute_option.attribute_name, current_attribute_option.attribute_value)
},
None => {
break;
},
};
}
xml_data_writer.write(xml_event);
// Checking if the line is an element
let caps = ELEMENT_REGEX.captures(current_line).unwrap();
let element_name = caps.get(2);
let xml_element_struct = Element {
indentation_count: current_indentation_count,
element_name: element_name.unwrap().as_str().to_string(),
};
xml_elements.push(xml_element_struct);
xml_event = XmlEvent::start_element(element_name.unwrap().as_str());
attribute_written_flag = false;
} else if ATTR_REGEX.is_match(current_line) {
// Checking if the line is an attribute
let caps = ATTR_REGEX.captures(current_line).unwrap();
let attr_name = caps.get(2);
let attr_value = caps.get(3);
// Saving attributes to a stack
attributes.push(Attribute{ attribute_name: attr_name.unwrap().as_str(), attribute_value: attr_value.unwrap().as_str() });
// xml_event.attr(attr_name.unwrap().as_str(), attr_value.unwrap().as_str());
}/* else if NEW_ATTR_SET_REGEX.is_match(current_line) {
let caps = NEW_ATTR_SET_REGEX.captures(current_line).unwrap();
let new_attr_set_name = caps.get(2);
let new_attr_set_value = caps.get(3);
current_xml_hash.insert("name".to_string(), new_attr_set_name.unwrap().as_str().to_string());
current_xml_hash.insert("value".to_string(), new_attr_set_value.unwrap().as_str().to_string());
} */
}
if attribute_written_flag {
xml_data_writer.write(xml_event);
}
for item in xml_elements.iter() {
let event = XmlEvent::end_element();
let event_name = item.element_name.to_string();
xml_data_writer.write(event.name(event_name.as_str()));
}
println!("OUTPUT");
println!("{:?}", target);
return "".to_string();
}
And here's the error:
error[E0382]: use of moved value: `xml_event`
--> src/main.rs:77:25
|
65 | let mut xml_event;
| ------------- move occurs because `xml_event` has type `StartElementBuilder<'_>`, which does not implement the `Copy` trait
...
77 | xml_event.attr(current_attribute_option.attribute_name, current_attribute_option.attribute_value)
| ^^^^^^^^^ --------------------------------------------------------------------------------------- `xml_event` moved due to this method call, in previous iteration of loop
|
note: this function takes ownership of the receiver `self`, which moves `xml_event`
--> /Users/defiant/.cargo/registry/src/github.com-1ecc6299db9ec823/xml-rs-0.8.4/src/writer/events.rs:193:24
|
193 | pub fn attr<N>(mut self, name: N, value: &'a str) -> StartElementBuilder<'a>
| ^^^^
From XmlEvent::start_element() documentation we see that it produces a StartElementBuilder<'a>.
From StartElementBuilder<'a>::attr() documentation we see that it consumes the StartElementBuilder<'a> (the first parameter is self, not &mut self) and produces a new StartElementBuilder<'a> (which is probably similar to self but considers the expected effect of .attr()).
This approach is known as the consuming builder pattern, which is used in Rust (for example std::thread::Builder).
The typical usage of such an approach consists in chaining the function calls: something.a().b().c().d() such as something is consumed by a(), its result is consumed by b(), the same about c() and finally d() does something useful with the last result.
The alternative would be to use mutable borrows in order to modify in place something but dealing with mutable borrows is known as difficult in some situations.
In your case, you can just reassign the result of .attr() to xml_event because otherwise the .attr() function would have no effect (its result is discarded) and xml_event would become unusable because it is consumed; reassigning it makes it usable again afterwards (at least i guess, i didn't try).

Rust lifetimes with closures using hyper

Been learning rust and having a problem with lifetime when passing conn to the request_handler. I get an error saying
error[E0312]: lifetime of reference outlives lifetime of borrowed content...
--> src/main.rs:33:70
|
33 | let request_handler = |req: Request<Body>| async { request_handler(conn, req).await };
| ^^^^
|
= note: ...the reference is valid for the static lifetime...
but I am not sure how to handle lifetimes with closures and why/how it is static. I have a loose understanding of lifetimes and borrowing but this seems like a more complex case. I also would just not use a closure, but the return type of one of the closures has a type that is not exported by the hyper crate, so i don't know how i would create a fn without being able to declare the return type.
Also I can confirm if i remove passing conn i can get everything to work, but I want to use the conn object in the request_handler.
use hyper::server::conn::AddrStream;
use hyper::service::make_service_fn;
use hyper::Version;
use hyper::{Body, Error, Method, Request, Response, Server};
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
mod http_models;
mod utils;
use hyper::service::service_fn;
async fn request_handler(
conn: &'static AddrStream,
req: Request<Body>,
) -> Result<Response<Body>, hyper::Error> {
println!("req: {:?}", req);
if req.method() == Method::CONNECT {
println!("Connect")
}
let res: Response<Body> = Response::builder()
.status(200)
.version(Version::HTTP_11)
.body(Body::empty())
.unwrap();
return Ok(res);
}
#[tokio::main]
async fn main() -> Result<(), Error> {
let ip = IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1));
let addr = SocketAddr::new(ip, 1337);
//let client = Client::new();
let make_service = make_service_fn(|conn: &AddrStream| async {
let request_handler = |req: Request<Body>| async { request_handler(conn, req).await };
let service = service_fn(request_handler);
Ok::<_, Error>(service)
});
let server = Server::bind(&addr).serve(make_service);
println!("Listening on http://{}", addr);
if let Err(e) = server.await {
eprintln!("server error: {}", e);
}
Ok(())
}
The conn argument of the closure passed to make_service_fn only lives as long as the closure body, but the return value of the closure (Ok(service)) references it. The closure must have the type FnMut(&Target) -> impl Future, which means it sadly is not permitted to return a value that references its argument.
The only solution is to copy/clone whatever you need from conn while setting up your request handler, since you cannot keep a reference to it once the closure returns.

Issue passing mutable Arc reference to hyper service_fn handler

I've been trying the following
Relevant imports and code shown
use std::sync::{Arc, Mutex};
use std::thread;
use hyper::rt::{self, Future, Stream};
use hyper::service::service_fn;
use hyper::{Body, Request, Response, Server, StatusCode};
pub struct ChallengeState;
pub struct ChallengeResponse;
type BoxFut<'a> = Box<Future<Item = Response<Body>, Error = hyper::Error> + Send + 'a>;
fn handle_challengeproof<'a>(
req: Request<Body>,
challenge: &Arc<Mutex<ChallengeState>>,
) -> BoxFut<'a> {
let resp = req.into_body().concat2().map(move |body| {
let challenge_lock = challenge.lock().unwrap();
Response::builder()
.status(StatusCode::OK)
.body(Body::from("test"))
.unwrap()
});
Box::new(resp)
}
fn handle<'a>(
req: Request<Body>,
challenge: &Arc<Mutex<ChallengeState>>,
) -> BoxFut<'a> {
handle_challengeproof(req, challenge)
}
pub fn run_listener(
challenge: Arc<Mutex<ChallengeState>>,
) -> thread::JoinHandle<()> {
let addr = ([127, 0, 0, 1], 9999).into();
let listener_service = move || {
let challenge = Arc::clone(&challenge);
service_fn(move |req: Request<Body>| {
handle(req, &challenge)
})
};
let server = Server::bind(&addr)
.serve(listener_service)
.map_err(|_| () );
thread::spawn(move || {
rt::run(server);
})
}
I've been trying to avoid an extra clone of Arc by passing a reference to the handle method but can't seem to get around this. Avoiding the lifetime on handle() got a different error regarding futures asking for static lifetime.
Code updated with only relevant stuff # https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=10ea31450e88a122455006760d7fcdd1
The whole point of an Arc is that it counts how many references there are, which happens when it is cloned. Passing around references to an Arc defeats the point.
Instead of passing references, pass the Arc itself. So handle's signature becomes:
fn handle<'a>(
req: Request<Body>,
challenge: Arc<Mutex<ChallengeState>>,
) -> BoxFut<'a>
Passing the Arc by references from the closure isn't possible because you would be referencing something that immediately goes out of scope. Instead, move the Arc into handle:
let listener_service = move || {
service_fn(move |req: Request<Body>| handle(req, challenge))
};

How to change value in a BTreeMap container

Code:
#[derive(Debug)]
struct Haha {
data: i32,
}
use std::collections::BTreeMap;
fn main() {
let mut map: BTreeMap<i8, Option<Box<Haha>>> = BTreeMap::new();
map.insert(1, Some(Box::new(Haha { data: 3 })));
map.insert(2, None);
for (key, value) in map.iter_mut() {
if value.is_none() { // if find `None`, change it to a `Some(Haha)`
value = Some(Box::new(Haha { data: 5 }));
}
}
}
I want to make a function that when I get the value None in BTreeMap.value, I change it to Some value, not a reference. But it comes out a mistake:
error[E0308]: mismatched types
--> Untitled.rs:15:12
|
15 | value = Some(Box::new(Haha{data: 5}));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected mutable reference, found enum `std::option::Option`
|
= note: expected type `&mut std::option::Option<std::boxed::Box<Haha>>`
found type `std::option::Option<std::boxed::Box<Haha>>`
= help: here are some functions which might fulfill your needs:
- .unwrap()
Because using map.iter_mut() I can only get the reference. How to send the true value Some(Haha) but not a reference to the mut reference value? If I change to value = & mut Some(Box::new(Haha { data: 5 })); Because Some(Box...) will be destructed, so it will come out another mistake.
Since value is a mutable reference (due to iter_mut()), just dereference it:
*value = Some(Box::new(Haha{data: 5}));
And it will work nicely.

Resources