I can not understand this error regarding lifetimes. Proxy is a struct:
impl Proxy {
pub fn new<A: ip::ToSocketAddr>(address: A) -> Proxy {
Proxy {
address: address.to_socket_addr().unwrap()
}
}
pub fn listen_and_proxy(&self) {
println!("listening {}", self.address);
for stream in TcpListener::bind(self.address).listen().incoming() {
let stream = stream.unwrap();
spawn(move || { // <- this causes an error
let mut stream = stream;
let name = stream.peer_name();
println!("client: {} -> {}", name, self.handle(stream));
});
}
}
}
Output error:
cannot infer an appropriate lifetime due to conflicting requirements
What is wrong in the code?
You are trying to call self.handle(stream) which would require moving self into the newly spawned thread. This cannot be done.
Related
We are using GStreamer to read the video stream off of network cameras and we observed a weird behaviour: our program would hang unexpectedly, without any kind of notice.
We, then, narrowed it down to the fact that our client had a very, very bad internet connection and we were able to reproduce the problem with just GStreamer and a tool to simulate bad networks.
I am attaching below both our tool to simulate bad networks and the example we've written to reproduce this behaviour.
Our tool to simulate bad networks uses tc to introduce a delay, packet loss and packet corruption.
Our example boils down to using a (subclassed) bin that can intercept EoS messages posted inside it, and a pipeline that reads a RTSP feed. The bin is then dinamically added and removed from the pipeline.
Are we doing something wrong? Could this be a bug in GStreamer itself?
crapnet.sh
sudo tc qdisc add dev wlp2s0 root netem \
delay 500ms 480ms distribution normal \
loss 30% 25% \
corrupt 2%
src/custom_bin.rs
mod implementation {
use {glib::subclass::prelude::*, gstreamer::subclass::prelude::*};
use {
glib::{glib_object_impl, glib_object_subclass, subclass::simple::ClassStruct},
gstreamer::{subclass::ElementInstanceStruct, Bin, Message, MessageView},
std::sync::{mpsc, Mutex},
};
pub struct CustomBin {
pub(super) eos_guard: Mutex<Option<mpsc::SyncSender<()>>>,
}
impl ObjectImpl for CustomBin {
glib_object_impl!();
}
impl ElementImpl for CustomBin {}
impl BinImpl for CustomBin {
fn handle_message(&self, bin: &Bin, message: Message) {
let mut eos_guard = self.eos_guard.lock().unwrap();
if let MessageView::Eos(_) = message.view() {
if let Some(eos_guard) = eos_guard.take() {
return eos_guard.send(()).unwrap_or(());
}
}
self.parent_handle_message(bin, message)
}
}
impl ObjectSubclass for CustomBin {
const NAME: &'static str = "GstCustomBin";
type ParentType = Bin;
type Instance = ElementInstanceStruct<Self>;
type Class = ClassStruct<Self>;
glib_object_subclass!();
fn new() -> Self {
Self {
eos_guard: Mutex::new(None),
}
}
}
}
use {
glib::{prelude::*, subclass::prelude::*, translate::*},
gstreamer::prelude::*,
};
use {
glib::{glib_bool_error, glib_wrapper, subclass::simple::ClassStruct, BoolError, Object},
gstreamer::{
event, subclass::ElementInstanceStruct, Element, GhostPad, PadDirection, State, StateChangeError,
StateChangeSuccess,
},
std::{sync::mpsc, time::Duration},
};
glib_wrapper! {
/// A subclass of `gstreamer::Bin` that has customized behaviour for intercepting end-of-stream messages. This is necessary so that we can remove a branch from a `tee` and not bring down the entire pipeline.
pub struct CustomBin(
Object<
ElementInstanceStruct<implementation::CustomBin>,
ClassStruct<implementation::CustomBin>,
CustomBinClass
>
) #extends gstreamer::Bin, gstreamer::Element, gstreamer::Object;
match fn {
get_type => || implementation::CustomBin::get_type().to_glib(),
}
}
unsafe impl Send for CustomBin {}
unsafe impl Sync for CustomBin {}
impl CustomBin {
/// Instantiates the structure.
pub fn new(name: Option<&str>) -> Self {
Object::new(Self::static_type(), &[("name", &name)])
.expect("Failed to instantiate `CustomBin` as an `Object`")
.downcast()
.expect("Failed to downcast `Object` to `CustomBin`")
}
/// Adds a ghost sink pad for the target element. It is assumed that the element belongs to the bin.
pub fn add_ghost_sink_pad(&self, element: &Element) -> Result<(), BoolError> {
let target_pad = element
.get_static_pad("sink")
.ok_or_else(|| glib_bool_error!("Failed to get sink pad from [{}]", element.get_name()))?;
let ghost_pad = GhostPad::new(Some("sink"), PadDirection::Sink);
ghost_pad.set_target(Some(&target_pad))?;
self.add_pad(&ghost_pad)?;
Ok(())
}
/// Installs an end-of-stream message guard, which will drop the end-of-stream message and then signal it was dropped.
pub fn install_eos_guard(&self) -> mpsc::Receiver<()> {
let super_self = implementation::CustomBin::from_instance(self);
let mut eos_guard = super_self.eos_guard.lock().unwrap();
let (sender, receiver) = mpsc::sync_channel(1);
eos_guard.replace(sender);
receiver
}
/// Sends an end-of-stream event to this bin. It will become an end-of-stream message once it reaches the sink.
pub fn send_eos_event(&self) {
let super_self = implementation::CustomBin::from_instance(self);
let mut eos_guard = super_self.eos_guard.lock().unwrap();
if !self.send_event(event::Eos::new()) {
if let Some(eos_guard) = eos_guard.take() {
eos_guard.send(()).unwrap_or(());
}
}
}
/// Flushes the data in the bin by sending an EoS event and then intercepting the resulting EoS message.
pub fn flush(&self) -> Result<(), BoolError> {
let eos_guard = self.install_eos_guard();
self.send_eos_event();
eos_guard
.recv_timeout(Duration::from_secs(5))
.map_err(|error| glib_bool_error!("Timed out waiting for the EoS guard during flush (error [{:?}])", error))
}
pub fn set_null_state(&self) -> Result<StateChangeSuccess, StateChangeError> {
self.set_state(State::Null)
}
}
src/main.rs
mod custom_bin;
use custom_bin::CustomBin;
use gstreamer::prelude::*;
use {
glib::glib_bool_error,
gstreamer::{ElementFactory, PadProbeReturn, PadProbeType, Pipeline, State},
std::{error::Error, sync::mpsc, thread, time::Duration},
};
fn rtsp_location() -> String {
String::from("rtspt://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov")
}
fn video_location(index: usize) -> String {
format!("/home/valmirpretto/videos/video_{}.mp4", index)
}
fn instantiate_mp4_writer(location: &str) -> Result<CustomBin, Box<dyn Error>> {
let queue = ElementFactory::make("queue", None)?;
let h264parse = ElementFactory::make("h264parse", None)?;
let mp4mux = ElementFactory::make("mp4mux", None)?;
let filesink = ElementFactory::make("filesink", None)?;
filesink.set_property("location", &location)?;
let bin = CustomBin::new(None);
bin.add(&queue)?;
bin.add(&h264parse)?;
bin.add(&mp4mux)?;
bin.add(&filesink)?;
queue.link(&h264parse)?;
h264parse.link(&mp4mux)?;
mp4mux.link(&filesink)?;
bin.add_ghost_sink_pad(&queue)?;
Ok(bin)
}
fn main() -> Result<(), Box<dyn Error>> {
gstreamer::init()?;
let rtspsrc = ElementFactory::make("rtspsrc", None)?;
let rtph264depay = ElementFactory::make("rtph264depay", None)?;
let tee = ElementFactory::make("tee", None)?;
rtspsrc.set_property("location", &rtsp_location())?;
tee.set_property("allow-not-linked", &true)?;
let pipeline = Pipeline::new(None);
pipeline.add(&rtspsrc)?;
pipeline.add(&rtph264depay)?;
pipeline.add(&tee)?;
rtspsrc.connect_pad_added({
let rtph264depay = rtph264depay.downgrade();
move |_, src_pad| {
if let Some(rtph264depay) = rtph264depay.upgrade() {
let sink_pad = rtph264depay
.get_static_pad("sink")
.expect("Element rtph264depay without sink pad");
if src_pad.can_link(&sink_pad) {
src_pad
.link(&sink_pad)
.expect("Failed to link after we checked it could");
}
}
}
});
rtph264depay.link(&tee)?;
pipeline.set_state(State::Playing)?;
(0..).try_for_each::<_, Result<(), Box<dyn Error>>>(|index| {
println!("Iteration [{}]", index);
// Add bin to the pipeline
let bin = instantiate_mp4_writer(&video_location(index))?;
pipeline.add(&bin)?;
bin.sync_state_with_parent()?;
tee.link(&bin)?;
thread::sleep(Duration::from_secs(5));
// Remove bin from the pipeline
let bin_sink_pad = bin
.get_static_pad("sink")
.ok_or_else(|| glib_bool_error!("Bin [{}] did not have a sink pad", bin.get_name()))?;
let tee_src_pad = bin_sink_pad
.get_peer()
.ok_or_else(|| glib_bool_error!("Bin [{}] sink pad did not have a peer", bin.get_name()))?;
let (signal_sender, signal_receiver) = mpsc::sync_channel(1);
tee_src_pad.add_probe(PadProbeType::IDLE, {
let bin = bin.downgrade();
let pipeline = pipeline.downgrade();
let tee = tee.downgrade();
move |tee_src_pad, _| {
if let Some(bin) = bin.upgrade() {
if let Some(pipeline) = pipeline.upgrade() {
bin.flush().expect(&format!("Could not flush bin [{}]", bin.get_name()));
pipeline
.remove(&bin)
.expect(&format!("Could not remove bin [{}] from pipeline", bin.get_name()));
bin.set_null_state()
.expect(&format!("Could not set null state of bin [{}]", bin.get_name()));
signal_sender
.send(())
.expect(&format!("Could not signal that bin [{}] is done", bin.get_name()));
}
}
if let Some(tee) = tee.upgrade() {
tee.release_request_pad(tee_src_pad);
}
PadProbeReturn::Remove
}
});
signal_receiver
.recv_timeout(Duration::from_secs(5))
.map_err(|error| glib_bool_error!("Timed out waiting for EoS (error [{:?}])", error))?;
Ok(())
})?;
Ok(())
}
I'm new to Rust and I'm trying to create a Server struct which listen to an address and starts a TCP socket connection. The problem is that I want to store the client connection inside a hash map so I can use it later..
I tried writing this:
use std::collections::HashMap;
use std::net::TcpListener;
use std::net::TcpStream;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
thread::spawn(move || {
let mut listener =
TcpListener::bind(self.clone().url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => self.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&mut self, stream: TcpStream) {
let id = self.id.read().unwrap();
self.connections.read().unwrap().insert(id, stream);
let id = self.id.write().unwrap();
*id += 1;
}
}
But of course this doesn't work.. There are 2 things that I don't understand, the first is how to pass the stream into my function and then store in my connections hash map so I can use it later and how to use my id inside my on_client_connect function.
You need to clone outside of thread::spawn and move the cloned instance in thread scope.
Also, on_client_connect do not need &mut self because the fields id and connections are already protected inside RwLock.
use std::net::TcpListener;
use std::net::TcpStream;
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
let me = self.clone(); // Clone it outside
thread::spawn(move || {
let mut listener = TcpListener::bind(&me.url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => me.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&self, stream: TcpStream) { // `&mut self` not needed as the id, connection are inside the lock
let mut id = self.id.write().unwrap();
self.connections.write().unwrap().insert(*id, stream);
*id += 1;
}
}
playground
There's quite a few issues that need minor fixes in this code.
The first one I ran into was the usage of self in the thread::spawn closure.
The thread::spawn needs its argument to have static lifetime, but we've no guarantee that the Server object lives that long.
I solved it by cloning the Server object and moving that into the closure. This is OK as all its data is already behind Arcs.
The next problem was self.connections.read().unwrap().insert(*id, stream); needs to get a write lock, not a read.
Finally id+=1 needs to dereference id.
Once these were fixed, it seems that storing the TcpStream is not an issue. (At least using nightly). I'd thought I'd need to box the TcpStream - but it seems OK as is.
You can see that it compiles in the playground
use std::collections::HashMap;
use std::net::TcpListener;
use std::net::TcpStream;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
let mut self_clone = self.clone();
thread::spawn(move || {
let mut listener =
TcpListener::bind(&self_clone.url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => self_clone.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&mut self, stream: TcpStream) {
let id = self.id.read().unwrap();
self.connections.write().unwrap().insert(*id, stream);
let mut id = self.id.write().unwrap();
*id += 1;
}
}
I am using hyper 0.12 to build a proxy service. When receiving a response body from the upstream server I want to forward it back to the client ASAP, and save the contents in a buffer for later processing.
So I need a function that:
takes a Stream (a hyper::Body, to be precise)
returns a Stream that is functionally identical to the input stream
also returns some sort of Future<Item = Vec<u8>, Error = ...> that is resolved with the buffered contents of the input stream, when the output stream is completely consumed
I can't for the life of me figure out how to do this.
I guess the function I'm looking for will look something like this:
type BufferFuture = Box<Future<Item = Vec<u8>, Error = ()>>;
pub fn copy_body(body: hyper::Body) -> (hyper::Body, BufferFuture) {
let body2 = ... // ???
let buffer = body.fold(Vec::<u8>::new(), |mut buf, chunk| {
buf.extend_from_slice(&chunk);
// ...somehow send this chunk to body2 also?
});
(body2, buffer);
}
Below is what I have tried, and it works until send_data() fails (obviously).
type BufferFuture = Box<Future<Item = Vec<u8>, Error = ()>>;
pub fn copy_body(body: hyper::Body) -> (hyper::Body, BufferFuture) {
let (mut sender, body2) = hyper::Body::channel();
let consume =
body.map_err(|_| ()).fold(Vec::<u8>::new(), move |mut buf, chunk| {
buf.extend_from_slice(&chunk);
// What to do if this fails?
if sender.send_data(chunk).is_err() {}
Box::new(future::ok(buf))
});
(body2, Box::new(consume));
}
However, something tells me I am on the wrong track.
I have found Sink.fanout() which seems like it is what I want, but I do not have a Sink, and I don't know how to construct one. hyper::Body implements Stream but not Sink.
What I ended up doing was implement a new type of stream that does what I need. This appeared to be necessary because hyper::Body does not implement Sink nor does hyper::Chunk implement Clone (which is required for Sink.fanout()), so I cannot use any of the existing combinators.
First a struct that contains all details that we need and methods to append a new chunk, as well as notify that the buffer is completed.
struct BodyClone<T> {
body: T,
buffer: Option<Vec<u8>>,
sender: Option<futures::sync::oneshot::Sender<Vec<u8>>>,
}
impl BodyClone<hyper::Body> {
fn flush(&mut self) {
if let (Some(buffer), Some(sender)) = (self.buffer.take(), self.sender.take()) {
if sender.send(buffer).is_err() {}
}
}
fn push(&mut self, chunk: &hyper::Chunk) {
use hyper::body::Payload;
let length = if let Some(buffer) = self.buffer.as_mut() {
buffer.extend_from_slice(chunk);
buffer.len() as u64
} else {
0
};
if let Some(content_length) = self.body.content_length() {
if length >= content_length {
self.flush();
}
}
}
}
Then I implemented the Stream trait for this struct.
impl Stream for BodyClone<hyper::Body> {
type Item = hyper::Chunk;
type Error = hyper::Error;
fn poll(&mut self) -> futures::Poll<Option<Self::Item>, Self::Error> {
match self.body.poll() {
Ok(Async::Ready(Some(chunk))) => {
self.push(&chunk);
Ok(Async::Ready(Some(chunk)))
}
Ok(Async::Ready(None)) => {
self.flush();
Ok(Async::Ready(None))
}
other => other,
}
}
}
Finally I could define an extension method on hyper::Body:
pub type BufferFuture = Box<Future<Item = Vec<u8>, Error = ()> + Send>;
trait CloneBody {
fn clone_body(self) -> (hyper::Body, BufferFuture);
}
impl CloneBody for hyper::Body {
fn clone_body(self) -> (hyper::Body, BufferFuture) {
let (sender, receiver) = futures::sync::oneshot::channel();
let cloning_stream = BodyClone {
body: self,
buffer: Some(Vec::new()),
sender: Some(sender),
};
(
hyper::Body::wrap_stream(cloning_stream),
Box::new(receiver.map_err(|_| ())),
)
}
}
This can be used as follows:
let (body: hyper::Body, buffer: BufferFuture) = body.clone_body();
I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!
I am trying to implement library for making TCP servers.
This is very simplified code with a problem:
#![crate_name="http_server2"]
#![crate_type="lib"]
use std::io::{TcpListener, Listener, Acceptor, TcpStream, IoResult, Reader, Writer};
use std::ops::Fn;
use std::sync::Arc;
pub trait Handler: Sized + Send {
fn do_it(s: TcpStream) -> IoResult<()>;
}
fn serve(handler: Arc<Handler + Sized>) -> IoResult<()>
{
let listener = TcpListener::bind("127.0.0.1", 1234);
for stream in try!(listener.listen()).incoming() {
let stream = try!(stream);
let handler = handler.clone();
spawn(proc() {
handler.do_it(stream);
});
}
Ok(())
}
Compiler totally ignores my specifications of Handler + Sized. If I implement structure with trait Handler and try to call serve with this structure, such advice about size will be ignored too ( http://is.gd/OWs22i ).
<anon>:13:1: 25:2 error: the trait `core::kinds::Sized` is not implemented for the type `Handler+'static+Sized`
<anon>:13 fn serve(handler: Arc<Handler + Sized>) -> IoResult<()>
<anon>:14 {
<anon>:15 let listener = TcpListener::bind("127.0.0.1", 1234);
<anon>:16
<anon>:17 for stream in try!(listener.listen()).incoming() {
<anon>:18 let stream = try!(stream);
...
<anon>:13:1: 25:2 note: the trait `core::kinds::Sized` must be implemented because it is required by `alloc::arc::Arc`
<anon>:13 fn serve(handler: Arc<Handler + Sized>) -> IoResult<()>
<anon>:14 {
<anon>:15 let listener = TcpListener::bind("127.0.0.1", 1234);
<anon>:16
<anon>:17 for stream in try!(listener.listen()).incoming() {
<anon>:18 let stream = try!(stream);
...
error: aborting due to previous error
How can I implement one template function with multithreading that will accept different handlers?
As I said in my comment above,
use std::io::{TcpListener, Listener, Acceptor, TcpStream, IoResult, Writer};
use std::sync::Arc;
pub trait Handler: Sized + Send {
fn do_it(&self, s: TcpStream) -> IoResult<()>;
}
fn serve<T: Handler + Sized + Send + Sync>(handler: Arc<T>) -> IoResult<()> {
let listener = TcpListener::bind("127.0.0.1", 1234);
for stream in try!(listener.listen()).incoming() {
let stream = try!(stream);
let handler = handler.clone();
spawn(proc() {
let _ = handler.do_it(stream);
});
}
Ok(())
}
struct Hello {
x: u32,
}
impl Handler for Hello {
fn do_it(&self, mut s: TcpStream) -> IoResult<()> { s.write_le_u32(self.x) }
}
fn main() {
let s = Arc::new(Hello{x: 123,});
let _ = serve(s);
}
compiles fine. (playpen)
Changes
Make do_it take &self.
Make serve generic, by adding a type parameter with the constraints you want.
Make the impl of Handler for Hello in do_it not discard the result of the write (remove ;).
Clarify with let _ = ... that we intentionally discard a result.
You will not be able to execute it in the playpen though (application terminated abnormally with signal 31 (Bad system call)), as the playpen forbids IO (network IO in this case). It runs fine on my local box though.