I am trying to use the read and write of a tcp stream in different threads. This is what I currently have:
use tokio::prelude::*;
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut stream = TcpStream::connect("localhost:8080").await?;
let (mut read, mut write) = stream.split();
tokio::spawn(async move {
loop {
let mut buf = [0u8; 32];
read.read(&mut buf).await.unwrap();
println!("{}", std::str::from_utf8(&buf));
}
});
Ok(())
}
Im going to use another thread for the write. My problem is that I get the error that 'stream' is dropped while still borrowed.
That happens due to the method signature of Tokio::split, as you can see it takes &mut self, so its part cannot be used in a tokio::spawn future argument due to the 'static bound. So, this is exactly what the error says.
What you are searching is tokio::io::split. Playground
use tokio::prelude::*;
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut stream = TcpStream::connect("localhost:8080").await?;
let (mut read, mut write) = tokio::io::split(stream);
tokio::spawn(async move {
loop {
let mut buf = [0u8; 32];
read.read(&mut buf).await.unwrap();
println!("{:?}", std::str::from_utf8(&buf));
}
});
Ok(())
}
Reading https://users.rust-lang.org/t/why-i-have-to-use-tokio-tcpstream-split-for-concurrent-read-writes/47755/3
And it suggests to use into_split it is more efficient.
tokio::io::split uses a Mutex which into_split apparently does not have (specific to TcpStream)
Related
I'm writing a program that will concurrently call an API and fill a VecDeque with integers from the response JSON. Since these will be running concurrently, I wrapped the VecDeque in a Mutex. When I test the code and try to push an integer into the VecDeque, I get an error.
#[tokio::main]
async fn main() -> Result<(), ()> {
let client = Client::new();
let mut tank = Mutex::new(VecDeque::new());
// let mut tank = VecDeque::with_capacity(10);
tank.push_back(2);
tank.push_back(3);
println!("{:?}", tank);
Ok(())
}
tank.push_back(2);
^^^^^^^^^ method not found in `std::sync::Mutex<VecDeque<_>>
Do I need to create the VecDeque inside a struct instead?
As per the suggestion, I needed to lock before changing the VecDeque. It works now:
#[tokio::main]
async fn main() -> Result<(), ()> {
let client = Client::new();
let mut tank = Mutex::new(VecDeque::new());
let mut tank = tank.lock().unwrap().push_back(2);
let mut tank = tank.lock().unwrap().push_back(3);
println!("{:?}", tank);
Ok(())
}
I have a TcpStream, and i'm trying to create a dedicated thread to read from TcpStream and the main thread should do the write only, so i wrapped the TcpStream inside an Arc to share ownership.
Here is my code
fn main() {
let mut tcp_stream = TcpStream::connect("127.0.0.1:8080").unwrap();
let tcp_stream = Arc::new(&mut tcp_stream);
}
fn read_from_stream(tcp_stream: Arc<&mut TcpStream>) {
let mut buffer = String::new();
tcp_stream.read_to_string(&mut buffer);
}
I got the error Cannot borrow data in Arc as mutable. But the tcp_stream needs to be mutable in order to call read_to_string method.
What should i do to make this work?
While Arc does inhibit mutability, in this case that's not an issue because &TcpStream implements Read.
Therefore you can wrap TcpStream (not &mut TcpStream) in Arc and call read_to_string() on the shared &TcpStream reference obtained with Arc::as_ref():
fn main() {
let tcp_stream = TcpStream::connect("127.0.0.1:8080").unwrap();
let tcp_stream = Arc::new(tcp_stream);
}
fn read_from_stream(tcp_stream: Arc<TcpStream>) {
let mut buffer = String::new();
tcp_stream.as_ref().read_to_string(&mut buffer);
}
Playground
From the docs:
Shared references in Rust disallow mutation by default, and Arc is no
exception: you cannot generally obtain a mutable reference to
something inside an Arc. If you need to mutate through an Arc, use
Mutex, RwLock, or one of the Atomic types.
In your case, I would go either with Mutex or RwLock.
Example using Mutex:
fn main()
{
let mut stream = TcpStream::connect("127.0.0.1:34254").unwrap();
let stream = Arc::new(Mutex::new(stream));
read_from_stream(stream);
}
fn read_from_stream(tcp_stream: Arc<Mutex<TcpStream>>)
{
let mut buffer = String::new();
let mut tcp_stream = tcp_stream.lock().unwrap();
tcp_stream.read_to_string(&mut buffer).unwrap();
}
I am trying to write a tcp server with async-std, in order to resolve the confilice of read-write borrow check, I found a strange way:
use async_std::prelude::*;
use async_std::task;
use async_std::net::TcpListener;
use async_std::io::BufReader;
fn main() {
task::block_on(async {
let listener = TcpListener::bind("0.0.0.0:9000").await.unwrap();
let mut incoming = listener.incoming();
while let Some(stream) = incoming.next().await {
let stream = stream.unwrap();
println!("Client Addr: {:?}", stream.peer_addr().unwrap());
task::spawn( async move {
let (reader, mut writer) = (&stream, &stream); // Stange here <-----
let reader = BufReader::new(reader);
let mut lines = reader.lines();
while let Some(line) = lines.next().await {
let mut line = line.unwrap();
line.push('x');
line.push('\n');
writer.write_all(line.as_bytes()).await.unwrap();
}
});
}
});
}
Strange:
let (reader, mut writer) = (&stream, &stream); // Stange here <-----
I wrote a verification program following this method:
fn main() {
let mut arr = vec![1, 2, 3, 4];
let (r, mut w) = (&arr, &arr);
for v in r.iter() {
if v == &2 {
w.push(5);
}
}
dbg!(arr);
}
got error:
error[E0596]: cannot borrow `*w` as mutable, as it is behind a `&` reference
can anyone explain it ?
This works because TcpStream has the following impls:
impl Read for TcpStream
impl Write for TcpStream
impl<'_> Read for &'_ TcpStream
impl<'_> Write for &'_ TcpStream
The first two are the "normal" ones, the last two are the "special" ones that allow you to do this reader/writer trick.
For example: The read() trait method is defined like this:
pub fn read<'a>(&'a mut self, buf: &'a mut [u8]) -> ImplFuture<Result<usize>>
When you do stream.read(buf), you're using the impl Read for TcpStream impl. Therefore, you're calling the method with self being of type &mut TcpStream.
When you do let reader = &stream; reader.read(buf), you're using the impl<'_> Read for &'_ TcpStream. Therefore, you're calling the method with self being of type &mut &TcpStream.
I want to modify a big vector from multiple threads in parallel.
Works fine: u32
use std::thread;
use std::sync::Arc;
fn main() {
let input = Arc::new([1u32, 2, 3, 4]);
let mut handles = Vec::new();
for t in 0..4 {
let inp = input.clone();
let handle = thread::spawn(move || unsafe {
let p = (inp.as_ptr() as *mut u32).offset(t as isize);
*p = inp[t] + t as u32 ;
});
handles.push(handle);
}
for h in handles {
h.join().unwrap();
}
println!("{:?}", input);
}
Segfaults: Vec<HashSet<&str>>
When I change the u32 to Vec<HashSet<&str>>, the pointer does not seem to work.
use std::thread;
use std::sync::Arc;
use std::collections::HashSet;
fn main() {
let mut a = HashSet::new();
a.insert("aaa");
let input = Arc::new(vec![a.clone(), a.clone(), a.clone(), a.clone()]);
let mut handles = Vec::new();
for _t in 0..4 {
let inp = input.clone();
let handle = thread::spawn(move || unsafe {
let p = (inp.as_ptr() as *mut Vec<HashSet<&str>>).offset(0);
(*p)[0].insert("bbb");
});
handles.push(handle);
}
for h in handles {
h.join().unwrap();
}
println!("{:?}", input);
}
What is the difference?
It is hard to say what is wrong with your initial code as it segfaults in the playground. You are likely invoking undefined behavior by taking a reference to immutable (!) vec and trying to mutate its elements by casting &Vec -> *mut Vec -> &mut Vec (on a method call). Multiple mutable references to the same thing are a big no-no. Besides, your code even uses the same HashSet ((*p)[0]) mutably in parallel, which, again, is undefined behavior.
The easiest way here would be to use crossbeam's scoped threads. They allow referencing stack variables, like your input. Vec can also give out distinct mutable references to its elements without using unsafe. Using this, your code seems to do the expected thing.
use crossbeam::thread;
use std::collections::HashSet;
fn main() {
let mut a = HashSet::new();
a.insert("aaa");
let mut input = vec![a.clone(), a.clone(), a.clone(), a.clone()];
thread::scope(|s| {
for set in &mut input {
s.spawn(move |_| {
set.insert("bbb");
});
}
}).unwrap();
println!("{:?}", input);
}
I have found the way:
use std::thread;
use std::sync::Arc;
use std::collections::HashSet;
fn main() {
let mut a = HashSet::new();
a.insert("aaa");
let input = Arc::new(vec![a.clone(), a.clone(), a.clone(), a.clone()]);
let mut handles = Vec::new();
for _t in 0..4 {
let inp = input.clone();
//let out = output.clone();
let handle = thread::spawn(move || unsafe {
let p = (inp.as_ptr() as *mut Vec<HashSet<&str>>).offset(0);
(*p)[0].insert("bbb");
});
handles.push(handle);
}
for h in handles {
h.join().unwrap();
}
println!("{:?}", input);
}
thanks for guys!
I am trying to read all content from a file into a vector using the async rust api:
let mut content : Vec<u8> = vec![];
let f = tokio::fs::File::open("myfilecontent")
.and_then(|mut myfile| {
myfile.read_buf(&mut content)
});
f.await;
But I keep getting this error:
error[E0515]: cannot return value referencing function parameter `myfile`
Which sounds reasonable, because the future returned by the closure must keep a reference to the file, but as this closure is the only user of the file it could take ownership. How can I convince rust to do the right thing?
You can use an async move block like so:
use futures::TryFutureExt;
use tokio::io::AsyncReadExt;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut content: Vec<u8> = vec![];
let f = tokio::fs::File::open("myfilecontent").and_then(
|mut myfile| async move { myfile.read_buf(&mut content).await },
);
f.await?;
Ok(())
}
or skip and_then and go straight for .await:
use tokio::io::AsyncReadExt;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut content: Vec<u8> = vec![];
let mut myfile = tokio::fs::File::open("myfilecontent").await?;
myfile.read_buf(&mut content).await?;
Ok(())
}