I've been building a websocket server library with Rust for about 4-6 weeks now. Today, I grabbed the latest rustc using the rustup.sh script, and build is breaking from new lifetime requirements. I'm not sure where to look for these changes to figure out what I need to specify in order to meet the requirements? This Week in Rust doesn't have a listing for the recent changes, and the Lifetimes Guide only goes so far with various examples.
Various Struct Definitions
pub struct Server<'a> {
pub sockets: Vec<Socket<'a>>,
pub events: Vec<Event<'a>>,
pub to_event_loop: Sender<Action<'a>>,
pub socket_id: String
}
impl<'a> Clone for Server<'a> {
fn clone(&self) -> Server<'a> {
Server {
sockets: self.sockets.clone(),
events: self.events.clone(),
to_event_loop: self.to_event_loop.clone(),
socket_id: self.socket_id.clone()
}
}
}
pub struct Action<'a> {
pub event: String,
pub socket: Socket<'a>,
pub message: Message
}
pub struct Event<'a> {
pub name: String,
pub execute: fn(data: json::Json, server: super::Server)
}
pub struct Socket<'a> {
pub id: String,
pub stream: TcpStream
}
Function Producing Error
pub fn start(server: Server, ip: &str, port: u16) {
/*
* Communication channel
* - From HTTP Server to Event Loop (Action Passed)
*/
let (to_event_loop, from_conn_pool): (Sender<Action>, Receiver<Action>) = channel();
// Start up event loop
let server_clone = server.clone();
let to_event_loop_clone = to_event_loop.clone();
spawn(proc() {
event_loop(server_clone, from_conn_pool, to_event_loop_clone)
});
// Start TCP server
let listener = TcpListener::bind(ip, port);
let mut acceptor = listener.listen();
for stream in acceptor.incoming() {
match stream {
Ok(stream) => {
let event_loop_msgr = to_event_loop.clone();
spawn(proc() {
process_new_connection(stream, event_loop_msgr)
})
}
Err(e) => {
println!("Error accepting connection: {}", e)
}
}
}
drop(acceptor);
}
Compiler Output
src/rustic_io.rs:47:24: 47:30 error: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
src/rustic_io.rs:47 let server_clone = server.clone();
^~~~~~
src/rustic_io.rs:50:9: 50:70 note: first, the lifetime cannot outlive the call at 50:8...
src/rustic_io.rs:50 event_loop(server_clone, from_conn_pool, to_event_loop_clone)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/rustic_io.rs:50:20: 50:32 note: ...so that argument is valid for the call
src/rustic_io.rs:50 event_loop(server_clone, from_conn_pool, to_event_loop_clone)
^~~~~~~~~~~~
src/rustic_io.rs:38:51: 70:2 note: but, the lifetime must be valid for the anonymous lifetime #1 defined on the block at 38:50...
src/rustic_io.rs:38 pub fn start(server: Server, ip: &str, port: u16) {
src/rustic_io.rs:39
src/rustic_io.rs:40 /*
src/rustic_io.rs:41 * Communication channel
src/rustic_io.rs:42 * - From HTTP Server to Event Loop (Action Passed)
src/rustic_io.rs:43 */
...
src/rustic_io.rs:47:24: 47:30 note: ...so that types are compatible (expected `&server::Server<'_>`, found `&server::Server<'_>`)
src/rustic_io.rs:47 let server_clone = server.clone();
^~~~~~
The compiler is very specific on what the problem is, I just have no idea what syntax it is expecting so that I can satisfy the requirements?
Related
I try to get the following code to compile.
Due to the buffer it wanted more explicit lifetimes inserted. So I tried that but still
cannot figure out yet how to make it also work in for_each.
use std::{sync::mpsc::Sender, ffi::OsStr};
use inotify::{Event, EventMask, Inotify, WatchMask};
pub struct Watcher<'a> {
buffer: [u8; 1024],
sender: Sender<Event<&'a OsStr>>,
}
impl <'a> Watcher<'a> {
pub fn new(sender: Sender<Event<&OsStr>>) -> Watcher{
Watcher{
buffer: [0; 1024],
sender: sender,
}
}
pub fn watch_for_led_directories(&mut self, dir: String) {
let sender = self.sender.clone();
let mut inotify = Inotify::init().expect("Error while initializing inotify instance");
// Watch for modify and close events.
inotify
.add_watch(dir, WatchMask::CREATE | WatchMask::DELETE)
.expect("Failed to add file watch");
// Read events that were added with `add_watch` above.
let events = inotify
.read_events_blocking(&mut self.buffer)
.expect("Error while reading events");
events.filter(|e| -> bool { !e.mask.intersects(EventMask::ISDIR) }).for_each(|e| sender.send(e).unwrap());
}
}
EDIT:
I ended up giving up on no-copy of strings:
use std::{ffi::OsStr, sync::mpsc::Sender};
use inotify::{Event, EventMask, Inotify, WatchMask};
pub struct Watcher {
buffer: [u8; 1024],
sender: Sender<Event<String>>,
}
impl Watcher {
pub fn new(sender: Sender<Event<String>>) -> Watcher {
Watcher {
buffer: [0; 1024],
sender: sender,
}
}
pub fn watch_for_led_directories(&mut self, dir: String) {
let sender = self.sender.clone();
let mut inotify = Inotify::init().expect("Error while initializing inotify instance");
// Watch for modify and close events.
inotify
.add_watch(dir, WatchMask::CREATE | WatchMask::DELETE)
.expect("Failed to add file watch");
// Read events that were added with `add_watch` above.
let events = inotify
.read_events_blocking(&mut self.buffer)
.expect("Error while reading events");
let filter = |e: &Event<&OsStr>| -> bool {
!e.mask.intersects(EventMask::ISDIR) || e.name.is_none()
};
events.filter(filter).for_each(|e| {
sender
.send(Event {
cookie: e.cookie,
mask: e.mask,
name: Some(e.name.unwrap().to_str().expect("Expected valid unicode string").into()),
wd: e.wd,
})
.unwrap()
});
}
}
The issue is that while self has the lifetime of 'a, &mut self in watch_for_led_directories() has a different lifetime.
inotify.read_events_blocking() returns events that has the same lifetime as &mut self.buffer, which, as a mutable borrow, has the lifetime of &mut self. But sender is bound to life time 'a. Therefore compiler cannot reconcile these two lifetimes.
One way to solve it is to provide an external buffer, instead of having it owned by Watcher, e.g.:
pub fn watch_for_led_directories(&self, buffer: &'a mut [u8], dir: String) {
...
let events: Events = inotify
.read_events_blocking(buffer)
.expect("Error while reading events");
...
Compiler derives the lifetime of events from buffer and therefore it will be 'a.
To use this method the externally allocated buffer must live at least as long as the Watcher object, which should not be too difficult to arrange.
I´m writing a smart contract with rust in near protocol. I have the following struct for the main contract:
#[near_bindgen]
#[derive(BorshDeserialize, BorshSerialize, PanicOnDefault)]
pub struct Contract {
pub total_transactions: u64,
pub owner_id: AccountId,
pub transactions_per_account: LookupMap<AccountId, UnorderedSet<TransactionId>>,
pub transaction_by_id: LookupMap<TransactionId, Transaction>,
pub transaction_metadata_by_id: UnorderedMap<TransactionId, TransactionMetadata>,
}
I have already written a function to initialize the contract as the following:
#[derive(BorshSerialize)]
pub enum StorageKey {
TransactionsPerAccount,
TransactionById,
TransactionMetadataById,
}
#[near_bindgen]
impl Contract {
#[init]
pub fn new(owner_id: AccountId) -> Self {
let this = Self {
total_transactions: 0,
owner_id,
transactions_per_account: LookupMap::new(StorageKey::TransactionsPerAccount.try_to_vec().unwrap()),
transaction_by_id: LookupMap::new(StorageKey::TransactionById.try_to_vec().unwrap()),
transaction_metadata_by_id: UnorderedMap::new(
StorageKey::TransactionMetadataById.try_to_vec().unwrap(),
),
};
//return the Contract object
this
}
}
But there the UnorderedSet that is inside the LookupMap transactions_per_account is not initialize because it is supposed that there will be an UnorderedSet that corresponds to each user, so it should be created when a user log in for first time and creates an account:
I tried to write that function as the following:
#[payable]
pub fn create_transaction( //implement storage management for transactions
&mut self,
seller_id: AccountId,
buyer_id: AccountId,
price: Price,
nft_id: TokenId,
nft_contract_id: AccountId,
) -> Transaction {
let sender = env::predecessor_account_id();
let transaction = Transaction {
transaction_id: self.total_transactions,
creator_id: sender.clone(),
seller_id: seller_id.clone(),
buyer_id: buyer_id.clone(),
price: price,
nft_id: nft_id.clone(),
nft_contract_id: nft_contract_id.clone(),
amount_in_escrow: false,
token_in_escrow: false,
transaction_status: TransactionStatus::Pending,
};
//let account_as_key = transaction.creator_id.try_to_vec().unwrap();
// update number of transactions
self.total_transactions += 1;
let unordered_set = self.transactions_per_account.get(&transaction.creator_id);
match unordered_set {
Some(mut set) => {
set.insert(&transaction.transaction_id);
},
None => {
let mut set = UnorderedSet::new(&*transaction.creator_id.try_to_vec().unwrap());
set.insert(&transaction.transaction_id);
self.transactions_per_account.insert(&transaction.creator_id, &set);
},
}
self.transaction_by_id.insert(&transaction.transaction_id, &transaction);
transaction
}
Cargo check let it pass with no error but when trying to compile I got the following error:
error[E0277]: the trait bound `UnorderedSet<u64>: Serialize` is not satisfied
--> src/lib.rs:133:1
|
133 | #[near_bindgen]
| ^^^^^^^^^^^^^^^ the trait `Serialize` is not implemented for `UnorderedSet<u64>`
|
note: required by a bound in `to_vec`
--> /home/walquer/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.78/src/ser.rs:2193:17
|
2193 | T: ?Sized + Serialize,
| ^^^^^^^^^ required by this bound in `to_vec`
= note: this error originates in the attribute macro `near_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
For more information about this error, try `rustc --explain E0277`.
I believe the error is caused by a erroneous initialization of the UnorderedSet collection that was defined inside the struct that do have the serialize trait.
Can someone tell me what im missing?
I have a Tokio client that talks to a remote server and is supposed to keep the connection alive permanently. I've implemented the initial authentication handshake and found that when my test terminates, I get an odd panic:
---- test_connect_without_database stdout ----
thread 'test_connect_without_database' panicked at 'Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.', /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.3.5/src/runtime/blocking/shutdown.rs:51:21
I'm at an absolute loss as to what might be causing this so I don't know what other bits of code to add for context.
Here's my minimal reproducible example (playground):
use std::cell::RefCell;
use std::net::{IpAddr, SocketAddr};
use tokio::net::TcpStream;
use tokio::prelude::*;
use tokio::runtime;
#[derive(PartialEq, Debug)]
pub struct Configuration {
/// Database username.
username: String,
/// Database password.
password: String,
/// Database name.
db_name: String,
/// IP address for the remove server.
address: IpAddr,
/// Remote server port.
port: u16,
/// Number of connections to open.
connections: u16,
}
impl Configuration {
pub fn new(
username: &str,
password: &str,
db_name: &str,
address: &str,
port: u16,
connections: u16,
) -> Result<Configuration, Box<dyn std::error::Error>> {
let address = address.to_string().parse()?;
let configuration = Configuration {
username: username.to_string(),
password: password.to_string(),
db_name: db_name.to_string(),
address,
port,
connections,
};
Ok(configuration)
}
pub fn socket(&self) -> SocketAddr {
SocketAddr::new(self.address, self.port)
}
}
#[derive(Debug)]
pub struct Session {
configuration: Configuration,
runtime: RefCell<runtime::Runtime>,
}
impl Session {
fn new(config: Configuration) -> Result<Session, io::Error> {
let runtime = runtime::Builder::new_multi_thread()
.worker_threads(6)
.enable_all()
.build()?;
let session = Session {
configuration: config,
runtime: RefCell::new(runtime),
};
Ok(session)
}
fn configuration(&self) -> &Configuration {
&self.configuration
}
}
#[derive(Debug)]
pub(crate) struct Connection<'a> {
/// Socket uses to read and write from.
session: &'a Session,
/// Connection to the remote server.
stream: TcpStream,
/// Flag that indicates whether the connection is live.
live: bool,
}
impl<'a> Connection<'a> {
async fn new(session: &Session) -> Result<Connection<'_>, Box<dyn std::error::Error>> {
let mut stream = TcpStream::connect(session.configuration().socket()).await?;
let conn = Connection {
session,
stream,
live: true,
};
Ok(conn)
}
fn live(&self) -> bool {
self.live
}
}
#[tokio::test]
async fn test_connect_without_database() -> Result<(), Box<dyn std::error::Error>> {
let config = Configuration::new("rust", "", "", "127.0.0.1", 2345, 2).unwrap();
let session = Session::new(config).unwrap();
let conn = Connection::new(&session).await?;
assert!(conn.live());
Ok(())
}
fn main() {
println!("{}", 65u8 as char)
}
As the error message states:
This happens when a runtime is dropped from within an asynchronous context
You have created nested runtimes:
From tokio::test
From runtime::Builder::new_multi_thread
The second runtime is owned by Session which is dropped at the end of the asynchronous test. You can observe this by skipping the destructor using mem::forget:
#[tokio::test]
async fn test_connect_without_database() -> Result<(), Box<dyn std::error::Error>> {
let config = Configuration::new("rust", "", "", "127.0.0.1", 2345, 2).unwrap();
let session = Session::new(config).unwrap();
// Note that the assert was removed!
std::mem::forget(session);
Ok(())
}
Don't spawn nested runtimes and don't drop one runtime from within another.
See also:
How can I create a Tokio runtime inside another Tokio runtime without getting the error "Cannot start a runtime from within a runtime"?
"cannot recursively call into `Core`" when trying to achieve nested concurrency using Tokio
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
try：cargo run --release rather than cargo run
https://dev.to/scyart/comment/1f88p
I'm new to Rust and I'm trying to create a Server struct which listen to an address and starts a TCP socket connection. The problem is that I want to store the client connection inside a hash map so I can use it later..
I tried writing this:
use std::collections::HashMap;
use std::net::TcpListener;
use std::net::TcpStream;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
thread::spawn(move || {
let mut listener =
TcpListener::bind(self.clone().url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => self.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&mut self, stream: TcpStream) {
let id = self.id.read().unwrap();
self.connections.read().unwrap().insert(id, stream);
let id = self.id.write().unwrap();
*id += 1;
}
}
But of course this doesn't work.. There are 2 things that I don't understand, the first is how to pass the stream into my function and then store in my connections hash map so I can use it later and how to use my id inside my on_client_connect function.
You need to clone outside of thread::spawn and move the cloned instance in thread scope.
Also, on_client_connect do not need &mut self because the fields id and connections are already protected inside RwLock.
use std::net::TcpListener;
use std::net::TcpStream;
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
let me = self.clone(); // Clone it outside
thread::spawn(move || {
let mut listener = TcpListener::bind(&me.url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => me.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&self, stream: TcpStream) { // `&mut self` not needed as the id, connection are inside the lock
let mut id = self.id.write().unwrap();
self.connections.write().unwrap().insert(*id, stream);
*id += 1;
}
}
playground
There's quite a few issues that need minor fixes in this code.
The first one I ran into was the usage of self in the thread::spawn closure.
The thread::spawn needs its argument to have static lifetime, but we've no guarantee that the Server object lives that long.
I solved it by cloning the Server object and moving that into the closure. This is OK as all its data is already behind Arcs.
The next problem was self.connections.read().unwrap().insert(*id, stream); needs to get a write lock, not a read.
Finally id+=1 needs to dereference id.
Once these were fixed, it seems that storing the TcpStream is not an issue. (At least using nightly). I'd thought I'd need to box the TcpStream - but it seems OK as is.
You can see that it compiles in the playground
use std::collections::HashMap;
use std::net::TcpListener;
use std::net::TcpStream;
use std::sync::{Arc, RwLock};
use std::thread;
#[derive(Clone, Debug)]
pub struct Server {
id: Arc<RwLock<u32>>,
connections: Arc<RwLock<HashMap<u32, TcpStream>>>,
url: String,
}
impl Server {
pub fn new(url: String) -> Server {
let server = Server {
id: Arc::new(RwLock::new(0)),
connections: Arc::new(RwLock::new(HashMap::new())),
url,
};
server
}
pub fn start(&self) {
let mut self_clone = self.clone();
thread::spawn(move || {
let mut listener =
TcpListener::bind(&self_clone.url).expect("Could not start the server");
println!("Server started succesfully");
for stream in listener.incoming() {
match stream {
Ok(stream) => self_clone.on_client_connect(stream),
Err(error) => eprintln!("Error when tried to use stream"),
}
}
});
}
fn on_client_connect(&mut self, stream: TcpStream) {
let id = self.id.read().unwrap();
self.connections.write().unwrap().insert(*id, stream);
let mut id = self.id.write().unwrap();
*id += 1;
}
}
I am writing a simple websocket server named BoltServer based on Rust websocket crate(Code is incomplete, I just started). I am using the example program as the base. However the example programs are not modular (having very long methods). So I am trying to break them up into structs and methods. I want to spawn two threads for each client. One sends messages and the other one receives messages. So here, I want to capture all variables used by the thread in a struct and then call the run method in the impl.
extern crate websocket;
extern crate time;
extern crate rustc_serialize;
pub mod ws {
use std::thread;
use std::sync::{Arc, Mutex};
use std::sync::mpsc;
use std::net::ToSocketAddrs;
use websocket;
use websocket::{Server, Message, Sender, Receiver};
use websocket::server::Connection;
use websocket::stream::WebSocketStream;
use std::str::from_utf8;
struct BoltUser {
user_id: u32,
my_tx: mpsc::Sender<String>,
}
struct Broadcaster {
my_rx: mpsc::Receiver<String>,
}
impl Broadcaster {
fn new(receiver: mpsc::Receiver<String>) -> Broadcaster {
Broadcaster { my_rx: receiver }
}
fn run(self) {
while let Ok(msg) = self.my_rx.recv() {
println!("Broadcaster got message: {}", msg);
}
}
}
struct SocketReader {}
impl SocketReader {
fn run(self) {}
}
struct SocketWriter {
my_rx: mpsc::Receiver<String>,
sender: Sender,
}
impl SocketWriter {
fn run(self) {
while let Ok(message) = self.my_rx.recv() {
}
}
}
pub struct BoltServer {
address: String,
connected_users: Arc<Mutex<Vec<BoltUser>>>,
}
impl BoltServer {
pub fn new(address: &str) -> BoltServer {
BoltServer {
address: address.to_string(),
connected_users: Arc::new(Mutex::new(vec![])),
}
}
fn handshake(&mut self,
connection: Connection<WebSocketStream, WebSocketStream>)
-> (SocketWriter, SocketReader) {
let request = connection.read_request().unwrap();
// println!("thread-> Accepting request...");
let response = request.accept();
let (mut sender, mut receiver) = response.send().unwrap().split();
let (user_tx, user_rx) = mpsc::channel::<String>();//Create a channel for writer
let socket_writer = SocketWriter {
my_rx: user_rx,
sender: sender,
};
let socket_reader = SocketReader {};
(socket_writer, socket_reader)
}
pub fn start(&mut self) {
println!("Starting");
let (broadcaster_tx, broadcaster_rx) = mpsc::channel::<String>();
let broadcaster = Broadcaster::new(broadcaster_rx);
let handle = thread::Builder::new()
.name("Broadcaster".to_string())
.spawn(move || broadcaster.run());
let server = Server::bind(&*self.address).unwrap();
let mut user_id: u32 = 0;
// Block and process connection request from a new client
for connection in server {
user_id = user_id + 1;//Create a new user id
let (socket_writer, socket_reader) = self.handshake(connection);
thread::Builder::new()
.name("Socket writer".to_string())
.spawn(move || socket_writer.run());
thread::Builder::new()
.name("Socket reader".to_string())
.spawn(move || socket_reader.run());
}
handle.unwrap().join();
println!("Finished");
}
}
}
The following code gives an idea of what I want to achieve.
// Block and process connection request from a new client
for connection in server {
user_id = user_id + 1;//Create a new user id
let (socket_writer, socket_reader) = self.handshake(connection);
thread::Builder::new().name("Socket writer".to_string()).spawn(move || {
socket_writer.run()
});
thread::Builder::new().name("Socket reader".to_string()).spawn(move || {
socket_reader.run()
});
}
Here I am stuck in the handshake method. I am not able to initialize the SocketWriter struct with the sender that I am getting by calling the split method in the library. I am getting the following compilation error:
error[E0038]: the trait `websocket::Sender` cannot be made into an object
--> src/lib.rs:46:9
|
46 | sender:Sender,
| ^^^^^^^^^^^^^ the trait `websocket::Sender` cannot be made into an object
|
= note: method `send_dataframe` has generic type parameters
= note: method `send_message` has generic type parameters
The error is telling you the immediate problem:
46 | sender:Sender,
| ^^^^^^^^^^^^^ the trait `websocket::Sender` cannot be made into an object
First of all, a variable/field can't have a plain trait type (but &Trait can be possible), but also the websocket::Sender trait is not object safe; it has generic methods which can't work dynamically (ie vtable methods have to have a fixed type).
Instead, you have to have a concrete type (you could also make it a generic struct).
It's not obvious what the right type is, so I like to get the compiler to tell me. So first try the simplest possible:
sender: (),
The compiler replies with some information:
| ^^^^^^ expected (), found struct `websocket::client::Sender`
Ok, let's plug that in:
sender: websocket::client::Sender,
That gives:
46 | sender: websocket::client::Sender,
| ^^^^^^^^^^^^^^^^^^^^^^^^^ expected 1 type arguments, found 0
Ok, that's a generic type. Next try:
sender: websocket::client::Sender<()>,
Finally it gives us the real type:
74 | sender:sender,
| ^^^^^^ expected (), found enum `websocket::WebSocketStream`
So finally we can finish SocketWriter:
struct SocketWriter {
my_rx: mpsc::Receiver<String>,
sender: websocket::client::Sender<websocket::WebSocketStream>,
}
There's a following compile error since the connection you get is a Result<> so you need to check for errors (it compiles if I change to self.handshake(connection.unwrap()), but that's obviously not the best practice.