I've read the documentation for std::net and mio, and I've found some methods like set_nodelay and set_keepalive, but I haven't found a way to set other socket options like SO_REUSEPORT and SO_REUSEADDR on a given socket. How can I do this?
Because SO_REUSEPORT isn't cross-platform, you will need to dip into platform-specific code. In this case, you can get the raw file descriptor from the socket and then use functions, types, and values from the libc crate to set the options you want:
extern crate libc; // 0.2.43
use std::{io, mem, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), io::Error> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
unsafe {
let optval: libc::c_int = 1;
let ret = libc::setsockopt(
listener.as_raw_fd(),
libc::SOL_SOCKET,
libc::SO_REUSEPORT,
&optval as *const _ as *const libc::c_void,
mem::size_of_val(&optval) as libc::socklen_t,
);
if ret != 0 {
return Err(io::Error::last_os_error());
}
}
Ok(())
}
I make no guarantee that this is the right place to set this option, or that I haven't screwed up something in the unsafe block, but it does compile and run on macOS 10.12.
A better solution may be to check out the nix crate, which provides nicer wrappers for most *nix-specific code:
extern crate nix; // 0.11.0
use nix::sys::socket::{self, sockopt::ReusePort};
use std::{error::Error, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), Box<Error>> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
socket::setsockopt(listener.as_raw_fd(), ReusePort, &true)?;
Ok(())
}
An even better solution may be to check out the net2 crate, which provides higher-level methods aimed specifically at networking-related code:
extern crate net2; // 0.2.33
use net2::{unix::UnixTcpBuilderExt, TcpBuilder};
fn main() -> Result<(), std::io::Error> {
let listener = TcpBuilder::new_v4()?
.reuse_address(true)?
.reuse_port(true)?
.bind("0.0.0.0:8888")?
.listen(42)?;
Ok(())
}
Related
I want to write a FFI wrapper for sn_api library, which contains async functions. It will be used in single-threaded non-async code written in Red.
I found, that the easy way is to use Runtime::new().unwrap().block_on(...) in every exported function, although it involves a lot of creating new Tokio runtimes and seem to be too heavy to be run on every call:
use std::os::raw::c_char;
use std::ffi::{CString, CStr};
use sn_api::{BootstrapConfig, Safe};
use tokio::runtime::Runtime;
#[no_mangle]
pub extern "C" _safe_connect(ptr: *const Safe, bootstrap_contact: *const c_char) {
assert!(!ptr.is_null());
let _safe = unsafe {
&*ptr
};
let bootstrap_contact = unsafe {
CStr::from_ptr(bootstrap_contact)
}
let mut bootstrap_contacts = BootstrapConfig::default();
bootstrap_contacts.insert(bootstrap_contact.parse().expect("Invalid bootstrap address"));
// how to reuse the Runtime in other functions?
Runtime::new().unwrap().block_on(_safe.connect(None, None, Some(bootstrap_contacts)));
}
Is it possible to run all async functions on a common Runtime? I imagine it would require creating some singleton / global, but my library is compiled with crate-type = ["cdylib"], which seems not a good place for globals. What would be the best approach?
I've decided for an approach, where I create a Tokio Runtime, and then pass it to every FFI function call containing async code:
use std::os::raw::c_char;
use std::ffi::{CString, CStr};
use sn_api::{BootstrapConfig, Safe};
use tokio::runtime::Runtime;
#[no_mangle]
pub extern "C" fn init_runtime() -> *mut Runtime {
Box::into_raw(Box::new(Runtime::new().unwrap()))
}
#[no_mangle]
pub extern "C" _safe_connect(rt_ptr: *mut Runtime, safe_ptr: *mut Safe, bootstrap_contact: *const c_char) {
assert!(!safe_ptr.is_null());
assert!(!rt_ptr.is_null());
let bootstrap_contact = unsafe {
CStr::from_ptr(bootstrap_contact)
}
let mut bootstrap_contacts = BootstrapConfig::default();
bootstrap_contacts.insert(bootstrap_contact.parse().expect("Invalid bootstrap address"));
unsafe {
let _safe = &mut *safe_ptr;
let rt = &mut *rt_ptr;
rt.block_on(_safe.connect(None, None, Some(bootstrap_contacts))).unwrap();
}
}
I faced the same issue. Here is my cut: export-tokio-to-lib.
plugin.rs:
use async_ffi::{FfiFuture, FutureExt};
use tokio::runtime::Handle;
/// # Safety
#[no_mangle]
pub unsafe extern "C" fn test(arg: f32, handle: *const Handle) -> FfiFuture<safer_ffi::String> {
let handle = &*handle;
async move {
let _enter = handle.enter();
tokio::time::sleep(std::time::Duration::from_secs_f32(arg)).await;
format!("slept {arg} secs").into()
}
.into_ffi()
}
Try this.
From this:
#[tokio::main]
async fn main() {
println!("hello");
}
Transformed into:
fn main() {
let mut rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async {
println!("hello");
})
}
Reference: https://tokio.rs/tokio/tutorial/hello-tokio#async-main-function
I have a future which wraps a TCP stream in a Framed using the LinesCodec.
When I try to wrap this in a test, I get the future blocking around 20% of the time, but because I have nothing listening on the socket I'm trying to connect to, I expect to always get the error:
thread 'tokio-runtime-worker-0' panicked at 'error: Os { code: 111, kind: ConnectionRefused, message: "Connection refused" }', src/lib.rs:35:24 note: Run with 'RUST_BACKTRACE=1' for a backtrace.
This is the test code I have used:
#[macro_use(try_ready)]
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
use std::io;
use std::net::SocketAddr;
use tokio::codec::{Framed, LinesCodec};
use tokio::net::TcpStream;
use tokio::prelude::*;
struct MyFuture {
addr: SocketAddr,
}
impl Future for MyFuture {
type Item = Framed<TcpStream, LinesCodec>;
type Error = io::Error;
fn poll(&mut self) -> Result<Async<Framed<TcpStream, LinesCodec>>, io::Error> {
let strm = try_ready!(TcpStream::connect(&self.addr).poll());
Ok(Async::Ready(Framed::new(strm, LinesCodec::new())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = MyFuture { addr: addr }
.and_then(|f| {
println!("connected");
let cn = f.get_ref();
cn.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
playground
I have seen patterns in other languages where the test binary itself offers a mechanism to return results asynchronously, but haven't found a good way of using a similar mechanism in Rust.
A simple way to test async code may be to use a dedicated runtime for each test: start it, wait for future completion and shutdown the runtime at the end of the test.
#[test]
fn my_case() {
// setup future f
// ...
tokio::run(f);
}
I don't know if there are consolidated patterns already in the Rust ecosystem; see this discussion about the evolution of testing support for future based code.
Why your code does not work as expected
When you invoke poll(), the future is queried to check if a value is available.
If a value is not available, an interest is registered so that poll() will be invoked again when something happens that can resolve the future.
When your MyFuture::poll() is invoked:
TcpStream::connect creates a new future TcpStreamNew
TcpStreamNew::poll is invoked immediately only once on the future's creation at step 1.
The future goes out of scope, so the next time you invoke MyFuture::poll you never resolve the previously created futures.
You have registered an interest for a future that, if not resolved the first time you poll it, you never ask back again (poll) for a resolved value or for an error.
The reason of the "nondeterministic" behavior is because the first poll sometimes resolve immediately with a ConnectionRefused error and sometimes it waits forever for a future connection event or a failure that it is never retrieved.
Look at mio::sys::unix::tcp::TcpStream used by Tokio:
impl TcpStream {
pub fn connect(stream: net::TcpStream, addr: &SocketAddr) -> io::Result<TcpStream> {
set_nonblock(stream.as_raw_fd())?;
match stream.connect(addr) {
Ok(..) => {}
Err(ref e) if e.raw_os_error() == Some(libc::EINPROGRESS) => {}
Err(e) => return Err(e),
}
Ok(TcpStream {
inner: stream,
})
}
When you connect on a non-blocking socket, the system call may connect/fail immediately or return EINPROGRESS, in this last case a poll must be triggered for retrieving the value of the error.
The issue is not with the test but with the implementation.
This working test case based on yours has no custom future implementation and only calls TcpStream::connect(). It works as you expect it to.
extern crate futures;
extern crate tokio;
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
use std::net::SocketAddr;
use tokio::net::TcpStream;
use tokio::prelude::*;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = TcpStream::connect(&addr)
.and_then(|f| {
println!("connected");
f.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
playground
You are connecting to the same endpoint over and over again in your poll() method. That's not how a future works. The poll() method will be called repeatedly, with the expectation that at some point it will return either Ok(Async::Ready(..)) or Err(..).
If you initiate a new TCP connection every time poll() is called, it will be unlikely to complete in time.
Here is a modified example that does what you expect:
#[macro_use(try_ready)]
extern crate futures;
extern crate tokio;
use std::io;
use std::net::SocketAddr;
use tokio::codec::{Framed, LinesCodec};
use tokio::net::{ConnectFuture, TcpStream};
use tokio::prelude::*;
struct MyFuture {
tcp: ConnectFuture,
}
impl MyFuture {
fn new(addr: SocketAddr) -> MyFuture {
MyFuture {
tcp: TcpStream::connect(&addr),
}
}
}
impl Future for MyFuture {
type Item = Framed<TcpStream, LinesCodec>;
type Error = io::Error;
fn poll(&mut self) -> Result<Async<Framed<TcpStream, LinesCodec>>, io::Error> {
let strm = try_ready!(self.tcp.poll());
Ok(Async::Ready(Framed::new(strm, LinesCodec::new())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = MyFuture::new(addr)
.and_then(|f| {
println!("connected");
let cn = f.get_ref();
cn.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
I'm not certain what you intend your future to do, though; I can't comment if it's the right approach.
to some degree, you can drop in tokio's test library to make this easier; it supports async/await in unit tests.
#[tokio::test]
async fn my_future_test() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
match MyFuture { addr }.poll().await {
Ok(f) => assert!("something good")
Err(e) => assert!("something bad")
}
}
https://docs.rs/tokio/0.3.3/tokio/attr.test.html
Here is an example program:
extern crate futures;
extern crate tokio_core;
use futures::{Async, Future, Stream};
use tokio_core::reactor::Core;
use tokio_core::net::TcpListener;
fn main() {
let mut core = Core::new().unwrap();
futures::sync::oneshot::spawn(
TcpListener::bind(&"127.0.0.1:5000".parse().unwrap(), &core.handle())
.unwrap()
.incoming()
.for_each(|_| {
println!("connection received");
Ok(())
}),
&core,
);
let ft = futures::future::poll_fn::<(), (), _>(|| {
std::thread::sleep_ms(50);
Ok(Async::NotReady)
});
core.run(ft);
}
As you can see, I call oneshot::spawn and then immediately drop its return value, which should theoretically cancel the future contained inside. However, when I run this program and then make a connection to 127.0.0.1:5000, it still prints "connection received." Why does it do this? I expected it to not print anything and drop the TcpListener, unbinding from the port.
This is a (now fixed) bug in the futures crate; version 0.1.18 should include the fix.
It used inverted values for keep_running: bool in SpawnHandle/Executor.
mexPrintf, just like printf, accepts a varargs list of arguments, but I don't know what the best way to wrap this is in Rust. There is a RFC for variadic generics, but what can we do today?
In this example, I want to print of the number of inputs and outputs, but the wrapped function just prints garbage. Any idea how to fix this?
#![allow(non_snake_case)]
#![allow(unused_variables)]
extern crate mex_sys;
use mex_sys::mxArray;
use std::ffi::CString;
use std::os::raw::c_int;
use std::os::raw::c_void;
type VarArgs = *mut c_void;
// attempt to wrap mex_sys::mexPrintf
fn mexPrintf(fmt: &str, args: VarArgs) {
let cs = CString::new(fmt).unwrap();
unsafe {
mex_sys::mexPrintf(cs.as_ptr(), args);
}
}
#[no_mangle]
pub extern "system" fn mexFunction(
nlhs: c_int,
plhs: *mut *mut mxArray,
nrhs: c_int,
prhs: *mut *mut mxArray,
) {
let hw = CString::new("hello world\n").unwrap();
unsafe {
mex_sys::mexPrintf(hw.as_ptr());
}
let inout = CString::new("%d inputs and %d outputs\n").unwrap();
unsafe {
mex_sys::mexPrintf(inout.as_ptr(), nrhs, nlhs);
}
mexPrintf("hello world wrapped\n", std::ptr::null_mut());
let n = Box::new(nrhs);
let p = Box::into_raw(n);
mexPrintf("inputs %d\n", p as VarArgs);
let mut v = vec![3];
mexPrintf("vec %d\n", v.as_mut_ptr() as VarArgs);
}
Contrary to popular belief, it is possible to call variadic / vararg functions that were defined in C. That doesn't mean that doing so is very easy, and it's definitely even easier to do something bad because there are even fewer types for the compiler to check your work with.
Here's an example of calling printf. I've hard-coded just about everything:
extern crate libc;
fn my_thing() {
unsafe {
libc::printf(b"Hello, %s (%d)\0".as_ptr() as *const i8, b"world\0".as_ptr(), 42i32);
}
}
fn main() {
my_thing()
}
Note that I have to very explicitly make sure my format string and arguments are all the right types and the strings are NUL-terminated.
Normally, you'll use tools like CString:
extern crate libc;
use std::ffi::CString;
fn my_thing(name: &str, number: i32) {
let fmt = CString::new("Hello, %s (%d)").expect("Invalid format string");
let name = CString::new(name).expect("Invalid name");
unsafe {
libc::printf(fmt.as_ptr(), name.as_ptr(), number);
}
}
fn main() {
my_thing("world", 42)
}
The Rust compiler test suite also has an example of calling a variadic function.
A word of warning specifically for printf-like functions: C compiler-writers realized that people screw up this particular type of variadic function call all the time. To help combat that, they've encoded special logic that parses the format string and attempts to check the argument types against the types the format string expect. The Rust compiler will not check your C-style format strings for you!
I had confused a "variable list of arguments" with a va_list. I'm going to avoid both if I can and in this situation, I'm just going to do the string formatting in Rust before passing it to interop. Here is what worked for me in this case:
#![allow(non_snake_case)]
#![allow(unused_variables)]
extern crate mex_sys;
use mex_sys::mxArray;
use std::ffi::CString;
use std::os::raw::c_int;
// attempt to wrap mex_sys::mexPrintf
fn mexPrintf(text: &str) {
let cs = CString::new(text).expect("Invalid text");
unsafe {
mex_sys::mexPrintf(cs.as_ptr());
}
}
#[no_mangle]
pub extern "C" fn mexFunction(
nlhs: c_int,
plhs: *mut *mut mxArray,
nrhs: c_int,
prhs: *mut *mut mxArray,
) {
mexPrintf(&format!("{} inputs and {} outputs\n", nrhs, nlhs));
}
I'm trying to write some Rust with #![no_std] set. I am trying to iterate over a str one character at a time; however none of the usual techniques seem to work. When I try to access the characters via a function provided by str, e.g. for c in s.char_indices() or for c in s.chars() I receive the error:
type &str does not implement any method in scope named ___
My understanding was that str was a part of core and therefore any traits it implements should be available in no_std. Is there a way to access this functionality, or an alternate way to iterate over a str?
You need to import the trait to be able to call their methods, e.g.
#![no_std]
extern crate core;
use core::str::StrExt;
fn foo(s: &str) {
for c in s.char_indices() {}
}
core also provides an alternative prelude, which includes the functionality in std::prelude that is available in core. You can import it like use core::prelude::*;.
Is core::str::StrExt automatically available though? I don't have the setup to compile something with ![no_std], but I got past a few hoops in the Playpen by adding
#![no_std]
#![feature(lang_items)]
extern crate core;
use core::str::StrExt;
fn main() {
let s = "Hello";
let mut cnt = 0u8;
for c in s.chars() {
cnt += 1;
}
}
#[lang = "stack_exhausted"] extern fn stack_exhausted() {}
#[lang = "eh_personality"] extern fn eh_personality() {}
#[lang = "panic_fmt"] fn panic_fmt() -> ! { loop {} }