I recognized that tokio::io::BufReader has a buffer() method for accessing its internal read buffer. However I cannot find such an interface for tokio::io::BufStream, nor can I access the internal BufReader of a BufStream by using the public API.
I wonder if this is by design, or if there is some other way to do it?
BTW, the following is my use case:
I want to implement a traffic dispatcher for a socket with the trait AsyncRead + AsyncWrite. The dispatcher will try to peek the first read of the underlying socket to determine where the traffic will be routed, and then return a BufStream that also supports AsyncRead + AsyncWrite, as if the original socket is untouched (no data consumed).
My plan is to first trigger a fill_buf() call to the BufStream, then to look at the internal read buffer (which I don't how to do).
I don't think this is an inherent limitation: BufStream is just BufReader<BufWriter<RW>>. You can do it yourself if you need and then you can access the buffers, but there is no need to: fill_buf() returns the filled buffer, so you can just use its returned value.
Related
Currently, we are switching the API gateway engine made in c to tokio, hyper, rustls of rust.
While analyzing the echo server example provided by hyper_rustls (tokio_rustls), there is a part that I do not understand and ask for help.
(I'm struggling with not many examples to refer to)
https://github.com/rustls/hyper-rustls/blob/main/examples/server.rs
Here's the flow I was thinking of:
When a POST request with a large body size is received, all http bodies are read → After reading up to content-length, the future code passed to make_service_fn is executed (echo operation in the example).
However, as soon as the request is received, the code passed to make_service_fn is executed and a response is sent to the client, and the poll_read function of the tokio::io::AsyncRead trait is executed many times. This has been run.
Q: When exactly does the make_fn_service code run, and is this something I can control?
Q: When using hyper, it seems to store the body accumulating in memory. Therefore, if the body size is very large, I would like to work with such as downloading it to a separate file. Is there a way to directly control every time the body comes?
Can I use the hyper::body::HttpBody trait?
About Q1:
let service = make_service_fn(|_| async { Ok::<_, io::Error>(service_fn(echo)) });
You call make_fn_service to convert an async function into something that can be passed to serve(). It gets an argument of type &AddrStream and can do all kind of fancy stuff such as filtering and throttling, but if you don't need any of these, just call service_fn with your async function.
Then your function, echo in the example, will be called once per client request.
About Q2:
The body is not accumulated in memory:
*response.body_mut() = req.into_body();
But these are of type Body, that implements Stream<Item=Result<Bytes>>, being these Bytes the chunks of the body request/response.
And by asigning a Stream to another Stream a very big ping should be painlessly streamed through the echo function, one chunk at a time.
If you want to manage the data yourself you can poll the stream (StreamExt::next()) and handle each piece of body individually. Just do not call Body::to_bytes() or Body::aggregate().
About using the HttpBody trait:
Sure you can use it directly, but it is non trivial. I think it is usually implemented so that you can get for example a JSON object directly from the request, or an XML or a urlencoded map or whatever your Content-Type dictates, without doing an intermediate byte array and a parse.
But as you can probably guess, processing huge XML/JSON payloads in async mode is not easy. And if you really need that you can probably make it easier just driving the byte chunks of a plain Body.
I have a struct which needs to be Send + Sync:
struct Core {
machines_by_id: DashMap<String, StateMachineManager>,
}
In my current implementation, StateMachineManager looks like this:
struct StateMachineManager {
protected: Arc<Mutex<StateMachines>>,
}
This works fine as long as StateMachines is Send. However, it doesn't need to be, and it complicates the implementation where I'd like to use Rc.
Performance wise, there's no reason all the StateMachines can't live on one thread forever, so in theory there's no reason they need to be Send - they could be created on a thread dedicated to them and live there until no longer needed.
I know I can do this with channels, but that would seemingly mean recreating the API of StateMachines as messages sent back and forth over that channel. How might I avoid doing that, and tell Rust that all I need to do is serialize access to the thread they all live on?
Here is a minimal example (where I have added the Send + Sync bounds to Shepmaster's comment which omitted them) -- DashMap is a threadsafe map.
What I ended up doing here was using channels, but I was able to find a way to avoid needing to recreate the API of StateMachines.
The technique is to use channels to pass a closure to a dedicated thread the StateMachines instances live on which accepts a &mut StateMachines argument, and sends the response back down a different channel, which lives on the stack while the access is happening.
Here is a playground implementing the key part
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e07972b928d0f0b7680b1e5a988dae84
The details of instantiating the machines on their dedicated thread are elided.
Just to get better understanding of the Send and Sync traits, are there examples of types that either:
Implement Send and do not implement Sync.
Implement Sync and do not implement Send.
First of all, it is important to realize that most structs (or enums) are Send:
any struct that does not contain any reference can be Send + 'static
any struct that contain references with a lower-bound lifetime of 'a can be Send + 'a
As a result, you would generally expect any Sync struct to be Send too, because Send is such an easy bar to reach (compared to the much harder bar of being Sync which requires safe concurrent modification from multiple threads).
However, nothing prevents the creator of a type to specifically mark it as not Send. For example, let's resuscitate conditions!
The idea of conditions, in Lisp, is that you setup a handler for a given condition (say: FileNotFound) and then when deep in the stack this condition is met then your handler is called.
How would you implement this in Rust?
Well, to preserve threads independence, you would use thread-local storage for the condition handlers (see std::thread_local!). Each condition would be a stack of condition handlers, with either only the top one invoked or an iterative process starting from the top one but reaching down until one succeeds.
But then, how would you set them?
Personally, I'd use RAII! I would bind the condition handler in the thread-local stack and register it in the frame (for example, using an intrusive doubly-linked list as the stack).
This way, when I am done, the condition handler automatically un-registers itself.
Of course, the system has to account for users doing unexpected things (like storing the condition handlers in the heap and not dropping them in the order they were created), and this is why we use a doubly-linked list, so that the handler can un-register itself from the middle of the stack if necessary.
So we have a:
struct ConditionHandler<T> {
handler: T,
prev: Option<*mut ConditionHandler<T>>,
next: Option<*mut ConditionHandler<T>>,
}
and the "real" handler is passed by the user as T.
Would this handler be Sync?
Possibly, depends how you create it but there is no reason you could not create a handler so that a reference to it could not be shared between multiple threads.
Note: those threads could not access its prev/next data members, which are private, and need not be Sync.
Would this handler be Send?
Unless specific care is taken, no.
The prev and next fields are not protected against concurrent accesses, and even worse if the handler were to be dropped while another thread had obtained a reference to it (for example, another handler trying to un-register itself) then this now dangling reference would cause Undefined Behavior.
Note: the latter issue means that just switching Option<*mut Handler<T>> for AtomicPtr<ConditionHandler<T>> is not sufficient; see Common Pitfalls in Writing Lock-Free Algorithms for more details.
And there you have it: a ConditionHandler<T> is Sync if T is Sync but will never be Send (as is).
For completeness, many types implement Send but not Sync (most Send types, actually): Option or Vec for example.
Cell and RefCell implement Send but not Sync because they can be safely sent between threads but not shared between them.
I have a driver, which handles several TCP connections.
Is there a way to perform something similar to user space application api's select/poll()/epoll() in kernel given a list of struct sock's?
Thanks
You may want to write your own custom sk_buff handler, which calls the kernel_select() that tries to lock the semaphore and does a blocking wait when the socket is open.
Not sure if you have already gone through this link Simulate effect of select() and poll() in kernel socket programming
On the kernel side it's easy to avoid using sys_epoll() interface outright. After all, you've got a direct access to kernel objects, no need to jump through hoops.
Each file object, sockets included, "overrides" a poll method in its file_operations "vtable". You can simply loop around all your sockets, calling ->poll() on each of them and yielding periodically or when there's no data available.
If the sockets are fairly high traffic, you won't need anything more than this.
A note on the API:
poll() method requires a poll_table() argument, however if you do not intend to wait on it, it can safely be initialized to null:
poll_table pt;
init_poll_funcptr(&pt, NULL);
...
// struct socket *sk;
...
unsigned event_mask = sk->ops->poll(sk->file, sk, &pt);
If you do want to wait, just play around with the callback set into poll_table by init_poll_funcptr().
To my current understanding, after calling MPI_Send, the calling thread should block until the variable is received, so my code below shouldn't work. However, I tried sending several variables in a row and receiving them gradually while doing operations on them and this still worked... See below. Can someone clarify step by step what is going on here?
matlab code: (because I am using a matlab mex wrapper for MPI functions)
%send
if mpirank==0
%arguments to MPI_Send_variable are (variable, destination, tag)
MPI_Send_variable(x,0,'A_22')%thread 0 should block here!
MPI_Send_variable(y,0,'A_12')
MPI_Send_variable(z,1,'A_11')
MPI_Send_variable(w,1,'A_21')
end
%recieve
if mpirank==0
%arguments to MPI_Recv_variable are (source, tag)
a=MPI_Recv_variable(0,'A_12')*MPI_Recv_variable(0,'A_22');
end
if mpirank==1
c=MPI_Recv_variable(0,'A_21')*MPI_Recv_variable(0,'A_22');
end
MPI_SEND is a blocking call only in the sense that it blocks until it is safe for the user to use the buffer provided to it. The important text to read here is in Section 3.4:
The send call described in Section 3.2.1 uses the standard communication mode. In this mode, it is up to MPI to decide whether outgoing messages will be buffered. MPI may buffer outgoing messages. In such a case, the send call may complete before a matching receive is invoked. On the other hand, buffer space may be unavailable, or MPI may choose not to buffer outgoing messages, for performance reasons. In this case, the send call will not complete until a matching receive has been posted, and the data has been moved to the receiver.
I highlighted the part that you're running up against in bold there. If your message is sufficiently small (and there are sufficiently few of them), MPI will copy your send buffers to an internal buffer and keep track of things internally until the message has been received remotely. There's no guarantee that when MPI_SEND is done, the message has been received.
On the other hand, if you do want to know that the message was actually received, you can use MPI_SSEND. That function will synchronize (hence the extra S both sides before allowing them to return from the MPI_SSEND and the matching receive call on the other end.
In a correct MPI program, you cannot do a blocking send to yourself without first posting a nonblocking receive. So a correct version of your program would look something like this:
Irecv(..., &req1);
Irecv(..., &req2);
Send(... to self ...);
Send(.... to self ...);
Wait(&req1, ...);
/* do work */
Wait(&req2, ...);
/* do more work */
Your code is technically incorrect, but the reason it is working correctly is because the MPI implementation is using internal buffers to buffer your send data before it is transmitted to the receiver (or matched to the later receive operation in the case of self sends). An MPI implementation is not required to have such buffers (generally called "eager buffers"), but most implementations do.
Since the data you are sending is small, the eager buffers are generally sufficient to buffer them temporarily. If you send large enough data, the MPI implementation will not have enough eager buffer space and your program will deadlock. Try sending, for example, 10 MB instead of a double in your program to notice the deadlock.
I assume that there is just a MPI_Send() behind MPI_Send_variable() and MPI_Receive() behind MPI_Receive_variable().
How do a process can ever receive a message that he sent to himself if both the send and receive operations are blocking ? Either send to self or receive to self are non-blocking or you will get a deadlock, and sending to self is forbidden.
Following answer of #Greginozemtsev Is the behavior of MPI communication of a rank with itself well-defined? , the MPI standard states that send to self and receive to self are allowed. I guess it implies that it's non blocking in this particular case.
In MPI 3.0, in section 3.2.4 Blocking Receive here, page 59, the words have not changed since MPI 1.1 :
Source = destination is allowed, that is, a process can send a message to itself.
(However, it is unsafe to do so with the blocking send
and receive operations described above, since this may lead to deadlock.
See Section 3.5.)
I rode section 3.5, but it's not clear enough for me...
I guess that the parenthesis are here to tell us that talking to oneself is not a good practice, at least for MPI communications !