I am building a multithreaded async HTTP server on top of mio.
How should I handle events on client TcpStream connections?
For is_readable it's pretty straightforward: I read the request and write the response.
For other kinds I am not sure. Should I shutdown() the stream if I get is_error or is_hup? What about is_none()?
All things that you mention have very precise meaning and map directly to POSIX/BSD Socket API. It's up to you to decide.
is_hup on Read mean the other side hanged-up it's sending side. Meaning it won't send you anything again. However it might have kept the reading open, and you might still want to send some data to it.
shutdown closes Reading/Writing/Both https://doc.rust-lang.org/std/net/enum.Shutdown.html , so it's up to you what and when you want to do.
TcpStream internally holds FileDesc and that will close the fd when you drop it, so if you don't shutdown manually everything will be closed anyway, as soon as you remove given TcpStream from usage. https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/fd.rs#L217
Related
I'm using https://github.com/websockets/ws
I've a silly question.
Can I do something like this to free resources quickly:
conn.send("otpreceived");
conn.close();
At the moment I'm doing like this:
conn.send("otpreceived");
setTimeout(function ()
{
conn.close();
}, 2000);
I'm not sure if I can close the websocket immediately after send message
Short Answer: Yes you can do this.
Long Answer:
Im going to assume you want to close it immediately for performance gains which is likely negligible. In socket programming, you want to open the socket and keep it open until you are finshed using it.
As you may already know, performing a close() on the connection will require you to open() that connection again if you ever want to send more messages or receieve a response from the other end. Now to answer your question, if you just want to send "otpreceived" without sending anything else or receiving, then yes close it. Otherwise just leave it open until the end of the program.
Now Ive only dealt with socket programming in c but I will say the following...
Does it save resources? Yes. Opening and closing sockets is a kernel function, so it accesses low memory components which is abstracted for you. You may not notice a difference in performance depending on many things like what it is your program is doing and how many concurrent connections exist at a given time. So with that said its usually best to keep your connection open until the very end.
I know there are already a lot similar questions in stackoverflow, but nothing seems convincing. Basically trying to understand under what circumstances I need to use one over the other or use both.
Also would like to understand if close() & shutdown() with shut_rdwr are the same.
Closing TCP connections has gathered so much confusion that we can rightfully say either this aspect of TCP has been poorly designed, or is lacking somewhere in documentation.
Short answer
To do it the proper way, you should use all 3: shutdown(SHUT_WR), shutdown(SHUT_RD) and close(), in this order. No, shutdown(SHUT_RDWR) and close() are not the same. Read their documentation carefully and questions on SO and articles about it, you need to read more of them for an overview.
Longer answer
The first thing to clarify is what you aim for, when closing a connection. Presumably you use TCP for a higher lever protocol (request-response, steady stream of data etc.). Once you decide to "close" (terminate) connection, all you had to send/receive, you sent and received (otherwise you would not decide to terminate) - so what more do you want? I'm trying to outline what you may want at the time of termination:
to know that all data sent in either direction reached the peer
if there are any errors (in transmitting the data in process of being sent when you decided to terminate, as well as after that, and in doing the termination itself - which also requires data being sent/received), the application is informed
optionally, some applications want to be non-blocking up to and including the termination
Unfortunately TCP doesn't make these features easily available, and the user needs to understand what's under the hood and how the system calls interact with what's under the hood. A key sentence is in the recv manpage:
When a stream socket peer has performed an orderly shutdown, the
return value will be 0 (the traditional "end-of-file" return).
What the manpage means here is, orderly shutdown is done by one end (A) choosing to call shutdown(SHUT_WR), which causes a FIN packet to be sent to the peer (B), and this packet takes the form of a 0 return code from recv inside B. (Note: the FIN packet, being an implementation aspect, is not mentioned by the manpage). The "EOF" as the manpage calls it, means there will be no more transmission from A to B, but application B can, and should continue to send what it was in the process of sending, and even send some more, potentially (A is still receiving). When that sending is done (shortly), B should itself call shutdown(SHUT_WR) to close the other half of the duplex. Now app A receives EOF and all transmission has ceased. The two apps are OK to call shutdown(SHUT_RD) to close their sockets for reading and then close() to free system resources associated with the socket (TODO I haven't found clear documentation taht says the 2 calls to shutdown(SHUT_RD) are sending the ACKs in the termination sequence FIN --> ACK, FIN --> ACK, but this seems logical).
Onwards to our aims, for (1) and (2) basically the application must somehow wait for the shutdown sequence to happen, and observe its outcome. Notice how if we follow the small protocol above, it is clear to both apps that the termination initiator (A) has sent everything to B. This is because B received EOF (and EOF is received only after everything else). A also received EOF, which is issued in reply to its own EOF, so A knows B received everything (there is a caveat here - the termination protocol must have a convention of who initiates the termination - so not both peers do so at once). However, the reverse is not true. After B calls shutdown(SHUT_WR), there is nothing coming back app-level, to tell B that A received all data sent, plus the FIN (the A->B transmission had ceased!). Correct me if I'm wrong, but I believe at this stage B is in state "LAST_ACK" and when the final ACK arrives (step #4 of the 4-way handshake), concludes the close but the application is not informed unless it had set SO_LINGER with a long-enough timeout. SO_LINGER "ON" instructs the shutdown call to block (be performed in the forground) hence the shutdown call itself will do the waiting.
In conclusion what I recommend is to configure SO_LINGER ON with a long timeout, which causes it to block and hence return any errors. What is not entirely clear is whether it is shutdown(SHUT_WR) or shutdown(SHUT_RD) which blocks in expectation of the LAST_ACK, but that is of less importance as we need to call both.
Blocking on shutdown is problematic for requirement #3 above where e.g. you have a single-threaded design that serves all connections. Using SO_LINGER may block all connections on the termination of one of them. I see 3 routes to address the problem:
shutdown with LINGER, from a different thread. This will of course complicate a design
linger in background and either
2A. "Promote" FIN and FIN2 to app-level messages which you can read and hence wait for. This basically moves the problem that TCP was meant to solve, one level higher, which I consider hack-ish, also because the ensuing shutdown calls may still end in a limbo.
2B. Try to find a lower-level facility such as SIOCOUTQ ioctl described here that queries number of unACKed bytes in the network stack. The caveats are many, this is Linux specific and we are not sure if it aplies to FIN ACKs (to know whether closing is fully done), plus you'd need to poll taht periodically, which is complicated. Overall I'm leaning towards option 1.
I tried to write a comprehensive summary of the issue, corrections/additions welcome.
TCP sockets are bidirectional - you send and receive over the one socket. close() stops communication in both directions. shutdown() provides another parameter that allows you to specify which direction you might want to stop using.
Another difference (between close() and shutdown(rw)) is that close() will keep the socket open if another process is using it, while shutdown() shuts down the socket irrespective of other processes.
shutdown() is often used by clients to provide framing - to indicate the end of their request, e.g. an echo service might buffer up what it receives until the client shutdown()s their send side, which tells the server that the client has finished, and the server then replies; the client can receive the reply because it has only shutdown() writing, not reading, through its socket.
Close will close both send and receving end of socket.If you want only sending part of socket should be close not receving part or vice versa you can use shutdown.
close()------->will close both sending and receiving end.
shutdown()------->only want to close sending or receiving.
argument:SHUT_RD(shutdown reading end (receiving end))
SHUT_WR(shutdown writing end(sending end))
SHUT_RDWR(shutdown both)
There is a lot of mixed information out there.
I just want to ensure that the data is fully sent. Should I be doing shutdown/close or close with SO_LINGER and timeout ?
I'm using non-blocking sockets with epoll under Linux and the same code (with defines) under OSX uses kqueue. It would seem SO_LINGER doesn't always work the same on all platforms?
Also when using SO_LINGER on a non-blocking socket, If you get back EWOULDBLOCK do you need to call close again until you don't get EWOULDBLOCK? or can I just ignore the EWOULDBLOCK error from close in that case?
I just want to ensure that the data is fully sent. Should I be doing shutdown/close
Yes, see below.
or close with SO_LINGER and timeout?
I don't find this at all useful, see below.
I'm using non-blocking sockets with epoll under Linux and the same code (with defines) under OSX uses kqueue. It would seem SO_LINGER doesn't always work the same on all platforms?
It should but who knows? There is a large paper on TCP implementation differences, and SO_LINGER isn't mentioned in it as far as I remember. But I'll check later when I can.
Also when using SO_LINGER on a non-blocking socket, If you get back EWOULDBLOCK do you need to call close again until you don't get EWOULDBLOCK? or can I just ignore the EWOULDBLOCK error from close in that case?
Theoretically you should call close() again in obedience to EAGAIN but I'm wondering whether it would actually work. It would make more sense to put the socket into blocking mode for the close, as there is nothing you can select on that will tell you when all the data has been written. But you still have the problem that if the timeout occurs, the socket is closed, the data may be lost, and there is now nothing you can do except log the possibility of a problem.
However back to your problem. The way to assure all data has been written and received before closing is to shutdown the socket for writing at both ends, then issue a read at both ends, which should return end of stream (recv() returns zero), then close the socket. That way you know both peers got to the close at the same time, having already read all the data. If the recv() returns data you have a bug in your application protocol. This should all be carried out in blocking mode in my opinion.
I'm implementing a tcp protocol in Node.
Full source:
https://github.com/roelandmoors/ads.js/blob/master/ads.js
specs:
http://infosys.beckhoff.com/content/1033/tcadsamsspec/html/tcadsamsspec_amstcppackage.htm?id=17754
The problem is that I use this to send a package:
this.tcpClient.write(buf);
If I send multiple commands, then multiple commands are combined into a single tcp packet.
This doesn't work.
There are more questions about this on SO, but they recommend using a delimeter.
But since I can't change the protocol this isn't an option.
Isn't there a simple solution to flush the socket?
socket.setNoDelay() doesn't help.
Edit: I also tried to use the drain event to send the next write, but the event is never called?
Update:
This seems to solve the problem, but is very uggly and I don't now if it always works.
Instead of writing it directly I write to a buffer:
this.writeFILO.push(buf);
Every cycle(?) I'm writing a package to the socket stream:
var sendCycle = function(ads) {
if (ads.writeFILO.length > 0) {
ads.tcpClient.write(ads.writeFILO.shift());
}
setTimeout(function() {
sendCycle(ads);
}, 0);
}
I refer to the socket.write(data, [encoding], [callback]) API:
The optional callback parameter will be executed when the data is finally written out - this may not be immediately.
So, set up a queue (array is fine) which holds messages to be sent.
When the above callback is being called, check the queue and send if needed..
This however does not guarantee what you're looking for, you'll have to test. Unfortunately the docs don't explicitly state when there's an acknowledgement from the remote end point that it actually received that message...
In the end, as you concluded, TCP is a stream.
An interesting idea which just came up to me now, however, if you're FORCED TO use an existing protocol, then open two TCP connections.
When one connection acknowledges (whatever the higher-level protocol is) receiving that message, send the next through the other one... and so forth..
Anyway, nice challenge :)
I was wrong. TCP is a stream and the protocol works like a stream, but I didn't handle it like a stream.
PS: sending seperate messages seemed to work with setImmediate()
I know that this is an old question, and I'm not 100% sure I understand what you are looking for, but there is a way to flush a socket in node. First you need to implement a Transform class.
See here for example: https://nodejs.org/api/stream.html#stream_implementing_a_transform_stream.
Then you can take your stream and pipe it through your transform before piping it into your socket.
I do not own this node module but I have seen an example of this here: https://github.com/yongtang/clamav.js/blob/master/index.js#L8
As a way to familiarize myself with Rust & networking in general I started writing a very basic telnet chat server. Everything appears to be going well, but right now I end up with blocks of unsafe code, and I wanted to know if there was a better way to do things.
I spawn a task to listen for connections similar to this: Example TCP server written in Rust Once a user connects and I get a TcpStream I put it in a Connection struct. A Connection struct uses channels to communicate with two tasks- one for reading from its TcpStream, and one for writing. The reading task blocks on the TcpStream's read() method and sends any input back to the Connection struct. The writer blocks on a Port's recv() method and writes anything it receives to the TcpStream. In this way, the main loop of the program can simply maintain a vector of Connection structs to check for user input and write to them at will. The issue is that with this implementation the TcpStream must be be shared by both the read & write tasks, and the mutable write() method called whilst the mutable read() method is still blocking in the other task. In 0.8 I did this with an Rc and unsafe_borrow_mut's, but I'd like to do it in a better fashion if I can- the objections of Rust's type system in this case may be completely valid, for all I know. Any comments on overall design would be welcome as well. Thanks.
There is currently no idiomatic way to do this because Rust's IO API currently does not support parallel read or writes.
See this issue for more info: https://github.com/mozilla/rust/issues/11165
There will be a redesign of the TcpStream API to allow such things in one or the other way (see issue) and which will then provide an "idiomatic way" to do this.