I'm looking to write my own multiplayer game server (most likely in nodejs) and I was wondering what protocol I should be using to transfer data? Are Datagrams the norm to send information (i realize they don't confirm delivery like HTTP, that can written on top of the protocol)? Any suggestions of performant proven systems would be a real help.
I guess I'm looking for successful techniques in handling the data transfer quickly and effectively (maintaining state on the server and scaling are a separate issue that I have a solid understanding of).
I'm looking to initially support desktop/mobile games (MacOS, iOS, and Android).
For all your protocol needs take a look at socketIO.
Basically your best solution is to rely on websockets which are TCP sockets. socketIO is just a nice cross-browser compliant abstraction.
Either you use standard long pulling techniques or html5 websockets. There is no access to UDP for browser <-> server.
There is a technology that is called RTMFP that Adobe introduced in the latest version of Flash (Flash 10). It allows you to do P2P connection and transfer data directly from a client to an other client without passing by the server. On top of that, it's using UDP to transfer data. I believe that this was originally designed to do video and audio streaming, but you can use it to pass data around.
However the main downside on this technology is the mobile since most of them don't support Flash. In this case you can use Socket.IO and use the server as a router of information as a fallback.
If you want to build your application in Javascript, you can still use it by bridging the functionnality to Javascript. If you want to take a look at a simple version of a bridge, you can take a look at this github project (I am the author).
As a C++ developer of Massive Multiplayer games for 10 years, I can tell you that most of your more advanced games, such as ones in which I was involved (Legends Of Kesmai, Magic: The Gathering Online, Airwarrior II, AVP, NTN Triva) to name a few, TCP is used for most communication simply because you need an ACK / NACK to be sure the data was received from the client. That is not to say UDP doesn't have it's place. In Legends we wrote the the protocol code to use UDP for out of bandwidth delivery of data which wasn't imperative that it be received in proper packet order and complete. Use UDP when you want to do things like update graphic files in the background while the user is playing, etc. This type of delivery is often used for such purposes and allows your TCP packets to arrive as required by your server.
Related
I am wondering what the benefit of a WebRTC based P2P system would be?
Is the performance improved?
Is the security stronger?
... as opposed to a traditional vanilla P2P solution in, fx, Java.
The benefit of WebRTC/RTCWeb (first is the implementation and w3c API, second is the ietf standard) is in its name: Web. It does not have anything special, it's just an standard inspired by SIP whose implementation is shared by all the main browsers. The benefit is that you don't have to code a client for it because it's already embedded in the browser next to GetUserMedia which enables you to easily capture video and audio streams from the devices.
Performance is determined mostly by the codecs and they use open source codecs you could use in your own implementation. It is also secure, but you can make any p2p connection secure through encryption.
It is just an standard meant to make web developers' lifes easier when dealing with p2p real time voice, video and data streaming.
I need a SIP server, hopefully simpler, more efficient and more secure than Asterisk or other full-featured PBX, to just terminate SIP calls and play audio, based on the incoming number. This would be in a linux environment.
I'm concerned about long-term support, and much-reduced attack surface. I'm a programmer but a no0b in the SIP world. I want to avoid the tons of security hassles of a full-featured PBX, let alone the crazy big bills after a hacker gets into it. Thanks in advance!
Personally I would build one with pjsip. Assuming you do not need built in registration server all calls would be handled by so called local account. You can use command-line pjsua with auto-answer and auto-play options for initial tests and then add some logic for audio wave selection and perhaps audio player allocation/deallocation (audio players could be active all the time and shared by multiple calls for efficiency if they e.g. contain something like background music).
For security disable not needed parts (TCP, TLS transports, not needed codecs?) and use port other than 5060. In my application I'm also using kind of application-level firewall that blocks requests from hosts that generate suspicious traffic (registrations, malformed SIP requests).
I have an Arduino board in a location, and a webserver.
I also have a website that is supposed to control the Arduino. What technique would allow users to take control of the Arduino board with the least amount of latency?
I have Node.js, socketserver, Jabber in mind to experiment with, but is this the right direction?
You should have a look at Socket.IO for implementing WebSockets on server- and client-side.
There's a great project called duino for accessing Arduino with Node.js, you "only" have to pipe all communication through WebSockets.
Update: In the meantime I have published a framework for sending commands to an Arduino with Node.JS, JavaScript and WebSockets. Make sure to have a look at Noduino!
I had good luck using node-serialport to talk to the Arduino. Using the serial port results in very low latency, and I used it to build a photobooth. The code is on GitHub if you want to check it out, though it's very poorly organized as I was rushing to get this done for my wedding and well... corners were cut.
I'm looking to a design a protocol for a client-server application and need some links to some resources that may help me.
The big part is I'm trying to create my own "packet" format so I can minimize the amount of information being sent. I'm looking for some resources to dissect their protocol, but it seems some completely lack packet design, such as SMTP (which just sends strings terminated by CLRF). What are the advantages/disadvantages of using a system like SMTP over a system that uses a custom made packet? Couldn't SMTP use only a couple bytes to cover all commands through bit flags and save bandwidth/space?
Just trying to get my head around all this.
True, but SMTP wasn't particularly optimized for space, nor is it a packet-based protocol. It sits atop TCP, and uses the stream functionality of TCP. You need to decide what is desirable in your protocol: is it performance sensitive? latency? bandwidth?
Is it going to need to run as superuser? If not, you'll probably want to use UDP or TCP.
Are you going to need guarantees on delivery? If so, TCP is probably your best option, unless you are dealing with fairly extreme performance or size issues.
Few protocols these days design individual packets, though many do send very specific data structures across the wire using TCP, or, less commonly, UDP.
If you want to really optimize for space or bandwidth, consider condensing your data as much as possible into individual bits and byte, and defining and packing structures to send it across TCP. Modern network adapters are so optimized for TCP anyway, that there is often little advantage to other strategies.
First of all, are you about to implement an enhanced transport protocol (like RTP on top of UDP) or an application protocol (like HTTP/SMTP)?
There are several things you should think about in both cases concerning your design of the protocol or the demands of your application:
Stream based or packet based,
unidirectional / bi-directional,
stateful and sessionful or stateles,
reliable or best effort,
timing demands,
flow/congestion control,
secure or plain.
Towards an application layer protocol, you should also think about:
Textual or binary data, mapping of application data to network data units/packets, security demands and integrity, etc.
SMTP, HTTP and other TCP based protocols do not concern themselves with packet design because they are stream based. So it makes more sense to talk about protocol design.
As for why use text based protocols vs binary protocols...
Readability of the protocol by packet sniffing programs like Wireshark is very useful.
Also it is often very useful to be able to simply telnet into your port and be able to communicate with the server by specifying plain text.
Also with a protocol like HTTP the actual resource is usually the payload of the communication, the resource can be in binary or any other specified format. So having just the headers and status in plain text is not a bad thing.
TCP is a stream based protocol, not packet based.
Using text with lines makes ad hoc debugging a lot easier
Using text with lines makes it possible to exercise your protocol with telnet
Is it possible to create a real time game with node.js that requires twitch reflexes. How high is the latency? How low can it realistically go?
It is possible to make a real-time game in node.js as you could with any other language/framework.
The problem here would be what kind of server and client you would use.
Using the http server feature for such game would be a bad idea and very difficult, but you could use the TCP server (now called net server) as you would in any other language.
The client would be on some platform where you can use sockets, like Flash, Java applets or desktop software.
Please notice that even with using a TCP socket server you might have problems with latency for a twitch game, but this is outside the area related to this question and more about games and networking.
PS: You could use web sockets since they should theoretically work like TCP sockets but there isn't yet a good support for them in the current modern browsers.
EDIT:
It seems I haven't explained myself correctly, you can make a browser accessible game like you said, you just need to use a protocol that allows you to quickly send data back and forth in real time.
If you want a "pure" browser game without any third party plugins the only way, like I said before, is using JavaScript with websockets which is not well supported yet by the major browsers. (You could use a Flash bridge and still have your game in JavaScript though.)
Using a third party plugin you have Flash and Java (besides the numerous less known plugins like unity and so on). Both have TCP sockets (not sure about UDP) and can be made to connect to a node.js net server (with some security limitations). Most people would say for you to go with Flash since there is a bigger support but Apple doesn't like it so no Flash in iPhone/iPad/iPod Touch or on other miscellaneous mobile devices (that support Java instead).
So yeah... good luck with this.
EDIT 2:
Websocket support in browsers is now pretty decent so I recommend it for realtime games if you want to use the browser as a client.
HTTP servers are typically optimised for throughput/bandwidth over latency. node.js is unlikely to be an exception, and HTTP is intrinsically poor for low latency anyway due to the structure of the protocol.
One informal benchmark using node.js supports this, showing latencies of hundreds of milliseconds. By comparison most twitch games support latencies of no more than 30 or 40ms, ideally less.
Therefore I'd recommend dropping the twitch aspect if you can't drop HTTP.
It's possible, but it depends on how much data must be transmitted between server and client and how fast (speaking of latency). Take a look at Sousaball by Creationix, for example.
Also, if you plan to use websockets, take a look at Socket.IO library by learnboost. It uses websockets when available and falls back to comet in other cases.