I am developing a P2P application, and I need to use STUN and/or TURN for NAT traversal. I have looked into the issues that an arise when using STUN only (basically it will not always work because it is UDP based and some firewalls won't like that - the reason is not that interesting as per my question), and I keep seeing recommendations about using STUN and TURN for fallback (the ICE scheme).
But everywhere I look I just see people STUN as "unlikely to always work". What I am looking for is some concrete numbers / stats. I might try to generate them myself but I don't have enough clients for a significant sample.
So I was wondering if anyone could shed some light on stats around the success rate of using STUN for NAT traversal. How many peers would not be able to connect if I fail to use TURN as fallback?
Depending on who your customers are, where they are, and the types of devices they use (PCs vs Mobile), the results can vary.
In practice (based on my experience), ICE connectivity with STUN alone is about 85% successful for desktop and laptop PCs. But if it works once for a particular pair of endpoints, it will be even more likely for subsequent connections for these same hosts (assuming network topologies have not changed). Things are a bit different for mobile devices.
Here are some factors that influence getting a successful "connection" (either UDP based or TCP) for P2P.
NAT type. If both endpoints are behind well behaved "port restricted" NATs or better, then chances are high for success with STUN. This is the usual case for home NATs with a good ISP such as those in the US. But mobile carriers and enterprise firewalls typically implement "symmetric NAT" as a result of having multiple layers of NATs and network configurations. This basically means that port mappings are not consistent - and are harder for a P2P algorithm like ICE to establish a connection with.
Firewalls or Enterprise configurations. Even if the firewall allows outbound UDP packets and accepts packets back, it is often a symmetric NAT configuration.
Mobile carriers. Are often (but not always) symmetric NAT types.
Related
These are many articles on internet, researches on the point that their is need to make IoT communication more secure. What are the difference in IoT communication and conventional communication, that there arise need of so much extra research, emergence of new communication protocols etc.. I may be missing some crucial point here.
IoT devices are cheap, small and have limited processing power. Therefore, their software typically doesn't contain the security features of desktop operating systems (implementing an SSL protocol is just not possible on many devices, because they don't have enough capacity). Despite that, IoT devices such as smartwatches transmit highly sensitive information, such as the whereabouts of its wearer and things like his night-time activities...
Additionally, many cheap IoT devices come with one preinstalled, unchangeable software. Or software that never gets any updates after the product is launched. This makes it easy for hackers to abuse these devices once a security leak is found in the code.
I am still trying to understand DDS and its concepts.
I have a configuration where 2 laptops run dds based application. My environment does not permit multicast so I decided to go for peer to peer connection(unicast). To bring both the laptops in the same network, I connected them using ethernet cable (Not sure if it was necessary or not).
Now I did not change anything in the QoS i.e. i did not do any settings for unicasting. But now my applications are communicating with each other.
Question :
How are the participants being discovered ? Multicasting ? as I did not do any settings for unicasting.
Was it necessary to bring them under one network i.e. connect with ethernet cable if I wanted to use unicasting ?
EDIT :
Configuration is as follows :
First laptop : Windows OS : Native DDS based application : Publisher : Multicast not allowed.
Second Laptop : Linux : ROS2 based subscriber : Multicast no problem
Out of the box, DDS is required to support Multicast and Unicast Discovery. Anonymous connections are handled through multicast. If you know the IP address of the recipient, you can manually configure those addresses into the unicast discovery list (each vendor will have their own way to name/process this list).
"Multicast is not permitted on our network", in most cases, means that your IT department has turned off multicast packet forwarding at the switch (or the switches) that define the fabric that is your network.
The as-shipped, standard-compliant DDS configuration, however, has no knowledge of this local policy (how could it?). If you haven't changed the configuration in line with your local policies, the DDS Participants are still going to try to connect via Multicast, because you haven't turned it off.
If the DDS-using machines are connected to the same hub, or to an unmanaged switch (defined here as one that your IT department doesn't care about, or is misconfigured), and the network topology does not cross a managed switch, and they are using the default configuration, and they find each other, then they are using Multicast anonymous discovery.
Figure out how to configure your DDS implementation, to add the unicast ip addresses of the machines that need to communicate. Because discovery is usually only needed in one direction (if A discovers B, then it is true that B has discovered A, assuming neither A nor B are configured to ignore the other[1]).
Once you have configured for unicast discovery, you can configure for no-multicast. If the machines are on IP hopping networks (WiFi, etc) it will be difficult unless the DDS implementation understands multipathing. Talk to the vendor to see if this is the case.
[1] DDS is nothing if not overly configurable.
How are the participants being discovered ? Multicasting ? as I did not do any settings for unicasting.
It is not possible for me to answer this question with complete certainty since you are using DDS as part of the ROS2 framework and I am not familiar with the exact details of how the two are set up to interact together. Having said that, from your description it does seem that the participants are indeed using multicast to discover each other.
The best way to get a conclusive answer is by sniffing the network -- assuming that you have the required privileges to do so. For example you can use Wireshark , which comes with an RTPS dissector that allows you to filter on RTPS messages. (RTPS is the name of the standardized DDS wire protocol.) Check out the destination address and see if you detect any addresses in the multicast range. You can do this while firing up a single DDS-based application. It will start announcing itself over the network immediately.
Was it necessary to bring them under one network i.e. connect with ethernet cable if I wanted to use unicasting ?
If you want to use unicasting, you will need to know IP addresses or host names of all peer nodes. As long as those peer nodes can reach each other over UDP, you are good to go. Often, but not always, ping will let you know whether this is the case. Firewalls are a typical cause of problems.
However, be aware that different types of network have their own specific properties that you might have to adjust your configuration to. Over WiFi for example, the likelihood of packets being dropped (especially with bursts of data) is much larger than when connecting nodes directly with a wire. DDS allows for tuning its protocol to deal with that.
is it possible to bond (aggregate) multiple connections over GPRS (usb stick) and use it as one link in linux?
It is technically possible. Linux has a module called bonding which can assemble several interfaces into one logical interface. If you set the mode of the bonding interface to balance-rr, it will distribute the packets between the two interfaces. You will also need a server somewhere to reassemble the traffic that will come from your two links.
However, in practice the results with such setups are awful, especially with high latency and high jitter links like GPRS. The main reason is that you get a lot of out of order delivery and protocols like TCP become crazy in these conditions. So the resulting throughput will never reach the total throughput of the two links.
Recently i had a case where i was trying to establish a p2p connection using Microsoft PNRP technology between two applications. One application was on Lan and another was on same Lan (diff computer but same Service provider) but was behind a WiFi router. Since, I registered the two peers in all clouds(Global & local links) on respective system but when i tried to resolve the another i could not find the respective peers. As far as i know those peers must be discoverable since i also registered them in global cloud (Internet). How can i achieve the aforesaid scenario ?
Using PNRP in this way depends on a couple of technologies, the most important of which is Teredo tunnelling. You've probably run into a restriction of Teredo tunnelling and how it works behind firewalls.
To summarise it, Teredo routes IPv6 traffic over UDP packets sent to a specific port with IPv4. Because of this only certain kinds of NAT are supported for direct connections. You'll probably find that each of your systems can resolve themselves and other services, but not each other within the firewall if they're on different networks.
The easiest way for you to resolve this will be to either make the computers connect to completely different networks, or have them on the same network (as PNRP also supports link-local discovery).
More information can be found on Wikipedia.
I'm looking to a design a protocol for a client-server application and need some links to some resources that may help me.
The big part is I'm trying to create my own "packet" format so I can minimize the amount of information being sent. I'm looking for some resources to dissect their protocol, but it seems some completely lack packet design, such as SMTP (which just sends strings terminated by CLRF). What are the advantages/disadvantages of using a system like SMTP over a system that uses a custom made packet? Couldn't SMTP use only a couple bytes to cover all commands through bit flags and save bandwidth/space?
Just trying to get my head around all this.
True, but SMTP wasn't particularly optimized for space, nor is it a packet-based protocol. It sits atop TCP, and uses the stream functionality of TCP. You need to decide what is desirable in your protocol: is it performance sensitive? latency? bandwidth?
Is it going to need to run as superuser? If not, you'll probably want to use UDP or TCP.
Are you going to need guarantees on delivery? If so, TCP is probably your best option, unless you are dealing with fairly extreme performance or size issues.
Few protocols these days design individual packets, though many do send very specific data structures across the wire using TCP, or, less commonly, UDP.
If you want to really optimize for space or bandwidth, consider condensing your data as much as possible into individual bits and byte, and defining and packing structures to send it across TCP. Modern network adapters are so optimized for TCP anyway, that there is often little advantage to other strategies.
First of all, are you about to implement an enhanced transport protocol (like RTP on top of UDP) or an application protocol (like HTTP/SMTP)?
There are several things you should think about in both cases concerning your design of the protocol or the demands of your application:
Stream based or packet based,
unidirectional / bi-directional,
stateful and sessionful or stateles,
reliable or best effort,
timing demands,
flow/congestion control,
secure or plain.
Towards an application layer protocol, you should also think about:
Textual or binary data, mapping of application data to network data units/packets, security demands and integrity, etc.
SMTP, HTTP and other TCP based protocols do not concern themselves with packet design because they are stream based. So it makes more sense to talk about protocol design.
As for why use text based protocols vs binary protocols...
Readability of the protocol by packet sniffing programs like Wireshark is very useful.
Also it is often very useful to be able to simply telnet into your port and be able to communicate with the server by specifying plain text.
Also with a protocol like HTTP the actual resource is usually the payload of the communication, the resource can be in binary or any other specified format. So having just the headers and status in plain text is not a bad thing.
TCP is a stream based protocol, not packet based.
Using text with lines makes ad hoc debugging a lot easier
Using text with lines makes it possible to exercise your protocol with telnet