As piece index is provided for pieces of file transferred using BitTorrent protocol, is there something similar to identify the transferred piece while using µTP. While capturing the network packets using Wireshark, it shows no messages similar to the peer messages of BitTorrent protocol.
Just from looking on their bugtracker it seems like treating µTP as a transport protocol isn't implemented:
https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=8792
Related
I've been reading on packets a lot today. I was confused for sometime because smtp, http, or ftp, for example, are all called protocols. But that they also somehow utilize transport protocols like TCP. I couldn't locate them on the packet 4 layers. Until I just discovered they're simply part of the application layer.
I want to know what exactly these "protocols" offer. I'm guessing a specific format for the data which applications on the client side know how to handle? If so, does this mean that realistically, I might have to create my own "protocols" if I created an application with a unique functionality?
A protocol, in this case, is just a structured way of communicating between two or multiple parties.
If you write, for example, a PHP-App and offer an API, you created a protocol to interact with your program. It defines how others interact with it and what response they can expect while doing so. Your self-created protocol depends on others, like the HTTP and TCP.
I suggest watching following video of LiveOverflow, explaining exactly this:
https://www.youtube.com/watch?v=d-zn-wv4Di8&ab_channel=LiveOverflow
I want to know what exactly these "protocols" offer.
You can read the definition of each protocol, if you really want to
I am assuming that after finding peers using that Infohash you send handshake messages until one peer establishes a connection. But I can't seem to find any packet in Wireshark that has the metadata in it. If possible can you show me how a metadata query looks like?
The only thing I've found about metadata was bep 9 but it was kinda hard for me to understand and I didn't find anything in Wireshark similar
You already found the relevant BEP 9. To implement it you first have to implement BEP 10 and then check the extension header whether the peer supports BEP 9 and if it does send a series of extension messages with the appropriate negotiated IDs carrying a metadata request message.
You should be seeing the extension handshake message in most bittorrent connections since many clients support it. You're less likely to see metadata request/data messages (unless you explicitly trigger them in a client) because it's simply a less common operation than downloading the torrent payload.
So WebRTC uses UDP and it works great if you are doing some video streaming, if you lose few frames that's ok but i wonder how that works when sending files like pictures.
The main problem being that UDP does not verify file integrity like TCP does and just by missing a packet you could end up with a corrupted file.
So how can you send pictures reliable between browsers and ensure that the files are whole?
you can use the datachannel to transfer files. They provide an abstraction providing reliable transfer. See https://webrtc.github.io/samples/src/content/datachannel/filetransfer/ for a sample.
I am trying to build a basic conference call system based on plain RTP.
_____
RTP IN #1 ______ | | _______ MIX RTP receiver #1
|______| MIX |_____|
______| | RTP | |_______ MIX RTP receiver #2
RTP IN #2 |_____|
I am creating RTP streams on Android via the AudioStream class and using a server written in Node.js to receive them.
The naive approach I've been using is that the server receives the UDP packets and forwards them to the participants of the conversation. This works perfectly as long as there are two participants, and it's basically the same as if the two were sending their RTP stream to each other.
I would like this to work with multiple participants, but forwarding the RDP packets as they arrive to the server doesn't seem to work, probably for obvious reasons. With more than two participants, the result of delivering the packets coming in from different sources to each of the participants (excluding the sender of such packet) results in a completely broken audio.
Without changing the topology of the network (star rather than mesh) I presume that the server will need to take care of carrying out some operations on the packets in order to extract a unique output RTP stream containing the mixed input RTP streams.
I'm just not sure how to go about doing this.
In your case I know two options:
MCU or Multipoint Control Unit
Or RTP simulcast
MCU Control Unit
This is middle box (network element) that gets several RTP streams and generate one or more RTP streams.
You can implement it by yourself but it is not trivial because you need to deal with:
Stream decoding (and therefore you need jitter buffer and codecs implementation)
Stream mixing - so you need some synchronisation between streams (collect some data from source 1 and source 2, mix them and send to destination 3)
Also there are several project that can do it for you (like Asterisk, FreeSWITCH etc), you can try to write some integration level with them. I haven't heard anything about something on Node.js
Simulcast
This is pretty new technology and their specifications available only in IETF drafts. Core idea here is to send several RTP streams inside one RTP stream simultaneously.
When destination receives several RTP streams it needs to do exactly the same as MCU does - decode all streams and mix them together but in this case destination may use hardware audio mixer to do that.
Main cons for this approach is bandwidth to the client device. If you have N participants you need:
either send all N streams to all other
or select streams based on some metadata like voice activity or audio level
First one is not efficient, second is very tricky.
The options suggested by Dimtry's answer were not feasible in my case because:
The middle box solution is difficult to implement, requires too many resources or requires to rely on an external piece of software, which I didn't want to have to rely on, especially because Android RTP stack should work out of the box with basic support from a server component, especially for hole punching
The simulcast solution cannot be used because the Android RTP package cannot handle that and ad far as my understanding goes it's only capable of handling simple RTP streams
Other options I've been evaluating:
SIP
Android supports it but it's more of a high level feature and I wanted to build the solution into my own custom application, without relying on additional abstractions introduced by a high level protocol such as SIP. Also, this felt just too complex to set up, and conferencing doesn't even seem to be a core feature but rather an extension
WebRTC
This is supposed to be the de-facto standard for peer 2 peer voice and video conferencing but looking through code examples it just looks too difficult to set up. Also requires support from servers for hole punching.
Our solution
Even though I had, and still have, little experience on this I thought there must be a way to make it work using plain RTP and some support from a simple server component.
The server component is necessary for hole punching, otherwise getting the clients to talk to each other is really tricky.
So what we ended up doing for conference calling is have the caller act as the mixer and the server component as the middle-man to deliver RTP packets to the participants.
In practice:
whenever a N-user call is started, we instantiate N-1 simple UDP broadcast servers, listening on N-1 different ports
We send those N-1 ports to the initiator of the call via a signaling mechanism built on socket.io and 1 port to each of the remaining participants
The server component listening on those ports will simply act as a relay: whenever it receives a UDP packet containing the RTP data it will forward it to all the connected clients (the sockets it has seen thus far) except the sender
The initiator of the call will receive and send data to the other participants, mixing it via the Android AudioGroup class
The participants will send data only to the initiator of the call, and they will receive the mixed audio (together with the caller's own voice and the other participants' voices) on the server port that has been assigned to them
This allows for a very simple implementation, both on the client and on the server side, with minimal signaling work required. It's certainly not a bullet proof conferencing solution, but given the simplicity and feature completeness (especially regarding common network issues like NAT traversal, which using a server aid is basically a non-issue) is in my opinion better than writing lots of code which requires many resources for mixing server-side, relying on external software like SIP servers, or using protocols like WebRTC which basically achieve the same with lots more effort implementation wise.
I'm writing a application in NodeJs where a client sends udp messages to a server with udp. I'm trying to find out how people normally handle different message types in NodeJs but can only find tons of examples of echo servers where the kind of message is not relevant.
The only example I have found so far is https://github.com/vbo/node-webkit-mp-game-template/tree/networking_1/networking
Maybe the best way is to send the udp messages as json?
User Datagram Protocol (UDP) is a network protocol and mechanism for sending short messages from one host to another without any guarantee of delivery. What you put in the message is entirely up to you.
While JSON can be used to encode your message, it suffers from two problems: it is not secure and is self-describing.
The first problem means that bad actors can easily see the content of your message while in flight and the second implies a substantial overhead for any message above and beyond its intended purpose.
Depending on your needs, a better choice might be to define your own binary protocol specific to your purpose using a node Buffer.
Another might be to use a more compact interchange format like thrift.