I am attempting to create my own Torrent client using the bit torrent protocol. I am currently at the stage where I can get a peer list from a tracker and get the IP and port of a peer.
The problem begins after I send the handshake message to the peer. I've followed the instructions and sent a handshake message of 68 bytes with the hash, a client id etc.
I get a response from the peer that is 370 bytes rather than the expected 68 byte handshake. When I parse the first 68 bytes of the response it resembles a handshake response. It has the same hash, a client id etc. However I don't have a clue what the remaining 302 bytes are. Is this normal?
Is there any documentation that clearly explains this? I am unable to find any information to help with this situation. The bit torrent protocol spec is vague to say the least.
This is the response I receive converted to string
\u0013BitTorrent protocol\0\0\0\0\0\u0010\0\u00051O???6H*\u00118$??U\r*?:?d-UT354S-X?\u001b\u0001???rwQ\t?\0\0\0R\u0005?????????????????????????????????????\u07ff??????????????????????????????????????????\0\0\0\u0005\u0004\0\0\u0002\"\0\0\0\u0005\u0004\0\0\0T\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\05\0\0\0\u0005\u0004\0\0\0\t\0\0\0\u0005\u0004\0\0\u0001Z\0\0\0\u0005\u0004\0\0\u00025\0\0\0\u0005\u0004\0\0\u0001*\0\0\0\u0005\u0004\0\0\0y\0\0\0\u0005\u0004\0\0\u0001I\0\0\0\u0005\u0004\0\0\u0002G\0\0\0\u0005\u0004\0\0\u0001?\0\0\0\u0005\u0004\0\0\0\u0001\0\0\0\u0005\u0004\0\0\u0002\f\0\0\0\u0005\u0004\0\0\u0001?\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\u00011\0\0\0\u0005\u0004\0\0\u0001W\0\0\0\u0005\u0004\0\0\0\u0016\0\0\0\u0005\u0004\0\0\0?\0\0\0\u0005\u0004\0\0\u0001?
The first 68 bytes are what I expected, the remaining 302 bytes I have no idea about
I've now realised what these extra bytes are.
They are a bitfield and multiple have messages! My issues was not realising this information can be sent immediately after the handshake
Related
I am assuming that after finding peers using that Infohash you send handshake messages until one peer establishes a connection. But I can't seem to find any packet in Wireshark that has the metadata in it. If possible can you show me how a metadata query looks like?
The only thing I've found about metadata was bep 9 but it was kinda hard for me to understand and I didn't find anything in Wireshark similar
You already found the relevant BEP 9. To implement it you first have to implement BEP 10 and then check the extension header whether the peer supports BEP 9 and if it does send a series of extension messages with the appropriate negotiated IDs carrying a metadata request message.
You should be seeing the extension handshake message in most bittorrent connections since many clients support it. You're less likely to see metadata request/data messages (unless you explicitly trigger them in a client) because it's simply a less common operation than downloading the torrent payload.
In the socket level, when the connection is built, the server will continue read data from the socket, my question is how the server will knows the client won’t send any data?
how the server will knows the client won’t send any data
It doesn't. One option is to scan for pre-agreed-upon "end of message" byte sequence. When server sees this byte sequence, it considers message fully received. If there's more data in the buffer, it belongs to another message.
Or the client must advertise its message length ahead of time. "I'm going to send you X bytes now. Here they are: ..." The server then reads only X bytes from the socket and considers message fully received.
Take a look at redis protocol for an example of the second scheme. It's very simple and fully functional. It is so simple, in fact, that a full client can be implemented in only 20 lines of ruby.
I been trying to make a Handshake request to the peer and although the I successfully getting the handshake response back.
For some reason data after the handshake response look weird.
Let me explain
Here is the handshake request
\x13BitTorrent protocol\x00\x00\x00\x00\x00\x00\x00\x00\x01G{\x96\er\xDB\x9Cd\x14O\xFCj\r/\x8D\xA7\xAE\xAE\xF166788486352714431784
and I'm getting a handshake response that look like this
\x13BitTorrent protocol\x00\x00\x00\x00\x00\x10\x00\x00\x01G{\x96\er\xDB\x9Cd\x14O\xFCj\r/\x8D\xA7\xAE\xAE\xF1-TR2840-e5rk4p002er6
Now here what making be baffled I read the next 200 bytes from socket
and I see this
\x00\x00\x00-\x14\x00d1:md6:ut_pexi1ee6:yourip4:o}\xD1)4:reqqi255ee\x00\x00\x00\x96\x14\x00d1:ei1e4:ipv616:$\x00a\x80\x01\x00\x00\xD0\x00\x00\x00\x00\x02\xA40\x011:md11:ut_metadatai3e6:ut_pexi1ee13:metadata_sizei166e1:pi51413e4:reqqi512e11:upload_onlyi0e1:v17:Transmission 2.
Assuming \x00\x00\x00- is the length of the (according to message structure of Bittorrent)
and "\x14" is the message_id = 20
But looking at the bittorrent protocol specification I fail to find a message with ID 20
I'm kind of stuck here and don't know where to proceed any help will highly appreciate.
The remote peer is showing non-compliant behavior. It is sending a extended handshake even though you have not signaled the capability in the reserved bitfield.
-TR2840-
This indicates that it is transmission 2.84. If it also occurs with current transmission versions I would report a bug to them. If it only occurs with old versions I would not bother handling this situation and just drop the connection.
I'm currently studying for a project involving a web-server and some raspberrys.
The challenge would basically be to watch raspberry "status" on a web interface.
Raspberrys are connected to the Internet through a GSM connection (mostly 3G).
I'm developping using Node.js on both the clients and the server, and I'd like to use websockets through socket.io in order to watch the raspberry connection status (actually, this is more like watching the raspberry capability to upload data through my application), dealing with "connected" and "disconnected" events.
Would an always-alive websocket connection be reliable for such a use-case?
Are websocket designed-to (or reliable-for) staying opened?
Since this is a hard-testable situation, does anyone know a data-consumption estimate for an always-alive websocket?
If I'm going in a wrong way, does anyone ever worked on such a use-case via another reliable way?
Would an always-alive WebSocket connection be reliable for such a use-case ? Are WebSocket designed-to (or reliable-for) staying opened ?
Yes, WebSocket was designed to stay open and yes it's reliable for your use-case, a WebSocket connection is just a TCP connection which transmits data in frames.
Since this is a hard-testable situation, does anyone know an data-consumption estimate for an always-alive websocket ?
As I wrote, data in WebSocket connections is transmitted using frames, each frame has a header and the payload. The data sent from the client to the server is always masked and like this adds 4 bytes (the masking key) to each frame. The length of the header depends on the payload length:
2 bytes header for <=125 bytes payload
4 bytes header for <=65535 bytes payload
10 bytes header for <=2^64-1 bytes payload (rarely used)
Base Framing Protocol: https://www.rfc-editor.org/rfc/rfc6455#section-5.2
To keep the connection open, the server sends at a specific timeout (depends on the implementation, usually ~30 seconds) ping frames which are 2-127 bytes long, usually 2 bytes (only the header, without payload) and the client responds with pong frames which are also 2-127 bytes long.
I'm developing implement Bittorrent without using any library. I have tried to send a handshake message to a peer. I think this handshake would be symmetric by both side. But the peers sends a little weird handshake to me. For example, when I send the following handshake message to a peer,
\x13BitTorrent protocol\0\0\0\0\0\0\0\0<sha1 info hash><20byte of my peer-id>
I expect the handshake message like
\x13BitTorrent protocol\0\0\0\0\0\0\0\0<sha1 info hash><20byte of peer-id>
But I get
\x13BitTorrent protocol\0\0\0\0\0\0\0\0<sha1 info hash><20byte of peer-id><long stream of many bytes>
What does the long stream mean? any specs have never explained this.
Also, whenever I receive a handshake message, that stream size is always different.
If you are the initiator and has already sent your handshake message, the other peer answer with a handshake and can then immediately start to send normal bittorrent messages. That is the long stream you see.