I have been receiving an odd/unknown message while attempting to communicate with some bittorrent peers. In this particular case I am in the middle of downloading pieces and all of a sudden this new/odd message pops up in front of a piece response.The message is odd because it doesn't appear to follow the protocol, all messages are supposed to look like this
'<length prefix><message ID><payload>'
length prefix is 4 bytes, message id is 1 byte and the payload. I am including a capture to show what I mean, on line 509 of the capture you will
see a request for a piece, on line 510 you will see the beginning of the response.
The first 4 bytes of the response are 00 00 00 00, ie 0 length message (Which is causing me issues), the next 4 bytes are the actual length of the message which is 30. The actual response to the piece request starts on line 513, so I get the piece I was requesting but this new/odd message is messing me up. I'm certain I can find a workaround but I would really like to understand what this means.
Also, I have no idea what the actual message means, and cannot find any information about it anywhere.
Here is the Wireshark capture.
https://1drv.ms/u/s!Agj06pa-wu0tnFqsYn_KnHmVz3x2
Data from packet 510:
0000 00 00 00 00 00 00 00 1e 14 01 64 35 3a 61 64 64 ..........d5:add
0010 65 64 36 3a 63 f2 7a 48 17 f4 37 3a 64 72 6f 70 ed6:c.zH..7:drop
0020 70 65 64 30 3a 65 ped0:e
00 00 00 00 4 bytes keep-alive message
00 00 00 1e message length 30 bytes
14 message type extended message (BEP10)
01 extended message ID = 1 as specified by the previous extension handshake: ut_pex
64 35 3a 61 64 64 65 64 36 3a 63 f2 7a 48 17 f4 37 3a 64 72 6f 70 70 65 64 30 3a 65
d5:added6:c.zH..7:dropped0:e
ut_pex message data (bencoded)
d
5:added
6:c.zH..
7:dropped
0:
e
ut_pex message data (bencoded with added white space)
The first 4 bytes of the response are 00 00 00 00, ie 0 length message (Which is causing me issues)
The bittorrent spec says
Messages of length zero are keepalives, and ignored.
Related
Using tcpdump im trying to sniff some packets. The result is this:
reading from file /tmp/prueba.pcap, link-type LINUX_SLL (Linux cooked v1)
13:35:51.767194 IP6 fdc1:41d:9c3:dbef:a6e9:69f0:59aa:b70a.47193 > fdc1:41d:9c3:dbef:0:ff:fe00:8c00.47193: UDP, length 63
0x0000: 6000 0000 0047 1140 fdc1 041d 09c3 dbef `....G.#........
0x0010: a6e9 69f0 59aa b70a fdc1 041d 09c3 dbef ..i.Y...........
0x0020: 0000 00ff fe00 8c00 b859 b859 0047 d42e .........Y.Y.G..
0x0030: 3f0c 0000 0dc2 50f1 0d7b 2254 696d 6522 ?.....P..{"Time"
0x0040: 3a5b 3136 3632 3033 3933 3531 2c22 225d :[1662039351,""]
0x0050: 2c22 4d6f 6417 0012 320f 00f0 0352 6f6c ,"Mod...2....Rol
0x0060: 6c22 3a5b 3533 302c 2264 c2ba 225d 7d l":[530,"d.."]}
The point is in the line with address 0x0050 we can read "Mod...2". That "Mod" means "Mode" but I don't understand why is not the whole word "Mode". ¿Where is the "e"? I need to read that message perfectly for automate a program reading values from there.
I discarded a puntual problem transmiting the message because every time I sniff a packet that contain that info, the format is exactly the same.
Regards,
There are indications that the packet is not correct in other ways than a missing e. For example, the ether type is 0x09c3 and not 0x86dd (IPv6).
Maybe this code to create a PCAP file can help. Using the raw packet you provided as input the output file is bad.pcap and you could use a tool like Wireshark to examine the packet in more detail, see here
import codecs
from scapy.all import wrpcap, Ether, IP, IPv6, UDP, Raw
data = (
'60 00 00 00 00 47 11 40 fd c1 04 1d 09 c3 db ef '
'a6 e9 69 f0 59 aa b7 0a fd c1 04 1d 09 c3 db ef '
'00 00 00 ff fe 00 8c 00 b8 59 b8 59 00 47 d4 2e '
'3f 0c 00 00 0d c2 50 f1 0d 7b 22 54 69 6d 65 22 '
'3a 5b 31 36 36 32 30 33 39 33 35 31 2c 22 22 5d '
'2c 22 4d 6f 64 17 00 12 32 0f 00 f0 03 52 6f 6c '
'6c 22 3a 5b 35 33 30 2c 22 64 c2 ba 22 5d 7d' )
data_list = data.split( " " )
data_s = codecs.decode(''.join(data_list), 'hex')
packet = Raw(load=data_s)
wrpcap('bad.pcap', [packet])
data = (
'3f 0c 00 00 0d c2 50 f1 0d 7b 22 54 69 6d 65 22 '
'3a 5b 31 36 36 32 30 33 39 33 35 31 2c 22 22 5d '
'2c 22 4d 6f 64 17 00 12 32 0f 00 f0 03 52 6f 6c '
'6c 22 3a 5b 35 33 30 2c 22 64 c2 ba 22 5d 7d' )
data_list = data.split( " " )
data_s = codecs.decode(''.join(data_list), 'hex')
packet = Ether(dst="60:00:00:00:00:47", src="11:40:fd:c1:04:1d") / IPv6(dst="fdc1:41d:9c3:dbef:0:ff:fe00:8c00", src="fdc1:41d:9c3:dbef:a6e9:69f0:59aa:b70a" ) / UDP(sport=47193, dport=47193, len=0x0047 ) / Raw(load=data_s)
wrpcap('better.pcap', [packet])
The hex dump begins with the IPv6 header; the link-layer header is not being dumped, so the Ethertype that would appear in the LINKTYPE_SLL_LINUX link-layer header isn't shown.
So the header is:
6000 0000: version (6), traffic class (0), flow label (0)
0047: payload length (0x47 = 71)
1140: next header (0x11 = 17 = UDP), hop limit (0x40 = 64)
fdc1 041d 09c3 dbef a6e9 69f0 59aa b70a: source IPv6 address (fdc1:41d:9c3:dbef:a6e9:69f0:59aa:b70a)
fdc1 041d 09c3 dbef 0000 00ff fe00 8c00: destination IPv6 address (fdc1:41d:9c3:dbef:0:ff:fe00:8c00)
The next header field is 17, so what follows that is a UDP header:
b859: source port (0x5859 = 47193)
b859: destination port (0x5859 = 47193)
0047: length (0x47 = 71)
d42e: checksum
Those 71 bytes are the UDP header (8 bytes) plus the UDP payload (71 - 8 = 63 bytes).
Port 47193 is a in the "registered" port range; however, it does not appear in the current list of well-known and registered ports.
It does, however, appear, from some web searches, to be the default gateway port for MQTT-SN. MQTT "is a lightweight, publish-subscribe, machine to machine network protocol", and MQTT-SN, according to that page, "is a variation of the main protocol aimed at battery-powered embedded devices on non-TCP/IP networks".
If this is MQTT-SN, then, according to the protocol specification for MQTT-SN, the payload would be:
3f0c: length (0x3f = 63), message type (0c = PUBLISH)
00: flags (0x0000)
000d: TopicId
c250: MsgId
f1 0d7b 2254 696d 6522 3a5b 3136 3632 3033 3933 3531 2c22 225d 2c22 4d6f 6417 0012 320f 00f0 0352 6f6c 6c22 3a5b 3533 302c 2264 c2ba 225d 7d: Data
So that data is:
0xf1 0x0d {"Time":[1662039351,""],"Mod 0x17 0x00 0x12 2 0x0f 0x00 0xf0 0x03 Roll":[530,"d 0xc2 0xba "]}
If the published data is intended to be ASCII text, it appears to have been damaged; if this is from a wireless low-power network, perhaps there was radio interference.
The answer is easy... The content of the pcap packet was compressed with lz4...
I have been trying to read the linux audit logs from go using mdlayher/netlink. I am able to make a connection and set the PID as well to be able to receive logs from the netlink socket over unicast and multicast.
The problem is, when the library tries to parse the messages from netlink, it fails, and not because of the library. I tried to dump the messages that were sent to my connection and this is what I found.
([]uint8) (len=48 cap=4096) {
00000000 1d 00 00 00 28 05 00 00 00 00 00 00 00 00 00 00 |....(...........|
00000010 61 75 64 69 74 28 31 36 31 32 30 33 31 38 36 32 |audit(1612031862|
00000020 2e 36 33 31 3a 32 37 31 30 34 29 3a 20 00 00 00 |.631:27104): ...|
}
This is one of the messages from the log stream. According to the packet structure, The first 16 bytes are part of the netlink message header nlmsghdr.
struct nlmsghdr {
__u32 nlmsg_len; /* Length of message including header */
__u16 nlmsg_type; /* Message content */
__u16 nlmsg_flags; /* Additional flags */
__u32 nlmsg_seq; /* Sequence number */
__u32 nlmsg_pid; /* Sending process port ID */
};
Note, how the nlmsg_len is marked as length of the message including header. If you look at the message dump, the first __u32 is 1d 00 00 00, which in the network byte order is 29. Which means the whole packet should be 29 bytes. But, if you count the bytes, it's 45 + 3 bytes of padding which is how the packets are aligned. The message ends at byte 45 which is (29 + 16) or 29 + length of the message header size.
But, the strange thing is, it only happens on audit log messages and not on audit control message replies. Refer to https://play.golang.com/p/mA7_MJdVSv8 for examples on how the packet structures are parsed.
Is this expected ? Looking at the go stdlib syscall.ParseNetlinkMessage, it seems to obey the header + body constraint. I can't make it out in the userspace-audit code which is responsible for auditd, auditctl and it's family of tools.
Another popular library, slackhq/go-audit seems to not rely on the header length and parse based on the size of the buffer read from the socket.
This diff on the mdlayher/netlink library seems to fix the above issue and can also get the byte dumps of the payload. But this shouldn't be the case.
diff --git a/conn_linux.go b/conn_linux.go
index ef18ef7..561ac69 100644
--- a/conn_linux.go
+++ b/conn_linux.go
## -11,6 +11,8 ## import (
"time"
"unsafe"
+ "github.com/davecgh/go-spew/spew"
+ "github.com/josharian/native"
"golang.org/x/net/bpf"
"golang.org/x/sys/unix"
)
## -194,7 +196,13 ## func (c *conn) Receive() ([]Message, error) {
raw, err := syscall.ParseNetlinkMessage(b[:n])
if err != nil {
- return nil, err
+ spew.Dump(b[:n])
+ bl := native.Endian.Uint32(b[:4]) + syscall.NLMSG_HDRLEN
+ native.Endian.PutUint32(b[:4], bl)
+ raw, err = syscall.ParseNetlinkMessage(b[:n])
+ if err != nil {
+ return nil, err
+ }
}
msgs := make([]Message, 0, len(raw))
code for reproducing the above behaviour
Code to read the audit log: https://play.golang.com/p/Kj4DOl0PKRQ
Code to see how different packets are parsed by golang: https://play.golang.com/p/mA7_MJdVSv8
The audit.h file: include/audit.h
uname -a: Linux linux-dev 4.15.0-135-generic #139-Ubuntu SMP Mon Jan 18 17:38:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Update 1
This above behavior seems to crop up when I try to alter the audit state via the AUDIT_SET message type. If I try to connect to the read-only multicast group AUDIT_NLGRP_READLOG, It doesn't seem to happen. Also, If I close the unicast connection and then try multicast, the issue is back again. Basically, as long as my PID is bound to the socket, this issue comes back up.
Sample dump when only connecting via multicast group
([]uint8) (len=76 cap=4096) {
00000000 49 00 00 00 1d 05 00 00 00 00 00 00 00 00 00 00 |I...............|
00000010 61 75 64 69 74 28 31 36 31 32 31 36 35 31 33 31 |audit(1612165131|
00000020 2e 30 31 36 3a 33 33 34 37 32 29 3a 20 61 72 67 |.016:33472): arg|
00000030 63 3d 32 20 61 30 3d 22 61 75 64 69 74 63 74 6c |c=2 a0="auditctl|
00000040 22 20 61 31 3d 22 2d 73 22 00 00 00 |" a1="-s"...|
}
Notice how the size if 0x49 = 73, the exact packet length.
so I don't know anything personally about this, but I found this question via a search, and one other result to that search was this blog article: https://blog.des.no/2020/08/netlink-auditing-and-counting-bytes/
To summarize, someone else found that behavior too, and it seems to be a bug.
Here's the relevant quote:
Bug #3: The length field on audit data messages does not include the
length of the header.
This is jaw-dropping. It is so fundamentally wrong. It means that
anyone who wants to talk to the audit subsystem using their own code
instead of libaudit will have to add a workaround to the Netlink layer
of their stack to either fix or ignore the error, and apply that
workaround only for certain message types.
How has this gone unnoticed? Well, libaudit doesn’t do much input
validation.
(...)
The odds of these bugs getting fixed is approximately zero, because
existing applications will break in interesting ways if the kernel
starts setting the length field correctly.
I'm trying to read the data off a contactless Mastercard Paypass card.
So I submit a SELECT PPSE command using 2PAY.SYS.DDF01. Here is my command APDU:
00 A4 04 00 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 00
In the APDU Response I get the error "6A82", which is "file not found".
I'm stuck, Can anyone help me to read data from the paypass card?
I am trying to write a parser to extract information from the following FLAC file:
$ hd audio.flac | head -n 6
00000000 66 4c 61 43 00 00 00 22 12 00 12 00 00 00 00 00 |fLaC..."........|
00000010 00 00 0a c4 42 f0 00 78 9f 30 00 00 00 00 00 00 |....B..x.0......|
00000020 00 00 00 00 00 00 00 00 00 00 84 00 02 64 1f 00 |.............d..|
00000030 00 00 47 53 74 72 65 61 6d 65 72 20 65 6e 63 6f |..GStreamer enco|
00000040 64 65 64 20 76 6f 72 62 69 73 63 6f 6d 6d 65 6e |ded vorbiscommen|
00000050 74 10 00 00 00 12 00 00 00 54 49 54 4c 45 3d 52 |t........TITLE=R|
Now, according to the specification, the format should be as follow (numbers are in bits):
<32> "fLaC", the FLAC stream marker in ASCII
<16> The minimum block size (in samples) used in the stream.
<16> The maximum block size (in samples) used in the stream.
<24> The minimum frame size (in bytes) used in the stream.
<24> The maximum frame size (in bytes) used in the stream.
<20> Sample rate in Hz.
<3> (number of channels)-1. FLAC supports from 1 to 8 channels
<5> (bits per sample)-1. FLAC supports from 4 to 32 bits per sample.
<36> Total samples in stream.
<128> MD5 signature of the unencoded audio data.
So, I start to write my parser and, while testing, get very strange results. So I test with a "real" metadata extractor:
$ metaflac --list audio.flac
METADATA block #0
type: 0 (STREAMINFO)
is last: false
length: 34
minimum blocksize: 4608 samples
maximum blocksize: 4608 samples
minimum framesize: 0 bytes
maximum framesize: 0 bytes
sample_rate: 44100 Hz
channels: 2
bits-per-sample: 16
total samples: 7905072
MD5 signature: 00000000000000000000000000000000
From the numbers, I can deduce the following:
66 4c 61 43 00 00 00 22 12 00 12 00 00 00 00 00
~~~~~~~~~~~ ~~~~~~~~~~~ ~~~~~ ~~~~~ ~~~~~~~~ ~~
^ ^ ^ ^ ^ ^
| | | | | |
| | | | | + Etc.
| | | | + Minimum frame size
| | | + Maximum block size
| | + Minimum block size
| + What is that ?!?
+ FLAC stream marker
Where does those 32 bits come from? I see they represent the length of the header, but isn't it against the standard to put it there (Taking into account that we already know the length: (32+16+16+24+20+3+5+36+128)/8)?
The 0x22 (34) is indeed the header block size in bytes as part of the METADATA_BLOCK_HEADER which follows the fLaC marker in the stream. Of the first 8 bits (00), bit 7 indicates that there are more metadatablocks to follow, the next 7 bits indicate that it's a STREAMINFO block. The following 3 bytes (00 00 22) is the length of the contents of the block;
16 + 16 + 24 + 24 + 20 + 3 + 5 + 36 + 128 = 272 bits
272 bits / 8 = 34 (0x22) bytes.
I've been using RFC 1035.4.1.3 as a reference for DNS RR format:
http://www.freesoft.org/CIE/RFC/1035/42.htm
The RFC says that RDLENGTH is "an unsigned 16 bit integer that specifies the length in octets of the RDATA field" but in the datagrams I'm getting RDLENGTH is sometimes 2 less than it should be. I've checked with wireshark to ensure that I'm getting the datagram correctly. Here's a CNAME record I got while looking up google:
C0 0C 00 05 00 01 00 03 95 FC 00 10 03 77 77 77
01 6C 06 67 6F 6F 67 6C 65 03 63 6F 6D 00
So that's the name: C0 0C (a pointer to www.google.com earlier in the dgram)
Then the type: 00 05 (CNAME)
Then the class: 00 01 (IN)
Then the TTL: 00 03 95 FC (whatever)
Then RDLENGTH: 00 10 (that's 16 bytes, yes?)
Then RDATA:
03 77 77 77 01 6C 06 67 6F 6F 67 6C 65 03 63 6F 6D 00 (www.l.google.com - format is correct)
As you can see, the RDATA is 18 bytes in length. 18 bytes is 0x12, not 0x10.
The type A records that come after that correctly report RDLENGTH 4 for the address data. Am I missing something here? I'd dismiss it as an error, but I get this for every DNS servers and every domain.
I guess really what I'm asking is why the RDATA is longer than RDLENGTH and what rules should I follow to adapt to it so I can parse any type of record. (Specifically, can I expect this kind of thing from other RR types?)
Thank you in advance to anyone who gives advice. :)
The response data appears to be messed up - either RDLENGTH should be 18 (0x00 0x12), or RDATA should be different.
I just ran a few google lookups from here, and I do not see that problem.
I get RDLENGHT of 7 and RDATA to match (a compressed name).
Is something messing with your packet data?