Formatting a CoAP packet
[RFC 7252 CoAP][1]
In RFC 7252 Section 3, Figure 7 the third row, bytes 9 ... 16 or maybe more, is the Options field. I am unable to find anything that specifies how long the options field is. I understand that it can change, but unlike the Token field who's length is specified by field TKL, I cannot recognize where the length of the Options is specified.
Yes, I see sections 3.1 and 3.2 but am not able to understand what they are telling me. The document states to reference the previous options. OK, what do you do for the first message where there is no previous packet and no previous option?
When my code needs to send a CoAP message, how do I determine what options can be sent? What values must be loaded into the packet to send, for example, no options?
If you see Figure 8 in sec 3.1 on the RFC, bits 4-7 denote the length of the option value.
0 1 2 3 4 5 6 7
+---------------+---------------+
| Option Delta | Option Length | 1 byte
+---------------+---------------+
Bits 0-3 will tell you which option it is. This nibble only gives you the delta compared to the previous option encoded in this message. For the first option in the message, there is no previous option so the bits 0-3 give you the Option number.
Lets consider an example where you need to encode 2 options Uri-Port with value 7000 and Uri-Path with value /temp in a CoAP message. Options are always encoded in increasing order of the Option numbers. So you first encode Uri-Port which has Option number 7 and then Uri-Path with Option number 11.
Uri-Port
As this is the first option in the message, the Option delta will be same as the Option number so Option delta = 0x7. Port value 7000 will take 2 bytes (0x1B58) so Option length = 0x2. So this Option will be encoded get encoded as 72 1b 58.
Uri-Path
This is not the first Option in this message. Option delta for this option will be this option number - prev option number i.e. 11 - 7 = 4. Encoding temp will take 4 bytes so Option length = 4. So this option would get encoded as 44 74 65 6d 70
Note that this was for a simplified case where the Option number and length are not more than 12 bytes. When either of these is more than 12 bytes, you encode using the extended option delta/length as specified in the RFC.
Related
I've been referring to RFC 4034 (https://www.rfc-editor.org/rfc/rfc4034#page-4), but there is something I quite don't understand about the flags field of a RDATA for a DNSKEY RR.
As stated in the RFC 2.1.1. The Flags Field, "Bits 0-6 and 8-14 are reserved: these bits MUST have value 0 upon creation of the DNSKEY RR and MUST be ignored upon receipt.".
But few lines under in 2.2. The DNSKEY RR Presentation Format, it states "The Flag field MUST be represented as an unsigned decimal integer. Given the currently defined flags, the possible values are: 0, 256, and 257."
How can bits 8-14 be required to have value of 0 if using 256 and 257 as the section value ?
Could someone explain to me if there is something I don't get please ?
I am currently building my own DNS server and I need to validate the format of a DNSKEY RR as well as craft packets on demand for them.
Thanks !
I am currently building my own DNS server
I hope it is mostly for learning, because otherwise this is a tricky matter!
You can (should) also study existing DNS libraries to understand implementation details.
As for the values, let us go back to the fields representation:
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|0| => 256 (ZSK case)
|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1| => 1 (SEP case)
|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|1| => 257 (KSK case)
Bits 8-14 are always 0, as are the 0-6 ones.
SEP alone can't happen, so value 1 is not possible, 256 and 257 are the usual one for DNSSEC, and 0 (which means bit 7 is 0 so a key but "not DNSSEC case").
Remember that most (all?) of IETF protocols are MSB, left-most bit is most significant, so bit 15 is 2⁰, bit 14 is 2¹, etc. from right to left up to bit 0 at left which encodes 2¹⁵.
You can find that repeated in RFC 1035 §2.3.2:
Whenever an octet represents a numeric quantity, the left most bit in
the diagram is the high order or most significant bit.
[..]
Similarly, whenever a multi-octet field represents a numeric quantity the left most bit of the whole field is the most significant bit. When a multi-octet quantity is transmitted the most significant octet is transmitted first.
I'm extracting the first 4 bytes from a pcap packet header, which should represent a quantity of seconds. Here they are, in order of appearance in the ByteStream (I'm using Haskell):
\192 (192)
\166 (166)
x (120)
SOH (01) (Start of Header)
My understanding is that the four bytes can be read as a 32-bit integer. However, the presence of SOH is throwing me off. If I interpret the 4 bytes as one integer, I get 2 billion, which is invalid (2 billion seconds = 63 years => invalid because UNIX times starts in 1970, about 45 years ago).
The packet header also ends with NUL (00).
I'm also not sure why the four bytes are reversed, maybe a side-effect of how I'm pulling bytes from the stream (using a Get function and getInt32). Shouldn't the SOH come first?
Did you check the magic number at the very beginning of the pcap file? Its purpose is 1. identify the file format; 2. allow you to determine the byte order. Here's a handy reference: https://wiki.wireshark.org/Development/LibpcapFileFormat#File_Format
For writing my own ogg-container-class (not using libogg), I try to understand the needed header format. According to the spec, at byte 27 of the stream (starting to count at 0) starts the "segment_table (containing packet lacing values)". This is the red marked byte 13. Concerning the Opus-data that I want to include, the Opus data must start with OpusHead (4F 70 75 73) on its beginning. Why doesn't it start on position 27 where the red 13 is placed? A 13 is a "device control 3" symbol that neither occurs in the Ogg spec, nor in the Opus spec.
EDIT: I found this link that describes the spec a little. There it becomes clear (which it is not from the first link imho) that the 13 (byte 27) is the size of the following segment.
That appears to be a single byte giving the length of the following segment_table data. So there is 13(hex) bytes (16 decimal) bytes of segment_table data.
RFC 3533 is a more verbose description of the format header.
Byte 26 says how many bytes the segment table occupies, so you read that, add 27, and that tells you where the first packet starts (or continues).
The segment table tells you the length(s) of the encapsulated packet(s). Basically you read through the table, adding together the values in each successive byte. If the value you just added is < 255 then that marks a packet boundary, so record the current value of the accumulator, reset it to zero, then continue until you reach the end of the table.
In your example, the segment table size in byte 26 is 1, so the data starts at 27+1 or byte 28, which is the start of the 'OpusHead' string. The value in the 1 byte segment table is 0x13, so the packet is 19 bytes long. 28+19 is 47 (or 0x2f) which is the start of the 'OggS' capture pattern at the start of the next header.
This slightly complicated algorithm is designed to store framing data for many small packets with bounded overhead while still allowing arbitrarily large packets. Note also that packets can be continued between pages, spanning 2 or more segment tables.
As an example, suppose a QR Code data stream contains 55 data words (each one byte in length) and 15 error correction words (again one byte). The data stream begins with a 12 bit header and ends with four 0 bits. So, 12 + 4 bits of header/footer and 15 bytes of error correction, leaves me 53 bytes to hold 53 alphanumeric characters. The 53 bytes of data and 15 bytes of ec are supplied in a string of length 68 (str68). The problem seems simple enough - concatenate 2 bytes of (right-shifted) header data with str68 and then left shift the entire 70 bytes by 4 bits.
This is the first time in many years of programming that I have ever needed to do something like this, I am a c and bit shifting noob, so please be gentle... I have done a little investigation and so far have not been able to figure out how to bitshift 70 bytes of data; any help would be greatly appreciated.
Larger QR codes can hold 2000 bytes of data...
You need to look at this 4 bits at a time.
The first 4 bits you need to worry about are the lower bits of the first byte. Fortunately this is an easy case because they need to end up in the upper bits of the first byte.
The next 4 bits you need to worry about are the upper bits of the second byte. These need to end up as the lower bits of the first byte.
The next 4 bits you need to worry about are the lower bits of the second byte. But fortunately you already know how to do this because you already did it for the first byte.
You continue in this vein until you have dealt with the lower bytes of the 70th byte.
I have a string of decimal digits like:
965854242113548732659745896523654789653244879653245794444524
length : 60 character
I want to send it to a function, but first I want reduce the length of it as much as possible. How can I do that?
I think about convert it to base-34, that will be 1RG7EEWTN7NW60EWIWMASEWWMEOSWC2SS8482WQE. That is 40 characters in length. Can I reduce it more some way?
Your number fits into 70 bits - for such a small payload compression seems nonsensical. Assuming that the server API supports arbitrary binary data, I would simply encode the value in binary and prefix it with the number of bytes needed.
1 byte length information - for 854657986453156789675, the example you gave initially, this would be 9
9 bytes of binary payload
→ 10 bytes of data transferred for your example.
Your example in hex:
09 2e 54 c3 1e 81 cf 05 fd ab
With the length given in bytes, this of course supports only decimals up to 255 bytes length, but I suppose this is sufficient. If your transport protocol has a built in concept of length of a packet, you could even skip the initial length byte.
Important: ensure that all sides use the same endianness. As you are transmitting your data over the network, network byte order (big endian) would be natural.
If you want to transmit very large numbers, keep in mind that you can use any compression algorithm you like on the binary representation of your data. However, your payload must be significantly larger in order to make compression feasible - for example, using zLib compression for the above 9 byte payload results in an 18 byte payload due to the overhead for the zLib datastructures.
If (and only if) you cannot use arbitrary bytes for your payload, you can encode your data (possibly after compression). Most modern libraries have built in support for Base64, so this would be a natual way of representing the data.