What is valid traceId in Brave? As I understand 16 and 32 length values are valid:
https://github.com/openzipkin/brave/blob/master/brave/README.md#128-bit-trace-ids
If service receives 15 lengths (or less) traceId in the header. Is it valid for Brave? or leading 0 will be added in standard implementation?
In other words, can I implement custom Propagation to pass 15 lengths (or less) traceId without leading 0 bit as per Brave specification?
Brave behaves as if it the ID has the upper hex digits as zeros:
https://github.com/openzipkin/brave/blob/5be287b91c3d18da9fc7a597d71b12b27be6043c/brave/src/main/java/brave/propagation/TraceContext.java#L440
Calls:
https://github.com/openzipkin/brave/blob/5be287b91c3d18da9fc7a597d71b12b27be6043c/brave/src/main/java/brave/internal/codec/HexCodec.java#L53...L78
Related
I've been referring to RFC 4034 (https://www.rfc-editor.org/rfc/rfc4034#page-4), but there is something I quite don't understand about the flags field of a RDATA for a DNSKEY RR.
As stated in the RFC 2.1.1. The Flags Field, "Bits 0-6 and 8-14 are reserved: these bits MUST have value 0 upon creation of the DNSKEY RR and MUST be ignored upon receipt.".
But few lines under in 2.2. The DNSKEY RR Presentation Format, it states "The Flag field MUST be represented as an unsigned decimal integer. Given the currently defined flags, the possible values are: 0, 256, and 257."
How can bits 8-14 be required to have value of 0 if using 256 and 257 as the section value ?
Could someone explain to me if there is something I don't get please ?
I am currently building my own DNS server and I need to validate the format of a DNSKEY RR as well as craft packets on demand for them.
Thanks !
I am currently building my own DNS server
I hope it is mostly for learning, because otherwise this is a tricky matter!
You can (should) also study existing DNS libraries to understand implementation details.
As for the values, let us go back to the fields representation:
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|0| => 256 (ZSK case)
|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1| => 1 (SEP case)
|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|1| => 257 (KSK case)
Bits 8-14 are always 0, as are the 0-6 ones.
SEP alone can't happen, so value 1 is not possible, 256 and 257 are the usual one for DNSSEC, and 0 (which means bit 7 is 0 so a key but "not DNSSEC case").
Remember that most (all?) of IETF protocols are MSB, left-most bit is most significant, so bit 15 is 2⁰, bit 14 is 2¹, etc. from right to left up to bit 0 at left which encodes 2¹⁵.
You can find that repeated in RFC 1035 §2.3.2:
Whenever an octet represents a numeric quantity, the left most bit in
the diagram is the high order or most significant bit.
[..]
Similarly, whenever a multi-octet field represents a numeric quantity the left most bit of the whole field is the most significant bit. When a multi-octet quantity is transmitted the most significant octet is transmitted first.
I am using slugId which is a node.js module for converting from UUID to base64 URL friendly text and vice-versa. (see: https://github.com/taskcluster/slugid) As one of our QAs was executing tests he found the following which I am unable to explain:
The slugs: aOSL2RT_Rhy-xNuoe3j7ag and aOSL2RT_Rhy-xNuoe3j7ah generate the same UUID: d2369f6c-1eea-4518-a641-33d6c2dc0493.
This is also applicable to more slugs. Example:
0jafbB7qRRimQTPWwtwEkw, 0jafbB7qRRimQTPWwtwEkx. (Both of them translate to UUID: d2369f6c-1eea-4518-a641-33d6c2dc0493)
The decode and decode functions of slugId look sound but I am unable to explain the above behaviour.
A "slugId" is 22 characters. Each character is base64, i.e. representing 6 bits, which means they have a total of 22×6=132 bits. However, UUIDs have only 128 bits; the last 4 bits of the slugId are discarded in the conversion, so there are 16 slugId values that map to each UUID value.
This means you need to sanitize all slugId values on input, e.g. by rejecting any value with one (or more) of those last 4 bits set. Presumably you are already validating them in other ways (e.g. too long, too short, invalid chars, etc.) so this is just one more minor test to be added to the list.
I am using MongoDB's build in id fields to label products and for ease of usage/typability, I would like to compress the _id field down from a hexadecimal string that looks like 5b69c35ac2cc78c8979a8a9b to something shorter and involving all letters of the alphabet (both uppercase and lowercase) and numbers. preferably it would involve no more than 10 or 12 characters. Are there any common methods of accomplishing this in Node.JS/MongoDB?
You could convert them to base64, that would make them 16 characters long.
Example:
Buffer.from('5b69c35ac2cc78c8979a8a9b', 'hex').toString('base64') // W2nDWsLMeMiXmoqb
It's better if you can directly access the Buffer - converting many ObjectIds from string could be costly.
The code 5b69c35ac2cc78c8979a8a9b is 24 bytes long (in hex), which means the absolute minimum number of bytes needed to represent this value without losing information is 12, ranging from 0-255 which is not what we want.
If we take a look at the ObjectId we could (maybe) eliminate some bytes:
a 4-byte value representing the seconds since the Unix epoch,
a 3-byte machine identifier,
a 2-byte process id, and
a 3-byte counter, starting with a random value.
Removing machine identifier and process id (if all id's are generated by the same process) would leave us with 7 bytes (0-255), which is still not ideal to encode in base64 or even base32.
So it would probably be better to just use a 32 bit unsigned integer for the product codes and display it as hex using 8 bytes (the leading zeros could be removed).
Encoding those 4 bytes in base64 wouldn't help much (every 3 bytes become 4 bytes), and personally I would prefer case insensitive id's for use in url's which would leave us only with base32.
For better ease of usage/typability than hexadecimal, those 4 bytes could be encoded in z-base-32 and would fit in 7 bytes without padding (7 * 5 bits = 35 bits).
Formatting a CoAP packet
[RFC 7252 CoAP][1]
In RFC 7252 Section 3, Figure 7 the third row, bytes 9 ... 16 or maybe more, is the Options field. I am unable to find anything that specifies how long the options field is. I understand that it can change, but unlike the Token field who's length is specified by field TKL, I cannot recognize where the length of the Options is specified.
Yes, I see sections 3.1 and 3.2 but am not able to understand what they are telling me. The document states to reference the previous options. OK, what do you do for the first message where there is no previous packet and no previous option?
When my code needs to send a CoAP message, how do I determine what options can be sent? What values must be loaded into the packet to send, for example, no options?
If you see Figure 8 in sec 3.1 on the RFC, bits 4-7 denote the length of the option value.
0 1 2 3 4 5 6 7
+---------------+---------------+
| Option Delta | Option Length | 1 byte
+---------------+---------------+
Bits 0-3 will tell you which option it is. This nibble only gives you the delta compared to the previous option encoded in this message. For the first option in the message, there is no previous option so the bits 0-3 give you the Option number.
Lets consider an example where you need to encode 2 options Uri-Port with value 7000 and Uri-Path with value /temp in a CoAP message. Options are always encoded in increasing order of the Option numbers. So you first encode Uri-Port which has Option number 7 and then Uri-Path with Option number 11.
Uri-Port
As this is the first option in the message, the Option delta will be same as the Option number so Option delta = 0x7. Port value 7000 will take 2 bytes (0x1B58) so Option length = 0x2. So this Option will be encoded get encoded as 72 1b 58.
Uri-Path
This is not the first Option in this message. Option delta for this option will be this option number - prev option number i.e. 11 - 7 = 4. Encoding temp will take 4 bytes so Option length = 4. So this option would get encoded as 44 74 65 6d 70
Note that this was for a simplified case where the Option number and length are not more than 12 bytes. When either of these is more than 12 bytes, you encode using the extended option delta/length as specified in the RFC.
I have a string of decimal digits like:
965854242113548732659745896523654789653244879653245794444524
length : 60 character
I want to send it to a function, but first I want reduce the length of it as much as possible. How can I do that?
I think about convert it to base-34, that will be 1RG7EEWTN7NW60EWIWMASEWWMEOSWC2SS8482WQE. That is 40 characters in length. Can I reduce it more some way?
Your number fits into 70 bits - for such a small payload compression seems nonsensical. Assuming that the server API supports arbitrary binary data, I would simply encode the value in binary and prefix it with the number of bytes needed.
1 byte length information - for 854657986453156789675, the example you gave initially, this would be 9
9 bytes of binary payload
→ 10 bytes of data transferred for your example.
Your example in hex:
09 2e 54 c3 1e 81 cf 05 fd ab
With the length given in bytes, this of course supports only decimals up to 255 bytes length, but I suppose this is sufficient. If your transport protocol has a built in concept of length of a packet, you could even skip the initial length byte.
Important: ensure that all sides use the same endianness. As you are transmitting your data over the network, network byte order (big endian) would be natural.
If you want to transmit very large numbers, keep in mind that you can use any compression algorithm you like on the binary representation of your data. However, your payload must be significantly larger in order to make compression feasible - for example, using zLib compression for the above 9 byte payload results in an 18 byte payload due to the overhead for the zLib datastructures.
If (and only if) you cannot use arbitrary bytes for your payload, you can encode your data (possibly after compression). Most modern libraries have built in support for Base64, so this would be a natual way of representing the data.