Experiencing weird behaviour with Buffer.compare / Buffer.from - node.js

I'm trying to modify buffers, however when modifying them I wish them to be in utf-8 so I attempt to do this via myBuffer.toString('utf8') however if I make no changes and attempt to convert it back via Buffer.from(myBuffer, 'utf8'), I am presented with a completely new buffer on occasions.
These occasions seem to be when parsing an image file, instead of a html file.
My next step was to accept a bug or erroneous behaviour by comparing the two buffers using the following code:
//myBuffer is the buffer is wish to attempt to modify
let testParse = Buffer.from(myBuffer.toString('utf8'), 'utf8');
let editable = Buffer.compare(myBuffer, testParse);
console.log(myBuffer);
console.log(testParse);
console.log(editable);
The following snippet however refuses to work and editable is always -1 here is an example output:
<Buffer 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 01 10 00 00 00 5c 08 02 00 00 00 29 85 7d e1 00 00 15 31 49 44 41 54 78 01 ed 5d 05 94 db c8 b2 ... >
<Buffer ef bf bd 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 01 10 00 00 00 5c 08 02 00 00 00 29 ef bf bd 7d ef bf bd 00 00 15 31 49 44 41 54 78 01 ef ... >
-1
As you can see the buffers are completely different however returns -1
another example where the buffers are both equal:
<Buffer 3c 21 64 6f 63 74 79 70 65 20 68 74 6d 6c 3e 3c 68 74 6d 6c 20 69 74 65 6d 73 63 6f 70 65 3d 22 22 20 69 74 65 6d 74 79 70 65 3d 22 68 74 74 70 3a 2f ... >
<Buffer 3c 21 64 6f 63 74 79 70 65 20 68 74 6d 6c 3e 3c 68 74 6d 6c 20 69 74 65 6d 73 63 6f 70 65 3d 22 22 20 69 74 65 6d 74 79 70 65 3d 22 68 74 74 70 3a 2f ... >
-1
As you can see both buffers are equal and -1 is still returned.
So my question is, what am I doing wrong so that the buffers cannot be compared properly? Any suggestions/criticism are welcome.

You have to compare in the same encoding :
//:Buffer Comparison
const fs = require('fs')
fs.readFile(__dirname+"/test.jpg",(e,buffer)=>{
let testParse = Buffer.from(buffer.toString('utf8'), 'utf8');
let editable = Buffer.compare(buffer, testParse);
console.log("----: wrong method :----")
console.log(buffer);
console.log(testParse);
console.log(editable);
// You have to compare them in the same encoding :
console.log("----: right method :----")
let goodParse = Buffer.from(buffer.toString('utf8'));
let editable2 = goodParse.compare(Buffer.from(buffer.toString('utf8')));
console.log(buffer);
console.log(goodParse);
console.log(editable2);
})
As you can see, we load an image as a buffer, then it is parsed into utf8, so if you modify it, and then want to compare it to the original buffer. Since the modified was parsed to utf8 the original must also be parsed to utf8 in the moment of the comparison.
I hope you understand that explanation.
Console output:
----: wrong method :----
<Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 43 00 08 06 06 07 06 05 08 07 07 07 09 09 08 0a 0c 14 0d 0c 0b 0b 0c 19 12 13 0f ... >
<Buffer ef bf bd ef bf bd ef bf bd ef bf bd 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ef bf bd ef bf bd 00 43 00 08 06 06 07 06 05 08 07 07 07 09 09 08 ... >
1
----: right method :----
<Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 43 00 08 06 06 07 06 05 08 07 07 07 09 09 08 0a 0c 14 0d 0c 0b 0b 0c 19 12 13 0f ... >
<Buffer ef bf bd ef bf bd ef bf bd ef bf bd 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ef bf bd ef bf bd 00 43 00 08 06 06 07 06 05 08 07 07 07 09 09 08 ... >
0

Related

How ImageMagic determines the ColorSpace of a PNG?

Suppose I create a simple PNG with:
convert -size 1x1 canvas:red red.png
Here is a similar image (bigger size) for reference:
Then run the command identify on it. It tells me the ColorSpace of the image is sRGB but there seems to be NO indication of this inside the file. In fact running
$ hexdump -C red.png
00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR|
00000010 00 00 00 01 00 00 00 01 01 03 00 00 00 25 db 56 |.............%.V|
00000020 ca 00 00 00 04 67 41 4d 41 00 00 b1 8f 0b fc 61 |.....gAMA......a|
00000030 05 00 00 00 20 63 48 52 4d 00 00 7a 26 00 00 80 |.... cHRM..z&...|
00000040 84 00 00 fa 00 00 00 80 e8 00 00 75 30 00 00 ea |...........u0...|
00000050 60 00 00 3a 98 00 00 17 70 9c ba 51 3c 00 00 00 |`..:....p..Q<...|
00000060 06 50 4c 54 45 ff 00 00 ff ff ff 41 1d 34 11 00 |.PLTE......A.4..|
00000070 00 00 01 62 4b 47 44 01 ff 02 2d de 00 00 00 07 |...bKGD...-.....|
00000080 74 49 4d 45 07 e5 01 0d 17 04 37 80 ef 04 02 00 |tIME......7.....|
00000090 00 00 0a 49 44 41 54 08 d7 63 60 00 00 00 02 00 |...IDAT..c`.....|
000000a0 01 e2 21 bc 33 00 00 00 25 74 45 58 74 64 61 74 |..!.3...%tEXtdat|
000000b0 65 3a 63 72 65 61 74 65 00 32 30 32 31 2d 30 31 |e:create.2021-01|
000000c0 2d 31 33 54 32 33 3a 30 34 3a 35 35 2b 30 30 3a |-13T23:04:55+00:|
000000d0 30 30 2d af d4 01 00 00 00 25 74 45 58 74 64 61 |00-......%tEXtda|
000000e0 74 65 3a 6d 6f 64 69 66 79 00 32 30 32 31 2d 30 |te:modify.2021-0|
000000f0 31 2d 31 33 54 32 33 3a 30 34 3a 35 35 2b 30 30 |1-13T23:04:55+00|
00000100 3a 30 30 5c f2 6c bd 00 00 00 00 49 45 4e 44 ae |:00\.l.....IEND.|
00000110 42 60 82 |B`.|
00000113
does not provide a clue, that I know of.
I understand that identifying the ColorSpace of an image, that does not contain that information, is a very hard problem -- see one proposed solution looking at the histogram of colors here.
So how identify, from the ImageMagick suite, determines the ColorSpace of this image?
It is common, but not standardized to assume that an image without an embedded or sidecar ICC profile or without an explicit encoding description is encoded according to IEC 61966-2-1:1999, i.e. sRGB specification.
This is just a bug in ImageMagick. You can use exiftool to check whether sRGB + intent chunk is present. In this case, no.
Gamma 2.2 is not sRGB. Thus ImageMagic is wrong here. That is a common problem on Wikipedia, all SVG images when converted to PNG have this and it destroys the colours. See: https://phabricator.wikimedia.org/T26768
We will have to reencode all images on Wikipedia, since we use ImageMagick. Sigh.

why DELETE Key APDU command returns 6A80 Status words?

I'm going to delete an existent key from my card's ISD. To do so I sent a DELETE Key APDU command with corresponding KeyID and KeyVersion to the ISD after a successful Mutual Authentication as below:
--> 00 A4 04 00 08 A0 00 00 01 51 00 00 00
<-- 6F 5B 84 08 A0 00 00 01 51 00 00 00 A5 4F 73 49 06 07 2A 86 48 86 FC 6B 01 60 0B 06 09 2A 86 48 86 FC 6B 02 02 02 63 09 06 07 2A 86 48 86 FC 6B 03 64 0B 06 09 2A 86 48 86 FC 6B 04 02 55 65 0B 06 09 2A 86 48 86 FC 6B 02 01 03 66 0C 06 0A 2B 06 01 04 01 2A 02 6E 01 03 9F 65 01 FF 90 00
--> 80 50 00 00 08 79 71 01 3C 63 9D 72 A3
<-- 00 00 90 30 09 0A 90 72 3D A3 01 02 00 00 60 AD 80 68 C2 A1 79 AE B9 E4 4A 4D B7 99 90 00
--> 84 82 00 00 10 AB E9 10 5B 60 7C DE C6 9C DC 15 E0 DA 9B 81 44
<-- 90 00
--> 80 E4 00 00 06 D0 01 01 D2 01 71
<-- 6A 80
As you see above, I received 6A80 status word which means Wrong Data. I've have tried the same command and data with a different card and it successfully returned 90 00 status words.
So
What is wrong with this card and how I can delete this key?
Is there anyway to list all existent keys on the card? As far as I know, GET DATA APDU command with Tag 66 (Key Information Template) does not return list of all available keys.
Some card are simply not supporting it. As alternative you can rotate the keys to a random value.
The tag for key templates is 00E0. You can use this with GET DATA. E.g. GPShell provides the command get_key_information_templates -keyTemplate index. Use 0 as index. This output returns a more readable list.

How would I "decode" the packet data from a pcap in NodeJS?

I'm wanting to make a PCap Analyzer script where it can detect what traffic is what from a pcap file.
The general idea is: HTTP(x10), DNS(x5), HTTPS(x20)
Now as you can see the majority of traffic is HTTPS based I want to be able to pull that from the pcap packet data to pass to another section of my analyzer script.
I don't have a clue nor any idea of what NPMs or anything that I can use, I have looked into pcap-parser which is a 9+ Yr old NPM this package , and only provides packet.data, packet.header.
I'm just completely losing all hope on making this script as I've tried ever potential resource even went into researching a potential API system to upload the pcap and bring the info I wish to obtain with no avail.
Example of packet.header
{
timestampSeconds: 1606145597,
timestampMicroseconds: 444357,
capturedLength: 60,
originalLength: 60
}
Example of packet.data (Buffer)
<Buffer 01 00 5e 7f ff fa 34 29 8f 99 09 70 08 00 45 00 00 a5 a4 76 00 00 04 11 10 f3 0a c8 06 1d ef ff ff fa ed 0c 07 6c 00 91 17 56 4d 2d 53 45 41 52 43 48 ... 129 more bytes>
<Buffer ff ff ff ff ff ff 34 29 8f 99 09 6e 08 06 00 01 08 00 06 04 00 01 34 29 8f 99 09 6e 0a c8 06 e6 00 00 00 00 00 00 0a c8 06 de 00 00 00 00 00 00 00 00 ... 10 more bytes>
<Buffer e0 55 3d 5e 95 a0 40 ec 99 d3 06 fd 08 00 45 00 05 6b a7 ed 40 00 80 06 00 00 0a 91 a6 ce 34 ef cf 64 e9 9f 01 bb a2 30 72 ed d9 06 6d cc 80 18 02 00 ... 1351 more bytes>
<Buffer 40 ec 99 d3 06 fd e0 55 3d 5e 95 a0 08 00 45 00 00 34 72 2d 40 00 70 06 e2 e3 34 ef cf 64 0a 91 a6 ce 01 bb e9 9f d9 06 6d cc a2 30 14 19 80 10 1b 25 ... 16 more bytes>
<Buffer e0 55 3d 5e 95 a0 40 ec 99 d3 06 fd 08 00 45 00 00 34 05 b4 40 00 80 06 00 00 0a 91 a6 ce 17 d9 8a 6c e9 a8 01 bb f0 0d cc ed 00 00 00 00 80 02 fa f0 ... 16 more bytes>

Socat corrupt image file

I use 'socat TCP4-LISTEN:8080,fork EXEC:./bashttpd' for http server. when try to receive image file from client socat remove some byte and corrupt my image.
correct:
01b0 0a 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 ..PNG........IHD
01c0 52 00 00 07 80 00 00 04 38 08 02 00 00 00 67 b1 R.......8.....g.
01d0 56 14 00 00 00 09 70 48 59 73 00 00 11 b0 00 00 V.
incorrect:(socat -> read line -> xxh)
00000000: 89 50 4e 47 0d 0a .PNG..
00000000: 1a 0a ..
00000000: 0d 49 48 44 52 07 80 04 38 08 02 67 b1 56 14 09 .IHDR...8..g.V
how to solve this problem?
thanks

DNS txt record invalid packet, FORMERR

I'm having trouble with my home made "for fun" nameserver. It's been a couple of months since I updated it so I'm a bit rusty and thought I'd ask here and see if someone else sees what's wrong. I'm getting a FORMERR when asking for a TXT record, and the same problem occur on different domains, so there's probably something wrong in the packet formatting. Anyone?
dig txt ffffff.com #ns1.ffffff.com
;; Got bad packet: FORMERR
1024 bytes
ce bf 84 00 00 01 00 01 00 02 00 00 06 66 66 66 .............fff
66 66 66 03 63 6f 6d 00 00 10 00 01 c0 0c 00 10 fff.com.........
00 01 00 00 02 58 00 13 12 57 65 6c 63 6f 6d 65 .....X...Welcome
20 74 6f 20 66 66 66 66 66 66 66 00 c0 0c 00 02 .to.fffffff.....
00 01 00 00 02 58 00 10 03 6e 73 31 06 66 66 66 .....X...ns1.fff
66 66 66 03 63 6f 6d 00 c0 0c 00 02 00 01 00 00 fff.com.........
02 58 00 10 03 6e 73 32 06 66 66 66 66 66 66 03 .X...ns2.ffffff.
63 6f 6d 00 00 00 00 00 00 00 00 00 00 00 00 00 com.............
In the example above supplied, I added an incorrect 00 (null terminator) at the end of the TXT-string. After removing the null terminator from the TXT records, the txt records now work on my nameserver.

Resources