A mqtt client send a binary message to certain topic. My node.js client subscribes to the topic and recieves the binary data. As our architecture payload is Int16Array. But I cannot cast it successfully to Javascript array.
//uint16 array [255, 256, 257, 258] sent as message payload, which contents <ff 00 00 01 01 01 02 01>
When I do this:
mqttClient.on("message", (topic, payload) => {
console.log(payload.buffer);
})
The output like:
ArrayBuffer {
[Uint8Contents]: <30 11 00 07 74 65 73 74 6f 7a 69 ff 00 00 01 01 01 02 01>,
byteLength: 19
}
which cant be cast to Int16Array because of odd length
It also contains more bytes than the original message
As it seems the original bytes exist at the end of the payload, which is offset for some reason.
Buffer also contains the offset and byte length information inside. By using them casting should be successful.
let arrayBuffer = payload.buffer.slice(payload.byteOffset,payload.byteOffset + payload.byteLength)
let int16Array = new Int16Array(arrayBuffer)
let array = Array.from(arrayBuffer)
Related
I am working on Change Streams introduced in MongoDB Version 3.6. Change Streams have a feature where I can specify to start streaming changes from a particular change in history. In native driver for Node.js, to resume change stream, it says (documentation here)
Specifies the logical starting point for the new change stream. This should be the _id field from a previously returned change stream document.
When I print it in console, this is what I am getting
{ _id:
{ _data:
Binary {
_bsontype: 'Binary',
sub_type: 0,
position: 49,
buffer: <Buffer 82 5a 61 a5 4f 00 00 00 01 46 64 5f 69 64 00 64 5a 61 a5 4f 08 c2 95 31 d0 48 a8 2e 00 5a 10 04 7c c9 60 de de 18 48 94 87 3f 37 63 08 da bb 78 04> } },
...
}
My problem is I do not know how to store the _id of this format in a database or a file. Is it possible to convert this binary object to string so I can use it later to resume my change stream from that particular _id. Example code would be greatly appreciated.
Convert BSON Binary to buffer and back
const Binary = require('mongodb').Binary;
const fs = require('fs');
Save _data from _id:
var wstream = fs.createWriteStream('/tmp/test');
wstream.write(lastChange._id._data.read(0));
wstream.close();
Then rebuild resumeToken:
fs.readFile('/tmp/test', void 0, function(err, data) {
const resumeToken = { _data: new Binary(data) };
});
I am trying to build quick nodejs script to look at some data in my network. Using node_pcap I manage to decode almost everything but the payload data that is end through the UDP protocol. This is my code (fairly basic but gives me the headers and payloads)
const interface = 'en0';
let filter = 'udp';
const pcap = require('pcap'),
pcap_session = pcap.createSession(interface, filter),
pcap_session.on('packet', function (raw_packet) {
let packet = pcap.decode.packet(raw_packet);
let data = packet.payload.payload.payload.data;
console.log(data.toString()); // not full data
});
When I try to print data using toString() method, it gives me most of the data but the beginning. I have something like this printed :
Li?��ddn-�*ys�{"Id":13350715,... I've cut the rest of the data which is the rest of the JSON.
But I am suspecting that the bit of data that I can't read contain some useful info such has how many packet, offset packet and so on..
I manage to get a part of it from the buffer from a payload :
00 00 00 01 52 8f 0b 4a 4d 3f cb de 08 00 01 00 00 00 04 a4 00 00 26 02 00 00 26 02 00 00 00 03 00 00 00 00 00 00 09 2d 00 00 00 00 f3 03 01 00 00 2a 00 02 00 79 00 05 73 01 d2
Although I have an idea of what kind of data it can be I have no idea of its structure.
Is there a way that I could decode this bit of the buffer ? I tried to look at some of the buffer method such as readInt32LE, readInt16LE but in vain. Is there some reading somewhere that can guide me through the process of decoding it?
[Edit] The more I looked into it, the more I suspect the data to be BSON and not JSON, that would explain why I can read some bit of it but not everything. Any chance someone manage to decode BSON from a packet ?
How does the library know which packet decoder to use?
It starts at Layer 2 of the TCP/IP stack by identifying which protocol is used https://github.com/node-pcap/node_pcap/blob/master/decode/pcap_packet.js#L29-L56
switch (this.link_type) {
case "LINKTYPE_ETHERNET":
this.payload = new EthernetPacket(this.emitter).decode(buf, 0);
break;
case "LINKTYPE_NULL":
this.payload = new NullPacket(this.emitter).decode(buf, 0);
break;
case "LINKTYPE_RAW":
this.payload = new Ipv4(this.emitter).decode(buf, 0);
break;
case "LINKTYPE_IEEE802_11_RADIO":
this.payload = new RadioPacket(this.emitter).decode(buf, 0);
break;
case "LINKTYPE_LINUX_SLL":
this.payload = new SLLPacket(this.emitter).decode(buf, 0);
break;
default:
console.log("node_pcap: PcapPacket.decode - Don't yet know how to decode link type " + this.link_type);
}
Then it goes upper and tries to decode the proper protocol based on the flags it finds in the header https://github.com/node-pcap/node_pcap/blob/master/decode/ipv4.js#L12-L17 in this particular case for the IPv4 protocol
IPFlags.prototype.decode = function (raw_flags) {
this.reserved = Boolean((raw_flags & 0x80) >> 7);
this.doNotFragment = Boolean((raw_flags & 0x40) > 0);
this.moreFragments = Boolean((raw_flags & 0x20) > 0);
return this;
};
Then in your case it would match with the udp protocol https://github.com/node-pcap/node_pcap/blob/master/decode/ip_protocols.js#L15
protocols[17] = require("./udp");
Hence, If you check https://github.com/node-pcap/node_pcap/blob/master/decode/udp.js#L32 the packet is automatically decoded and it exposes a toString method
UDP.prototype.toString = function () {
var ret = "UDP " + this.sport + "->" + this.dport + " len " + this.length;
if (this.sport === 53 || this.dport === 53) {
ret += (new DNS().decode(this.data, 0, this.data.length).toString());
}
return ret;
};
What does this mean for you?
In order to decode a udp(any) packet you just call the high level api pcap.decode.packet(raw_packet) and then call toString method to display the decoded body payload
pcap_session.on('packet', function (raw_packet) {
let packet = pcap.decode.packet(raw_packet);
console.log(packet.toString());
});
Below, you see a simple applet that returns 0x6781 to incoming APDU commands with INS=0x70 or INS=0x71:
package testPack;
import javacard.framework.*;
public class TestApp extends Applet
{
public static void install(byte[] bArray, short bOffset, byte bLength)
{
new TestApp().register(bArray, (short) (bOffset + 1), bArray[bOffset]);
}
public void process(APDU apdu)
{
if (selectingApplet())
{
return;
}
byte[] buf = apdu.getBuffer();
switch (buf[ISO7816.OFFSET_INS])
{
case (byte)0x70:
ISOException.throwIt((short)0x6781);
break;
case (byte)0x71:
ISOException.throwIt((short)0x6781);
break;
default:
ISOException.throwIt(ISO7816.SW_INS_NOT_SUPPORTED);
}
}
}
The problem is that, I receive 0x6C01 to the APDU command with INS=0x70:
Send: 00 A4 04 00 07 01 02 03 04 05 00 00 00
Recv: 90 00
Send: 00 70 00 00 00
Recv: 6C 01
Send: 00 70 00 00 01
Recv: 01 90 00
Send: 00 71 00 00 00
Recv: 67 81
I tried two different Java Cards (One is NXP JCOP v2.4.2 r3 and another is a KONA java card) through both contact and contactless interfaces and using two different cap file generated inside two different laptops via two different IDE!!!( How Suspicious am I? :D ) But the response is equal.
I suspect to the PCSC or Card Manager for this weird response. Because in the simulator, even the process method doesn't called for this special INS value.
What's wrong with it?
INS = 70with CLA = 00 is the MANAGE CHANNEL command according to the ISO-7816 specification, as well as INS = A4 means SELECT.
If you want to use these INS codes, you must use CLA >= 0x80 in order to specify that it is your proprietary command.
I think if class represent interindustry class then only INS will work as defined in standard,Here CLA - 00 represent interindusty command therefore, card response behvaiour was like same behaviour as Manage channel command because you used INS = 70.
6.16.4 Response message (nominal case)
Table 73 - MANAGE CHANNEL response APDU
Data field Logical channel number if P1-P2='0000'
Empty if P1-P2!='0000'
SW1-SW2 Status bytes
actually your card was returning logical channel no -01 to you. ManageChannel
it seems to me that if class is propriatry. here INS will not treated as defined INS in standard. Hope CLA 80 with same INS 0x70 will give you require result.
hope it helps.
[Bit 8 set to 1 indicates the proprietary class]
I am using bson-ext as a native binding.
Building the native .node file has worked well so far. I installed GCC, make and ran node-gyp configure and node-gyp build. I am now including the file in my node app like so:
var bson = require('./bson/build/Release/bson.node');
var object = bson.BSON.prototype;
console.log(object.serialize('hey'))
The problem is, I am getting this error:
console.log(object.serialize({test: "turkey"}))
TypeError: Illegal invocation
I get confused here because when I do console.log(object)
It outputs:
BSON {
calculateObjectSize: [Function: calculateObjectSize],
serialize: [Function: serialize],
serializeWithBufferAndIndex: [Function: serializeWithBufferAndIndex],
deserialize: [Function: deserialize],
deserializeStream: [Function: deserializeStream] }
These are the native functions from C++ that were built to bson.node, right? But, now I am just not sure how to call them. I have checked their github page as well for documentation, but with no luck.
Edit: The following:
var bson = require('./bson/build/Release/bson');
console.log(bson.BSON)
outputs:
[Function: BSON]
Next, I run:
console.log(bson.BSON().serialize({turkey: 'hey'}));
But receive:
console.log(bson.BSON().serialize({turkey: 'hey'}));
TypeError: First argument passed in must be an array of types
Then I run:
console.log(bson.BSON(['BSON']).serialize({turkey: 'hey'}));
And receive:
Segmentation fault
I found a solution inside the mongodb package.
(Specifically at \mongodb\node_modules\mongodb-core\lib\replset.js # line 15).
They are using .native()
BSON = require('bson').native().BSON;
var newBSON = new bson();
console.log(newBSON.serialize('hello'));
Which does work:
<Buffer 32 00 00 00 02 30 00 02 00 00 00 68 00 02 31 00 02 00 00 00 65 00 02 32 00 02 00 00 00 6c 00 02 33 00 02 00 00 00 6c 00 02 34 00 02 00 00 00 6f 00 00>
However, (Slightly off-topic, but, I made a question yesterday Why is JSON faster than BSON, and the conclusion was native code vs non-native code. But, now, even after installing bson natively, the performance results are roughly the same... bummer.
I have been given a text file containing hex data which I know forms a jpeg image. Below is an example of the format:
FF D8 FF E0 00 10 4A 46 49 46 00 01 02 00 00 64 00 64 00 00 FF E1 00 B8 45 78 69 00 00 4D
This is only a snippet but you get the idea.
Does anyone know how I could convert this back into the original jpeg?
To convert from a hex string to a byte you use the Convert.ToByte with a base 16 parameter.
To convert a byte array to a Bitmap you put it in a Stream and use the Bitmap(Stream) constructor:
using System.IO;
//..
string hexstring = File.ReadAllText(yourInputFile);
byte[] bytes = new byte[hexstring.Length / 2];
for (int i = 0; i < hexstring.Length; i += 2)
bytes[i / 2] = Convert.ToByte( hexstring.Substring(i, 2), 16);
using (MemoryStream ms = new MemoryStream(bytes))
{
Bitmap bmp = new Bitmap(ms);
// now you can do this:
bmp.Save(yourOutputfile, System.Drawing.Imaging.ImageFormat.Jpeg);
bmp.Dispose(); // dispose of the Bitmap when you are done with it!
// or that:
pictureBox1.Image = bmp; // Don't dispose as long as the PictureBox needs it!
}
I guess that there are more LINQish way but as long as it works..