I'm struggling to wrap my head around this and I've read all the posts I can find on it, but still am not making sense of the results I'm getting. I'm using react-signature-canvas which produces the output into a base64 string. I then seem to be successfully saving this through Mongoose with:
let data = req.body.submittedSig.data;
let split = data.split(',');
let base64string = split[1];
const buffer = Buffer.from(base64string, 'base64');
//...//
submittedSig: {
data: buffer,
contentType: plan.submittedSig.contentType
},
this shows in the db as:
submittedSig
Object
data
BinData(0, 'iVBORw0KGgoAAAANSUhEUgAAAJYAAABsCAYAAACICEudAAANIElEQVR4Xu2dSwgsRxWGjaj4CKjgxpVzjYrowidowOAoxMcmKgqi…')
contentType
"image/png"
When trying to decode, I've used several approaches such as .toBase64() - https://www.mongodb.com/docs/atlas/app-services/functions/globals/#bson--binary-json- mongo function, but I get a 'not a function' error, and otherwise I've played with various combinations of Buffer.from(data, 'base64') and .toString('base64'). https://nodejs.org/api/buffer.html#static-method-bufferfromstring-encoding
const servicePlan = await ServicePlan.findOne({ _id: req.params.planId })
// let sig = Buffer.from(servicePlan.submittedSig.data, 'base64').toString('base64');
// sig = servicePlan.submittedSig.data.toString('base64');
sig = servicePlan.submittedSig.data.toBase64();
servicePlan.submittedSig.data = sig
console.log("sig plan: ", servicePlan)
res.send(servicePlan)
For any combination of the Buffer.from or .toString- whether used separetely or together, the console.logs the output as
data: new Binary(Buffer.from("6956424f5277304b ... 414141456c46546b5375516d4343", "hex"), 0),
and the return comes as:
"submittedSig": {
"data": {
"type": "Buffer",
"data": [
137,
80,
78,
71,
13,
10,
26,
10,
0,
0,
0,
13,
...
]
So as far as I can tell, the encoding takes it into binary, but all of my output is not decoding back to base64. Let alone, the end of the console.log of shows 'hex' which had me trying to convert the binary as if it was stored as a hex object, but that hasn't worked either. I'm sure its something simple that I just don't get with these, appreciate any guidance.
Related
I’m building a MERN App and trying to display image saved in MongoDB on the fronted. I use Multer and Sharp to process images when saved to database like so:
const buffer = await sharp(req.file.path)
.resize({ width: 160, height: 160 })
.png()
.toBuffer();
group.groupImgUrl = Buffer.from(buffer, "binary").toString("base64");
await group.save();

when I console.log(group.groupImgUrl), it gives me the following output: new Binary(Buffer.from("6956….73d3d", "hex"), 0)
On the frontend side, I receive the groupImgUrl in different formats, sometimes in typed array, sometimes in string. when the response is in shape: groupImgUrl: { type: Buffer; data: Array }, I can transform the typed array and display it properly:
let base64Str = "";
if ("data" in group.groupImgUrl) {
response.group.groupImgUrl.data.forEach((index: number) => {
base64Str += String.fromCharCode(index)
})
}
<img src = `data:image/png;base64,${src}` />
However, when I receive the groupImgUrl in string format, it’s somehow tampered and cannot display correctly using src = data:image/png;base64,${src}
My question is, how come I receive the same image in different formats, and how to display them properly?
I made on node.js, this program, which has an anomaly. When I do decodeAllSync I get a vector of decimal numbers
which has one fewer numbers than I get by reconverting the vector with encodeAsync. Why don't I get the same vector?
Thank you
const cbor = require('cbor');
...
const results = cbor.decodeAllSync(output);
const input = cbor.encodeAsync(results);
console.log(output);
input.then(
function (x) {
var v=new Uint8Array(x);
console.log(v);
},
function () {
console.log("fail ");
});
I receive for the printout of the input:
Uint8Array [
129,
210,
132,
77,
...
I receive for the printout of the output:
Uint8Array [
210,
132,
77,
...
cbor.decodeAllSync with n arguments returns an array with n entries. So what you encode has an extra pair of [] around it.
> cbor.decodeAllSync(cbor.encode(1))
[ 1 ]
> cbor.decode(cbor.encode(1))
1
I receive the below object in my service but when I am parsing this object I get the error
SyntaxError: Unexpected token : in JSON at position 603069
Code:
var data1 = [];
// Process a chunk of data. This may be called multiple times.
req
.on("data", function(chunk) {
// Append to buffer
data1.push(chunk);
})
.on("end", function() {
var buffer = Buffer.concat(data1);
console.info("Buffer Data Request Body: " + buffer);
buffer = buffer.toString("utf8");
var partsOfStr = buffer.split("&");
//This line gives error
var obj = JSON.parse(
decodeURI(buffer.replace(/&/g, '","').replace(/=/g, '":"'))
);
Object:
{
"type" : "NewThreadVoice",
"incidentId": "398115",
"channel" : "Mobile",
"data": a huge base 64 string
"fileName": "1.aac",
"contentType" : "aac",
"contactId" : "954344"
}
When I reduce the base64 (value of data) to half it works.
A base64 string is not necessary to contain only one "=" character. This character is used for padding (for more information see Why does a base64 encoded string have an = sign at the end )
For example, the codification of home in base64 is aG9tZQ==. Using your code ( .replace(/=/g, '":"') ), this will be transformed into aG9tZQ":"":"
You should use .replace(/=+/g, '":"') for replacing all consecutive = chars.
I'm trying to get calibrateCamera working on my Node.js backend. The library is working fine, but I'm having trouble with the OpenCV functions not giving any error messages if I have faulty inputs. I'm kind of flying in the dark.
But that's beside the point. I've taken 17 images of the chessboard calibration pattern, and got the code to detect the pattern in all of the images. Everything works just fine until I call cv.calibrateCamera(), probably due to me not knowing what I should use as the required inputs for cameraMatrix and distCoeff (4th and 5th input parameters). However, I can't be 100% this is the issue because of not receiving any error messages from errors in the cv... functions.
I tried to follow the example at https://docs.opencv.org/3.1.0/dc/dbb/tutorial_py_calibration.html , but on python in the tutorial you can use None as inputs to cameraMatrix and distCoeff. I tried to use null, but that didn't work either.
Any help would be appreciated.
const size = new cv.Size(9,6);
let mat = null;
let objpt = [];
for(let i=0;i<9;i++) {
for(let j=0;j<6;j++) {
objpt.push(cv.Point(2.5*i,2.5*j,0))
}
}
let objectPoints = [];
let imagePoints =[];
for (let i=0; i < 17;i++) {
mat = cv.imread('./calib/calib'+(i+1)+'.jpg');
let smallmat = mat.resize(756,1008);
const corners = smallmat.findChessboardCorners(size);
if (corners.returnValue) {
objectPoints = objectPoints.concat(objpt);
imagePoints = imagePoints.concat(corners.corners);
}
}
// THIS IS WHERE EXECUTION JUST STOPS WITH NO ERROR MESSAGE
cv.calibrateCamera(
objectPoints,
imagePoints,
new cv.Size(756,1008),
new cv.Mat(3, 3, cv.CV_32FC1,0),
[0,0,0,0,0]
);
According to the test parameters should be passed like this:
[_objectPoints, _objectPoints],
[imagePoints, imagePoints],
imageSize,
_cameraMatrix,
distCoefficients
where
const _cameraMatrix = new cv.Mat([
[800, 0, 100],
[0, 800, 100],
[0, 0, 1]
], cv.CV_64F);
and
const distCoefficients = [0, 0.5, 1.0, 1.0];
When I parse this little piece of JSON:
{ "value" : 9223372036854775807 }
This is what I get:
{ hello: 9223372036854776000 }
Is there any way to parse it properly?
Not with built-in JSON.parse. You'll need to parse it manually and treat values as string (if you want to do arithmetics with them there is bignumber.js) You can use Douglas Crockford JSON.js library as a base for your parser.
EDIT2 ( 7 years after original answer ) - it might soon be possible to solve this using standard JSON api. Have a look at this TC39 proposal to add access to source string to a reviver function - https://github.com/tc39/proposal-json-parse-with-source
EDIT1: I created a package for you :)
var JSONbig = require('json-bigint');
var json = '{ "value" : 9223372036854775807, "v2": 123 }';
console.log('Input:', json);
console.log('');
console.log('node.js bult-in JSON:')
var r = JSON.parse(json);
console.log('JSON.parse(input).value : ', r.value.toString());
console.log('JSON.stringify(JSON.parse(input)):', JSON.stringify(r));
console.log('\n\nbig number JSON:');
var r1 = JSONbig.parse(json);
console.log('JSON.parse(input).value : ', r1.value.toString());
console.log('JSON.stringify(JSON.parse(input)):', JSONbig.stringify(r1));
Output:
Input: { "value" : 9223372036854775807, "v2": 123 }
node.js bult-in JSON:
JSON.parse(input).value : 9223372036854776000
JSON.stringify(JSON.parse(input)): {"value":9223372036854776000,"v2":123}
big number JSON:
JSON.parse(input).value : 9223372036854775807
JSON.stringify(JSON.parse(input)): {"value":9223372036854775807,"v2":123}
After searching something more clean - and finding only libs like jsonbigint, I just wrote my own solution. Is not the best, but it solves my problem. For those that are using Axios you can use it on transformResponse callback (this was my original problem - Axios parses the JSON and all bigInts cames wrong),
const jsonStr = `{"myBigInt":6028792033986383748, "someStr":"hello guys", "someNumber":123}`
const result = JSON.parse(jsonStr, (key, value) => {
if (typeof value === 'number' && !Number.isSafeInteger(value)) {
let strBig = jsonStr.match(new RegExp(`(?:"${key}":)(.*?)(?:,)`))[1] // get the original value using regex expression
return strBig //should be BigInt(strBig) - BigInt function is not working in this snippet
}
return value
})
console.log({
"original": JSON.parse(jsonStr),
"handled": result
})
A regular expression is difficult to get right for all cases.
Here is my attempt, but all I'm giving you is some extra test cases, not the solution. Likely you will want to replace a very specific attribute, and a more generic JSON parser (that handles separating out the properties, but leaves the numeric properties as strings) and then you can wrap that specific long number in quotes before continuing to parse into a javascript object.
let str = '{ "value" : -9223372036854775807, "value1" : "100", "strWNum": "Hi world: 42 is the answer", "arrayOfStrWNum": [":42, again.", "SOIs#1"], "arrayOfNum": [100,100,-9223372036854775807, 100, 42, 0, -1, 0.003] }'
let data = JSON.parse(str.replace(/([:][\s]*)(-?\d{1,90})([\s]*[\r\n,\}])/g, '$1"$2"$3'));
console.log(BigInt(data.value).toString());
console.log(data);
you can use this code for change big numbers to strings and later use BigInt(data.value)
let str = '{ "value" : -9223372036854775807, "value1" : "100" }'
let data = JSON.parse(str.replace(/([^"^\d])(-?\d{1,90})([^"^\d])/g, '$1"$2"$3'));
console.log(BigInt(data.value).toString());
console.log(data);