I am trying to consume Azure Forms Recognizer API, where I have to provide the body in the form of "[Binary PNG data]" as stated here.
The connection seems the be working fine, however I am getting this response:
{"error":{"code":"InvalidImage","innerError":{"requestId":"73c86dc3-51a3-48d8-853b-b6411f54c51e"},"message":"The input data is not a valid image or password protected."}}
I am using a png that is my local directory and I've tried converting it in many different ways including:
fs.readFile('test.png', function(err, data){
if (err) throw err;
// Encode to base64
let encodedImage = new Buffer(data, 'binary').toString('base64');
// Decode from base64
var decodedImage = new Buffer(encodedImage, 'base64').toString('binary');});
or
let data_string = fs.createReadStream('test.png');
and many others. None of them seem to work and I always get the same response from my post request.
I would appreciate if anyone could share how to convert this png into the correct format. Thank you in advance
To base 64:
const file = fs.readFileSync('/some/place/image.png')
const base64String = Buffer.from(file).toString('base64')
Then pass the base64String to Azure
If you want just a BLOB so a binary file, you can do this
const file = fs.readFileSync('/some/place/image.png')
const blob = Buffer.from(file)
const processFile = (file: any) => {
const reader = new FileReader();
reader.readAsArrayBuffer(file);
reader.onload = function(){
const binaryData = Buffer.from(reader.result as string,'binary');
console.log(binaryData);
};
}
Related
I need to read a content of a password-protected archive (zip is preferred) to a node-js app, without writing the protected content to a file
In addition, the app is cross-platform so solution such this doesn't help
I looked also here but there is no code in the answer
The only library I can find that supports encryption is: https://github.com/rf00/minizip-asm.js
Unfortunately, it isn't well maintained.
This solution will read the file buffer which you can get from base64 or by reading the zip file, after that unzipping and opening the password-protected file is done in-memory. I hope this helps -
const unzipper = require("unzipper");
const unzipAndUnlockZipFileFromBuffer = async (zippedFileBase64, password) => {
try {
const zipBuffer = Buffer.from(zippedFileBase64, "base64"); // Change base64 to buffer
const zipDirectory = await unzipper.Open.buffer(zipBuffer); // unzip a buffered file
const file = zipDirectory.files[0]; // find the file you want
// if you want to find a specific file by path
// const file = zipDirectory.files.find((f) => f.path === "filename");
const extracted = await file.buffer(password); // unlock the file with the password
console.log(extracted.toString()); // file content
} catch (e) {
console.log(e);
}
};
const zippedFileBase64 = "{{BASE64}}";
const password = "1234";
unzipAndUnlockZipFileFromBuffer(zippedFileBase64, password);
I need to update orientation tag(EXIF data) for the uploaded image. I am using "PIEXIF" for this. I am not using express but swagger. The code I've written is:
//Get the uploaded buffer
var _originalBuffer = req.swagger.params.uploadedFile.value.buffer;
let Duplex = require('stream').Duplex;
//Create stream from buffer. This stream is required later to send to cloud.
let _uploadedFileStream = new Duplex();
_uploadedFileStream.push(_originalBuffer);
_uploadedFileStream.push(null);
//Create base 64 string so that "PIEXIF" can read exif data from it.
const jpegData = "data:image/jpeg;base64, " + createStringFromBuffer(_originalBuffer, 'base64');
//Read exif data.
var _exifData = piexif.load(jpegData);
//Create a copy of exif data. Will be used to create a new image with updated orientation tag.
var _exifDataCopied = {};
for (var key in _exifData) {
_exifDataCopied[key] = _exifData[key];
}
//Update orientation tag.
if (_exifDataCopied["0th"][piexif.ImageIFD.Orientation])
_exifDataCopied["0th"][piexif.ImageIFD.Orientation] = 1;
//Example taken from https://www.npmjs.com/package/piexifjs
//From here onwards, there seems to be an issue.
var exifbytes = piexif.dump(_exifDataCopied);
var newData = piexif.insert(exifbytes, createStringFromBuffer(_originalBuffer, 'binary'));
var newJpeg = new Buffer(newData);
//Create a new stream and save it as image back.
let _updatedFileStream = new Duplex();
_updatedFileStream.push(newJpeg);
_updatedFileStream.push(null);
var fs = require('fs');
var writeStream = fs.createWriteStream("./uploads/" + "Whatever.jpg")
The issue is there is no error thrown by the code. The image is also getting saved in the directory but it is corrupted. I can not preview it. Since, the code does not breaks anywhere, I am confused what could be the issue? The function to convert buffer to string with different encoding(since I need it a lot) is:
var createStringFromBuffer = function(_buffer, _encoding) {
return Buffer.from(_buffer).toString(_encoding);
}
Can someone point out where I am mistaking? I am using the example given Here
I am writing an iOS app in Swift 2.2 using Xcode 7.3.1. Then, for backend services I am using AWS Mobilehub (S3 and Lambda)
What the app should do: Take a screenshot, send it to an AWS S3 bucket, send the screenshot through SendGrid by using an AWS Lambda function trigger.
My problem: I can't seem to attach the damn image in the S3 bucket to the email. Locally it works fine, but when uploaded to Lambda is throws the following error:
Error: ENOENT: no such file or directory, open '...'
Using the following code:
'use strict';
var fs = require('fs');
console.log('Loading function');
let aws = require('aws-sdk');
let s3 = new aws.S3({ apiVersion: '2006-03-01' });
exports.handler = (event, context, callback) => {
// Get the object from the event and show its content type
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
var helper = require('sendgrid').mail;
let from_email = new helper.Email("from#gmail.com");
let to_email = new helper.Email("to#gmail.com");
let subject = "Hello World from the SendGrid Node.js Library";
let content = new helper.Content("text/plain", "Email content");
let mail = new helper.Mail(from_email, subject, to_email, content);
var bse64 = base64_encode(INeedThisFrigginPath);
let attachment = new helper.Attachment();
attachment.setContent(bse64);
attachment.setType("image/png");
attachment.setFilename(key);
attachment.setDisposition("attachment");
mail.addAttachment(attachment);
var sg = require('sendgrid')('SG.sengridkey');
var requestBody = mail.toJSON();
var emptyRequest = require('sendgrid-rest').request;
var requestPost = JSON.parse(JSON.stringify(emptyRequest));
requestPost.method = 'POST';
requestPost.path = '/v3/mail/send';
requestPost.body = requestBody;
sg.API(requestPost,
function (error, response)
{
console.log(response.statusCode);
console.log(response.body);
console.log(response.headers);
}
);
};
// function to encode file data to base64 encoded string
function base64_encode(file)
{
// read binary data
var bitmap = fs.readFileSync(file);
// convert binary data to base64 encoded string
return new Buffer(bitmap).toString('base64');
}
Specifically this line:
var bse64 = base64_encode(INeedThisFrigginPath);
It's quite obvious what the problem is, so what I need to know is, what is the correct path to my image.
I have tried using the key value, and the image link:
https://s3.eu-central-1.amazonaws.com/bucketname/public/test0.png
No luck.
It would be great if anyone can help me by supplying code, a tutorial or just general pointers to boost me into the right direction. Maybe using AWS S3, AWS Lambda and SendGrid isn't necessarily the best technologies to use here?
Thanks a bunch!
You are trying to build a path to that file and then you are trying to open it as a local file. The file isn't local, it's on S3, so you can't do that. You have two options:
Download the file from S3 to Lambda first (into the /tmp directory) and then refer to the local path when you open the file.
Open a ReadStream to the file using the AWS S3 SDK.
On a node server I would like to save uploaded datauri data as an image. To do this I've tried decoding the content of this png-
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAMAAAACCAIAAAFlEcHbAAAAB3RJTUUH1gMWFjk7nUWcXQAAAAlwSFlzAABOIAAATiABFn2Z3gAAAARnQU1BAACxjwv8YQUAAAAeSURBVHjaY7h79y7DhAkTGIA04/Tp0xkYGJ49ewYAgYwLV/R7bDQAAAAASUVORK5CYII=
And saving it as a .png extension. Looks like there is more too it than that. How do I decode the datauri and save it as a file?
I've created a library to be used with Node.js that helps with encoding and decoding of data URI schemes. I believe it can help you, check:
https://github.com/DiegoZoracKy/image-data-uri
Using this library, in your case, the code would be:
'use strict';
const ImageDataURI = require('image-data-uri');
const dataURI = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAMAAAACCAIAAAFlEcHbAAAAB3RJTUUH1gMWFjk7nUWcXQAAAAlwSFlzAABOIAAATiABFn2Z3gAAAARnQU1BAACxjwv8YQUAAAAeSURBVHjaY7h79y7DhAkTGIA04/Tp0xkYGJ49ewYAgYwLV/R7bDQAAAAASUVORK5CYII=';
const fileName = 'decoded-image.png';
ImageDataURI.outputFile(dataURI, filePath);
I was trying to decode the data using atob and saving this as a png file. I'm instead saving it base64 encoded but specifying the encoding in the write buffer.
fs.writeFileSync('tmp/myfile.png', new Buffer(data, 'base64'));
You can convert your data uri to a blob using below code:
function dataURItoBlob(dataURI) {
var byteStr;
if (dataURI.split(',')[0].indexOf('base64') >= 0)
byteStr = atob(dataURI.split(',')[1]);
else
byteStr = unescape(dataURI.split(',')[1]);
var mimeStr = dataURI.split(',')[0].split(':')[1].split(';')[0];
var arr= new Uint8Array(byteStr.length);
for (var i = 0; i < byteStr.length; i++) {
arr[i] = byteStr.charCodeAt(i);
}
return new Blob([arr], {type:mimeStr});
}
and then you can append this blob data to from data and upload it as a file:
var blob = dataURItoBlob(dataURI);
var fd = new FormData(document.forms[0]);
fd.append("image", blob);
I've got a data URL like this:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...
What's the easiest way to get this as binary data (say, a Buffer) so I can write it to a file?
Put the data into a Buffer using the 'base64' encoding, then write this to a file:
var fs = require('fs');
var string = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==";
var regex = /^data:.+\/(.+);base64,(.*)$/;
var matches = string.match(regex);
var ext = matches[1];
var data = matches[2];
var buffer = Buffer.from(data, 'base64');
fs.writeFileSync('data.' + ext, buffer);
Try this
const dataUrl = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==";
const buffer = Buffer.from(dataUrl.split(",")[1], 'base64');
I also met such questions (parsing and validating data URL) recently and found the following workaround: https://gist.github.com/bgrins/6194623
I created 2 packages to make working with data URL easier in the code. Here they are:
https://github.com/killmenot/valid-data-url
https://github.com/killmenot/parse-data-url
Check out examples
I was looking into the sources of Node.js and stumbled upon this code that decodes a data URL into a Buffer. Although the function is not public and exclusively intended to parse encoded ES modules, it sheds light on aspects of data URLs that are apparently not considered by some other answers: the content of data URLs must not be base64 encoded and may be URL encoded, and it may even be unencoded.
Essentially, the Node.js logic boils down to something like the code below plus error handling:
const parsed = new URL(url);
const match = /^[^/]+\/[^,;]+(?:[^,]*?)(;base64)?,([\s\S]*)$/.exec(parsed.pathname);
const { 1: base64, 2: body } = match;
const buffer = Buffer.from(decodeURIComponent(body), base64 ? 'base64' : 'utf8');
This will correctly handle different encodings of a Javascript file with the content console.log("Node.js");:
data:text/javascript,console.log("Node.js");
data:text/javascript,console.log(%22Node.js%22)%3B
data:text/javascript;base64,Y29uc29sZS5sb2coIk5vZGUuanMiKTs=
The resulting buffer can be converted into a string if required with buffer.toString().
This method works for me
function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
var data = dataURI.split(',')[1];
var byteString = Buffer.from(data, "base64");
// separate out the mime component
var mimeString = dataURI.split(",")[0].split(":")[1].split(";")[0];
// write the ArrayBuffer to a blob, and you're done
var blob = new Blob([byteString], { type: mimeString });
return blob;
}
to use
var uri = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==';
dataURItoBlob(uri)