I am using Node Red to implement a web service and I am racking my brains to convert a base64 string to byte array (uint8array) or convert buffer to uint8array.
One "node" of my node-red flow outputs an image as a buffer or a base64 string.
I need to pass the responsed image into a web service which requires an uint8array base image.
There is a lot of answers using atob and btoa but node red does not support it.
Below is the output buffer format that I need to convert
And this is the buffer format what I want:
Here is the documentation of the web service I want to call:
https://demous-cdb.thereforeonline.com/theservice/v0001/restun/help/operations/CreateDocument
I have tryed a lot of ways:
function toArrayBuffer(myBuf) {
var myBuffer = new ArrayBuffer(myBuf.length);
var res = new Uint8Array(myBuffer);
for (var i = 0; i < myBuf.length; ++i) {
res[i] = myBuf[i];
}
return myBuffer;
}
=====
Also tryed to use...
var buf = Buffer.from(b64string, 'base64')
The solution above does not generates an uint8array
Do you have any idea?
The base64 node (https://flows.nodered.org/node/node-red-node-base64) should convert the base64 to buffer which you can then use as input to what ever node you are using to send to the webservice.
The NodeJS Buffer is a subclass of Uint8Array, they are just ways to represent collections of bytes.
Related
I'm trying to pass a file (image) from my Reactjs app to Firebase Functions to then upload it to Pinata using their Node.js sdk but my cloud function keeps returning: ERR_INVALID_ARG_TYPE
Is it possible to pass a file to Firebase Functions?
I need to eventually convert it to a readable stream to use Pinata.
Image file:
Callable Cloud Functions only accept JSON types, so you can't pass a File object to them.
What you can do is:
either read the bytes from the file in your client, and then pass the byte[] (which is a JSON type) or pass it as a base64 encoded String.
or you can write the file to Cloud Storage through Firebase, and then pass the path to that file to your Cloud Function
Used Frank van Puffelen's advice and first uploaded the file to Firebase Storage, then downloaded it in my Firebase Function, then converted the file to an array buffer.
Download from Google Cloud Storage
const [bufferFile] = await admin
.storage()
.bucket('my-bucket-name')
.file('file-path)
.download()
Convert to Array Buffer
const toArrayBuffer = (buf: Buffer) => {
const ab = new ArrayBuffer(buf.length)
const view = new Uint8Array(ab)
for (let i = 0; i < buf.length; ++i) {
view[i] = buf[i]
}
return ab
}
Then you can pass the Array Buffer to IPFS or arweave
Just started exploring Google Cloud Vision APIs. From their guide:
const client = new vision.ImageAnnotatorClient();
const fileName = 'Local image file, e.g. /path/to/image.png';
const [result] = await client.textDetection(fileName);
However, I wanna use base64 representation of binary image data, since they claim that it's possible to use.
I found this reference on SO:
Google Vision API Text Detection with Node.js set Language hint
Instead of imageUri I used "content": stringas mentioned here. But the SO sample uses const [result] = await client.batchAnnotateImages(request);method. I tried using same technique on const [result] = await client.textDetection( method and it gave me an error.
So my question is: Is it possible to use base64 encoded string to represent image in order to perform TEXT_DETECTION ? And if so, how?
Any kind of help is highly appreciated.
You can use the quickstart guide and from there edit the lines after the creation of the client for the following:
// Value of the image in base64
const img_base64 = '/9j/...';
const request = {
image: {
content: Buffer.from(img_base64, 'base64')
}
};
const [result] = await client.textDetection(request);
console.log(result.textAnnotations);
console.log(result.fullTextAnnotation);
You can take a look at the function here, read the description of the request parameter, in particular the following part:
A dictionary-like object representing the image. This should have a
single key (source, content).
If the key is content, the value should be a Buffer.
Which leads to the structure used in the sample code from before. Opposed to when using imageUri or filename, which have to be inside of another object which key is source, as shown in the sample.
content field need to be Buffer.
You use the nodejs client library. The library use the grpc API internally, and grpc API expect bytes type at content field.
However, JSON API expect base64 string.
References
https://cloud.google.com/vision/docs/reference/rpc/google.cloud.vision.v1#image
https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection
I am sending DICOM images to my API by encoding as base64 from the frontend, which is in Angular CLI. Also, I have Rest API to get those encoded DICOM images and decode them back before had some process with them. But after decoding the DICOM image into the memory stream, metadata of DICOM images are lost. It is appreciatable if I got a better solution. Please find my codes below.
//Angular code
var file = event.dataTransfer ? event.dataTransfer.files[i] :
event.target.files[0];
//var pattern = /.dcm/;
var reader = new FileReader();
reader.onload = this._handleReaderLoaded.bind(this);
reader.readAsDataURL(file);
//Web API Code
[HttpPost("UploadFile/{Id}")]
public async Task<IActionResult> UploadFile(int Id, [FromBody] DICOMFiles
dicomfiles)
{
String base64Encoded = encodedImage;
string output =
encodedImage.Substring(encodedImage.IndexOf(',') + 1);
byte[] data = Convert.FromBase64String(output);
MemoryStream stream = new MemoryStream(data);
client.UploadFile(stream, "Projects/test_images/Test.dcm");
}
At last, I found a solution for this. The problem is not about decode from base64. The actual problem is with the client.UploadFile() method call.
Before using the client.uploadfile(), we need to make sure that the memory stream object is pointing to position "0". This will allow the client.UploadFile() method to create and write all the content of the mentioned file from the start of the byte[] array. we can do this as mentioned below.
stream.Position = 0;
I use javamail to get message, when I get the message i have:
com.sun.mail.util.BASE64DecoderStream,
I know that is part of multipart message, in the source of message I have
Content-Type: image/png; name=index_01.png
Content-Transfer-Encoding: base64
How encode this message??
edit:
I have that code:
else if (mbp.getContent() instanceof BASE64DecoderStream){
InputStream is = null;
ByteArrayOutputStream os = null;
is = mbp.getInputStream();
os = new ByteArrayOutputStream(512);
int c = 0;
while ((c = is.read()) != -1) {
os.write(c);
}
System.out.println(os.toString());
}
And that code return strange string, for example:
Ř˙á?Exif??II*????????????˙ě?Ducky???????˙á)
com.sun.mail.util.BASE64DecoderStream is platform dependent. You cannot rely on that always being the type of class that handles base64 decoding.
Rather the javamail APIs already support decoding for you:
// part is instanceof javax.mail.Part
ByteArrayOutputStream bos = new ByteArrayOutputStream();
part.getDataHandler().writeTo(bos);
String decodedContent = bos.toString()
What are you expecting when you read the contents of an image part? The image is stored in the message in an encoded format, but JavaMail is decoding the data before returning the bytes to you. If you store the bytes in a file, you can display the image with many image viewing/editing applications. If you want to display them with your Java program, you'll need to convert the bytes to an appropriate Java Image object using (e.g.) APIs in the java.awt.image package.
That Sun's base 64 encoder is in the optional package and can be moved or renamed at any time without warning, also may be missing in alternative Java runtimes, also access to these packages may be disabled. It is much better not to rely on it.
I would say, use Base64 from Apache Commons instead, should do the same. Hope you can rebuild and fix the source code.
I've got a data URL like this:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...
What's the easiest way to get this as binary data (say, a Buffer) so I can write it to a file?
Put the data into a Buffer using the 'base64' encoding, then write this to a file:
var fs = require('fs');
var string = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==";
var regex = /^data:.+\/(.+);base64,(.*)$/;
var matches = string.match(regex);
var ext = matches[1];
var data = matches[2];
var buffer = Buffer.from(data, 'base64');
fs.writeFileSync('data.' + ext, buffer);
Try this
const dataUrl = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==";
const buffer = Buffer.from(dataUrl.split(",")[1], 'base64');
I also met such questions (parsing and validating data URL) recently and found the following workaround: https://gist.github.com/bgrins/6194623
I created 2 packages to make working with data URL easier in the code. Here they are:
https://github.com/killmenot/valid-data-url
https://github.com/killmenot/parse-data-url
Check out examples
I was looking into the sources of Node.js and stumbled upon this code that decodes a data URL into a Buffer. Although the function is not public and exclusively intended to parse encoded ES modules, it sheds light on aspects of data URLs that are apparently not considered by some other answers: the content of data URLs must not be base64 encoded and may be URL encoded, and it may even be unencoded.
Essentially, the Node.js logic boils down to something like the code below plus error handling:
const parsed = new URL(url);
const match = /^[^/]+\/[^,;]+(?:[^,]*?)(;base64)?,([\s\S]*)$/.exec(parsed.pathname);
const { 1: base64, 2: body } = match;
const buffer = Buffer.from(decodeURIComponent(body), base64 ? 'base64' : 'utf8');
This will correctly handle different encodings of a Javascript file with the content console.log("Node.js");:
data:text/javascript,console.log("Node.js");
data:text/javascript,console.log(%22Node.js%22)%3B
data:text/javascript;base64,Y29uc29sZS5sb2coIk5vZGUuanMiKTs=
The resulting buffer can be converted into a string if required with buffer.toString().
This method works for me
function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
var data = dataURI.split(',')[1];
var byteString = Buffer.from(data, "base64");
// separate out the mime component
var mimeString = dataURI.split(",")[0].split(":")[1].split(";")[0];
// write the ArrayBuffer to a blob, and you're done
var blob = new Blob([byteString], { type: mimeString });
return blob;
}
to use
var uri = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==';
dataURItoBlob(uri)