I am making a react app where I have a field that allows users to upload PDF files.
I have successfuly uploaded and sent the files as base64 string to the server and I do receive it, however I am having trouble with saving the file back to pdf, here is what I have tried:
const fs = require("fs");
const invoice = { fileData: "data:application/pdf;base64,JVBERi0xLjandtherestofthedatastring..." };
const invoiceFileContents = new Buffer.from(invoice.fileData, "base64");
fs.writeFileSync(__dirname + "invoicetest.pdf", invoiceFileContents);
This does create a pdf file, but I am unable to open it, Adobe says its broken.
I managed to solve it, the appended string infront of the whole data string data:application/pdf;base64, should be trimmed:
const invoiceFileContents = new Buffer.from(
invoice.fileData.substring(invoice.fileData.indexOf("base64") + 7),
"base64"
);
Related
Goal: Try to download a pdf file from Amazon S3 to my local machine via a NodeJS/VueJS application without creating a file on the server's filesystem.
Server: NodeJs(v 18.9.0) Express (4.17.1)
Middleware function that retrieves the file from S3 and converts the stream into a base64 string and sends that string to the client:
const filename = 'lets_go_to_the_snackbar.pdf';
const s3 = new AWS.S3(some access parameters);
const params = {
Bucket: do_not_kick_this_bucket,
Key: `yellowbrickroad/${filename}`
}
try {
const data = await s3
.getObject(params)
.promise();
const byte_string = Buffer.from(data.Body).toString('base64');
res.send(byte_string);
} catch (err) {
console.log(err);
}
Client: VueJS( v 3.2.33)
Function in component receives byte string via an axios (v 0.26.1) GET call to the server. The code to download is as follows:
getPdfContent: async function (filename) {
const resp = await AxiosService.getPdf(filename) // Get request to server made here.
const uriContent = `data:application/pdf;base64,${resp.data}`
const link = document.createElement('a')
link.href = uriContent
link.download = filename
document.body.appendChild(link) // Also tried omitting this line along with...
link.click()
link.remove() // ...omitting this line
}
Expected Result(s):
Browser opens a window to allow a directory to be selected as the file's destination.
Directory Selected.
File is downloaded.
Ice cream and mooncakes are served.
Actual Results(s):
Browser opens a window to allow a directory to be selected as the file's destination
Directory Selected.
Receive Failed - Network Error message.
Lots of crying...
Browser: Chrome (Version 105.0.5195.125 (Official Build) (x86_64))
Read somewhere that Chrome will balk at files larger than 4MB, so I checked the S3 bucket and according to Amazon S3 the file size is a svelte 41.7KB.
After doing some reading, a possible solution was presented that I tried to implement. It involved making a change to the VueJs getPdfContent function as follows:
getPdfContent: async function (filename) {
const resp = await AxiosService.getPdf(filename) // Get request to server made here.
/**** This is the line that was changed ****/
const uriContent = window.URL.createObjectURL(new Blob([resp.data], { type: 'application/pdf' } ))
const link = document.createElement('a')
link.href = uriContent
link.download = filename
document.body.appendChild(link) // Also tried omitting this line along with...
link.click()
link.remove() // ...omitting this line
}
Actual Results(s) for updated code:
Browser opens a window to allow a directory to be selected as the file's destination
Directory Selected.
PDF file downloaded.
Trying to open the file produces the message:
The file “lets_go_to_the_snackbar.pdf” could not be opened.
It may be damaged or use a file format that Preview doesn’t recognize.
I am able to download the file directly from S3 using the AWS S3 console with no problems opening the file.
I've read through similar postings and tried implementing their solutions, but found no joy. I would be highly appreciative if someone can
Give me an idea of where I am going off the path towards reaching the goal
Point me towards the correct path.
Thank you in advance for your help.
After doing some more research I found the problem was how I was returning the data from the server back to the client. I did not need to modify the data received from the S3 service.
Server Code:
let filename = req.params.filename;
const params = {
Bucket: do_not_kick_this_bucket,
Key: `yellowbrickroad/${filename}`
}
try {
const data = await s3
.getObject(params)
.promise();
/* Here I did not modify the information returned */
res.send(data.Body);
res.end();
} catch (err) {
console.log(err);
}
On the client side my VueJS component receives a Blob object as the response
Client Code:
async getFile (filename) {
let response = await AuthenticationService.downloadFile(filename)
const uriContent = window.URL.createObjectURL(new Blob([response.data]))
const link = document.createElement('a')
link.setAttribute('href', uriContent)
link.setAttribute('download', filename)
document.body.appendChild(link)
link.click()
link.remove()
}
In the end the goal was achieved; a file on S3 can be downloaded directly to a user's local machine without the application storing a file on the server.
I would like to mention Sunpun Sandaruwan's answer which gave me the final clue I needed to reach my goal.
I have a variable in my NodeJS application which contains base64 string of an image. I need to send a form to some server with POST request containing this image. The problem is I can't convert base64 string to an image without writing it to a filesystem. Here's my code:
const imagePath = path.resolve(__dirname, '../../../images/anomalies/' + Date.now() + '.png')
fs.writeFileSync(imagePath, img, { encoding: 'base64' })
setTimeout(() => {
fs.unlinkSync(imagePath)
}, 30_000)
const form = new FormData()
form.append('photo', fs.createReadStream(imagePath))
As you can see, I need to write base64 string to a file and then grab it with fs.createReadStream. Otherwise file won't upload. I tried converting it to ReadStream via stream-buffers (but server still not accepted that data) and also I tried to make a Blob from it but Node don't have blobs and all these modules on npm are either too old or don't have typings which is not great at all. I also tried Buffer.from(base64, 'base64'), which doesn't work as well.
Is there any way to create an uplodable image file from base64 encoded string without accessing filesystem in NodeJS?
You should be able to convert the Buffer object returned by fs.readFileSync to a base64 string
const base64Image = fs.readFileSync(imagePath).toString('base64')
I am trying to upload a string which stores contents of a pdf in base64 to server. In my earlier approach I was using following code
var form = new FormData();
form.append("file", fs.createReadStream(filePath));
let response=await axios.post(url, form, {headers: {
...form.getHeaders(),
}});
But in this approach I had to write string contents to temporary file first. I want to avoid it. So I need some help. The data in string is obtained from PDFMake tool.
I want to upload pdf file created by pdfmake to remote server. For that I am using following code roughly.
const doc = printer.createPdfKitDocument(docDefinition);
doc.pipe(fs.createWriteStream(filename))
var form = new FormData();
form.append("file", fs.createReadStream(filename));
let response=await axios.post(url, form, {headers: {
...form.getHeaders(),
}});
But issue with above approach is it requires to create file locally. I want to avoid that and want to take output from pdfmake and send it directly to server. How can I do that ?
For some reason I need to store some files (mostly images or pdfs) into my database (PG 9.2.20).
Those files are uploaded by users and when I download them back, they are corrupted.
I'm working with nodejs.
The column type I store the file is BYTEA.
This is how I store them :
const { files, fields } = await asyncBusboy(ctx.req);
const fileName = files[0].filename;
const mimeType = files[0].mimeType;
const bufferedFile = fs.readFileSync(files[0].path, { encoding: 'hex' });
const fileData = `\\x${bufferedFile}`;
//Just a basic insert into with knex.raw
const fileId = await storageModel.create(fields.name, fields.description, fileName, mimeType, fileData, ctx.user);
And this is how I retrieve my file :
const file = await storageModel.find(ctx.params.fileId, ctx.user);
ctx.body = Buffer.from(file.file_bin, 'hex');
ctx.set('Content-disposition', `attachment; filename=${file.file_name}`);
The file is corrupted, and of course, if I look closely, the uploaded file and the one I downloaded are different.
See hex screenshot, there is some additional data at the start of the downloaded one : http://imgur.com/a/kTRAB
After some more testing I can tell the problem lies into the koa part, when I put the buffer into the ctx.body. It got corrupted (???)
EDIT : I was working with Swagger UI : https://github.com/swagger-api/swagger-ui/issues/1605
You should not use bytea as a regular text string. You should pass in type Buffer directly, and have the driver escape it for you correctly.
Not sure which driver you are using, but for example...
pg-promise does it automatically, see the example
node-postgres is supposed to do it automatically, which it does mostly, but I know there were issues with arrays, recently fixed here.
massive.js - based on pg-promise since v3.0, so the same story - it just works.