I have been looking through multiple tutorials and stack overflow questions but for some reason I just cannot make this work. I have issues with uploading a file, so maybe fixing that first would solve the whole issue.
I tried a few options of sending a file from the front end to the back end, but it seems to always "get lost" before reaching the back end.
I have decided to use multer at the NodeJS backend to upload the file. Not sure if I am calling multer upload single right or not. Currently this is the code which I have for it:
const multer = require('multer');
const storage = multer.diskStorage({
destination: './uploadedImages',
filename: function(req,file,cb){
cb(null,file.originalname)
}
}) ;
const upload = multer({storage: storage})
exports.saveDrawing = async(req, res, next) => {
try
{
//save image
//tried a few different logs, but with FormData it seems like everything always empty
console.log("Image:");
console.log(req.body.drawingElement);
console.log(req.file);
upload.single('body.file');
return res.status(200).json({message: element});
}
}
catch (err)
{
console.log("Error at drawing save: " + err)
return res.status(500).json({message: "Error - Could not add/edit Drawing"});
}
}
And this is how it is sent from the Angular front end:
setDrawing(params, image): Observable<any> {
const formData = new FormData();
formData.append('file', image)
formData.append('data', params)
console.log("File: ");
console.log(formData.get('file'));
console.log("Data: ");
console.log(formData.get('data'));
return this.http.post<any>(`api/v1/structure/drawing/save`, formData);
}
At this stage printing out the data shows the right values. And the browser shows the right payload too:
At the back end I cannot see them in the req, req.body is empty, there is no req.form. For this api call before I have tried to include any files without the FromData I have accessed the data from req.body.
Am I looking for the data at the right place?
You're not using multer correctly, it's not doing anything.
To implement it as a middleware which you call from your handler, check the example from the docs
So, your handler should look something like this:
// setup multer middleware, set file field name
const upload = multer({storage: storage}).single('file');
exports.saveDrawing = async(req, res, next) => {
// now use the middleware, handle errors
upload(req, res, function (err) {
if (err instanceof multer.MulterError) {
// A Multer error occurred when uploading.
return res.status(500).json({message: "Error - Could not add/edit Drawing"});
} else if (err) {
// An unknown error occurred when uploading.
return res.status(500).json({message: "Error - Could not add/edit Drawing"});
}
// Everything went fine.
console.log("Image:");
console.log(req.body.drawingElement);
console.log(req.file);
return res.status(200).json({message: element});
});
});
Related
I have been trying to reproduce this error using postman in the Nodejs server on my local but I am not able to do so.
QA was able to produce it once or twice by terminating the application while uploading the file which in turn threw the uncaught exception error on Kibana APM logs and crashed the server.
My aim is to crash the server on local once so that I can add exception handlers to avoid crashes on prod env.
I am using Multer with Multer S3
"multer": "^1.4.2",
"multer-s3": "^2.9.0",
here is my code
uploadToS3: (fileFilter, limits, bcktName, awsS3Path) => { return multer({
storage: multerS3({
s3: s3,
bucket: bcktName,
contentType: multerS3.AUTO_CONTENT_TYPE,
key: function (req, file, cb) {
cb(null, awsS3Path + Date.now().toString())
}
}),
fileFilter:fileFilter,
limits:{...limits}
})
},
here is the complete error
Unexpected end of multipart data
node_modules/dicer/lib/Dicer.js in global.<anonymous> at line 62
var self = this;
self.emit('error', new Error('Unexpected end of multipart data'));
if (self._part && !self._ignoreData) {
var type = (self._isPreamble ? 'Preamble' : 'Part');
internal/process/task_queues.js in processTicksAndRejections at line 75
controller code:
let upload = util.uploadToS3(fileFilter, limits, process.env.Bucket, path).single('file');
upload(req, res, function(err) {
console.log("inside upload");
try {
if (err instanceof multer.MulterError || (err && MULTER_ERROR_CODES[err.code || err.message])) {
// A Multer error occurred when uploading.
return res
.status(400)
.send(util.sendFormattedErrorResponse(MULTER_ERROR_CODES[err.code || err.message]));
} else if (err) {
// log error in kibana
console.log(err);
// An unknown error occurred when uploading.
return res.status(500).send(util.sendFormattedErrorResponse(MULTER_ERROR_CODES['UNKNOWN']));
}
} catch (e) {
// log error in kibana
console.log(e);
}
Thanks in advance.
PS: if you can also point me where to add the exception handlers it would be great!
You can add a callback function to handle these errors,
Here is a reference link
https://github.com/expressjs/multer#error-handling
const multer = require('multer')
const upload = multer().single('avatar')
app.post('/profile', function (req, res) {
upload(req, res, function (err) {
if (err instanceof multer.MulterError) {
// A Multer error occurred when uploading.
} else if (err) {
// An unknown error occurred when uploading.
}
// Everything went fine.
})
})
You can add a global error handler, Details about it is mentioned in this link
Global Error Handler
So I'm trying to make the html form:
<form action="blahblah" encblah="multipart/form-data" whatever>
Thats not the problem, I need to make that form send the blob to express
app.post('/upload/avatars', async (req, res) => {
const body = req.body;
console.log(req.file);
console.log(body);
res.send(body);
});
So I can access the blob, create a read stream, pipe it to the cloud, and bam, upload the file without downloading anything on the express server it self.
Is that possible?
If yes, please tell me how.
If no, please tell me other alternatives.
On the client we do a basic multi-part form upload. This example is setup for a single image but you could call uploadFile in sequence for each image.
//client.ts
const uploadFile = (file: File | Blob) => {
const formData = new FormData();
formData.append("image", file);
return fetch("/upload", {
method: "post",
body: formData,
});
};
const handleUpload = (event: any) => {
return event.target.files.length ? uploadFile(event.target.files[0]) : null;
};
On the server we can use multer to read the file without persisting it to disk.
//server.js
const express = require("express");
const app = express();
const multer = require("multer");
const upload = multer();
app.post(
"/upload",
upload.fields([{ name: "image", maxCount: 1 }]),
(req, res, next) => {
console.log("/upload", req.files);
if (req.files.image.length) {
const image = req.files.image[0]; // { buffer, originalname, size, ...}
// Pipe the image.buffer where you want.
res.send({ success: true, count: req.files.image.originalname });
} else {
res.send({ success: false, message: "No files sent." });
}
}
);
For larger uploads I recommend socket.io, but this method works for reasonably sized images.
it is possible, but when you have a lot of traffic it would overwhelm your express server (in case you are uploading videos or big files ) but if it's for uploading small images (profile image, etc...) you're fine. either way you can use Multer npm
I'd recommend using client-side uploading on ex: s3-bucket, etc..., which returned a link, and therefore using that link.
Looking for help on Uploading and Retrieving Images from MongoDb using multer.
My front end is ReactNative.(Not sure if this is needed but just to be sure.)
Multer
Problem: After looking and following tutorials i'm able to encode my path to base64 and upload it to my DB but now i'm confused how to retrieve the file from my DB. I saw some tutorials about decoding it from base64 but I don't quite understand how do I go about retrieving an image and displaying it in postman. (I tried looking but haven't found anything that gives me an answer. I'm sorry if this is a duplicated question. If you could point me in a direction or give me some advice I would be really greatful.)
**POST**
route.post("/sad", upload.single("image"), (req, res, next) => {
console.log(req.file);
const img = fs.readFileSync(req.file.path);
const img_enc = img.toString('base64');
const obj = {
usrImage: {
data: new Buffer.from(img_enc, 'base64'),
contentType: "image/jpg",
},
};
console.log(obj);
const newAccout = new account(obj);
newAccout.save();
});
**RETRIEVE**
route.get('/sad',(req,res)=>{
img.find({}).then((img)=>{
res.json(img)
//How do decode my buffer to show an image in Postman?
})
}
)
I am trying to create a userprofile where a username,password and image is saved. If you can help save an Image and then retrieve it from my accounts collection.
Hey I would advise that you start using a 3rd party for file upload like cloudinary very good way of managing files i.e images or video...
I am not that well of with multer but I can give a quick code example using Formidable does the same work as multer
Before you can start you'd need to make an account on cloudinary.com(don't worry its free)
Code below is how you could handle file upload
const Formidable = require("formidable"); //Meant for body parsing
const cloudinary = require("cloudinary").v2; // file uploader
//This below is your connection/configuration to get access to your cloudinary account so cloud_name, api_key and api_secret you'll get in your home dashboard(Cloudinary)
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
});
router.post('/api/file-upload', (req, res)=>{
const form = new Formidable.InconmingForm();
form.parse(req, (error, fields, files)=>{
const {file} = files
cloudinary.uploader.upload(file.path, {folder:"/"}, (err, res)=>{
const file_url = res.secure_url //This would be the url for your file given back by cloudinary
})
})
})
This script should upload your file and the file_url will be having the url of the file that you upload having ssl then after that you can now continue saving to mongoDB
Cloudinary docs for NodeJS
https://cloudinary.com/documentation/node_integration
Nice clear and understandable docs
Shameless plug
If you get lost you can check this video out on YouTube that I made handling file upload with cloudinary then save url given back to mongoDB
https://youtu.be/mlu-tbr2uUk
First call api find one
you will need fs module to complete following query
const fs = require('fs');
let data = await db.user.findOne({
where: {
id = req.body.id
}
})
// _________________ base 64 string data from findone query data
// |
let buff = new Buffer(data.image, 'base64');
let name = name.jpeg
let path = `tmp/${name}`; // <--- destination and file name you want to give to your file
fs.writeFileSync(path, buff);// < --this will write file to given path
fs.readFile(path, function (err, content) {// <------to send file in postman response
if (err) {
res.writeHead(400)
console.log(err);
res.end("No such image");
} else {
//specify the content type in the response will be an image
res.writeHead(200);
res.end(content);
}
});
fs.unlink(path, (err) => { // <-----to delete file from tmp directory
if (err) {
console.log(err)
}
})
Try this and switch to preview tab in postman.
I haven't tried it but maybe it helps.
route.get('/sad',(req,res)=>{
img.find({}).then((img)=>{
res.setHeader('contentType','image/jpg').send(img)
})
})
I have Express API which used to upload file to the files directory. Whenever I call my API res.send redirecting to a new page. How can I perform this API with reloading my current page?
app.post('/upload', function(req, res) {
let sampleFile;
let uploadPath;
if (Object.keys(req.files).length == 0) {
res.status(400).send('No files were uploaded.');
return;
}
console.log('req.files >>>', req.files); // eslint-disable-line
sampleFile = req.files.sampleFile;
console.log('lusu', sampleFile); // eslint-disable-line
uploadPath = __dirname + '/uploads/' + sampleFile.name;
sampleFile.mv(uploadPath, function(err) {
if (err) {
return res.status(500).send(err);
}
res.send('File uploaded to ' + uploadPath);
});
});
First of all, I would like to recommend multer package for file upload in node js.
instead of res.send(), try res.status(200).json({message:"successfully uploaded"})
try debugging at front end, suppose you have a function for file upload like below,
function fileUpload(){
http.post('url',{headers:headers}).then(res){
// Try to handle the response here. Do not write anything that reloads the page.
}
}
Currently i am using putObject to upload the large file to AWS s3 with REST api call.
var params ={
Bucket:'lambdacushbu',
Key:req.files.image.name,
Body:req.files.image.data
}
s3.putObject(params,function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else {
console.timeEnd('Uploadtime');
console.log("uploaded",data);
res.json({
'status':'Uploaded',
'url':data.Location
});
} // successful response
});
But its looks like asynchronous i want the above in synchronous mode also a timeout is occurred but the file is being uploaded to the AWS s3.
So how can i increase the timeout value?? tried with connect-timeout package
app.use(timeout('600000'));
But it dosen't worked
Try using upload function instead of putObject. That should solve your timeout problem.
Here is a documentation for that function: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#upload-property
Synchronous call will definitely lower your app's performance. Can you provide me more details about your problem so we can find an async solution?
EDIT:
Here is how you should return response in your controller:
router.post('/your-route',
//additional middlewares
function(req, res, next) {
var params = {
Bucket:'lambdacushbu',
Key:req.files.image.name,
Body:req.files.image.data
}
s3.upload(params,function(err, data) {
if (err) { res.json(err); }
else {
res.json({
'status':'Uploaded',
'url':data.Location
}
});
}
);
And make sure you don't call res.json() or res.send() anywhere else in this route