Current Situation
In the front-end side of the application I have a simple form, from where I can get 3 parameters for the data before uploading it to the back-end:
The form submition function
e.preventDefault();
const data = new FormData(e.currentTarget) //contains a string key called title
data.append("file", JSON.stringify(file)) //file object
data.append("description", JSON.stringify(editorState)) //description object
handleSubmit(e, data)
To prevent any error, I have some client-side validations (cannot upload if some parameters are empty). This is how I fetch the data:
handleSubmit
await fetch(process.env.NEXT_PUBLIC_DR_HOST, {
method: 'POST',
body: body, //body is the data argument
}).then(res =>
{
// window.location = res.url;
})
After that, the data will be sent to the server:
The server
router.post('/', upload.single('file'), validateDbData, tryAsync(async (req, res, next) =>
And using the multer middleware(upload.single('file')), the file will be uploaded respectively (storage).
The Problem
As you can see, after the upload.single('file') middleware, comes another one called validateDbData. This is the code for it:
const declarationSchema = Joi.object({
title: Joi.string().required(),
description: Joi.object().required(),
file: Joi.object()
})
const { error } = declarationSchema.validate(
req.body
)
console.log(error)
if (error)
{
const msg = error.details.map(e => e.message).join(',') //
throw new ServerError("Invalid Data", 400)
}
else
{
next()
}
And if there is something wrong (say the title is empty) an error is thrown, so the server-side validation works. But, the upload.single('file') middleware is called before the validateDbData validation midleware. This means that even if there is an error and everything got canceled, the file remained uploaded in the cloud (storage).
Question
How can I validate the data inside the FormData on the server before uploading the file parameter to the cloud (storage)? The multer middleware accepts only multipart/form-data, so reversing the middleware order will not work.
Potential solution: express-fileupload
file remained uploaded in the cloud.
What is meant by cloud here? Multer lets you store the file onto a storage or on memory, and then subsequent middleware calls get the file and its info in the req.file field.
Your validator validateDbData requires the presence of the file, which leaves you with the option of housekeeping(to be performed in validateDbData middleware) if the form is not valid, depending on the use-case you should carefully chose where multer middleware stores the file, on memory or on a storage.
Related
I´ve got a weird problem.
Using Node, React, Express, MongoDB -> MERN Stack.
So my page generates a PDF file which then gets send to the backend (as blob data) and is being stored on there.
The problem I have, now I need to send a payment ID along with that blob data to save the order in the data base. I need both in one post request, to make it as smooth as possible:
await axios
.post(process.env.REACT_APP_SERVER_API + '/payment/cash', {
blobData,
paymentId
})
.then(async (res) => ...
like so.
Before, when I just sent the blob data, I could simply access the data in the backend by writing:
exports.createCashOrder = async (req, res) => {
const { filename } = req.file; // THIS RIGHT HERE
const fileHash = await genFileHash(filename);
try {
await saveOrder(filename, fileHash, "cash", paymentId);
//await sendOrderCreatedEmail(req.body, fileHash);
//await sendOrderReceivedConfirmEmail(req.body, fileHash);
res.send({ filename: filename });
}
But that doesn't work anymore. I dont have access to that file object anymore when sending that request object.
Neither by trying
req.body.blobData
req.body.blobData.file
req.file
Any idea how to achieve that, except from making two seperate post requests?
Glad for any help, cheers!
Send the data as a form
await axios
.postForm(process.env.REACT_APP_SERVER_API + '/payment/cash', {
blobData,
paymentId
})
.then(async (res) => ...
And then use multer middleware to handle the form in express.
So I'm trying to make the html form:
<form action="blahblah" encblah="multipart/form-data" whatever>
Thats not the problem, I need to make that form send the blob to express
app.post('/upload/avatars', async (req, res) => {
const body = req.body;
console.log(req.file);
console.log(body);
res.send(body);
});
So I can access the blob, create a read stream, pipe it to the cloud, and bam, upload the file without downloading anything on the express server it self.
Is that possible?
If yes, please tell me how.
If no, please tell me other alternatives.
On the client we do a basic multi-part form upload. This example is setup for a single image but you could call uploadFile in sequence for each image.
//client.ts
const uploadFile = (file: File | Blob) => {
const formData = new FormData();
formData.append("image", file);
return fetch("/upload", {
method: "post",
body: formData,
});
};
const handleUpload = (event: any) => {
return event.target.files.length ? uploadFile(event.target.files[0]) : null;
};
On the server we can use multer to read the file without persisting it to disk.
//server.js
const express = require("express");
const app = express();
const multer = require("multer");
const upload = multer();
app.post(
"/upload",
upload.fields([{ name: "image", maxCount: 1 }]),
(req, res, next) => {
console.log("/upload", req.files);
if (req.files.image.length) {
const image = req.files.image[0]; // { buffer, originalname, size, ...}
// Pipe the image.buffer where you want.
res.send({ success: true, count: req.files.image.originalname });
} else {
res.send({ success: false, message: "No files sent." });
}
}
);
For larger uploads I recommend socket.io, but this method works for reasonably sized images.
it is possible, but when you have a lot of traffic it would overwhelm your express server (in case you are uploading videos or big files ) but if it's for uploading small images (profile image, etc...) you're fine. either way you can use Multer npm
I'd recommend using client-side uploading on ex: s3-bucket, etc..., which returned a link, and therefore using that link.
I am submitting a form with an image. Using the below code.
router.post("/", upload.upload('image').single('categoryLogo'), categoryRules.categoryCreationRules(), validate, categoryController.createCategory);
It is working fine, but is some validation comes then still image is saving in disk.
so what I tried is :
router.post("/", categoryRules.categoryCreationRules(), validate,upload.upload('image').single('categoryLogo'), categoryController.createCategory);
But in this express validator getting blank body so it throws validation error very time.
What should I do for it, I search on google but I did not found any helpful info I am new in the node.
Rules code:
const categoryCreationRules = () => {
return [
check('name')
.isLength({ min: 1 })
.trim()
.withMessage("Category name is required."),
check('name').custom((name)=>{
return CategoryModel.findOne({name: name}).collation({locale:'en',strength: 2})
.then(category=>{
if(category){
return Promise.reject(category.name+" category already exsist.");
}
})
}),
check('name')
.isLength({max: 100})
.trim()
.withMessage("Category name should not exceed more then 100 characters."),
check('description')
.isLength({max: 255})
.trim()
.withMessage("Category description should not exceed more then 255 characters.")
];
}
In theory, running categoryCreationRules and validate middlewares before multer would be enough. Therefore, you would only need a verification in the request body and if it contains any errors, you just return a bad request response, not letting the request pass to the next middleware (in this case the multer).
A simple example what i'm talking about: (Just to let it clear, the below code won't work)
router.post("/", categoryRules.categoryCreationRules(), validate, upload.upload('image').single('categoryLogo'), categoryController.createCategory);
const validator = (req, res, next) => {
try {
validationResult(req).throw();
// Continue to next middleware, in this case, multer.
next();
} catch (errors) {
// return bad request
res.status(400).send(errors);
}
};
this won´t work because your req.body will be undefined, as you are sending the data as a multipart/form-data (generally used to upload files). And in this case, errors will always be true.
But with multer this will not happen - you will be able to access body fields like description and name and then do the validation code.
This occurs because multer, internally, parses the multipart/form-data request to body with a library called busboy, and with this you can access fields through req.body.
Because of that, i think the best approach here is call multer middleware before your validations middlewares:
router.post("/", upload.upload('image').single('categoryLogo'), categoryRules.categoryCreationRules(), validate, categoryController.createCategory);
And after that, if the validation has an error, you delete the file created from multer and return a bad request response, something like that:
const fs = require("fs");
const validator = (req, res, next) => {
try {
validationResult(req).throw();
// continue to next middleware
next();
} catch (errors) {
fs.unlink(req.file.path, (err) => {
if (err) {multipart/form-data
/* HANLDE ERROR */
}
console.log(`successfully deleted ${req.file.path}`);
});
// return bad request
res.status(400).send(errors);
}
};
You can get more info about this in the below links:
node-js-with-express-bodyparser-unable-to-obtain-form-data-from-post-request
req-body-undefined-multipart
html-multipart-form-data-error-in-req-body-using-node-express
To upload an image , you have set the enctype of form to multipart/form-data . But if you use multer later, you don't have the form data parsed, hence probaby giving undefined.
Please check multiparty , an npm module
https://www.npmjs.com/package/multiparty
It also parses other fields along with file uploads and validation might be easy to set.
I send JSON data to route:
const data=[{name:this.name}]
//qs.stringify(payload)
axios
.post('/users',{name:this.name})
.then(
response => {
console.log(response.data);
}
)
.catch(
// error=>console.log(error)
)
And try get data:
router.post('/', function (req, res, next) {
var data = req.body; // here is your data
var obj=JSON.parse(data);
console.log(obj.toString());
res.toString("ok");
});
And I got error 500.
Why not get the data?
Your client side code is fine, except that the constant named data is entirely unused. On server-side however, req.body almost certainly contains a parsed JSON object (provided you have included a body paring middleware already). If you haven't included any body parsing middleware then req.body would be undefined.
Additionally, the toString() method in res doesn't send any response, it simply returns a string representation of the response.
You would need to make following changes to your code:
Include a body parsing middleware (eg. body-parser) to your express middleware chain, if one isn't already included.
Don't call JSON.parse(req.body). body-parser would already have done that. Calling it again will only throw exception, and return 500.
To convert an Object to JSON string, use JSON.stringify() method. obj.toString() will probably only return [object Object].
Send response using one of .send(), .json(), .end() methods on res. Since you need to send a string back, res.send("ok") seems most appropriate.
The changed code should appear something like this:
router.post('/', function (req, res, next) {
var data = req.body; // here is your data
console.log(JSON.stringify(data));
res.send("ok");
});
I am implementing a web app using MEAN Stack and Angular 6. There I want to submit a form with file upload. '.png' files should be uploaded.
I want to save the file in a different file server and send the url to the image.Currently I upload files into a folder in my project and save the image in db (I used ng2fileupload and multer for that.). Then it saves like this.
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAV4AAAFUCAYAAABssFR8AAAK..."
But I want to save the image url and the image should be retrived by the url. Does anyone can explain a proper method for that?
I faced the same problem a month ago and find out a solution to this problem. Though I haven't used multer in the app.
From my frontend, I will be sending an object to Node API endpoint /event which will look like:-
let img = {
content: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg...",
filename: 'yourfile.png'
}
At the backend, I'm using Cloudinary to store my images (Its free plan allows 10GB storage) and returns secure https URLs. So install it using npm i cloudinary and require in your api.js file.
And add the below configuration
cloudinary.config({
cloud_name: 'yourapp',
api_key: 'YOUR_KEY',
api_secret: 'YOUR_SECRET_KEY'
});
Last Step:- (Not so optimized code)
Let say I have an event Schema which has images array, where I'll be storing the URLs returned by cloudinary.
app.post('/event', (req, res) => {
try {
if (req.body.images.length > 0) {
// Creating new Event instance
const event = new Event({
images: [],
});
// Looping over every image coming in the request object from frontend
req.body.images.forEach((img) => {
const base64Data = img.content.split(',')[1];
// Writing the images in upload folder for time being
fs.writeFileSync(`./uploads/${img.filename}`, base64Data, 'base64', (err) => {
if (err) {
throw err;
}
});
/* Now that image is saved in upload folder, Cloudnary picks
the image from upload folder and store it at their cloud space.*/
cloudinary.uploader.upload(`./uploads/${img.filename}`, async (result) => {
// Cloudnary returns id & URL of the image which is pushed into the event.images array.
event.images.push({
id: result.public_id,
url: result.secure_url
});
// Once image is pushed into the array, I'm removing it from my server's upload folder using unlinkSync function
fs.unlinkSync(`./uploads/${img.filename}`);
// When all the images are uploaded then I'm sending back the response
if (req.body.images.length === event.images.length) {
await event.save();
res.send({
event,
msg: 'Event created successfully'
});
}
});
});
}
} catch (e) {
res.status(400).send(e);
}
});
P.S. Go ahead and suggest some optimization solution for this code here