How to solve Multer Error: Unexpected end of form? - node.js

I found other similar questions regarding Multer, but with no answers. I'm trying to upload a file with next.js (front-end) and node.js (back-end). The data is being posted via the network tab when using dev tools.
Below is my setup:
app.js
const express = require('express');
const bodyParser = require('body-parser');
// Get routes
const routeUser = require('./routes/user');
// Create an express server
var app = express();
// Add necessary middleware
app.use(bodyParser.urlencoded({ extended: true })); // Support encoded bodies
app.use(bodyParser.json({
type: ["application/x-www-form-urlencoded", "application/json"], // Support json encoded bodies
}));
// Custom routes
routeUser(app);
// Start server on port 1234
app.listen(1234, () => {
console.log("API is running.");
});
route/user.js
const multer = require('multer');
module.exports = function(app) {
const upload = multer({
storage: multer.diskStorage({
destination: (req, file, cb) => {
cb(null, "./user_photos");
},
filename: (req, file, cb) => {
cb(null, file.originalname)
}
})
});
app.post('/user/update', upload.single('user_photo'), (req, res) => {
console.log(req.body, req.file);
});
}
Form
submit(event) {
event.preventDefault();
let form_values = new FormData(event.target);
fetch(this.props.env.api + "/user/update", {
method: 'post',
headers: {
'Content-Type': 'multipart/form-data; boundary=MyBoundary',
},
body: form_values
}).then((response) => {
return response.json();
}).then((response) => {
console.log(response);
});
}
------
<form onSubmit={this.submit}>
<input type="file" name="user_photo"/>
<button type="submit">Upload</button>
</form>
I've tried several tutorials, I'm setting it up according to the docs, yet I keep getting the following error:
Error: Unexpected end of form
at Multipart._final (...\node_modules\busboy\lib\types\multipart.js:588:17)
If I remove multipart/form-data as Content-Type, the form submits with no problem, but with no file. My guess it has something to do with the way the formData object is being received.

Hey #SReca, just faced this issue today and hope I can help you out here and anybody else reading this.
Resolving Unexpected end of form
Regarding the original issue about Unexpected end of form, you are correct in removing the Content-Type as a solution. This is because sending FormData() with regular Fetch or XMLHttpRequest will automatically have the header set by the Browser. The Browser will also attach the boundary needed in all multipart/form-data requests in order indicate when a form starts and ends. More details can be found on MDN's docs about Using FormData Objects... there's a warning about explicitly setting Content-Type.
Potential fix for missing file
Now, about the missing file, it's possible that it has an incorrect reference to the path you're expecting. I am not using diskStorage, but I am using regular dest option to save my file and had the same problem (wanted to save images to ./assets/images). What worked for me was to supply the complete directory path. Maybe changing your callback function in the destination property to
// Assuming the the path module is 'required' or 'imported' beforehand
cb(null, path.resolve(__dirname, '<path-to-desired-folder>'));
will solve the issue. Hope this helps!

What caused this too me was because I was using multiple middleware on the same route, I was using global middleware and then applied another middleware in sub route. so this caused conflits.

In my case the problem magically went away by downgrading Multer to 1.4.3.
See https://github.com/expressjs/multer/issues/1144

Related

problem while POSTing to the server in Express

I'm learning Express and I face an issue which I can't understand.
When I route to /addPerson I expect to log the name: 'Mike', age: 30 to the console. Instead I got nothing logged to the console. What's wrong in my code?
here's the server.js code
const Express = require('express'),
app = Express(),
PORT = process.env.PORT || 5000,
parser = require('body-parser'),
data = []
// initialize the main project folder
app.use(Express.static('public'))
// running the server
app.listen(PORT, () => {
console.log(`Server is running at port ${PORT}`);
})
// include body parser to handle POST requests
app.use(parser.urlencoded({extended: false}))
app.use(parser.json())
// setup CORS
const cors = require('cors')
app.use(cors())
// GET request
app.get('/', (req, res) => {
res.send('<h1>Home Page</h1>')
})
app.get('/addPerson', (req, res) => {
res.send('<h1>Hello Hany</h1>')
})
// POST request
app.post('/addPerson', (req, res) => {
data.push(req.body)
console.log(data);
})
and here is the client side app.js code
const postData = async ( url = '', data = {})=>{
console.log(data);
const response = await fetch(url, {
method: 'POST',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json',
},
// Body data type must match "Content-Type" header
body: JSON.stringify(data),
});
try {
const newData = await response.json();
console.log(newData);
return newData;
}catch(error) {
console.log("error", error);
}
}
postData('/addPerson', {name: 'Mike', age: 30});
this the files structure
Alright, I've taken a look at your code and this is what I've noticed. Within your server.js file you have this code block:
app.get('/addPerson', (req, res) => {
res.send('<h1>Hello Hany</h1>')
})
That is sending back a static H1 tag when the user creates a get request to localhost:5000/addPerson. Then, directly below that you have your post route but you're never fully accessing that from anywhere (I looked through all your app.js code to double check).
Instead, I have changed your code to return a static html file with a button that allows you to call this function (just as an example so you can see that your routes do in fact work). This isn't the cleanest solution to your problem but I just wanted to make sure you see where the problem lies as I've been in your shoes before when I first started working with express. You can take a look at the CodeSandbox I setup below to replicate your issue and take a look through all the code to get an understanding.
To properly solve your issue using the app.js file you would have to serve the javscript file as your "frontend". Personally I'm a big fan of React so I usually serve my frontend with React, while my backend is express. You can also very easily serve this file using NodeJS in a similar fashion that you are with your "backend". If you were to take the React approach you would be able to modify this code:
app.get("/addPerson", (req, res) => {
res.sendFile(path.resolve(__dirname, "public", "index.html"));
});
To find the frontend section you desire using React (I can recommend react-router if you require multiple routes but I don't want to overwhelm you with too much information yet) and complete the same function. If you have any questions feel free to reach out and let me know! Hopefully this helps!

Downloading a file from a Node.JS API REST (express) with React.JS (Axios Get)

I have a React JS application that as a Backend has an API REST made with Node JS.
Currently, my objective is to be able to download files that are on the server.
The correct behavior should be that the user, after clicking on "Download file", should receive the file (Download with browser).
On the server-side, I have something like this (obviously, I'm gonna simplify it by removing JWT middleware, DB queries, etc..):
const express = require('express');
const router = express.Router();
const bodyParser = require("body-parser");
const cors = require("cors");
const app = express();
app.use(cors({ origin: "http://localhost:3000" }));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
router.get('/download', (req, res, next) => {
res.download("\\\\folder\\subfolder\\myfile.txt");
});
app.use('/api', router);
const PORT = 3001;
app.listen(PORT, function() {
console.log("Server is running on port "+PORT);
});
Then, as I have said, I have a React JS application working as a Frontend:
apihelperdownload () {
return axios.get(API_URL + "download").then(function (response) {
return response;
})
}
.............
function downloadFile() {
apihelperdownload().then(
(res) => {
// Should I do something here with response?
},
(error) => {
}
)
}
<button className="download" onClick={() => downloadFile()}>
Download File
</button>
I have two files on my server, one TXT and one JPG.
Both have the correct path (I'm not getting any "file/path not found" error) and I am receiving a "200 status OK" in both cases... But I can't download the files.
Also:
In the JPG case, in Network Tab, on preview sub-Tab I can see the image (so the browser is receiving the image).
And the response looks like this:
(ignore the params and the different url, it's just that here is not simplified)
- In the TXT case, in Network Tab, on preview sub-Tab I can just see a white page.
And the response looks like this:
As you can see, in this second case (.txt file), the data is "empty" ( "" )
Data is the correct text.. I didn't save the txt file.. So it was empty..
I have checked several related questions like this Download a file from NodeJS Server using Express
But unfortunately, I haven't found how to solve my issue.
1) What am I doing wrong on the server-side?
2) What I have to do with the response on client-side?
Thanks
I have found how to solve it without third-party libraries and in quite an "easy way".
First of all, I have changed the request to POST (since I just made GET because I thought it was the only way).
After that, on the Axios request, we have to indicate the responseType as blob:
function apihelperdownload () {
return axios.post(API_URL + "download",{ key: 'value', headers: authHeader(), responseType: 'blob' }).then(function (response) {
return response;
})
}
Then, when we receive the response, we have to create an URL object as a Blob and a link element to download it.
function downloadFile(filename) {
apihelperdownload().then(
(res) => {
const url = window.URL.createObjectURL(new Blob([res.data]));
const link = document.createElement('a');
link.href = url;
if (typeof window.navigator.msSaveBlob === 'function') {
window.navigator.msSaveBlob(
res.data,
filename
);
} else {
link.setAttribute('download', filename);
document.body.appendChild(link);
link.click();
}
},
(error) => {
alert("Something went wrong");
}
)
}
With this, we can download almost any kind of file very easily.
You can use js-file-download module.
const FileDownload = require('js-file-download');
Axios.get(API_URL + "download")
.then((response) => {
FileDownload(response.data, 'file.txt');
});
Check this response for more: https://stackoverflow.com/a/41940307/6512445

Express req.body is empty

I've tried many StackOverflow answers, and this method normally works using body-parser, however I've been having issues with getting any output from req.body with either AJAX or form data.
In server.js:
app.use(helmet()); // Helmet middleware
app.use('/assets', express.static('resources/web/assets')); // Makes /assets public
app.use(require('./resources/modules/session.js')); // Custom session middleware
app.set('view engine', 'ejs'); // Sets EJS to the view engine
app.set('views', `${__dirname}/resources/web/pages`); // Sets the views folder
app.use(cookieParser()); // cookie-parser middleware
app.use(bodyParser.urlencoded({ extended: true })); // body-parser's middleware to handle encoded data
app.use(bodyParser.json()); // body-parser's middleware to handle JSON
app.use(fileUpload({ limits: { fileSize: 100 * 1024 * 1024 } })); // express-fileupload middleware (bushboy wrapper)
app.use('/api', require('./resources/routes/api.js')); // External API router
// ...
app.post('/login', (req, res) => {
console.log(req.body);
res.render('login', {
config,
page: {
name: 'Login'
},
error: ''
});
res.end();
});
My login.ejs code:
<form method="POST">
<div class="input-group">
<i class="las la-user"></i>
<input placeholder="Username" name="username" type="text" required>
</div>
<div class="input-group">
<i class="las la-lock"></i>
<input placeholder="Password" name="password" type="password" required>
</div>
<button type="submit">
<i class="las la-paper-plane"></i> Login
</button>
</form>
No matter what I try, I always get an empty {} in the console with no avail. I've tried debugging; I need a fresh pair of eyes to see what I've done wrong.
Here's the form data:
And I've tried using jQuery's AJAX ($.get) too:
$.post('', {username:'test', password:'test'})
.fail(console.error)
.done(() => console.log('Success'));
Edit: After trying multer's app.use(require('multer')().array()); and app.use(require('multer')().none()); middleware, I'm still at the same old issue, except with multer req.body is now undefined instead of {}. This is due to the data being sent as application/x-www-form-urlencoded instead of what I previously thought was application/form-data. As that is the case, the body-parser middleware method should work. If contributing, please do not contribute an answer relating to parsing application/form-data!
Edit 2: For those asking for the session.js code, here it is:
const enmap = require('enmap'),
sessions = new enmap('sessions');
module.exports = (req, res, next) => {
if (!req.cookies) next();
const { cookies: { session: sessionID } } = req;
if (sessionID) {
const session = sessions.get(sessionID);
if (session) {
req.session = session;
} else {
req.session = undefined;
};
} else {
req.session = undefined;
};
next();
};
I'm attaching the whole source code as you people claim to be able to reproduce it somehow. Download it at https://dropfile.nl/get/F7KF (DM me on Discord if it isn't working - PiggyPlex#9993).
For the specific case you're talking about, you usually need only 'body-parser' module to be able to access the form input fields. The minimum example that I advice you to build above it is the following:
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.urlencoded({extended : true}));
app.use(bodyParser.json());
app.get('/login', (req, res) => { /* ... */ });
app.post('/login', (req, res) => {
console.log(req.body);
// ...
});
app.listen(3000);
So my advice is to narrow on the cause of the problem by removing any other middleware except for the bodyParser. Try to comment them one-by-one and then you will be able to find the guilty!
Also note that no need to bother yourself trying to make it work with Ajax as it will make no difference. Keep it simple and just try the normal browser submission.
When you found the problematic middleware, debug it. If it's made by you, make sure that you don't make any changes to the req.body. If it a thirdparty middleware, so please consult their installation steps very carefully and I'm happy for further explanation and support
Edit: Some other hints
Make sure that the request is being submitted with the header Content-Type: application/x-www-form-urlencoded
Check if there any CORS problems
File upload middleware to be on a separate path just like the assets
Removing enctype="multipart/form-data" from the form elements works for me, also after registering these middlewares:
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
Sometimes the main point this doesn't work is - when your data passed in the body is in text format, or as in my case in the headers I had 'Content-Type': 'application/x-www-form-urlencoded', but your req.body is expecting JSON data - so please make sure to double-check the 'Content-Type':'application/json' is set on the request headers.
You need to use other module to deal with multipart/form-data
https://github.com/pillarjs/multiparty
app.post('/login', function (req, res) {
const form = new multiparty.Form();
form.parse(req, function(err, fields, files) {
console.log(fields);
});
})
Please use body parser to get the input from the form
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.urlencoded({extended : true}));
app.use(bodyParser.json());
app.post('/anyRoute', (req, res) => {
console.log(req.body); //your input value
});
app.listen(3000);
I had this same error and what fixed it was by removing enctype="multipart/form-data" from the form element. If you are not sending a file alongside the form with the enctype attribute, then you get such errors

Upload PDF file to Express server

I've built a basic browser form allowing users to upload a PDF file. I then want to send that file to an Express backend. It seems like this should be a pretty basic action, but I'm unfamiliar with the end-to-end process so I'm not sure which piece is failing. I've searched through a number of SO questions/answers, but haven't found any that provide a complete solution, and I haven't been able to cobble together a solution either.
Update: It looks like the file is getting to the server, but the encoding is messed up. My guess is that FileReader.readAsText is the wrong method to use. FileReader.readAsBinaryString got me a little closer, but still not quite right (and it's deprecated). FileReader.readAsArrayBuffer seems like potentially the way to go, but I'm not sure how to correctly handle the buffer in Express.
Client/Browser
The form is built in React and just uses an onChange handler on the input itself. When a file has been added, the handler reads the file, adds it to the form data and sends the post request to the server.
// React form
<input
name="upload"
onChange={this._handleUpload}
type="file"
/>
_handleUpload = (e) => {
const { files, name } = e.target;
// Read the file
const reader = new FileReader();
reader.onload = (e) => {
const file = e.target.result;
// Now that we have the file's contents, append to the form data.
const formData = new FormData();
formData.append('file', file);
formData.append('type', name);
axios
.post('/upload', formData)
.then(res => {
// Handle the response...
})
.catch(err => console.log(err));
};
// Reading as text. Should this be something else?
reader.readAsText(files[0]);
}
Express App
The express app uses multer middleware to process the upload:
const app = express();
const upload = multer({});
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(cors());
app.post('/upload', upload.any(), handleUpload);
Middleware
Finally, I have my own middleware that gets the file from multer. I'm testing this piece by just writing the file I've received to disk. It has contents, but it's not a readable PDF file.
const handleUpload = (req, res, next) => {
// The file shows up on req.body instead of req.file, per multer docs.
const { file } = req.body;
// File is written, but it's not a readable PDF.
const tmp = fs.writeFileSync(
path.join(__dirname, './test.pdf'),
file,
);
}
Is there some piece that I'm getting obviously wrong here? eg: Do PDFs need to be handled in a special way? Any tips for where to focus my debugging?
See if that solves your problem.
_handleUpload = (e) => {
const dataForm = new FormData();
dataForm.append('file', e.target.files[0]);
axios
.post('http://localhost:4000/test', dataForm)
.then(res => {
})
.catch(err => console.log(err));
}
render() {
return (
<div className="App">
<input
onChange={this._handleUpload}
type="file"
/>
</div>
)
}
server:
router.post('/test', upload.any(), (req, res) => {
console.log(req.files)
res.send({sucess: true})
})
No need to send the file type, the multer identifies the name and type for you.

How to handle binary post data in Express?

I am wanting to post an image in the form of binary to my Express app.
I'm assuming it should come through in the req.body object but will need some form of middleware to be able to handle binary data?
When I send an image as binary from postman and try log req.body, the object is empty.
I am using express-generator as a boilder plate which comes with body-parser like so:
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
I had a look at Multer but think that is just for multipart data
Also looked at busboy but couldn't figure out if that will handle binary data.
Am I correct that the post data will still come through in req.body?
And what middleware do I need to handle binary data?
Thanks
The method I ended up using:
const multer = require('multer')
const storage = multer.memoryStorage()
const upload = multer({ storage: storage })
router.post('/upload', upload.single('image'), function(req, res, next) {
const image = req.file.buffer
});
Unfortunately, you can't use the body-parser to handle the binary data like files and stuff like that. But wut you can do is use a module call formidable to handle this
Example snipper
app.post('/', (req, res) => {
const form = new formidable.IncomingForm();
form.parse(req, (error, fields, files) => {
if(error){
console.log(error)
}
console.log(fields.name)
const cuteCat = files.cat_image;
console.log(cuteCat.name) // The origin file name
console.log(cuteCat.path) // The temporary file name something like /tmp/<random string>
})
});
<input name="cat_image" type="file" />
<input name="name" type="text" />

Resources