Increase body limit for specific route with ExpressJS - node.js

We have a REST API built with ExpressJS. One of our routes is getting files as binary data in the JSON request body. Therefore we want to increase the limit of the body.
Simple version of our starting index.js file
const start = async () => {
try {
// Connect to database
await connectDB(process.env.DATABASE_URL);
// Create app
const app = express();
// Body parser
app.use(express.json()); //100kb default
app.use(express.urlencoded({ extended: true }));
// Mount routes
app.use(`/${process.env.API_VERSION}`, authRoutes);
app.use(`/${process.env.API_VERSION}`, profileRoutes);
app.use(`/${process.env.API_VERSION}`, filesRoutes);
app.use(`/${process.env.API_VERSION}`, ticketRoutes);
// Error handler
app.use(errorHandler);
// Not found handler
app.use(notFoundHandler);
// Start server
const server = app.listen(process.env.PORT, () => {
console.log(`Server running in ${process.env.NODE_ENV} mode on http://${process.env.HOST}:${process.env.PORT}/${process.env.API_VERSION}`)
});
} catch(error) {
console.log(error);
}
}
start();
At this point the limit for all routes is 100kb.
Inside our filesRoutes we have 1 specific upload route where we want to increase this limit to 200mb.
import express from 'express';
import { uploadFiles, getFiles, downloadFile, deleteFile } from '../controllers/filesController.js';
import authorize from '../middleware/authorizeHandler.js';
const router = express.Router();
router.get('files', authorize, getFiles);
router.get('files/:id/download', authorize, downloadFile);
router.post('files/upload', [authorize, express.json({ limit: '200mb' })], uploadFile);
router.delete('files/:id', authorize, deleteFile);
export default router;
I added express.json({ limit: '200mb' }) to the upload route but I'm getting an error back request entity too large.

Based on #vighnesh153 comment we could fix the issue by changing app.use(express.json()); to app.use(/^(?!\/api\/files\/upload$)/, express.json()); and add express.json({ limit: '200mb' }) as middleware to the api/files/upload route

Related

How to write Dynamic routes for express server ( nodejs)

I have written crud routes
`
const express = require('express')
const bodyParser = require('body-parser')
const app = express()
const db = require('./user_crud_queries')
const port = 3000
app.use(bodyParser.json())
app.use(
bodyParser.urlencoded({
extended: true,
})
)
app.get('/', (request, response) => {
response.json({
info: 'Node.js, Express, and Postgres API'
})
})
app.get('/users', db.getUsers)
app.get('/users/:id', db.getUserById)
app.post('/users', db.createUser)
app.put('/users/:id', db.updateUser)
app.delete('/users/:id', db.deleteUser)
app.listen(port, () => {
console.log(`App running on port ${port}.`)
})
`
if i have 100 tables do i have to include 100 files? Can it be imported dynamically based on call?
I did create generic crud but i dont want to give that to client because they expect it to be detailed. So 100 tables means 100 separate file for each table. if sever include all 100 tables , will the memory is enough?
So what you can instead do is create a file called users.routes.js/ts and include all of it like this :
import { Router } from "express";
router
.route("/users")
.get(db.getUsers)
.post(db.createUser)
router
.route("/users/:id")
.get('/users/:id', db.getUserById)
.delete('/users/:id', db.deleteUser)
export default router;
and then in the app.js/ts you can user it like this :
import userRouter from "./routes/user.routes";
app.use("/api", userRouter);
So now you can use the following endpoint :
/api/users or /api.users/:id
But yes, if there are 100 tables you will have to use 100 different files for all the CRUD operation that is there, only this will make the code readability easy. Also, you can use different microservice for the same job and reduce the tables for a single server to handle.

express cant handle too many requests?

Im currently working on my new MERN stack project.
Im trying to get information about time spent on every location and everything seemed ok until i' ve discovered a bug. After changing location and sending post request to server about 10 times whole localhost server is frozing. Any calls to the server stop working. This is very frustrating and i cant figure it out. Do you guys have any ideas?
UseEffect calling action:
useEffect(() => {
dispatch(savePageStats())
}, [location])
Redux action call:
export const savePageStats = () => async (dispatch) => {
try{
const arr = []
await api.savePageSession(arr)
}catch(err){
console.log(err)
}
Axios api:
export const savePageSession = (arr) => API.post('/stats/savepagesession', arr)
Express router:
const app = express()
app.use(bodyParser.json({ limit: '30mb', extended: true}))
app.use(bodyParser.urlencoded({ limit: '30mb', extended: true}))
app.use(cors())
app.use('/users', usersRoutes)
app.use('/messages', messagesRoutes)
app.use('/stats', statsRouter)
dotenv.config()
mongoose.connect(process.env.CONNECTION_URL)
.then(() => app.listen(process.env.PORT, () => console.log(`server running on port ${process.env.PORT}`) ))
.catch((err) => console.log(err))
Express controler
export const savePageSession = async (req, res) => {
try{
console.log('im here')
}catch(err){
res.status(500).json({message: 'something went wrong'})
}
}
The savePageSession route handler doesn't send any response back to the client. That means the browser will still be waiting for a response and will eventually stop sending requests to your server until the previous requests finish or eventually timeout.
Add at least something simple like res.send("ok"); to the route. All http request handlers on your server MUST send some kind of response back to the client or it will mess up the client.

Preventing posts from origin other than domain with axios

I'm working on my first website, and am using axios to send post/get requests to the backend. I'm using React on the front-end and node/express on the back-end. I'm wondering if there is a way to prevent posts from a source other than my site.
For example, if I make this exact request through postman I am still be able to post comments, meaning that someone could post with names and ID's other than themselves.
Here is a typical post request made on the front-end:
axios.post('/api/forumActions/postComment', {}, {
params: {
postUserID: this.props.auth.user.id,
name: `${this.props.auth.user.firstName} ${this.props.auth.user.lastName}`,
commentContent: this.state.commentContent,
respondingToPost: this.state.postID,
respondingToComment: this.state.postID
}
})
And here is how it gets processed on the back-end
app.use(
bodyParser.urlencoded({
extended: false
})
);
app.use(bodyParser.json());
app.use(passport.initialize());
require("./config/passport")(passport);
app.post('/postComment', (req, res)=>{
var commentData={
postUserID: req.query.postUserID,
name: req.query.name,
commentContent: req.query.commentContent,
respondingToPost: req.query.respondingToPost,
respondingToComment: req.query,respondingToComment
}
//Write commentData to database
})
const port = process.env.PORT || 80;
const server = app.listen(port, () => console.log(`Server running on port ${port} !`));
I'm wondering if there is anything I can do to ramp up security to prevent post requests being made from anywhere?
You can use cors to accomplish this. This is a pretty good guide on how to configure it, specifically this section. You can configure it for certain routes, or all across the board.
CORS sets the Access-Control-Allow-Origin header, which you can read more about here - it only allows requests from specified origins.
Keep in mind you don't need that package to accomplish this.. you could always build your own middleware for this.
Something like:
app.use((req, res, next) => {
res.header("Access-Control-Allow-Origin", "http://yourdomain.com");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
Within the Express documentation, they provide the following demo code, which you should be able to use as a helper.
Client
Server
You could use a makeshift middleware with special headers.. but then all someone has to do is read your client side source code, or look at the network tab in their browser to figure out which headers you're sending, so then can duplicate them. It would prevent random people from snooping, though..
const express = require('express');
const path = require('path');
const app = express();
const port = process.env.PORT || 3000;
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
// Custom special middleware..
function blockBadHosts({ host, whitelistHeader, whitelistHeaderValue }) {
return (req, res, next) => {
if(req.headers['host'] === host) {
if(whitelistHeader && req.headers[whitelistHeader] === whitelistHeaderValue) {
next();
} else {
res.status(301).send('BAD REQUEST');
}
} else {
res.status(301).send("BAD REQUEST");
}
}
}
// Options for our custom middleware
const badHostOptions = {
host: "localhost:3000",
whitelistHeader: "x-my-special-header", // Request must contain this header..
whitelistHeaderValue: "zoo" // .. with this value
}
// This should succeed
app.get('/success', (req, res) => {
res.status(200).send("from /success");
});
// This should fail even if sent from Postman without correct headers
app.get('/failure', blockBadHosts(badHostOptions), (req, res) => {
res.status(200).send("from /failure");
});
// 404 route
app.use((req, res) => {
res.status(404).send("Uh oh can't find that");
})
app.listen(port, () => {
console.log(`App listening on port: '${port}'`);
});

Express app won't let me check what's inside req

I know my problem may seem not very specific, but I'm having problem describing what's happening because I don't understand it :(
So I've written small express app with ssr (for react) and jwt authentication. The SSR part works nice but the rest is crap.....
import 'babel-polyfill';
import express from 'express';
import bodyParser from 'body-parser';
import logger from 'morgan';
import authRouter from './auth/authRouter';
const app = express();
app.use(express.static('public'));
app.use(logger('dev'));
app.use(bodyParser.json({ type: '*/*' }));
app.use(bodyParser.urlencoded({ extended: false }));
//a lot of irrelevant ssr code
authRouter(app);
app.listen(3000, function(err) {
if (err) {
console.log(err);
}
console.log('main server on port 3000');
});
This is my main server file. My first problem is I don't see ANY console.logs from my files. There's nothing in my terminal. That's the reason I can't see how does my requests look like in my app. I'm testing it using postman like that:
And that's the authRouter I'm using above in main server file:
import express from 'express';
import { signup, signin } from '../controllers/authentication';
import { jwtLogin, localLogin } from '../services/passport';
import passport from 'passport';
passport.use(jwtLogin);
passport.use(localLogin);
//if user is auth'd do not create session for him
const requireAuth = passport.authenticate('jwt', { session: false });
const requireSignin = passport.authenticate('local', { session: false });
const authRouter = function(app) {
app.get('/auth', requireAuth, function(req, res) {
res.send({ hi: 'there' });
});
app.post('/auth/signin', requireSignin, signin); // irrelevant right now
app.post('/auth/signup', signup);
};
export default authRouter;
And that's signup function I'm using in router:
const signup = (req, res, next) => {
console.log('reqqqqqqq', req);
const { email, password, passwordCheck } = req.body; //code crashes here
//some code I deleted for everyones convenience
};
Every request I make my app crashes because req.body is undefined. I can't log req because I can't see any logs. I also tried sending back stringified version of my request body but every time i get "TypeError: Converting circular structure to JSON".
I'll be happy to add any information you may need
EDIT:
I'm gonna check that later at home but now I'm thinking there is something wrong with ssr part of my app because I don't even see that 'main server on port 3000' log..... At the same time server responds with right html, js files and routing works well so.....anyway I'm gonna look it up later
Try using util.inspect from node:
const util = require('util');
// usage in your code
const signup = (req, res, next) => {
console.log(util.inspect(req, { showHidden: true, depth: null }));
const { email, password, passwordCheck } = req.body;
...
};
I solved it.....unexpectedly the problem was lying in my package.json file.
My start script looked like this:
"scripts": {
"start": "nodemon build/server.bundle.js | nodemon build/api.bundle.js",
...
}
And because of that I had these weird errors. Only api.bundle.js file was running correctly......
Anyway thanks for your help ;)

Express route in controller not resolved

I'm using express 4.16.3 and trying to make sense of why one request to a controller works and a request doesn't.
in my server.js i've got the following. There's no semi-colons because used prettier beforehand.
import express from 'express'
import bodyParser from 'body-parser'
import cors from 'cors'
import PriceCheckerController from './controllers/PriceChecker'
import PersonalLibraryController from './controllers/PersonalLibrary'
const app = express()
app.set('port', process.env.PORT || 5000)
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({extended: true}))
app.use(cors())
// routes definition
app.use('/api/books',PersonalLibraryController) // this does not
app.use('/api/stock-prices', PriceCheckerController) // this works
//
app.listen(app.get('port'), error => {
if (error) {
logger.error(`error fcc-isqa:${error}`)
} else {
logger.info(`fcc-isqa is running on port ${app.get('port')}`)
}
})
export default app
In PriceCheckerController i've implemented it like so.
import 'babel-polyfill' // mocha required
import express from 'express'
import logger from '../logger'
const PriceCheckerController = express.Router()
PriceCheckerController.use((req, res, next) => {
logger.info(
`date=>${new Date()}\n method=>${req.method}nsender:${req.ip}`
)
})
PriceCheckerController.get('/', async (req, res) => {
return res.status(200).json({message:'soon'})
})
export default PriceCheckerController
In PersonalLibraryController i've implemented it like so
import 'babel-polyfill'
import express from 'express'
import logger from '../logger'
const PersonalLibraryController = express.Router()
PersonalLibraryController.use((req,res,next)=>{
logger.info(
`library date=>${
new Date()}method=>${req.method}url=>${req.baseUrl}${req.path}`
)
})
PersonalLibraryController.route('/test')
.get(async (req, res) => {
return res.status(200).json({message: 'get soon'})
})
.post(async (req,res)=>{
return res.status(200).json({message: 'post soon'})
})
export default PersonalLibraryController
A request to /api/stock-prices returns ok with message soon.
A request to /api/books/test is logged by the middleware but a response is not sent back. It eventually gives a timeout
Can anyone give me any insights/ help in understanding what is the problem and how to fix it?
Thanks in advance.
Your middleware functions need to call next() to carry on the route execution.
e.g
PriceCheckerController.use((req, res, next) => {
logger.info(
`date=>${new Date()}\n method=>${req.method}nsender:${req.ip}`
)
next();
})
https://expressjs.com/en/guide/using-middleware.html
"If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function. Otherwise, the request will be left hanging."

Resources