I have a doubt in my design pattern on my Express app,
so i wrap my controller in try and catch, and the catch method is emitting (req, res) handler from controller and later will be handled by a function that send response back to the client.
the code is more or less like this :
const errorExceptionHandler = fn => (req, res, next) => {
fn(req, res, next).catch((err) => {
emitter.emit('onControllerError', {
err: err,
req: req,
res: res,
next: next
})
})
}
the code above emtting req, res, and next, the default parameters that express provided.
emitter.on('onControllerError', params => {
const err = params.err
const req = params.req
const res = params.res
const next = params.next
if (!res.headerSent) {
res.send({
status: 500,
url: process.env.DEBUG ? req.url : undefined,
message: process.env.DEBUG ? err.message : "Something went wrong!"
})
}
})
and above is how the 'onControllerError' event is handled, my concern is, will this cause trouble later if the traffic goes up? or will it send a wrong response to the client?
Increased traffic wouldn't matter here as each request is still handled independently, plus all the necessary data is being passed directly to the event handler.
So no, based on your code I can't think of any reason why it would start to fail.
Related
Here's my frigo controller :
const fs = require('fs');
const mqtt = require('mqtt');
const transporter = require('../params/mail')
const winston = require('../params/log');
const User = require("../models/User");
const { cli } = require('winston/lib/winston/config');
exports.OpenTheCase = async (req, res) => {};
exports.AddCard = async (req, res) => {};
exports.ShowCurrentTemperature = async (req, res) => {};
exports.ShowCurrentHumidity = async (req, res) => {};
exports.SetAlarm = async (req, res) => {};
exports.AlarmIsOn = async (req, res) => {};
const options = {
clientId: 'backendserver1032',
key: fs.readFileSync('./certs/mqtt_cert/client.key'),
cert: fs.readFileSync('./certs/mqtt_cert/client.crt'),
ca: [ fs.readFileSync('./certs/mqtt_cert/ca.crt') ]
}
const client = mqtt.connect('mqtts://localhost:8883', options);
exports.OpenTheCase = async (req, res) => {
try {
client.publish('RFID', 'RFID_OPEN');
res.status(200).json({ 'case':"opened" });
}
catch(e){
res.status(200).json({ 'state':"something went wrong" });
}
}
exports.AddCard = async (req, res) => {
try {
client.publish('RFID', 'RFID_ADD');
res.status(200).json({ 'card':"will be added" });
}
catch(e){
res.status(200).json({ 'state':"something went wrong" });
}
}
exports.ShowCurrentTemperature = async (req, res) => {
try {
client.subscribe('temperature');
client.on('message', (topic, message, packet) => {
res.status(200).json({ 'temperature': message.toString('ascii') })
client.unsubscribe('temperature')
})
}
catch(e){
res.status(200).json({ 'state':"something went wrong" });
}
return
}
exports.ShowCurrentHumidity = async (req, res) => {
try {
client.subscribe('humidity');
client.on('message', (topic, message) => {
res.status(200).json({"temperature": message.toString('ascii')});
client.unsubscribe('humidity')
});
}
catch(e){
res.status(200).json({ 'state':"something went wrong" });
}
return
}
The problem is : when I try to get "ShowCurrentTemperature", it works once and after it. It says that the http header was already send.
Here's my route :
router.get("/frigo/Temperature",auth.verifyToken, frigoController.ShowCurrentTemperature)
I really thank you.
I had try several things, like adding return or trying to end the connection but none of them works.
I'm getting out of idea. If someone can help me through this.
This design is never going to work.
You are basically leaking client.on('message',...) handlers. Every call to the HTTP endpoint adds a new handler which holds a reference to it's local res object.
Calling unsubscribe() does not remove the message handler, so the instant you call subscribe() again in either of the HTTP routes the very next message to arrive will be delivered to ALL the old handlers which will ALL try to call res.send() on a already complete HTTP transaction.
You are trying to map an inherently asynchronous protocol (MQTT) into a synchronous HTTP request, which is a really bad idea.
You may be able to get it to work by swapping all the client.on('message', ...) calls to client.once('message', ....) so the handlers only fire once, but this is a truly UGLY hack. (EDIT: On reflection, this still has a race condition where you may end up handling the wrong message if both HTTP endpoints are called too close together, so I stand by my first statement that this design can never work properly)
The right thing to do is to run the whole MQTT client totally independently of the HTTP requests and just have the single background client.on('message',...) handler update some global variables with the latest temperature and humidity that the HTTP routes can just return the latest value, rather than try and grab the next published message.
In a nutshell, this error happens when you try to send more than one response using res. This could happen in your code when there is an exception during client.unsubscribe('temperature'), because then the execution will flow into the catch where you are sending another response. You can avoid that by moving the unsubscribe after the try-catch:
exports.ShowCurrentTemperature = async(req, res) => {
try {
client.subscribe('temperature');
client.on('message', (topic, message, packet) => {
res.status(200).json({
'temperature': message.toString('ascii')
})
})
} catch (e) {
res.status(200).json({
'state': "something went wrong"
});
}
client.unsubscribe('temperature')
}
Update:
Actually, the more likely explanation is that you receive more than one message before you unsubscribe, and hence the first res.. is executed multiple times. So probably best to unsubscribe before sending a response. Since more handlers could queue up in the meantime, though, you probably need to add a guard as well to make sure you never send more than one response:
exports.ShowCurrentTemperature = async(req, res) => {
let done = false;
try {
client.subscribe('temperature');
client.on('message', (topic, message, packet) => {
client.unsubscribe('temperature')
!done && res.status(200).json({
'temperature': message.toString('ascii')
})
done = true;
})
} catch (e) {
!done && res.status(200).json({
'state': "something went wrong"
});
}
}
BTW, you could also just use mqtt directly on the client, which would be more elegant, if that's fine from an authorization perspective.
I am reading a code that has two files like below:
first file that uses the currentuser middleware:
const router = express.Router();
router.get("/api/users/currentuser", currentUser, (req, res) => {
res.send({ currentUser: req.currentUser || null });
});
export { router as currentUserRouter };
Second file that defines the middleware:
interface UserPayload {
id: string;
email: string;
}
declare global {
namespace Express {
interface Request {
currentUser?: UserPayload;
}
}
}
export const currentUser = (
req: Request,
res: Response,
next: NextFunction
) => {
if (!req.session?.jwt) {
return next();
}
try {
const payload = jwt.verify(
req.session.jwt,
process.env.JWT_KEY!
) as UserPayload;
req.currentUser = payload;
} catch (err) {}
next();
};
I understand that if there is a verified jwt token, the middleware will take the the payload out of it and add it to the req object. But what if it fails and it can't add the payload/current user to the req? What would happen for the following request and what will the res object look like?
router.get("/api/users/currentuser", currentUser, (req, res) => {
res.send({ currentUser: req.currentUser || null });
});
Could you edit this get request to show how can I catch the probable error if I am not the writer of the middleware?
If you had a catchall exception handler, and your middleware threw an exception, you would determine the response.
If your middleware threw an exception and you did not catch it, the system might just exit the process.
If your middleware did not throw an exception, and did not call next(), and did not respond, the request would hang.
If your middleware returned a response, and did not call next(), your send function would never get invoked.
The bottom line is that you need to dump the response on your server and see exactly how your middleware handles this.
In most of my auth middleware, I choose to not call next(), and return a 403 error. But there are some benefits by throwing an exception, then returning a 403 from a catchall handler.
You need to respond with an error HTTP status code, and an error message in the body. The exact status and message depends on the type of the exception and its parameters, so you need to catch it and check it.
The current express middleware does not handle errors, it just does not set the req.currentUser = payload;, so you won't know about the user. I don't think this is a proper solution for an authentication error.
In the documentation you can see how error are handled:
https://expressjs.com/en/guide/using-middleware.html
app.use((err, req, res, next) => {
console.error(err.stack)
res.status(500).send('Something broke!')
})
So I would rewrite the code and if the JWT verification fails, then I return for example 401 unauthorized. https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401
I guess you are using this JWT library: https://github.com/auth0/node-jsonwebtoken According to the docs and the code there are 3 types of errors: TokenExpiredError, JsonWebTokenError, NotBeforeError for verify. Here you can check when they are thrown: https://github.com/auth0/node-jsonwebtoken/blob/master/verify.js , here are their definitions: https://github.com/auth0/node-jsonwebtoken/tree/master/lib
So in the catch block you just check the type of the error with instanceof e.g. if (err instanceof jwt.JsonWebTokenError) ... and send the message accordingly with the res.status(401) and put the next() to the end of the try block, because it should be called only if the verification does not fail.
From inside my expressJS application I have to verify that a cookie token is valid with a back-end server. So the relevant code involved in this is as follows:
app.get('*', (req, res, next) => {
console.log('GET: ' + req.path);
// ...
const payload = JSON.stringify({ authnToken: token });
const opts = { ... authServerOptions };
opts.headers['Content-Length'] = payload.length;
// build request
const restReq = https.request(authServerOptions, result => {
console.log('back-end response' + result.statusCode);
result.on('data', data => {
next(); // token is good now proceed.
});
result.on('error', error => {
res.redirect('somewhere'); // token is bad or timeout
});
});
restReq.write(token);
restReq.end();
}
So the main get function sets the REST request in motion and then just returns without calling next() or anything.
Questions:
Is this the right code for doing this? What happens if the callbacks are never called?
Is the application blocked from processing other requests until the back-end server returns or times out?
If so is there some way of freeing up the thread to process more requests?
Thanks in advance for any help. I haven't found many examples for this code pattern so if there is one a link would be appreciated.
Yes, I think the general idea of your implementation is correct.
I would also suggest, as done in the comments, to use a client such as axios to handle the request in a less verbose and more comprehensive manner, which would leave your code looking something like this:
const axios = require('axios');
app.get('*', (req, res, next) => {
const payload = JSON.stringify({ authnToken: token });
const opts = { ... authServerOptions };
opts.headers['Content-Length'] = payload.length;
axios.post(url, payload, opts)
.then(response => next())
.catch(error => {
console.error(error);
res.redirect('somewhere');
});
});
A bit more to the point, but functionally almost equivalent to your implementation. The one thing you are missing is the onerror callback for your request object, which currently may fail and never return a response as you correctly suspected. You should add:
restReq.on('error', error => {
console.error(error);
res.redirect('somewhere');
});
On the same vein, it would probably be more fitting to call next on result end, instead of doing so while reading response data:
result.on('end', () => {
next();
});
Then you'd be covered to guarantee that a callback would be invoked.
Neither implementation blocks the processing of future requests, as the call to the token validation service is done asynchronously in both cases.
I am rather new with express together with the request-promise module,
and need to create a service S
that is called from serverA
and after S has asked ServerB for some additional info,
it redirects the request of serverA to ServerC.
Since I get a
Error: Can't set headers after they are sent.
even though I do not add something by myself, I wonder someone could help me to get this workflow straight.
This is the code:
`
const express = require('express')
const rp = require('request-promise')
...
app.get('/dispatch', cors(), (req, res, next) => {
var options = {
uri: 'https://ServerB/calc-something..',
headers: {
'User-Agent': 'its-me',
'Data': data_from_serverA
},
resolveWithFullResponse: true, // Get statuscode
json: true // Parse the JSON string in the response
};
rp(options) // Do request to serverB
.then(function (response) {
console.log(`ServerB responded with statuscode ${response.statusCode}`)
// No error, so redirect original res
res.redirect('https://serverC/...') // error occurs here
return next(response)
})
.catch(function (err) {
console.log(`ServerB responded with error ${err}`)
return next(err) // send 500 to serverA
})
})
`
Your cors() middleware is setting CORS headers. This is causing the headers to be sent while your promise is resolving.
A redirect ALSO sends headers, and this is the issue. A redirect sets a location header, but you've already sent the headers so that won't work.
The solution is to split your final middleware into two. First, check to see if a redirect is needed and if so, do that. Otherwise, set whatever data you need on the req object and handle this AFTER the cors call.
Your final route will look something like:
app.get('/dispatch', checkRedirect, cors(), (req, res, next) => {
//do something useful, or send your error
})
The contents of your checkRedirect function will be pretty similar to what you have above. However, you do not pass data to the next() function. That just passes control to the next middleware. Instead, put any data you need on the req object and handle it in the final middleware, AFTER cors. If all you are doing is setting a 500 error, you don't even need CORS for that.
According to #Rampant 's answer,
this is how I did it with request-promise (rp):
function checkPrecondition(req, res, next){
req.precondition = false
rp({ method: 'POST',
...
})
.then((data) => {
...
req.precondition = true
next()
})
.catch((data) => {
...
next()
})
}
and in the express handler:
app.post('/query', checkPrecondition, cors(), (req, res, next) => {
if(!req.precondition){
res.status(400).send(JSON.stringify({status: 'insufficient'}))
return
}
res.redirect('target.host')
})
Thanks for clearifying the CORS issue.
I need to modify the request body asynchronously. Something along the lines of this:
proxy.on('proxyReq', function(proxyReq, req, res, options) {
if(req.body) {
new Promise(function(resolve){
setTimeout(function() { // wait for the db to return
'use strict';
req.body.text += 'test';
let bodyData = JSON.stringify(req.body);
proxyReq.setHeader('Content-Type','application/json');
proxyReq.setHeader('Content-Length', Buffer.byteLength(bodyData));
// stream the content
proxyReq.write(bodyData);
resolve();
},1);
});
}
});
When I run this I get the error saying cannot modfiy headers once they have been set. Which makes sense.
How can I halt the sending of the request until I'm ready? I've looked at removing various listeners from proxyReq without success..
By looking at the source code #-) it seems like it's not really possible because the proxyReq event is sent and then the code moves on.
If it would instead wait for a promise, it would be possible (if you'd return that promise as well).
A minimal fork on this lib could be for example:
// Enable developers to modify the proxyReq before headers are sent
proxyReq.on('socket', function(socket) {
if(server) { server.emit('proxyReq', proxyReq, req, res, options); }
});
(proxyReq.proxyWait || Promise.resolve())
.then( ... // rest of the code inside the callback
And then
proxy.on('proxyReq', function(proxyReq, req, res, options) {
if(req.body) {
proxyReq.proxyWait = new Promise(function(resolve){
setTimeout(function() { ...
But depending on your use case, there might be other solutions as well. For example, consider if it's really necessary that you use this proxy library. It You could alternatively use http directly, where you have all the control on the events and callbacks.
You can set selfHandleResponse: true inside the HttpProxy.createProxyServer. This then allows (and forces) you to handle the proxyRes manually!
const proxy = HttpProxy.createProxyServer({selfHandleResponse: true});
proxy.on('proxyRes', async (proxyReq, req, res, options) => {
if (proxyReq.statusCode === 404) {
req.logger.debug('Proxy Request Returned 404');
const something = await doSomething(proxyReq);
return res.json(something);
}
return x;// return original proxy response
});
I came here looking for the solution to a slightly different problem: Modifying the request headers (not body) before proxying.
I post this here in case that it is helpful to others. And maybe the code can be adapted to also modify the request body.
const http = require('http');
const httpProxy = require('http-proxy');
var proxy = httpProxy.createProxyServer({});
var server = http.createServer(function(req, res) {
console.log(`${req.url} - sleeping 1s...`);
setTimeout(() => {
console.log(`${req.url} - processing request`);
req.headers['x-example-req-async'] = '456';
proxy.web(req, res, {
target: 'http://127.0.0.1:80'
});
}, 1000);
});
server.listen(5050);