I'm working on a homemade RBAC system with Nodejs Express, based on two levels:
First, verify if the user has the right role to perform this action.
Second, verify if the user has the right plan to perform this action.
, I create a middleware like this:
exports.can = function (resource, action) {
return function (request, response, next) {
action = action || request.method;
if (!request.user) {
return next(new errors.UnauthorizedError());
}
request.user.can(request.user.role, request.user.currentPack, resource, action, function (can, error) {
if (error) return next(error);
if (!can) {
return next(new errors.UnauthorizedError());
}
return can;
});
return next();
};
};
I added to my User model this method:
const rbac = new RBAC(rbacJson);
const pack = new PACK(packJson);
schema.method('can', function (role, userPack, resource, action, next) {
let can = false;
action = action.toUpperCase();
can = rbac.can(role, resource, action);
if (can) {
can = pack.can(userPack, resource, action, function (can) {
return next(can);
});
}
return next(can);
});
In my method pack.can(...) I need to execute a mongoose query like this:
PACK.prototype.can = function (pack, resource, action, next) {
let can = true;
// some sequantial code
Trader.count({/* some conditions */}, function (err, count) {
if(count == 0) return next(true);
return next(false);
});
return can;
};
My problem is when the return of Mongoose query is next(false), I have this error:
Error: Can't set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:356:11)
at ServerResponse.header (/home/invoice/node_modules/express/lib/response.js:730:10)
at ServerResponse.send (/home/invoice/node_modules/express/lib/response.js:170:12)
at ServerResponse.json (/home/invoice/node_modules/express/lib/response.js:256:15)
at ServerResponse.response.apiResponse (/home/invoice/server/config/middlewares/api.js:10:14)
at /home/invoice/server/controllers/api/invoice/traders.js:130:21
at /home/invoice/node_modules/mongoose/lib/model.js:3835:16
at /home/invoice/node_modules/mongoose/lib/services/model/applyHooks.js:162:20
at _combinedTickCallback (internal/process/next_tick.js:73:7)
at process._tickDomainCallback (internal/process/next_tick.js:128:9)
After an investigation, I found that the error is maybe due to the double callback Call:
can = pack.can(userPack, resource, action, function (can) {
return next(can);
});
return next(new errors.UnauthorizedError());
But I don't know how to solve this.
I hope that I well explained my problem.
So let's start with the error:
Can't set headers after they are sent.
Nine times out of ten this is caused by trying to send two responses to the same request. The reference to headers is a little misleading, though technically true. The first thing the second response will try to do is set some headers, which will fail because the first response has already sent them back to the client.
The stack trace gives you a clear indication of where the second response originates, including filenames and line numbers. More difficult is to track down the first response, generally I'd just throw in some extra console logging to figure it out.
In the question you mention that you believe you've found the source of the problem but it looks to me like it might just be the tip of the iceberg. You've repeatedly used the same pattern and even if you fix it in one place that may not be sufficient.
Before I get into that, let's start with this:
return next();
For the purposes of this example it doesn't really matter whether you pass an error, e.g. return next(err);, the point is the same. First it invokes next(), which returns undefined. It then returns undefined from the surrounding function. In other words, it's just a convenient shorthand for this:
next();
return;
The reason why we return is to ensure that nothing else happens after we've called next(), we always try to ensure that calling next() is the last thing we do in our handler, not least because otherwise we can have bugs where we try to send the response twice.
The (anti-)pattern you've used looks a bit like this:
obj.doSomething(function() {
return next(); // next 1
});
return next(); // next 2
Again, it doesn't really matter whether you're calling next() or next(err), it all works out much the same. The key thing to note is that the return for next 1 is just returning from the function passed to doSomething. It isn't doing anything to prevent next 2 from being hit. Both next 1 and next 2 will be called.
In places your code seems unclear about whether it's trying to be synchronous or asynchronous, using callbacks and return values at the same time. It makes it a little bit difficult to say for sure what the 'correct' code should look like. Specifically, is the value of can supposed to be returned synchronously or passed to the callback asynchronously? I suspect it's the latter but the current code seems torn between the two. It's important to ensure that you don't call next() until you're ready for the next thing to happen, so if you're waiting on a DB query you mustn't call next until that comes back.
Personally I'd rename your callbacks so they aren't all called next, I find that really confusing. When I see next I'm expecting it to be an Express next function, not just an arbitrary callback.
It's a bit of a guess but I'd suggest your middleware should look something like this:
exports.can = function (resource, action) {
return function (request, response, next) {
action = action || request.method;
if (!request.user) {
return next(new errors.UnauthorizedError());
}
request.user.can(request.user.role, request.user.currentPack, resource, action, function (can, error) {
if (error) {
next(error);
}
else if (can) {
next();
}
else {
next(new errors.UnauthorizedError());
}
});
// Do not call next() here
};
};
The relevant section of the User model would then be:
if (can) {
pack.can(userPack, resource, action, function (can) {
next(can);
});
}
else {
next(can);
}
Related
I'm writing an API where I'm having a bit of trouble with the error handling. What I'm unsure about is whether the first code snippet is sufficient or if I should mix it with promises as in the second code snippet. Any help would be much appreciated!
try {
var decoded = jwt.verify(req.params.token, config.keys.secret);
var user = await models.user.findById(decoded.userId);
user.active = true;
await user.save();
res.status(201).json({user, 'stuff': decoded.jti});
} catch (error) {
next(error);
}
Second code snippet:
try {
var decoded = jwt.verify(req.params.token, config.keys.secret);
var user = models.user.findById(decoded.userId).then(() => {
}).catch((error) => {
});
user.active = true;
await user.save().then(() => {
}).catch((error) => {
})
res.status(201).json({user, 'stuff': decoded.jti});
} catch (error) {
next(error);
}
The answer is: it depends.
Catch every error
Makes sense if you want to react differently on every error.
e.g.:
try {
let decoded;
try {
decoded = jwt.verify(req.params.token, config.keys.secret);
} catch (error) {
return response
.status(401)
.json({ error: 'Unauthorized..' });
}
...
However, the code can get quite messy, and you'd want to split the error handling a bit differently (e.g.: do the JWT validation on some pre request hook and allow only valid requests to the handlers and/or do the findById and save part in a service, and throw once per operation).
You might want to throw a 404 if no entity was found with the given ID.
Catch all at once
If you want to react in the same way if a) or b) or c) goes wrong, then the first example looks just fine.
a) var decoded = jwt.verify(req.params.token, config.keys.secret);
b) var user = await models.user.findById(decoded.userId);
user.active = true;
c) await user.save();
res.status(201).json({user, 'stuff': decoded.jti});
I read some articles that suggested the need of a try/catch block for each request. Is there any truth to that?
No, that is not required. try/catch with await works conceptually like try/catch works with regular synchronous exceptions. If you just want to handle all errors in one place and want all your code to just abort to one error handler no matter where the error occurs and don't need to catch one specific error so you can do something special for that particular error, then a single try/catch is all you need.
But, if you need to handle one particular error specifically, perhaps even allowing the rest of the code to continue, then you may need a more local error handler which can be either a local try/catch or a .catch() on the local asynchronous operation that returns a promise.
or if I should mix it with promises as in the second code snippet.
The phrasing of this suggests that you may not quite understand what is going on with await because promises are involved in both your code blocks.
In both your code blocks models.user.findById(decoded.userId); returns a promise. You have two ways you can use that promise.
You can use await with it to "pause" the internal execution of the function until that promise resolves or rejects.
You can use .then() or .catch() to see when the promise resolves or rejects.
Both are using the promise returns from your models.user.findById(decoded.userId); function call. So, your phrasing would have been better to say "or if I should use a local .catch() handler on a specific promise rather than catching all the rejections in one place.
Doing this:
// skip second async operation if there's an error in the first one
async function someFunc() {
try {
let a = await someFunc():
let b = await someFunc2(a);
return b + something;
} catch(e) {
return "";
}
}
Is analogous to chaining your promise with one .catch() handler at the end:
// skip second async operation if there's an error in the first one
function someFunc() {
return someFunc().then(someFunc2).catch(e => "");
}
No matter which async function rejects, the same error handler is applied. If the first one rejects, the second one is not executed as flow goes directly to the error handler. This is perfectly fine IF that's how you want the flow to go when there's an error in the first asynchronous operation.
But, suppose you wanted an error in the first function to be turned into a default value so that the second asynchronous operation is always executed. Then, this flow of control would not be able to accomplish that. Instead, you'd have to capture the first error right at the source so you could supply the default value and continue processing with the second asynchronous operation:
// always run second async operation, supply default value if error in the first
async function someFunc() {
let a;
try {
a = await someFunc():
} catch(e) {
a = myDefaultValue;
}
try {
let b = await someFunc2(a);
return b + something;
} catch(e) {
return "";
}
}
Is analogous to chaining your promise with one .catch() handler at the end:
// always run second async operation, supply default value if error in the first
function someFunc() {
return someFunc()
.catch(err => myDefaultValue)
.then(someFunc2)
.catch(e => "");
}
Note: This is an example that never rejects the promise that someFunc() returns, but rather supplies a default value (empty string in this example) rather than reject to show you the different ways of handling errors in this function. That is certainly not required. In many cases, just returning the rejected promise is the right thing and that caller can then decide what to do with the rejection error.
I'm slowly working my way through Node.js. I've got a basic iOS app (swift) that pushes a message to Firestore and I'm using Node to send a notification when the database is updated.
exports.updateRequestToJoin = functions.firestore
.document('/chats/{chatId}')
.onUpdate(event => {
if(userId != sentBy)
{
return db.collection('users').doc(userId).get().then(doc => {
var payload = {
notification:{
title: "msg",
body: "send msg"
}
};
admin.messaging().sendToDevice(fcm_token, payload)
.then(function(response) {
// See the MessagingDevicesResponse reference documentation for
// the contents of response.
console.log("Successfully sent message:", response);
})
.catch(function(error) {
console.log("Error sending message:", error);
});
};
return 0
});
Initially, I just had return at the end of the function but was getting the error "Function returned undefined, expected Promise or value" until I used return 0 and that seemed to silence the error.
The things I was unsure about are:
Is returning a value like 0 ok practice?
When the error says "expected Promise or value" does the promise refer to the .then?
In the if statement, I return the db.collection - is that necessary to return that or can I skip the return keyword?
Thanks.
You have good questions here. The place to start is with the function caller. You are exporting updateRequestToJoin(). What is the caller expecting? Are you thinking in terms of exiting the function with a success or failure code? Node apps tend to work differently than scripting environments that return only boolean values. The whole node ethos is about supporting a single thread environment that is happy to execute asynchronously. So, node functions tend either to return Promises, with their built-in resolve or reject methods; or they run a callback.
If you want to merely return a success code in your then statement, or a failure code in your catch statement, you could something as simple as this:
if(userId != sentBy)
{
db.collection('users').doc(userId).get().then(doc => {
var payload = {
notification:{
title: "msg",
body: "send msg"
}
};
admin.messaging().sendToDevice(fcm_token, payload)
.then(function(response) {
// See the MessagingDevicesResponse reference documentation for
// the contents of response.
console.log("Successfully sent message:", response);
return true;
})
.catch(function(error) {
console.log("Error sending message:", error);
return false;
});
};
The success or failure is returned in the then/catch of your admin.messaging chain.
That's sort of the answer to your first question and a partial answer to your second. A more complete answer to your second question requires a bit of reading. Promise is a native Javascript object that gives you convenience methods to handle asynchronous execution. The db.collection chain you are calling, and the admin.messaging chain are Promises (or some sort of thenable construct) that are returning a resolve in the then, and a reject in the catch (you are not using the catch side of the db.collection call, but you are using the catch side of the admin.messaging call).
Finally, as to whether you need to return the db.collection chain, you would do this if the db.collection call was a part of a Promise.all execution of an array of function calls. In this case, however, it looks like you want to simply return a success or failure code, based on the success or failure of your admin.messaging call.
I am making some assumptions about your goal here. Hopefully, at least, this moves the discussion toward clarity.
router.post("/application_action", function(req,res){
var Employee = req.body.Employee;
var conn = new jsforce.Connection({
oauth2 : salesforce_credential.oauth2
});
var username = salesforce_credential.username;
var password = salesforce_credential.password;
conn.login(username, password, function(err, userInfo, next) {
if (err) { return console.error(err); res.json(false);}
// I want this conn.query to execute first and then conn.sobject
conn.query("SELECT id FROM SFDC_Employee__c WHERE Auth0_Id__c = '" + req.user.id + "'" , function(err, result) {
if (err) { return console.error(err); }
Employee["Id"] = result.records[0].Id;
});
//I want this to execute after the execution of above query i.e. conn.query
conn.sobject("SFDC_Emp__c").update(Employee, function(err, ret) {
if (err || !ret.success) { return console.error(err, ret);}
console.log('Updated Successfully : ' + ret.id);
});
});
I have provided my code above. I need to modify Employee in the conn.query and use it in conn.sobject. I need to make sure that my first query executes before 2nd because I am getting value from 1st and using in the 2nd. Please do let me know if you know how to accomplish this.
New Answer Based on Edit to Question
To execute one query based on the results of the other, you put the second query inside the completion callback of the first like this:
router.post("/application_action", function (req, res) {
var Employee = req.body.Employee;
var conn = new jsforce.Connection({
oauth2: salesforce_credential.oauth2
});
var username = salesforce_credential.username;
var password = salesforce_credential.password;
conn.login(username, password, function (err, userInfo, next) {
if (err) {
return console.error(err);
res.json(false);
}
// I want this conn.query to execute first and then conn.sobject
conn.query("SELECT id FROM SFDC_Employee__c WHERE Auth0_Id__c = '" + req.user.id + "'", function (err, result) {
if (err) {
return console.error(err);
}
Employee["Id"] = result.records[0].Id;
//I want this to execute after the execution of above query i.e. conn.query
conn.sobject("SFDC_Emp__c").update(Employee, function (err, ret) {
if (err || !ret.success) {
return console.error(err, ret);
}
console.log('Updated Successfully : ' + ret.id);
});
});
});
});
The only place that the first query results are valid is inside that callback because otherwise, you have no way of knowing when those asynchronous results are actually available and valid.
Please note that your error handling is unfinished since you don't finish the response in any of the error conditions and even in the success case, you have not yet actually sent a response to finish the request.
Original Answer
First off, your code shows a route handler, not middleware. So, if you really intend to ask about middleware, you will have to show your actual middleware. Middleware that does not end the request needs to declare next as an argument and then call it when it is done with it's processing. That's how processing continues after the middleware.
Secondly, your console.log() statements are all going to show undefined because they execute BEFORE the conn.query() callback that contains the code that sets those variables.
conn.query() is an asynchronous operation. It calls its callback sometime IN THE FUTURE. Meanwhile, your console.log() statements execute immediately.
You can see the results of the console.log() by putting the statements inside the conn.query() callback, but that is probably only part of your problem. If you explain what you're really trying to accomplish, then we could probably help with a complete solution. Right now, you're just asking questions about flawed code, but not explaining the higher level problem you're trying to solve so you're making it hard for us to give you the best answer to your actual problem.
FYI:
app.locals - properties scoped to your app, available to all request handlers.
res.locals - properties scoped to a specific request, available only to middleware or request handlers involved in processing this specific request/response.
req.locals - I can't find any documentation on this in Express or HTTP module. There is discussion of this as basically serving the same purpose as res.locals, though it is not documented.
Other relevants answers:
req.locals vs. res.locals vs. res.data vs. req.data vs. app.locals in Express middleware
Express.js: app.locals vs req.locals vs req.session
You miss the basics of the asynchronous flow in javascript. All the callbacks are set to the end of event loop, so the callback of the conn.query will be executed after console.logs from the outside. Here is a good article where the the basic concepts of asynchronous programming in JavaScript are explained.
I'm writing a JavaScript function that makes an HTTP request and returns a promise for the result (but this question applies equally for a callback-based implementation).
If I know immediately that the arguments supplied for the function are invalid, should the function throw synchronously, or should it return a rejected promise (or, if you prefer, invoke callback with an Error instance)?
How important is it that an async function should always behave in an async manner, particularly for error conditions? Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
e.g:
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
throw new Error('userId is not valid')
}
// make async call
}
// OR...
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
return cb(new Error('userId is not valid'))
}
// make async call
}
Ultimately the decision to synchronously throw or not is up to you, and you will likely find people who argue either side. The important thing is to document the behavior and maintain consistency in the behavior.
My opinion on the matter is that your second option - passing the error into the callback - seems more elegant. Otherwise you end up with code that looks like this:
try {
getUserById(7, function (response) {
if (response.isSuccess) {
//Success case
} else {
//Failure case
}
});
} catch (error) {
//Other failure case
}
The control flow here is slightly confusing.
It seems like it would be better to have a single if / else if / else structure in the callback and forgo the surrounding try / catch.
This is largely a matter of opinion. Whatever you do, do it consistently, and document it clearly.
One objective piece of information I can give you is that this was the subject of much discussion in the design of JavaScript's async functions, which as you may know implicitly return promises for their work. You may also know that the part of an async function prior to the first await or return is synchronous; it only becomes asynchronous at the point it awaits or returns.
TC39 decided in the end that even errors thrown in the synchronous part of an async function should reject its promise rather than raising a synchronous error. For example:
async function someAsyncStuff() {
return 21;
}
async function example() {
console.log("synchronous part of function");
throw new Error("failed");
const x = await someAsyncStuff();
return x * 2;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
There you can see that even though throw new Error("failed") is in the synchronous part of the function, it rejects the promise rather than raising a synchronous error.
That's true even for things that happen before the first statement in the function body, such as determining the default value for a missing function parameter:
async function someAsyncStuff() {
return 21;
}
async function example(p = blah()) {
console.log("synchronous part of function");
throw new Error("failed");
const x = await Promise.resolve(42);
return x;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
That fails because it tries to call blah, which doesn't exist, when it runs the code to get the default value for the p parameter I didn't supply in the call. As you can see, even that rejects the promise rather than throwing a synchronous error.
TC39 could have gone the other way, and had the synchronous part raise a synchronous error, like this non-async function does:
async function someAsyncStuff() {
return 21;
}
function example() {
console.log("synchronous part of function");
throw new Error("failed");
return someAsyncStuff().then(x => x * 2);
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
But they decided, after discussion, on consistent promise rejection instead.
So that's one concrete piece of information to consider in your decision about how you should handle this in your own non-async functions that do asynchronous work.
How important is it that an async function should always behave in an async manner, particularly for error conditions?
Very important.
Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
Yes, I personally think it is OK when that is a very different error from any asynchronously produced ones, and needs to be handled separately anyway.
If some userids are known to be invalid because they're not numeric, and some are will be rejected on the server (eg because they're already taken) you should consistently make an (async!) callback for both cases. If the async errors would only arise from network problems etc, you might signal them differently.
You always may throw when an "unexpected" error arises. If you demand valid userids, you might throw on invalid ones. If you want to anticipate invalid ones and expect the caller to handle them, you should use a "unified" error route which would be the callback/rejected promise for an async function.
And to repeat #Timothy: You should always document the behavior and maintain consistency in the behavior.
Callback APIs ideally shouldn't throw but they do throw because it's very hard to avoid since you have to have try catch literally everywhere. Remember that throwing error explicitly by throw is not required for a function to throw. Another thing that adds to this is that the user callback can easily throw too, for example calling JSON.parse without try catch.
So this is what the code would look like that behaves according to these ideals:
readFile("file.json", function(err, val) {
if (err) {
console.error("unable to read file");
}
else {
try {
val = JSON.parse(val);
console.log(val.success);
}
catch(e) {
console.error("invalid json in file");
}
}
});
Having to use 2 different error handling mechanisms is really inconvenient, so if you don't want your program to be a fragile house of cards (by not writing any try catch ever) you should use promises which unify all exception handling under a single mechanism:
readFile("file.json").then(JSON.parse).then(function(val) {
console.log(val.success);
})
.catch(SyntaxError, function(e) {
console.error("invalid json in file");
})
.catch(function(e){
console.error("unable to read file")
})
Ideally you would have a multi-layer architecture like controllers, services, etc. If you do validations in services, throw immediately and have a catch block in your controller to catch the error format it and send an appropriate http error code. This way you can centralize all bad request handling logic. If you handle each case youll end up writing more code. But thats just how I would do it. Depends on your use case
I try to understand how I can stop or exit on error in a Nodejs route.
In the code below, I check if header UID is sent and if the fiels group is also sent.
The problem is that nodejs continues to execute the rest of the code even if I use res.end () + return; I wish that Node Js stop everything when I display an error. Perhaps because I do not know much in Node Js I myself take it badly and I have to work otherwise. Can you explain to me and give me an example of how I should do?
var uid;
var group;
if (typeof req.headers['uid'] == 'undefined' || req.headers['uid'] == '') {
res.status(404);
res.json('user_id not set');
res.end();
return;
}
else
{
uid = req.headers['uid'];
User.find({_id:uid}).exec(function(err, data)
{
if(err){
res.status(404);
res.json('user not found');
res.end();
return;
}
});
}
if (typeof req.body.group == 'undefined' || req.body.group == '') {
res.status(500);
res.json('group not defined');
res.end();
return;
}
else
{
group = req.body.group;
}
it is unclear exactly what you mean by "node js stop everything". If you want to stop the entire nodejs process, you can use process.exit().
If you're trying to keep some code after your error from executing and your error occurs in an async callback, then you can't do that. The other code has already executed.
If you want to serialize asynchronous operations so that you complete one async operation BEFORE you decide whether to start the next operation, then you will need to code that differently. You will need to execute the second block of code from within the completion callback of the first async operation.
One aspect of your code that you may not understand is that this block is asynchronous:
User.find({_id:uid}).exec(function(err, data)
{
if(err){
res.status(404);
res.json('user not found');
res.end();
return;
}
});
The callback you pass to .exec() is called sometime LATER. Meanwhile, the rest of your JS has already executed. In addition, when you do a return from within that callback that doesn't return from your outer function, it only returns from that callback function back into the bowels of .exec(). It stops any more of the callback from executing, but has no effect at all on the outer function because that outer function.
So, if you want the .find() operation to finish before you execute the rest of your code in that function, you have to put that code inside the callback function.