I'm still learning the node.js ropes and am just trying to get my head around what I should be deferring, and what I should just be executing.
I know there are other questions relating to this subject generally, but I'm afraid without a more relatable example I'm struggling to 'get it'.
My general understanding is that if the code being executed is non-trivial, then it's probably a good idea to async it, as to avoid it holding up someone else's session. There's clearly more to it than that, and callbacks get mentioned a lot, and I'm not 100% on why you wouldn't just synch everything. I've got some ways to go.
So here's some basic code I've put together in an express.js app:
app.get('/directory', function(req, res) {
process.nextTick(function() {
Item.
find().
sort( 'date-modified' ).
exec( function ( err, items ){
if ( err ) {
return next( err );
}
res.render('directory.ejs', {
items : items
});
});
});
});
Am I right to be using process.nextTick() here? My reasoning is that as it's a database call then some actual work is having to be done, and it's the kind of thing that could slow down active sessions. Or is that wrong?
Secondly, I have a feeling that if I'm deferring the database query then it should be in a callback, and I should have the actual page rendering happening synchronously, on condition of receiving the callback response. I'm only assuming this because it seems like a more common format from some of the examples I've seen - if it's a correct assumption can anyone explain why that's the case?
Thanks!
You are using it wrong in this case, because .exec() is already asynchronous (You can tell by the fact that is accepts a callback as a parameter).
To be fair, most of what needs to be asynchronous in nodejs already is.
As for page rendering, if you require the results from the database to render the page, and those arrive asynchronously, you can't really render the page synchronously.
Generally speaking it's best practice to make everything you can asynchronous rather than relying on synchronous functions ... in most cases that would be something like readFile vs. readFileSync. In your example, you're not doing anything synchronously with i/o. The only synchronous code you have is the logic of your program (which requires CPU and thus has to be synchronous in node) but these are tiny little things by comparison.
I'm not sure what Item is, but if I had to guess what .find().sort() does is build a query string internally to the system. It does not actually run the query (talk to the DB) until .exec is called. .exec takes a callback, so it will communicate with the DB asynchronously. When that communication is done, the callback is called.
Using process.nextTick does nothing in this case. That would just delay the calling of its code until the next event loop which there is no need to do. It has no effect on synchronicity or not.
I don't really understand your second question, but if the rendering of the page depends on the result of the query, you have to defer rendering of the page until the query completes -- you are doing this by rendering in the callback. The rendering itself res.render may not be entirely synchronous either. It depends on the internal mechanism of the library that defines the render function.
In your example, next is not defined. Instead your code should probably look like:
app.get('/directory', function(req, res) {
Item.
find().
sort( 'date-modified' ).
exec(function (err, items) {
if (err) {
console.error(err);
res.status(500).end("Database error");
}
else {
res.render('directory.ejs', {
items : items
});
}
});
});
});
Related
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
I realize that the standard practice for promises in Mongoose is to use exec(), but the following works (or at least appears to) and I want to understand the flow. I'm not against using exec, I'm just exploring this a bit to learn.
In the following Mongoose operation:
let id:string;
SomeDocument.remove({}, (err) => { //clears collection
someDoc = new SomeDocument(data); //creates a new doc for the collection. Id is created here from what I understand.
someDoc.save( (err, result) => { doSomething(); });//doSomething gets called sometime in the future.
id = someDoc._id.toString();
}).then( (result) => {doSomethingElse(id)});//This works - but will doSomethingElse always be called after the first annonymous callback completes?
I get that doSomething() will just get called at some future point - no problem. The question is, will the first callback to the remove call complete prior to doSomethingElse in the then call being called. It seems to be, in that the id is correctly populated in doSomethingElse, but I want to make sure that isn't just a fluke of timing - i.e. I want to know if I can rely on that callback completing prior to the then. I'm using standard ES6 promises (NodeJS.Global.Promise).
The alternative is that maybe then is called after the remove operation completes, but prior to the callback completing (doesn't seem to - but I want to confirm).
Set me straight if I'm explaining this incorrectly.
Yes, as #JaromandaX explained in the comments the order of callbacks is deterministic.
However, it's still bad code, and you should not rely on this behaviour. If you are using promises, don't pass a callback to remove at all; only pass callbacks to then!
SomeDocument.remove({})
.then(() => {
const someDoc = new SomeDocument(data);
someDoc.save().then(doSomething); // doSomething will get called in the future.
return someDoc._id.toString();
//^^^^^^
})
.then(doSomethingElse); // doSomethingElse will get passed the id
doSomethingElse will get called with the result of the previous callback, which is guaranteed to have been completed for that.
I have looked around quite extensivly and could not find any example on how to use query results from the marklogic module inside node.js...
Most examples do a console.log() of results and thats it, but what if I need the query results (say in a JSON array and use these results later on?
Seems I am missing some node.js ascynch stuff here...
Example :
var marklogic = require('marklogic');
var my = require('./my-connection.js');
var db = marklogic.createDatabaseClient(my.connInfo);
var qb = marklogic.queryBuilder;
db.documents.query(
qb.where(qb.parsedFrom('oslo'))
).result( function(results) {
console.log(JSON.stringify(results, null, 2));
});
// I would like to use the results here
// console.log(JSON.stringify(results, null, 2))
Now question is I would like to use the results object later on in this script. I have tried using .then(), or passing it to a variable and returning that variable but no luck.
Regards,
hugo
Simple answer: you need to continue your business logic from the result() callback.
In more detail, your goal is to do something with the result of an asynchronous computation or request. Since JS has no native async capabilities (such as threads), callbacks are typically used to resume an operation asynchronously. The most important thing to realize is that you cannot return the result of an async computation or request, but must resume control flow after it completes. Defining lots of functions can help make this kind of code easier to read and understand.
This example illustrates what's happening:
process.nextTick(function() {
console.log('second')
})
console.log('first')
That program will log first, then second, because process.nextTick() asynchronously invokes the callback function provided to it (on the next turn of the event loop).
The answers at How do I get started with Node.js offer lots of resources to better understand async programming with node.js.
I'm new to javascript, and jumped right into node.js. I've read a lot of theory, and began well with the practical side (I'm writing an API for a mobile app), but I have one basic problem, which has lead me to middleware. I've successfully implemented a middleware function, but I would like to know if the use I'm giving the idea of middleware is OK, and also resolve the original problem which brought me to middleware. My question is two-fold, it's as follows:
1) From what I could gather, the idea of using middleware is repeating a process before actually processing the request. I've used it for token verification, as follows:
Only one of my urls doesn't receive a token parameter, so
app.js
app.get('/settings', auth.validateToken, auth.settings);
auth.js
function validateToken(req, res, next){ //code };
In validateToken, my code checks the token, then calls next() if everything is OK, or modifies res as json to return a specific error code.
My questions regarding this are: a) Is this a correct use of middleware? b) is there a [correct] way of passing a value onto the next function? Instead of calling next only if everything is OK, is there a [correct] way of calling next either way, and knowing from inside the next function (whichever it is), if the middleware was succesful or not? If there is, would this be a proper use of middleware? This precise point brings me to my original problem, and part two of this question, which is encapsulating functions:
THIS PART WAS FIXED, SEE MY SECOND COMMENT.
2) I discovered middleware trying to simply encapsulate validateToken, and be able to call it from inside the functions that the get handlers point to, for example auth.settings.
I'm used to common, sequential programming, and not in javascript, and haven't for the life of me been able to understand how to do this, taking into account the event-based nature of node.js.
What I want to do right now is write a function which simply verifies the user and password. I have it perfectly written inside a particular handler, but was about to copy-paste it to another one, so I stopped. I want to do things the right way from scratch, and understand node.js. One of the specific problems I've been having, is that the error code I have to return when user and password don't match are different depending on the parent function, so I would need this function to be able to tell the callback function "hey, the password and user don't match", so from the parent function I can respond with the correct message.
I think what I actually want is to write an asynchronous function I can call from inside another one.
I hope I've been clear, I've been trying to solve this on my own, but I can't quite finish wrapping my head around what my actual problem is, I'm guessing it's due to my recent introduction to node.js and JS.
Thanks in advance! Jennifer.
1) There is res.locals object (http://expressjs.com/api.html#res.locals) designed to store data local to the request and to pass them from one middleware to another. After request is processed this object is disposed of. If you want to store data within the session you can use req.session.
2) If I understand your question, you want a function asynchronously passing the response to the caller. You can do it in the same way most node's functions are designed.
You define a function in this way:
function doSomething(parameters, callback) {
// ... do something
// if (errorConddition()) err = errorCode();
if (callback) callback(err, result)
}
And the caller instead of using the return value of the function passes callback to this function:
function caller(req, res, next) {
//...
doSomething(params, function(err, result) {
if (! err && result) {
// do something with the result
next();
} else {
// do something else
next();
// or even res.redirect('/error');
}
});
}
If you find yourself writing similar callback functions you should define them as function and just pass the function as parameter:
//...
doSomething(param, processIt);
function processIt(err, result) {
// ...
}
What keeps you confused, probably, is that you don't treat functions as values yet, which is a very specific to JavaScript (not counting for languages that are little used).
In validateToken, my code checks the token, then calls next() if everything is OK, or modifies res as json to return a specific error code.
a) Is this a correct use of middleware?
b) is there a [correct] way of passing a value onto the next function?
Yes that is the correct way of using middleware, although depending on the response message type and specifications you could use the built in error handling of connect. That is in this example generate a 401 status code by calling next({status:401,stack:'Unauthorized'});
The middleware system is designed to handle the request by going through a series of functions until one function replies to the request. This is why the next function only takes one argument which is error
-> if an error object is passed to the next function then it will be used to create a response and no further middleware will be processed. The manner in which error response is created is as follows
// default to 500
if (res.statusCode < 400) res.statusCode = 500;
debug('default %s', res.statusCode);
// respect err.status
if (err.status) res.statusCode = err.status;
// production gets a basic error message
var msg = 'production' == env
? http.STATUS_CODES[res.statusCode]
: err.stack || err.toString();
-> to pass values down the middleware stack modifying the request object is the best method. This ensures that all processing is bound to that specific request and since the request object goes through every middleware function it is a good way to pass information down the stack.
According to the meteor docs, inserts block:
On the server, if you don't provide a callback, then insert blocks
until the database acknowledges the write, or throws an exception if
something went wrong. If you do provide a callback, insert still
returns the ID immediately.
So this would be wrong:
Meteor.methods({
post: function (options) {
return Stories.insert(options)
}
});
I need to do this:
Meteor.methods({
post: function (options) {
return Stories.insert(options, function(){})
}
});
Can somebody confirm that this is the case? The former will block the ENTIRE SERVER until the db returns?
Yeah, it will block, but not the entire server.
In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.
So, if you are worried about that it will block the entire server as it will do in typical Node, don't be.