it might be obvious, but right now I'm not able to either find it in the docs or google it...
I'm using mongodb with the nodejs-driver and have a potentially long operation (> 10 minutes) pertaining to a cursor which does get a timeout (as specified in http://docs.mongodb.org/manual/core/cursors/#cursor-behaviors).
In the nodejs-driver API Documentation (http://mongodb.github.io/node-mongodb-native/2.0/api/Cursor.html) a method addCursorFlag(flag, value) is mentioned to be called on a Cursor.
However, there's no example on how to do it, and simply calling e.g.
objectCollection.find().limit(objectCount).addCursorFlag('noCursorTimeout', true).toArray(function (err, objects) {
...
}
leads to a TypeError: Object #<Cursor> has no method 'addCursorFlag'.
So how to go about making this Cursor exist longer than those 10 minutes?
Moreover, as required by the mongodb documentation, how do I then manually close the cursor?
Thanks!
The example you've provided:
db.collection.find().addCursorFlag('noCursorTimeout',true)
..works fine for me on driver version 2.14.21. I've an open cursor for 45 minutes now.
Could it be you were using 1.x NodeJS driver?
so I've got a partial solution for my problem. it's doesn't say so in the API docs, but apparently you have to specify it in the find() options like so:
objectCollection.find({},{timeout: false}).limit(objectCount).toArray(function (err, objects) {
...
}
however still, what about the cleanup? do those cursors ever get killed? is a call to db.close() sufficient?
Related
I am a bit new to JavaScript web dev, and so am still getting my head around the flow of asynchronous functions, which can be a bit unexpected to the uninitiated. In my particular use case, I want execute a routine on the list of available databases before moving into the main code. Specifically, in order to ensure that a test environment is always properly initialized, I am dropping a database if it already exists, and then building it from configuration files.
The basic flow I have looks like this:
let dbAdmin = client.db("admin").admin();
dbAdmin.listDatabases(function(err, dbs){/*Loop through DBs and drop relevant one if present.*/});
return await buildRelevantDB();
By peppering some console.log() items throughout, I have determined that the listDatabases() call basically puts the callback into a queue of sorts. I actually enter buildRelevantDB() before entering the callback passed to listDatabases. In this particular example, it seems to work anyway, I think because the call that reads the configuration file is also asynchronous and so puts items into the same queue but later, but I find this to be brittle and sloppy. There must be some way to ensure that the listDatabases portion resolves before moving forward.
The closest solution I found is here, but I still don't know how to get the callback I pass to listDatabases to be like a then as in that solution.
Mixing callbacks and promises is a bit more advanced technique, so if you are new to javascript try to avoid it. In fact, try to avoid it even if you already learned everything and became a js ninja.
Dcumentation for listDatabases says it is async, so you can just await it without messing up with callbacks:
const dbs = await dbAdmin.listDatabases();
/*Loop through DBs and drop relevant one if present.*/
The next thing, there is no need to await before return. If you can await within a function, it is async and returns a promise anyway, so just return the promise from buildRelevantDB:
return buildRelevantDB();
Finally, you can drop database directly. No need to iterate over all databases to pick one you want to drop:
await client.db(<db name to drop>).dropDatabase();
I have seen three ways to return a JSON object or end a Lambda function. My trigger is Smart Home Alexa.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
Others just uses the return response_JSON. I have not used this one.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
context.succeed()/fail() causes the Lambda function to terminate immediately. However, I have not seen this documented in the context object docs, so it may get deprecated in later Node versions (?).
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
This one probably doesn't work for you because by default Node.js waits for the event loop to be empty before executing the callback statement. This may be due to open network/database connection. As per the doc, set the context.callbackWaitsForEmptyEventLoop variable to false to send the response right away.
Others just uses the return response_JSON. I have not used this one.
This should be used with async handlers. Read more about async and non-async handlers here: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html
I am trying to understand the node.js documentation specifically for the https.get() method. https://nodejs.org/dist/latest-v8.x/docs/api/https.html#https_https_get_options_callback
What is unclear to me is the callback. The example in the document indicates the callback can take a res (response) object as its parameter but I am unsure if this is the only parameter it can take or more importantly where I can find the definition of the res object so I can know what properties and methods I can access on this object.
Is there a straightforward way to identify this?
I have read this thread: Trying to understand nodejs documentation. How to discover callback parameters and the answers seem to suggest that if there is a non-error argument that a callback can take it will be documented, but I am assuming that answer is outdated.
I've run into the same issue with many Node/NPM packages. Documentation sometimes does not describe the parameters well.
So, welcome to JavaScript in 2018! It's gotten a lot better, though, to be honest.
My go-to method is to try the methods and dump the information myself.
Try a console.dir(res) in your callback:
https.get('https://encrypted.google.com/', (res) => {
console.dir(res);
});
Alternatively, you can set a breakpoint in the callback and inspect it yourself. You can then probe the arguments object* to see what else, if anything, was passed as an argument, or do another console dump:
https.get('https://encrypted.google.com/', function (res) {
console.dir("args:", arguments);
console.dir("res:", res);
});
EDIT: Wait, apparently the arguments variable is not available to arrow functions, fixed the second example.
*From MDN:
The arguments object is not an Array. It is similar to an Array, but
does not have any Array properties except length.
From your link https://nodejs.org/dist/latest-v8.x/docs/api/https.html#https_https_get_options_callback, you can see that it works like the http version :
Like http.get() but for HTTPS.
With http.get() clickable.
On that page (https://nodejs.org/dist/latest-v8.x/docs/api/http.html#http_http_get_options_callback), we can see this :
The callback is invoked with a single argument that is an instance of http.IncomingMessage
With http.IncomingMessage clickable, linking this page :
https://nodejs.org/dist/latest-v8.x/docs/api/http.html#http_class_http_incomingmessage
I agree the Node documentation is not very clear about the callbacks in general, and that is a shame. You can still use IDEs with good intellisense (and JSDoc to identify the type of the function parameters), like VSCode.
Or you can use a debugger, always works :)
Edit: If you want to see all the parameters sent to a function, you can use the spread syntax like this :
function foo(...params) {
// Here params is an array containing all the parameters that were sent to the function
}
If you want the absolute truth, you can look at the implementation. Though that's fairly time consuming.
If you find that the documentation is wrong, or in this case could be improved by adding a sentence about the callback parameter to https.get(), please open an issue, or, better yet, a pull request. This is where the change needs to be made:
https://github.com/nodejs/node/blob/67790962daccb5ff19c977119d7231cbe175c206/doc/api/https.md
I'm using mongoose 3.8. I need to fetch 100 documents, execute the callback function then fetch next 100 documents and do the same thing.
I thought .batchSize() would do the same thing, but I'm getting all the data at once.
Do I have to use limit or offset? If yes, can someone give a proper example to do it?
If it can be done with batchSize, why is it not working for me?
MySchema.find({}).batchSize(20).exec(function(err,docs)
{
console.log(docs.length)
});
I thought it would print 20 each time, but its printing whole count.
This link has the information you need.
You can do this,
var pagesize=100;
MySchema.find().skip(pagesize*(n-1)).limit(pagesize);
where n is the parameter you receive in the request, which is the page number client wants to receive.
Docs says:
In most cases, modifying the batch size will not affect the user or the application, as the mongo shell and most drivers return results as if MongoDB returned a single batch.
You may want to take a look at streams and perhaps try to accumulate subresults:
var stream = Dummy.find({}).stream();
stream.on('data', function (dummy) {
callback(dummy);
})
When I first used MongoDB I managed to connect successfully, however then I wanted to carry out the most basic query such as:
db.users.find()
I got an error saying TypeError: Cannot read property 'find' of undefined
Basically meaning I cannot use a collection as a property to the object db.
So i tried this:
var user_col = db.collection('users');
user.col.find();
which works absolutely fine.
Over the last few days I have kept having to look up other ways of doing things as the standard documented way doesn't seem to work. Just now I wanted to get the total users on the app, so like it says in the documentation I should do this:
var count = db.runCommand( { count: 'users' } );
console.log(count);
however this gave the error:
TypeError: undefined is not a function
Is there a problem with MongoDB you have seen like this before or am I just being stupid? I do not want to have to keep finding other, less efficient ways of doing things so finally I ask here what is going on.
Thank you.
It appears you are confusing the Mongo shell API with the node.js native driver API. While both are JavaScript, the shell is sync while node.js is async so they're totally different.