I am a bit new to JavaScript web dev, and so am still getting my head around the flow of asynchronous functions, which can be a bit unexpected to the uninitiated. In my particular use case, I want execute a routine on the list of available databases before moving into the main code. Specifically, in order to ensure that a test environment is always properly initialized, I am dropping a database if it already exists, and then building it from configuration files.
The basic flow I have looks like this:
let dbAdmin = client.db("admin").admin();
dbAdmin.listDatabases(function(err, dbs){/*Loop through DBs and drop relevant one if present.*/});
return await buildRelevantDB();
By peppering some console.log() items throughout, I have determined that the listDatabases() call basically puts the callback into a queue of sorts. I actually enter buildRelevantDB() before entering the callback passed to listDatabases. In this particular example, it seems to work anyway, I think because the call that reads the configuration file is also asynchronous and so puts items into the same queue but later, but I find this to be brittle and sloppy. There must be some way to ensure that the listDatabases portion resolves before moving forward.
The closest solution I found is here, but I still don't know how to get the callback I pass to listDatabases to be like a then as in that solution.
Mixing callbacks and promises is a bit more advanced technique, so if you are new to javascript try to avoid it. In fact, try to avoid it even if you already learned everything and became a js ninja.
Dcumentation for listDatabases says it is async, so you can just await it without messing up with callbacks:
const dbs = await dbAdmin.listDatabases();
/*Loop through DBs and drop relevant one if present.*/
The next thing, there is no need to await before return. If you can await within a function, it is async and returns a promise anyway, so just return the promise from buildRelevantDB:
return buildRelevantDB();
Finally, you can drop database directly. No need to iterate over all databases to pick one you want to drop:
await client.db(<db name to drop>).dropDatabase();
Related
Top-level await support was added to Node.js in 14.3.0 via --experimental-top-level-await and later to --harmony-top-level-await.
The Problem
I need to use a top level await in my ESM script file, if it is supported by the current Node.js runtime. And further, I need to set a boolean flag to indicate that the promise was successfully awaited at the top level.
An example of what I mean:
let topLevelAwaitEnabled;
try {
await Promise.resolve(); // replaced with an actual promise
topLevelAwaitEnabled = true;
} catch (ignored) {
topLevelAwaitEnabled = false;
}
console.log(topLevelAwaitEnabled);
// carry on with the rest of the application, regardless of success or failure
// at some point, topLevelAwaitEnabled is checked to conditionally execute some code
If top level await support is enabled, this succeeds fine. However, if it is not supported, this will result in the following error during parsing and cannot be caught at runtime with a try/catch:
$ node test.js...\test.js:3
await Promise.resolve(); // replaced with an actual promise
^^^^^
SyntaxError: await is only valid in async function
So the question is: How can I use a top level await if it is supported, without creating incompatibility issues with Node.js runtimes that do not support top level await (either no CLI flag was specified or simply no runtime support)?
If the answer is "it is not possible", I would like an explanation as to why this is impossible.
In the case I am actually committing an XY problem, the underlying issue is I need a top-level dynamic import.
Note: I am well aware that top level await is not recommended for a variety of reasons, however it is crucial for a specific functionality of my application and does not impose any issue with my use case. Alternatives will likely not suffice.
Attempts
I have tried the following methods, to no avail:
eval: I have tried replacing the await line with an eval("await Promise.resolve()"), in the hope the code was evaluated in the current context. Unfortunately, even if top level await is supported, this will result in the same error, as it does not seem to inherit the current context.
vm.compileFunction: Same issue was eval(), top level await is not supported.
vm.SourceTextModule: Evaluation is asynchronous and would need to be awaited at the top level to check if it is supported... which is a catch 22.
conditional execution of the await based on process.version and process.execArgv: The error during parsing - it never actually executes the code, so conditional execution is ruled out.
As far as I know this is not possible because the parser will simply error out. The compiler will not understand the await directive and will not complete its cycle. This is probably similar to using a word that's simply not a recognized keyword.
The closest you can get is using an anonymous function.
Seems like you might be able to check the version of node being used at runtime like so process.version and then you can use also use process.argv to check for any flags passed in when starting the process.
Using the above methods, you can first check the version of node being used, and then if relevant, you can check for the needed flags.
Node docs on checking for CLI args.
(async()=>{await promise})()
I'm writing an AWS Lambda function in TypeScript using the Node.js runtime. I'm using a "batchDelete" function from a DynamoDB ORM library which returns an AsyncIterableIterator type.
According to the documentation here https://github.com/awslabs/dynamodb-data-mapper-js#batchDelete, I should invoke the method with a for await loop like this:
for await (const found of mapper.batchDelete(toRemove)) {
// items will be yielded as they are successfully removed
}
This all works great but the problem comes in where if I enable ESLint on my project. The default rules throw an error because the for await block is empty. I also get a warning because the found constant is never used. I have no use for the found constant and don't want to log it. I was wondering if there was another way to call an AsyncIterableIterator function where we disregard what is returned and don't have the empty block?
If you don't care about the results of the iteration, then you should probably just do something like this:
await Promise.all(toRemove.map(item => mapper.delete(item));
To use the mapper.batchDelete(toRemove) result more directly, you have to allow for multiple levels of promises. Perhaps you could do this:
await Promise.all(await mapper.batchDelete(toRemove)[Symbol.asyncIterator]());
In doing this, await mapper.batchDelete(toRemove)[Symbol.asyncIterator](), that would get you the default async Iterator and then passing it to Promise.all() would iterate it to get an iterable of promises. Unfortunately, in building it to make this easier:
for await (const found of mapper.batchDelete(toRemove))
they made it a bit more difficult to just get an array of promises out of it.
FYI, here's a link to the code for the .batchDelete() method if you want to look at how it's implemented.
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
I have seen three ways to return a JSON object or end a Lambda function. My trigger is Smart Home Alexa.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
Others just uses the return response_JSON. I have not used this one.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
context.succeed()/fail() causes the Lambda function to terminate immediately. However, I have not seen this documented in the context object docs, so it may get deprecated in later Node versions (?).
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
This one probably doesn't work for you because by default Node.js waits for the event loop to be empty before executing the callback statement. This may be due to open network/database connection. As per the doc, set the context.callbackWaitsForEmptyEventLoop variable to false to send the response right away.
Others just uses the return response_JSON. I have not used this one.
This should be used with async handlers. Read more about async and non-async handlers here: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html
I just started developing nodejs. I'm confused to use async model. I believe there is a way to turn most of SYNC use cases into ASYNC way. Example, by SYNC, we load some data and wait until it returns then show them to user; by ASYNC, we load data and return, just tell the user data will be presented later. I can understand why ASYNC is used in this scenario.
But here I have a use case. I'm building an web app, allowing user to place a order (buying something). Before saving the order data into db, I want to put some user data together with order data (I'm using document NoSql db by the way). So I think by SYNC, after I get order data, I make a SYNC call to database and wait for its returned user data. After I get returned data, integrate them together and ingest into db.
I think there might be an issue if I make ASYNC call to db to query user data because user data may be returned after I save data to db. And that's not what I want.
So in this case, how can I do this thing ASYNCHRONOUSLY?
Couple of things here. First, if your application already has the user data (the user is already logged in), then this information should be stored in session so you don't have to access the DB. If you are allowing the user to register at the time of purchase, you would simply want to pass a callback function that handles saving the order into your call that saves the user data. Without knowing specifically what your code looks like, something like this is what you would be looking for.
function saveOrder(userData, orderData, callback) {
// save the user data to the DB
db.save(userData, function(rec) {
// if you need to add the user ID or something to the order...
orderData.userId = rec.id; // this would be dependent on your DB of choice
// save the order data to the DB
db.save(orderData, callback);
});
}
Sync code goes something like this. step by step - one after other. There can be ifs and loops (for) etc. all of us get it.
fetchUserDataFromDB();
integrateOrderDataAndUserData();
updateOrderData();
Think of async programming with nodejs as event driven. Like UI programming - code (function) is executed when an event occurs. E.g. On click event - framework calls back registered clickHandler.
nodejs async programming can also be thought on these lines. When db query (async) execution completes, your callback is called. When order data is updated, your callback is called. The above code goes something like this:
function nodejsOrderHandler(req,res)
{
var orderData;
db.queryAsync(..., onqueryasync);
function onqueryasync(userdata)
{
// integrate user data with order data
db.update(updateParams, onorderudpate);
}
function onorderupdate(e, r)
{
// handler error
write response.
}
}
javascript closure provides the way to keep state in variables across functions.
There is certainly much more to async programming and there are helper modules that help with basic constructs like chain, parallel, join etc as you write more involved async code. but this probably gives you a quick idea.