I'm working on an Express app. At startup it connects to a Redis server and to a PostgreSQL sever. I'd like to wait for both connections to succeed before starting the express server. Now, if I were waiting on just one callback, I could initiate the Express server within that callback. But how to best do this when waiting on more than one async operation? I'm new to this. Is there a good idiomatic pattern for doing this sort of thing? Maybe grouping promises together or..?
Promises are what you want.
You can use .all() on an array of promises to wait for them all to complete. You didn't mention what Promise library you're using, but it's fairly universal. here's the Bluebird documentation: https://github.com/petkaantonov/bluebird/blob/master/API.md#all---promise
Promises are probably the idiomatic way to solve this. You will have to "promisify" your functions that return callbacks to turn them into something that returns and resolves promises, but then Promise.all() will work just fine. I personally use Bluebird for my nodejs development and regularly promisify existing functions or whole modules that use asynchronous callbacks.
If you aren't otherwise using a library that can promisify non-promised functions, then you can also just use a counter to keep track of when all your callbacks are done. This is the "older fashioned" way of doing things, but works just fine too. Such an operation works like this:
function setup(fn) {
// initialize counter to number of async operations
var counter = 3;
function checkDone() {
--counter;
if (counter <= 0) {
fn();
}
}
firstAsyncFunction(...., checkDone);
secondAsyncFunction(...., checkDone);
thirdAsyncFunction(...., checkDone);
}
setup(function() {
// this will get called when all the initial async operations are done
});
Related
I am fairly new to node and using express with it.
I have a simple API which gets data from mongo in this way:
From a Router I call a async method in AccountServices(which is in a different file) called getAccountDetails
getAccountDetails in turn gets data from mongo making an async call.
I don't want to pass req,res objects to the getAccountDetails function, So I've use async await and it works perfectly.
What I want to understand is does my whole thread/api goes into wait state for that async await function to resolve?
If So, how can I overcome this.
async/await functions are asynchronous which means that the rest of your code doesn't wait for them to finish. They are very useful for operations like fetching data since you can't predict the time it'll take which would potentially end up freezing your app for seconds or even minutes.
I am using child process exec, which I believe is asynchronous based on the documentation, to run a Python script in my Node.js server.
However, when I searched on the web and Stack Overflow, I saw many instances of using promises and async/await for child processes in Node.js.
It just made me curious that if the child processes (exec, execFile, spawn, fork) are asynchronous functions already, why would you want to use promises or async/await on them.
Its really depending upon your need..please find below code for clarification about async/await
Without async/await
function (){
rs = f1();
rs1 = f2(); // this function executes without waiting for f1()
return rs + rs1;
}
With async/await
function (){
rs = await f1();
rs1 = f2(); // this function call wait untill f1() executes
return rs + rs1;
}
Why would you want to use promises or async/await on them?
by "them", you mean the child process, yes the child process is asynchronous. Then we can ask this question:
Why would you want to use promises or async/await on asynchronous function?
One reason is to avoid callback hell, make the coding structure easy to read by making them look synchronous.
There is 2 type of function execution: synchronous and asynchronous.
There is 3 way to call the asynchronous function: callback, promises, async/await.
Hope that helps your understanding.
I would like to add to previous answers some consideration why not using callbacks that do use asynchronous functions(setTiemout, setInterval etc) as are child processes and so the answer why sometimes you can find more Promises or async/await patterns. Those are well described here https://exploringjs.com/impatient-js/ch_promises.html:
The type signatures of Promise-based functions and methods are cleaner: if a function is callback-based, some parameters are about input, while the one or two callbacks at the end are about output. With Promises, everything output-related is handled via the returned value.
Chaining asynchronous processing steps is more convenient.
Promises handle both asynchronous errors (via rejections) and synchronous errors: Inside the callbacks for new Promise(), .then(), and .catch(), exceptions are converted to rejections. In contrast, if we use callbacks for asynchronicity, exceptions are normally not handled for us; we have to do it ourselves.
Promises are a single standard that is slowly replacing several, mutually incompatible alternatives. For example, in Node.js, many functions are now available in Promise-based versions. And new asynchronous browser APIs are usually Promise-based.
I am trying to use async-await style of code throughout my app which is simple way to do async operation without any callback methods. I was able to do this for creating a wrapper apis for some external apis using "request-promise" module. But I am not sure if there are any similar thing available for contacting to database. I am completely trying to avoid callbacks.
Please let me know if anyone can help me in this regard. Let me know if you need any more information.
await requires its passed function to return a promise.
Seems like there are 2 things you can do.
Make your function, the one that interacts with the db(the one that performs asynchronous task) return a new Promise and then await the promise returned.
Use a library that performs MYSQL operations and return a promise . ex: https://www.npmjs.com/package/promise-mysql
This code works because system-sleep blocks execution of the main thread but does not block callbacks. However, I am concerned that system-sleep is not 100% portable because it relies on the deasync npm module which relies on C++.
Are there any alternatives to system-sleep?
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100) // without this line the while loop causes problems because it is a spin wait
console.log('sleeping')
}
console.log('If this is displayed then it works!')
PS Ideally, I want a solution that works on Node 4+ but anything is better than nothing.
PPS I know that sleeping is not best practice but I don't care. I'm tired of arguments against sleeping.
Collecting my comments into an answer per your request:
Well, deasync (which sleep() depends on) uses quite a hack. It is a native code node.js add-on that manually runs the event loop from C++ code in order to do what it is doing. Only someone who really knows the internals of node.js (now and in the future) could imagine what the issues are with doing that. What you are asking for is not possible in regular Javascript code without hacking the node.js native code because it's simply counter to the way Javascript was designed to run in node.js.
Understood and thanks. I am trying to write a more reliable deasync (which fails on some platforms) module that doesn't use a hack. Obviously this approach I've given is not the answer. I want it to support Node 4. I'm thinking of using yield / async combined with babel now but I'm not sure that's what I'm after either. I need something that will wait until the callback is callback is resolved and then return the value from the async callback.
All Babel does with async/await is write regular promise.then() code for you. async/await are syntax conveniences. They don't really do anything that you can't write yourself using promises, .then(), .catch() and in some cases Promise.all(). So, yes, if you want to write async/await style code for node 4, then you can use Babel to transpile your code to something that will run on node 4. You can look at the transpiled Babel code when using async/await and you will just find regular promise.then() code.
There is no deasync solution that isn't a hack of the engine because the engine was not designed to do what deasync does.
Javascript in node.js was designed to run one Javascript event at a time and that code runs until it returns control back to the system where the system will then pull the next event from the event queue and run its callback. Your Javascript is single threaded with no pre-emptive interruptions by design.
Without some sort of hack of the JS engine, you can't suspend or sleep one piece of Javascript and then run other events. It simply wasn't designed to do that.
var one = 0;
function delay(){
return new Promise((resolve, reject) => {
setTimeout(function(){
resolve('resolved')
}, 2000);
})
}
while (one == 0) {
one = 1;
async function f1(){
var x = await delay();
if(x == 'resolved'){
x = '';
one = 0;
console.log('resolved');
//all other handlers go here...
//all of the program that you want to be affected by sleep()
f1();
}
}
f1();
}
I've been confused on this for a month and searched everything but could not find an answer.
I want to get control of what runs first in the node.js. I know the way node deals with the code is non-blocking. I have the following example:
setTimeOut(function(){console.log("one second passed");}, 1000);
console.log("HelloWorld");
Here I want to run first one, output "one second passed", and then run the second one. How can I do that? I know setTimeOut is a way to solve this problem but that's not the answer I am looking for. I've tried using callback but not working. I am not sure about if I got the correct understanding of callbacks. Callbacks just mean function parameters to me and I don't think that will help me to solve this problem.
One possible way to solve this problem is to define a function that contains the "error first callback" like the following example:
function print_helloworld_atend(function helloworld(){
console.log("HelloWorld");
}){
setTimeOut(function(){console.log("one second passed");}, 1000);
helloworld();
}
Can I define a function with a callback who will know when the previous tasks are done. In the above function, how to make the callback function: helloworld to run after the "setTimeOut" expression?
If there is a structure that can solve my problem, that's my first choice. I am tired of using setTimeOuts.
I would really appreciate if anyone can help. Thanks for reading
You should be using promises. Bluebird is a great promise library. Faster than native and comes with great features. With promises you can chain together functions, and know that one will not be called until the previous function resolves. No need to set timeouts or delays. Although you can if you'd like. Below is example of a delay. Function B wont run until 6 seconds after A finishes. If you remove .delay(ms) B will run immediately after A finishes.
var Promise = require("bluebird");
console.time('tracked');
console.time('first');
function a (){
console.log('hello');
console.timeEnd('first');
return Promise.resolve();
}
function b (){
console.log('world');
console.timeEnd('tracked');
}
a().delay(6000)
.then(b)
.catch(Promise.TimeoutError, function(e) {
console.log('Something messed up yo', e);
});
This outputs:
→ node test.js
hello
first: 1.278ms
world
tracked: 6009.422ms
Edit: Promises are, in my opinion, the most fun way of control flow in node/javascript. To my knowledge there is not a .delay() or .timeout() in native javascript promises. However, there are Promises in general. You can find their documentation on mozilla's site. I would recommend that you use Bluebird instead though.
Use bluebird instead of native because:
It's faster. Petka Antonov, the creator of bluebird, has a great understanding of the V8 engines two compile steps and has optimized the library around it's many quirks. Native has little to no optimization and it shows when you compare their performance. More information here and here.
It has more features: Nice things like .reflect(), .spread(), .delay(), .timeout(), the list goes on.
You lose nothing by switching: all features in bluebird which also exist in native function in exactly the same way in implementation. If you find yourself in a situation where only native is available to you, you wont have to relearn what you already know.
Just execute everything that you want to execute after you log "one second passed", after you log "one second passed":
setTimeOut(function(){
console.log("one second passed");
console.log("HelloWorld");
}, 1000);
You can use async module to handle the callbacks.
To understand callbacks I'll give you a high level glance:
function: i want to do some i/o work.
nodejs: ok, but you shouldn't be blocking my process as I'm single threaded.
nodejs: pass a callback function, and I will let you know from it when the i/o work is done.
function: passes the callback function
nodejs: i/o work is done, calls the callback function.
function: thanks for the notification, continue processing other work.