This code works because system-sleep blocks execution of the main thread but does not block callbacks. However, I am concerned that system-sleep is not 100% portable because it relies on the deasync npm module which relies on C++.
Are there any alternatives to system-sleep?
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100) // without this line the while loop causes problems because it is a spin wait
console.log('sleeping')
}
console.log('If this is displayed then it works!')
PS Ideally, I want a solution that works on Node 4+ but anything is better than nothing.
PPS I know that sleeping is not best practice but I don't care. I'm tired of arguments against sleeping.
Collecting my comments into an answer per your request:
Well, deasync (which sleep() depends on) uses quite a hack. It is a native code node.js add-on that manually runs the event loop from C++ code in order to do what it is doing. Only someone who really knows the internals of node.js (now and in the future) could imagine what the issues are with doing that. What you are asking for is not possible in regular Javascript code without hacking the node.js native code because it's simply counter to the way Javascript was designed to run in node.js.
Understood and thanks. I am trying to write a more reliable deasync (which fails on some platforms) module that doesn't use a hack. Obviously this approach I've given is not the answer. I want it to support Node 4. I'm thinking of using yield / async combined with babel now but I'm not sure that's what I'm after either. I need something that will wait until the callback is callback is resolved and then return the value from the async callback.
All Babel does with async/await is write regular promise.then() code for you. async/await are syntax conveniences. They don't really do anything that you can't write yourself using promises, .then(), .catch() and in some cases Promise.all(). So, yes, if you want to write async/await style code for node 4, then you can use Babel to transpile your code to something that will run on node 4. You can look at the transpiled Babel code when using async/await and you will just find regular promise.then() code.
There is no deasync solution that isn't a hack of the engine because the engine was not designed to do what deasync does.
Javascript in node.js was designed to run one Javascript event at a time and that code runs until it returns control back to the system where the system will then pull the next event from the event queue and run its callback. Your Javascript is single threaded with no pre-emptive interruptions by design.
Without some sort of hack of the JS engine, you can't suspend or sleep one piece of Javascript and then run other events. It simply wasn't designed to do that.
var one = 0;
function delay(){
return new Promise((resolve, reject) => {
setTimeout(function(){
resolve('resolved')
}, 2000);
})
}
while (one == 0) {
one = 1;
async function f1(){
var x = await delay();
if(x == 'resolved'){
x = '';
one = 0;
console.log('resolved');
//all other handlers go here...
//all of the program that you want to be affected by sleep()
f1();
}
}
f1();
}
Related
I need to make part of my method's code asynchronous, so it will execute in a non-blocking manner. For this purpose I've tried to create a "dummy" Promise and put the specified code in then block. I have something like this:
public static foo(arg1, arg2) {
const prom = new Promise((resolve, reject) => {
if (notValid(arg1, arg2)) {
reject();
}
resolve(true);
});
prom.then(() => {
...my code using arg1 and arg2...
});
}
However, then block always executes synchronously and blocks whole app, even though each and every JS documentation tells that then always runs asynchronously. I've also tried to replace the Promise with this:
Promise.resolve().then(() => {
...my code using arg1 and arg2...
});
but got same result.
The only way I've managed to make then block work asynchronously is by using setTimeout:
const pro = new Promise(resolve => {
setTimeout(resolve, 1);
});
pro.then(() => {
...my code using arg1 and arg2...
})
What can be the reason behind then block working synchronously? I don't want to proceed with using setTimeout, because it is kind of a "dirty" solution, and I think that there should be a way to make then run asynchronously.
Application is NodeJS with Express, written using Typescript.
I need to make part of my method's code asynchronous, so it will execute in a non-blocking manner. For this purpose I've tried to create a "dummy" Promise and put the specified code in then block.
Promises don't really make things asynchronous in and of themselves. What they do is wrap around something that's already asynchronous, and give a convenient way to tell when that thing is done. If you wrap a promise around something synchronous, your code is mostly still synchronous, just with a few details about when the .then callback executes.
even though each and every JS documentation tells that then always runs asynchronously.
By that, they mean that it waits for the current call stack to finish, and then runs the code in the .then callback. The technical term for what it's doing is a "microtask". This delay is done so that the order of operations of your code is the same whether the promise is already in a resolved state, or if some time needs to pass before it resolves.
But if your promise is already resolved (eg, because it's wrapped around synchronous code), the only thing your .then callback will be waiting for is the currently executing call stack. Once the current call stack synchronously finishes, your microtask runs synchronously to completion. The event loop will not be able to progress until you're done.
The only way I've managed to make then block work asynchronously is by using setTimeout
setTimeout will make things asynchronous*, yes. The code will be delayed until the timer goes off. If you want to wrap this in a promise you can, but the promise is not the part that makes it asynchronous, the setTimeout is.
I don't want to proceed with using setTimeout, because it is kind of a "dirty" solution
Ok, but it's the right tool for the job.
* when i say setTimeout makes it asynchronous, i just mean it delays the execution. This is good enough for many cases, but when your code eventually executes, it will tie up the thread until it's done. So if it takes a really long time you may need to write the code so it just does a portion of the work, and then sets another timeout to resume later
I've been confused on this for a month and searched everything but could not find an answer.
I want to get control of what runs first in the node.js. I know the way node deals with the code is non-blocking. I have the following example:
setTimeOut(function(){console.log("one second passed");}, 1000);
console.log("HelloWorld");
Here I want to run first one, output "one second passed", and then run the second one. How can I do that? I know setTimeOut is a way to solve this problem but that's not the answer I am looking for. I've tried using callback but not working. I am not sure about if I got the correct understanding of callbacks. Callbacks just mean function parameters to me and I don't think that will help me to solve this problem.
One possible way to solve this problem is to define a function that contains the "error first callback" like the following example:
function print_helloworld_atend(function helloworld(){
console.log("HelloWorld");
}){
setTimeOut(function(){console.log("one second passed");}, 1000);
helloworld();
}
Can I define a function with a callback who will know when the previous tasks are done. In the above function, how to make the callback function: helloworld to run after the "setTimeOut" expression?
If there is a structure that can solve my problem, that's my first choice. I am tired of using setTimeOuts.
I would really appreciate if anyone can help. Thanks for reading
You should be using promises. Bluebird is a great promise library. Faster than native and comes with great features. With promises you can chain together functions, and know that one will not be called until the previous function resolves. No need to set timeouts or delays. Although you can if you'd like. Below is example of a delay. Function B wont run until 6 seconds after A finishes. If you remove .delay(ms) B will run immediately after A finishes.
var Promise = require("bluebird");
console.time('tracked');
console.time('first');
function a (){
console.log('hello');
console.timeEnd('first');
return Promise.resolve();
}
function b (){
console.log('world');
console.timeEnd('tracked');
}
a().delay(6000)
.then(b)
.catch(Promise.TimeoutError, function(e) {
console.log('Something messed up yo', e);
});
This outputs:
→ node test.js
hello
first: 1.278ms
world
tracked: 6009.422ms
Edit: Promises are, in my opinion, the most fun way of control flow in node/javascript. To my knowledge there is not a .delay() or .timeout() in native javascript promises. However, there are Promises in general. You can find their documentation on mozilla's site. I would recommend that you use Bluebird instead though.
Use bluebird instead of native because:
It's faster. Petka Antonov, the creator of bluebird, has a great understanding of the V8 engines two compile steps and has optimized the library around it's many quirks. Native has little to no optimization and it shows when you compare their performance. More information here and here.
It has more features: Nice things like .reflect(), .spread(), .delay(), .timeout(), the list goes on.
You lose nothing by switching: all features in bluebird which also exist in native function in exactly the same way in implementation. If you find yourself in a situation where only native is available to you, you wont have to relearn what you already know.
Just execute everything that you want to execute after you log "one second passed", after you log "one second passed":
setTimeOut(function(){
console.log("one second passed");
console.log("HelloWorld");
}, 1000);
You can use async module to handle the callbacks.
To understand callbacks I'll give you a high level glance:
function: i want to do some i/o work.
nodejs: ok, but you shouldn't be blocking my process as I'm single threaded.
nodejs: pass a callback function, and I will let you know from it when the i/o work is done.
function: passes the callback function
nodejs: i/o work is done, calls the callback function.
function: thanks for the notification, continue processing other work.
I'm working on an Express app. At startup it connects to a Redis server and to a PostgreSQL sever. I'd like to wait for both connections to succeed before starting the express server. Now, if I were waiting on just one callback, I could initiate the Express server within that callback. But how to best do this when waiting on more than one async operation? I'm new to this. Is there a good idiomatic pattern for doing this sort of thing? Maybe grouping promises together or..?
Promises are what you want.
You can use .all() on an array of promises to wait for them all to complete. You didn't mention what Promise library you're using, but it's fairly universal. here's the Bluebird documentation: https://github.com/petkaantonov/bluebird/blob/master/API.md#all---promise
Promises are probably the idiomatic way to solve this. You will have to "promisify" your functions that return callbacks to turn them into something that returns and resolves promises, but then Promise.all() will work just fine. I personally use Bluebird for my nodejs development and regularly promisify existing functions or whole modules that use asynchronous callbacks.
If you aren't otherwise using a library that can promisify non-promised functions, then you can also just use a counter to keep track of when all your callbacks are done. This is the "older fashioned" way of doing things, but works just fine too. Such an operation works like this:
function setup(fn) {
// initialize counter to number of async operations
var counter = 3;
function checkDone() {
--counter;
if (counter <= 0) {
fn();
}
}
firstAsyncFunction(...., checkDone);
secondAsyncFunction(...., checkDone);
thirdAsyncFunction(...., checkDone);
}
setup(function() {
// this will get called when all the initial async operations are done
});
Can someone explain/ redirect me, what is the difference between Node.js's async model(non blocking thread) vs any other language for example c#'s asynchronous way of handling the I/O. This looks to me that both are same model. Kindly suggest.
Both models are very similar. There are two primary differences, one of which is going away soon (for some definition of "soon").
One difference is that Node.js is asynchronously single-threaded, while ASP.NET is asynchronously multi-threaded. This means the Node.js code can make some simplifying assumptions, because all your code always runs on the same exact thread. So when your ASP.NET code awaits, it could possibly resume on a different thread, and it's up to you to avoid things like thread-local state.
However, this same difference is also a strength for ASP.NET, because it means async ASP.NET can scale out-of-the-box up to the full capabilities of your sever. If you consider, say, an 8-core machine, then ASP.NET can process (the synchronous portions of) 8 requests simultaneously. If you put Node.js on a souped-up server, then it's common to actually run 8 separate instances of Node.js and add something like nginx or a simple custom load balancer that handles routing requests for that server. This also means that if you want other resources shared server-wide (e.g., cache), then you'll need to move them out-of-proc as well.
The other major difference is actually a difference in language, not platform. JavaScript's asynchronous support is limited to callbacks and promises, and even if you use the best libraries, you'll still end up with really awkward code when you do anything non-trivial. In contrast, the async/await support in C#/VB allow you to write very natural asynchronous code (and more importantly, maintainable asynchronous code).
However, the language difference is going away. The next revision of JavaScript will introduce generators, which (along with a helper library) will make asynchronous code in Node.js just as natural as it is today using async/await. If you want to play with the "coming soon" stuff now, generators were added in V8 3.19, which was rolled into Node.js 0.11.2 (the Unstable branch). Pass --harmony or --harmony-generators to explicitly enable the generator support.
The difference between Node.js's async model and C#'s async/await model is huge. The async model that has Node.js is similar to the old async model in C# and .Net called Event-based Asynchronous Pattern (EAP). C# and .Net has 3 async models, you can read about them at Asynchronous Programming Patterns. The most modern async model in C# is Task-based with C#'s async and await keywords, you can read about it at Task-based Asynchronous Pattern.
The C#'s async/await keywords make asynchronous code linear and let you avoid "Callback Hell" much better than in any of other programming languages. You need just try it, and after that you will never do it in other way. You just write code consuming asynchronous operations and don't worry about readability because it looks like you write any other code.
Please, watch this videos:
Async programming deep dive
Async in ASP.NET
Understanding async and Awaitable Tasks
And please, try to do something asynchronous in both C# and then Node.js to compare. You will see the difference.
EDIT:
Since Node.js V8 JavaScript engine supports generators, defined in ECMAScript 6 Draft, "Callback Hell" in JavaScript code also can be easily avoided. It brings some form of async/await to life in JavaScript
With nodejs, all requests go in the event queue. Node's event loop uses a single thread to process items in the event queue, doing all non-IO work, and sending to C++ threadpool (using javascript callbacks to manage asynchrony) all IO-bound work. The C++ threads then add to the event queue its results.
The differences with ASP.NET (the two first apply pretty much to all web servers that allow async IO) is that :
ASP.NET uses a different thread for each incoming requests, so you get an overhead of context switching
.NET doesn't force you to use async to do IO-bound work, so it isn't as idiomatic as nodejs where IO-bound api calls are de facto async (with callbacks)
.NET' "await-async" add's a step at compile time to add "callbacks", so you can write linear code (no callback function passing), in contrast with nodejs
There are so much places on the web that describe node's architecture, but here's something : http://johanndutoit.net/presentations/2013/02/gdg-capetown-nodejs-workshop-23-feb-2013/index.html#1
The difference between async in Nodejs and .NET is in using preemptive multitasking for user code.
.NET uses preemptive multitasking for user code, and Nodejs does not.
Nodejs uses an internal thread pool for serving IO requests, and a single thread for executing your JS code, including IO callbacks.
One of the consequences of using preemptive multitasking (.NET) is that a shared state can be altered by another stack of execution while executing a stack. That is not the case in Nodejs - no callback from an async operation can run simultaneously with currently executing stack. Another stacks of execution just do not exist in Javascript. A result of an async operation would be available to the callbacks only when current stack of execution exits completely. Having that, simple while(true); hangs Nodejs, because in this case current stack does not exit and the next loop is never initiated.
To understand the difference consider the two examples, one for js an one for net.
var p = new Promise(function(resolve) { setTimeout(resolve, 500, "my content"); });
p.then(function(value) { // ... value === "my content"
In this code, you can safely put a handler (then) after you "started" an async operation, because you can be sure, that no callback code that is initiated by an async operation would ever execute until the entire current call stack exits. The callbacks are handled in next cycles. As for the timer callbacks, they are treated the same. Async timer event justs puts callback processing on queue to be processed in a following cycle.
In .NET it's different. There are no cycles. There is preemptive multitasking.
ThreadPool.QueueUserWorkItem((o)=>{eventSource.Fire();});
eventSource.Fired += ()=>{
// the following line might never execute, because a parallel execution stack in a thread pool could have already been finished by the time the callback added.
Console.WriteLine("1");
}
Here is a Hello World .NET a-la Nodejs code to demonstrate async processing on single thread and using a thread pool for async IO, just like node does.
(.NET includes TPL and IAsyncResult versions of async IO operations, but there's no difference for the purposes of this example. Anyway everything ends up with different threads on a thread pool.)
void Main()
{
// Initializing the test
var filePath = Path.GetTempFileName();
var filePath2 = Path.GetTempFileName();
File.WriteAllText(filePath, "World");
File.WriteAllText(filePath2, "Antipodes");
// Simulate nodejs
var loop = new Loop();
// Initial method code, similar to server.js in Nodejs.
var fs = new FileSystem();
fs.ReadTextFile(loop, filePath, contents=>{
fs.WriteTextFile(loop, filePath, string.Format("Hello, {0}!", contents),
()=>fs.ReadTextFile(loop,filePath,Console.WriteLine));
});
fs.ReadTextFile(loop, filePath2, contents=>{
fs.WriteTextFile(loop, filePath2, string.Format("Hello, {0}!", contents),
()=>fs.ReadTextFile(loop,filePath2,Console.WriteLine));
});
// The first javascript-ish cycle have finished.
// End of a-la nodejs code, but execution have just started.
// First IO operations could have finished already, but not processed by callbacks yet
// Process callbacks
loop.Process();
// Cleanup test
File.Delete(filePath);
File.Delete(filePath2);
}
public class FileSystem
{
public void ReadTextFile(Loop loop, string fileName, Action<string> callback)
{
loop.RegisterOperation();
// simulate async operation with a blocking call on another thread for demo purposes only.
ThreadPool.QueueUserWorkItem(o=>{
Thread.Sleep(new Random().Next(1,100)); // simulate long read time
var contents = File.ReadAllText(fileName);
loop.MakeCallback(()=>{callback(contents);});
});
}
public void WriteTextFile(Loop loop, string fileName, string contents, Action callback)
{
loop.RegisterOperation();
// simulate async operation with a blocking call on another thread for demo purposes only.
ThreadPool.QueueUserWorkItem(o=>{
Thread.Sleep(new Random().Next(1,100)); // simulate long write time
File.WriteAllText(fileName, contents);
loop.MakeCallback(()=>{callback();});
});
}
}
public class Loop
{
public void RegisterOperation()
{
Interlocked.Increment(ref Count);
}
public void MakeCallback(Action clientAction)
{
lock(sync)
{
ActionQueue.Enqueue(()=>{clientAction(); Interlocked.Decrement(ref Count);});
}
}
public void Process()
{
while(Count > 0)
{
Action action = null;
lock(sync)
{
if(ActionQueue.Count > 0)
{
action = ActionQueue.Dequeue();
}
}
if( action!= null )
{
action();
}
else
{
Thread.Sleep(10); // simple way to relax a little bit.
}
}
}
private object sync = new object();
private Int32 Count;
private Queue<Action> ActionQueue = new Queue<Action>();
}
We are using Node for developing and 95% of code is Async, working fine.
For some 5% (one small module), which is sync in nature [and depends on other third party software],
and we are looking for
1. "Code to block until call back is finished"
2. At a time only one instance of function1 + its callback should be executed.
PS 1: I do completely agree, Node is for async work, We should avoid that, but this is separate non-realtime process.
PS 2: If not with Node any other Serverside JS framework? Last option is to use other lang like python, but if anything in JS possible, we are ready to give it a shot!
SEQ should solve your problem.
For an overview about sync modules please look at http://nodejsrocks.blogspot.de/2012/05/how-to-avoid-nodejs-spaghetti-code-with.html
Seq()
.seq(function () {
mysql.query("select * from foo",[], function(err,rows,fields) {
this(null, rows);
});
})
.seq(function(mysqlResult) {
console.log("mysql callback returnes:"+mysqlResult);
})
There are lots and lots of options, look at node-async, kaffeine async support, IcedCoffeescript, etc.
I want to make a plug for IcedCoffeeScript since I'm its maintainer. You can get by with solutions like Seq, but in general you'll wind up encoding control flow with function calls. I find that approach difficult to write and maintain. IcedCoffeeScript makes simple sequential operations a breeze:
console.log "hello, just wait a sec"
await setTimeout defer(), 100
console.log "ok, what did you want"
But more important, it handles any combination of async code and standard control flow:
console.log "Let me check..."
if isRunningLate()
console.log "Can't stop now, sorry!"
else
await setTimeout defer(), 1000
console.log "happy to wait, now what did you want?"
resumeWhatIWasDoingBefore()
Also loops work well, here is serial dispatch:
for i in [0...10]
await launchRpc defer res[i]
done()
And here is parallel dispatch:
await
for i in [0...10]
launchRpc defer res[i]
done()
Not only does ICS make sequential chains of async code smoother, it also encourages you to do as much as possible in parallel. If you need to change your code or your concurrency requirements, the changes are minimal, not a complete rewrite (as it would be in standard JS/CS or with some concurrency libraries).