I have a code that looks like the following
console.time('setfoo')
redis.set('foo', 'bar')
...
let result = await redis.get('foo')
console.timeEnd('setfoo')
Response time for this get is 263ms but when I remove await it goes down to 0.037ms What could be the reason?
Using await basically means "wait for this instruction to be completed before you do anything else".
Not using it would make your script execute quickly, but the result variable will only contain an unresolved Promise. More on promises here
In this case, redis does not use Promises natively, await is not needed.
What I don't get is your bit of code here:
let result = await redis.get(
redis.get('foo')
)
You pass a call to redis as a parameter to another call to redis. Redis natively uses callbacks, so if you want to display the variable you got from redis, try:
let result = await redis.get('foo', (result) => { console.log(result) })
Also, since redis.set() is asynchronous, by the time you try to recover your data, 'foo' might not have been set yet.
This might help you use promises with redis
EDIT: Your code re-written according to the redis documentation
const { promisify } = require("util");
const getAsync = promisify(client.get).bind(redis);
const setAsync = promisify(client.set).bind(redis);
...
console.time('setFoo')
await setAsync('foo', 'bar')
...
let result = await getAsync('foo', 'bar')
console.timeEnd('setfoo')
Just to throw my 2 cents here: V8 (the JavaScript engine behind browsers and NodeJS) is using an event-loop (powered by libuv) which does not necessarily execute tasks on a separate thread. What this event loop does is essentially accepting jobs and executing them when it can. More info on the event loop here.
With that being said, .get call adds a task to the event loop, which translates into a JavaScript Promise object. If you do not await that, the task is just added to the event loop. From your example:
console.time('foo')
let result = redis.get('foo')
console.timeEnd('foo')
What you are actually measuring here is how much time does it take for the event loop task to be submitted?, which is insignificant.
Awaiting on the .get call will not only add the task to the event loop, but also block until the task executes:
console.time('foo')
let result = await redis.get('foo')
console.timeEnd('foo')
So, to answer your question: the two times should not be equal at all, because one example is measuring how long it takes for Redis to process the request and return the result, while the other example is measuring just how long it takes for the job to be submitted to the event loop.
Related
I have made a few questions about this already, but maybe this question would result in better answers(i'm bad at questions)
I have one class, called FOO, where I call an async Start function, that starts the process that the class FOO was made to do. This FOO class does a lot of different calculations, as well as posting/getting the calculations using the node.js "requets" module.
-I'm using electron UI's (by pressing buttons, that executes a function etc..) to create and Start the FOO class-
class FOO {
async Start(){
console.log("Start")
await this.GetCalculations();
await this.PostResults()
}
async PostResults(){
//REQUESTS STUFF
const response = {statusCode: 200} //Request including this.Cal
console.log(response)
//Send with IPC
//ipc.send("status", response.statusCode)
}
async GetCalculations(){
for(var i = 0; i < 10; i++){
await this.GetCalculation()
}
console.log(this.Cal)
}
async GetCalculation(){
//REQUEST STUFF
const response = {body: "This is a calculation"} //Since request module cant be used in here.
if(!this.Cal) this.Cal = [];
this.Cal.push(response)
}
}
var F1 = new FOO();
F1.Start();
Now imagine this code but with A LOT more steps and more requests ect. where it might take seconds/minutes to finish all tasks in the class FOO.
-Electron got a stop button that the user can hit when he wants the calculations to stop-
How would I go about stopping the entire class from continuing?
In some cases, the user might stop and start right after, so I have been trying to figure out a way to STOP the code from running entirely, but where the user would still be able to create a new class and start that, without the other class running in the background.
I have been thinking about "tiny-worker" module, but on the creation of the worker, it takes 1-2 seconds, and this decreases the purpose of a fast calculation program.
Hopefully, this question is better than the other ones.
Update:
Applying the logic behind the different answers I came up with this:
await Promise.race([this.cancelDeferred, new Promise( async (res, req) => {
var options ={
uri: "http://httpstat.us/200?sleep=5000"
}
const response = await request(options);
console.log(response.statusCode)
})])
But even when the
this.cancelDeferred.reject(new Error("User Stop"));
Is called, the response from the request "statuscode" still gets printed out when the request is finished.
The answares I got, shows some good logic, that I didn't know about, but the problem is that they all only stop the request, the code hanlding the request response will still execute, and in some cases trigger a new request. This means that I have to spam the Stop function until it fully stops it.
Framing the problem as a whole bunch of function calls that make serialized asynchronous operations and you want the user to be able to hit a Cancel/Stop button and cause the chain of asynchronous operations to abort (e.g. stop doing any more and bail on getting whatever eventual result it was trying to get).
There are several schemes I can think of.
1. Each operation checks some state property. You make these operations all part of some object that has a aborted state property. The code for every single asynchronous operation must check that state property after it completes. The Cancel/Stop button can be hooked up to set this state variable. When the current asynchronous operation finishes, it will abort the rest of the operation. If you are using promises for sequencing your operations (which it appears you are), then you can reject the current promise causing the whole chain to abort.
2. Create some async wrapper function that incorporates the cancel state for you automatically. If all your actual asynchronous operations are of some small group of operations (such as all using the request module), then you can create a wrapper function around whichever request operations you use that when any operation completes, it checks the state variable for you or merges it into the returned promise and if it has been stopped, it rejects the returned promise which causes the whole promise chain to abort. This has the advantage that you only have to do the if checks in one place and the rest of your code just switches to using your wrapped version of the request function instead of the regular one.
3. Put all the async steps/logic into another process that you can kill. This seems (to me) like using a sledge hammer for a small problem, but you could launch a child_process (which can also be a node.js program) to do your multi-step async operations and when the user presses stop/cancel, then you just kill the child process. Your code that is monitoring the child_process and waiting for a result will either get a final result or an indication that it was stopped. You probably want to use an actual process here rather than worker threads so you get a full and complete abort and so all memory and other resources used by that process gets properly reclaimed.
Please note that none of these solutions use any sort of infinite loop or polling loop.
For example, suppose your actual asynchronous operation was using the request() module.
You could define a high scoped promise that gets rejected if the user clicks the cancel/stop button:
function Deferred() {
let p = this.promise = new Promise((resolve, reject) => {
this.resolve = resolve;
this.reject = reject;
});
this.then = this.promise.then.bind(p);
this.catch = this.promise.catch.bind(p);
this.finally = this.promise.finally.bind(p);
}
// higher scoped variable that persists
let cancelDeferred = new Deferred();
// function that gets called when stop button is hit
function stop() {
// reject the current deferred which will cause
// existing operations to cancel
cancelDeferred.reject(new Error("User Stop"));
// put a new deferred in place for future operations
cancelDeferred = new Deferred();
}
const rp = require('request-promise');
// wrapper around request-promise
function rpWrap(options) {
return Promise.race([cancelDeferred, rp(options)]);
}
Then, you just call rpWrap() everywhere instead of calling rp() and it will automatically reject if the stop button is hit. You need to then code your asynchronous logic so that if any reject, it will abort (which is generally the default and automatic behavior for promises anywa).
Asynchronous functions do not run code in a separate thread, they just encapsulate an asynchronous control flow in syntactic sugar and return an object that represents its completion state (i.e. pending / resolved / rejected).
The reason for making this distinction is that once you start the control flow by calling the async function, it must continue until completion, or until the first uncaught error.
If you want to be able to cancel it, you must declare a status flag and check it at all or some sequence points, i.e. before an await expression, and return early (or throw) if the flag is set. There are three ways to do this.
You can provide a cancel() function to the caller which will be able set the status.
You can accept an isCancelled() function from the caller which will return the status, or conditionally throw based on the status.
You can accept a function that returns a Promise which will throw when cancellation is requested, then at each of your sequence points, change await yourAsyncFunction(); to await Promise.race([cancellationPromise, yourAsyncFunction()]);
Below is an example of the last approach.
async function delay (ms, cancellationPromise) {
return Promise.race([
cancellationPromise,
new Promise(resolve => {
setTimeout(resolve, ms);
})
]);
}
function cancellation () {
const token = {};
token.promise = new Promise((_, reject) => {
token.cancel = () => reject(new Error('cancelled'));
});
return token;
}
const myCancellation = cancellation();
delay(500, myCancellation.promise).then(() => {
console.log('finished');
}).catch(error => {
console.log(error.message);
});
setTimeout(myCancellation.cancel, Math.random() * 1000);
I'm developing a simple discord bot and I am attempting to print some general data about a certain player. I recently learned about async/await and I attempted to implement it into my code. It does not seem to work however because when I first trigger this code, it prints null but on subsequent triggers it will print the correct data, indicating that my function did not wait for the Promise to resolve.
async function stats(){
data = await NBA.stats.playerInfo({ PlayerID: curry.playerId });
}
stats();
data = JSON.stringify(data);
console.log(data);
The variable data is a global variable declared at the top of my program and initially initialized to null.
If I understand your intention correctly, what you want is to asynchronously fetch some data into data and then display it on the console.
Your implementation of stats is doing the fetch correctly. But your problem is, the part where you log to console has no dependence on the fetch being completed.
When you call a function that has been declared async, you are saying that you want it to execute "in the background" so to speak. The interpreter won't wait for that function to finish executing.
stats(); // begin executing this function
data = JSON.stringify(data); // but don't wait for it to finish before moving on to this line
console.log(data);
Clearly, that's not what you want. Instead, you want to wait for stats to finish what it's doing before logging data. stats being an async function returns a Promise, so you can do this:
function logData() {
console.log(JSON.stringify(data));
}
stats().then(logData);
Are there some advantages of one of this options?
1.
const fs = require('fs')
const testFunc1 = async () => {
fs.writeFileSync('text.txt', 'hello world')
}
2.
const fs = require('fs')
const util = require('util')
const writeFilePromisified = util.promisify(fs.writeFile)
const testFunc2 = async () => {
await writeFilePromisified('text.txt', 'hello world')
}
I am aware the difference betweeen writeFile and writeFileSync. The question is are there some diffs between promisses that return testFunc1 and testFunc2. So ist it the same to calling
testFunc1.then(...) // or await testFunc1
or
testFunc2.then(...) // or await testFunc2
These both promisses will be fullfilled when file writing is done.
fs already contains promisified API that doesn't need promisify.
const fsPromises = require("fs/promises");
await fsPromises.writeFile(file, data[, options])
Asynchronous promise-based version requires to use it as a part of promise-based control flow, while synchronous version doesn't impose this requirement.
Asynchronous readFile/writeFile is non-blocking, while synchronous readFileSync/writeFileSync is blocking but allows to complete a job faster. This may be noticeable during intensive IO operations.
To illustrate the difference between two promises that returns functions:
const fs = require('fs')
const util = require('util')
const testFunc1 = async () => {
fs.writeFileSync('text.txt', 'hello world')
console.log('file write done with writeFileSync')
}
const writeFilePromisified = util.promisify(fs.writeFile)
const testFunc2 = async () => {
await writeFilePromisified('text.txt', 'hello world')
console.log('file write done with promisified writeFile')
}
console.log('start test1')
testFunc1().then(() => {
console.log('promise 1 is fullfiled')
})
console.log('start test2')
testFunc2().then(() => {
console.log('promise 2 is fullfiled')
})
console.log('stop')
Output would be:
start test1
file write done with writeFileSync
start test2
stop
promise 1 is fullfiled
file write done with promisified writeFile
promise 2 is fullfiled
So like estus said testFunc1 blocks the execution of the main thread. testFunc2 do not block.
fs.readFile takes a callback function, which means it will not block the execution of your script.
fs.readFileSync however does not take a callback, which means that the execution of your script will be paused untill the process is finished.
Using promisfy is one way to solve this issue, for small files it wont make a difference, but for large files you might want to transform fs.readFileSync into a promise so you wont block the execution.
Hope that helps.
The difference between writeFile() and writeFileSync() is as explained by others that writeFileSync() "blocks". So, what is the difference between "blocking" and "not blocking"?
I wrote a small test where I compared writeFile() and writeFileSync() in terms of speed. I measured the time it took for writeFileSync() to return a result vs. the time it took from calling writeFile() until its callback-argument got called. To my (initial) surprise there was no noticeable difference between the two. writeFile() did not seem any faster than writeFileSync(). So why should I use it when it seems to complicate my program-flow?
But then I measured the time writeFile() took if I didn't wait for its callback-argument to get called. There was a big difference in speed, maybe 10x.
Having a big or no difference thus depends on the rest of your program. If you make a call to writeFileSync() that takes a long time, your program can do nothing else while it is waiting for writeFileSync() to return. If you do writeFile() and wait for the callback to complete before doing anything else, the outcome is no different. But writeFile() is much faster if you don't (need to) wait for its callback to get called.
This would have a big difference if you write a server of some kind that calls writeFileSync(). The server could not service any new requests while it is waiting for a writeFileSync() to complete. And if most requests caused a call to writeFileSync() that could have a big impact on the performance of your server.
fs.writeFileSync( file, data, options )
fs.writeFile( file, data, options, callback)
The Asynchronous function has a callback function as the last
parameter which indicates the completion of the asynchronous function.
The ‘fs’ module implements the File I/O operation. Methods in the fs module can be synchronous as well as asynchronous.
Developers prefer asynchronous methods over synchronous methods as they never block a program during its execution, whereas the later does. Blocking the main thread is "malpractice" in Node.js, thus synchronous functions should only be used for "debugging" or when no other options are available.
The fs.writeFileSync() is a synchronous method & creates a new file if
the specified file does not exist while fs.writeFile() is an
asynchronous method.
I understand what a callback is and what asynchronous means, what I don't get is how to run asynchronous functions in node.
For example, how is this
var action = (function(data,callback) {
result = data+1;
callback(result);
});
http.createServer(function (req, res) {
action(5, function(r){
res.end(r.toString());
});
}).listen(80);
different from this
var action = (function(data) {
result = data+1;
return result;
});
http.createServer(function (req, res) {
var r = action(5);
res.end(r.toString());
}).listen(80);
?
I guess in the first example I'm doing it asynchronously, yet I don't know how Node knows when to do it sync or async... is it a matter of the return? or the fact that in the sync mode we're doing var x = func(data);?
And also: when to use sync or async? Because obviously you don't want to use it when adding +1... is it OK to use async just when performing IO tasks, such as reading from DB?
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
I guess in the first example I'm doing it asynchronously...
Your first example isn't async :) Merely passing a callback and calling it when you're done doesn't make a function asynchronous.
Asynchronous means that, basically, you're telling Node: "here, do this for me, and let me know when you're done while I continue doing other stuff".
Your example is not handing anything to Node for future completion. It's doing a calculation and calling the callback immediately after that. That's functionally the same as your second example, where you return the result of the calculation.
However, you can change your first example to something that is asynchronous:
var action = (function(data,callback) {
setTimeout(function() {
result = data + 1;
callback(result);
}, 1000);
});
Here, you're telling Node to delay calling the callback for one second, by using setTimeout. In the mean time, Node won't get stuck waiting for a second; it will happily accept more HTTP requests, and each one will be delayed one second before the response is sent.
When to use sync or async?
Asynchronous code is "viral": if you rely on functions that are async, your own code that uses those functions will also have to be async (generally by accepting a callback, or using another mechanism to deal with asynchronicity, like promises).
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
This depends on which function you're using. AFAIK, most encryption functions in crypto aren't asynchronous, so you can't "make" them asynchronous yourself.
Both examples will work synchronous. Simple async operations are setTimout and setInterval.
Node actually doesn't care what code are you running. You can block or not (blocking/non-blocking).
In other words - you have event loop. If your process is async he will pass the program control to the event loop, so it can execute any other action node needs to be done. If not - he wont.
if you want a function to work asynchronously, you can do that using promises, look at the code below :
function is_asynch(){
return new Promise((resolve,reject)=>{
resolve( here_your_synch_function() )
})
}
I have task to search meaning of certain words via lookup on a database for which i am making asynchronous calls to the database, every request would look for sat n number of terms.
The issue i have is I want to call another call back, say, grand_callback, the goal of this callback is to aggregate data from all other callbacks and process the next set of codes after aggregating all the data.
Is there a way I can implement the same..
Some details:
terms = [........] // 1000 terms
grand_callback = () ->
#called with aggreagted data.
getbucket_data = (bucket ,callback) ->
#some treatment over terms
callback null , data
some_func = (term) ->
bucket.push term
if bucket.length is 15{
getbucket_data bucket , (err, data)->
#i need to aggregate this data
}
_.map terms , some_func
You can, of course, just keep track of which callbacks are done manually, but this can be a pain and pretty error prone if you have to do it a lot. Perhaps you could benefit from one of these solutions:
1. Use an async library
I personally like using async by caolan. It has a bunch of functions that can be useful for managing asynchronous operations; the one you're looking for is probably parallel (docs):
parallel(tasks, [callback])
Run the tasks array of functions in parallel, without waiting until the previous
function has completed. If any of the functions pass an error to its
callback, the main callback is immediately called with the value of the error.
Once the tasks have completed, the results are passed to the final callback as an
array.
So you would do something like:
async.parallel([
asyncFunctionOne,
asyncFunctionTwo,
asyncFunctionThree
], function(error, results) {
// ...
});
2. Use promises
Promises are a nice abstraction on top of asynchronous operations. A promise represents a value that either exists now or will exist in the future; you don't have to care which. You can wait for multiple promises to complete by creating a new promise that contains them all. One of the most popular promise libraries for Node is Q by kriskowal.
// creating a promise manually
var deferred1 = Q.defer();
var promise1 = deferred1.promise;
asyncFunction1(function(err, value) {
if (err) deferred1.reject(err);
else deferred1.resolve(value);
});
// wrapping a Node-style function to create promises
var promise2 = Q.nfcall(asyncFunction2, arg1, arg2);
// Create a promise that waits for all other promises
var grandPromise = Q.all([promise1, promise2]);
grandPromise.then(Q.spread(function(promise1result, promise2result) {
// ...
}));