Im currently working on a project and got stuck at the setTimeout() function in a recursive function. Im rather new to promises so may i did implement this part not corretly too.
The programm should do:
add a listener to a stream event 'readable'
write a request to the stream if specific periodic data is read
resolve promise and remove listener after some other data (answer) is
recieved
Send message from Stream to another process
repeat by calling the same method recursivly with a delay of 10secs
Bassicly im trying to poll from a stream every 10 seconds.
The code looks simplyfied like this:
class XYZ {
myFunction(commands, intervall, i) {
var self = this;
var promise = new Promise((resolve, reject) => {
// I have to write to a Stream and listen for an answer
self.dataStream.write(someData, () => {
self.dataStream.addListener('readable', handleStuff);
});
// Function that handles incoming data from the Stream
var handleStuff = function () {
if (self.dataStream == someFormat) {
self.dataStream.write(commands[i]);
} else {
self.dataStream.removeListener('readable', hadleStuff);
resolve(self.dataStream.read());
}
}
});
// Resolving by sending msg and calling recursivly
promise.then((message) => {
self.send(message);
if (i + 1 > resetValue) {
setTimeout(() => {
self.myFunction(commands, intervall, 0);
}, intervall);
} else {
self.myFunction(commands, intervall, i + 1);
}
});
}
};
And i call it like this:
var myXYZ = new XYZ();
myXYZ.myFunction(myCommands, 10000, 0);
Now when i run this the initial call, it works just fine and sends the message from the dataStream to another process. But when the setTimeout() function is called the function gets "stuck" after writing data to the stream for the first time and the promise is not resolved neither rejected.
My guess is that im mixing up the context (this/self) in my code. Theres sadly no error message, so that i think my logic is faulty. It also works fine if i just remove the setTimeout() function. My Question now is how does setTimeout() change the context from which the code operates?
Related
I'm building a socket.io Node JS application and my socket.io server will be listening for data from many socket.io clients, I need to save data to an API via my socket.io server as quickly as possible and figure that async/await is the best way forward.
Right now, I've got a function inside my .on('connection'), but is there a way I can make this an async function rather than have a nested function inside?
io.use((socket, next) => {
if (!socket.handshake.query || !socket.handshake.query.token) {
console.log('Authentication Error 1')
return
}
jwt.verify(socket.handshake.query.token, process.env.AGENT_SIGNING_SECRET, (err, decoded) => {
if (err) {
console.log('Authentication Error 2')
return
}
socket.decoded = decoded
next()
})
}).on('connection', socket => {
socket.on('agent-performance', data => {
async function savePerformance () {
const saved = await db.saveToDb('http://127.0.0.1:8000/api/profiler/save', data)
console.log(saved)
}
savePerformance()
})
})
Sort of, but you'll probably want to keep your current code if there can be multiple agent-performance events. You can modify the following, but it'd be messy and less readable. Event emitters still exist for a reason, they're not made obsolete by the introduction of promises. If it's performance you're after, your current code is probably faster and more resistant to backpressure and easier to error-handle.
events.on is a utility function that takes an event emitter (like socket) and returns an iterator that yields promises. You can await those with for await of.
events.once is a utility function that takes an event emitter (like socket) and returns a promise that resolves when the specified event is executed.
const { on, once } = require('events');
(async function() {
// This is an iterator that can emit infinite number of times.
const iterator = on(io, 'connection');
// Yield a promise, await it, run what is between `{ }` and repeat.
for await (const socket of iterator) {
const data = await once(socket, 'agent-performance');
const saved = await db.saveToDb(/* etc */);
}
})();
As the names imply, on is similar to socket.on and once is similar to socket.once. In the above example:
connected user 1, first agent-performance event: OK
connected user 1, second agent-performance event: not handled, there's no more event handler, since once is "used up".
connected user 2, first agent-performance event: OK
The documentation for on has a note about concurrency when using for await (x of on(...)), but I don't know if that would be a problem in your usecase.
// The execution of this inner block is synchronous and it
// processes one event at a time (even with await). Do not use
// if concurrent execution is required.
In Rust there are relatively new features, async and await.
This syntax is a godsend but I am not sure how to use it in an evented i/o pattern in Rust.
One such example of a use is where there are websockets and you send a message and are waiting for a particular message to return, but there is just one onMessage and one sendMessage for the entire application.
So I have tried to make a Javascript equivalent where I use a setInterval to run a clock (to simulate multiple onMessage events in the case of websockets.
How would I write this Javascript in Rust/or Tokio?. Again I'm not actually going for this code but going for the conceptual sendMessage to a websocket server such that the method call waits a corresponding answer.
let setAlarm = (seconds) => {
return new Promise((resolve, reject) {
if(seconds > 60 || seconds < 0) {
return reject(new Error("Invalid seconds"));
}
//Represents going through many messages
//waiting for the matching id indicating
//its a reply to my original message
let handle = setInterval(function() {
let date = new Date();
if(date.getSeconds() == seconds) {
clearInterval(handle);
resolve(date);
}
}, 1000);
});
}
async function main() {
//Wait for 45th second
await setAlam(45);
console.log("Running tasks now");
}
So I'm working with websockets to process data from website's API. For every new event I also send some http requests back to the website in order to obtain more data. Up untill now everything has worked fine, but now that I started using async requests to speed it up a bit things got a bit different. My code used to process one event and then move on to the next one (these events come in extremely quick - around 10 per second) but now it just seems to ignore the async (non blocking) part and move on to the next event and that way it just skips over half of the code. Note that the code works fine outside the Pusher. I'm using the 'pusher-client' module. My code looks like this:
var Request = require("request");
var requestSync = require('sync-request');
var Pusher = require('pusher-client');
var events_channel = pusher.subscribe('inventory_changes');
events_channel1.bind('listed', function(data)
{
var var2;
//Async request (to speed up the code)
function myFunction(callback){
request("url", function(error, response, body) {
if (!error && response.statusCode == 200)
{
result = JSON.stringify(JSON.parse(body));
return callback(null, result);
}
else
{
return callback(error, null);
}
});
}
myFunction(function(err, data){
if(!err)
{
var2 = data
return(data);
}
else
{
return(err);
}
});
//The part of the code below waits for the callback and the executes some code
var var1 = var2;
check();
function check()
{
if(var2 === var1)
{
setTimeout(check, 10);
return;
}
var1 = var2;
//A CHUNK OF CODE EXECUTES HERE (connected to the data from the callback)
}
});
In conclusion the code works, but not inside the pusher due to the pusher skipping the asynchronous request. How would I make the pusher wait for my async request to finish, before processing the next event (I have no idea)? If you happen to know, please let me know :)
You need to implement a queue to handle events one after another. I'm curious how it worked before, even without Pusher you'd have to implement some queue mechanism for it.
const eventsQueue = []
events_channel1.bind('listed', function(data) {
eventsQueue.push(data)
handleNewEvent()
})
let processingEvent = false
function handleNewEvent() {
if (processingEvent) return // do nothing if already processing an event
processingEvent = true
const eventData = eventsQueue.shift() // pick the first element from array
if (!eventData) return // all events are handled at the moment
... // handle event data here
processingEvent = false
handleNewEvent() // handle next event
}
Also, you should call clearTimeout method to clear your timeout when you don;t need it anymore.
And it's better to use promises or async/await instead of callbacks. Your code will be much easier to read and maintain.
I'm developing an app with the following node.js stack: Express/Socket.IO + React. In React I have DataTables, wherein you can search and with every keystroke the data gets dynamically updated! :)
I use Socket.IO for data-fetching, so on every keystroke the client socket emits some parameters and the server calls then the callback to return data. This works like a charm, but it is not garanteed that the returned data comes back in the same order as the client sent it.
To simulate: So when I type in 'a', the server responds with this same 'a' and so for every character.
I found the async module for node.js and tried to use the queue to return tasks in the same order it received it. For simplicity I delayed the second incoming task with setTimeout to simulate a slow performing database-query:
Declaration:
const async = require('async');
var queue = async.queue(function(task, callback) {
if(task.count == 1) {
setTimeout(function() {
callback();
}, 3000);
} else {
callback();
}
}, 10);
Usage:
socket.on('result', function(data, fn) {
var filter = data.filter;
if(filter.length === 1) { // TEST SYNCHRONOUSLY
queue.push({name: filter, count: 1}, function(err) {
fn(filter);
// console.log('finished processing slow');
});
} else {
// add some items to the queue
queue.push({name: filter, count: filter.length}, function(err) {
fn(data.filter);
// console.log('finished processing fast');
});
}
});
But the way I receive it in the client console, when I search for abc is as follows:
ab -> abc -> a(after 3 sec)
I want it to return it like this: a(after 3sec) -> ab -> abc
My thought is that the queue runs the setTimeout and then goes further and eventually the setTimeout gets fired somewhere on the event loop later on. This resulting in returning later search filters earlier then the slow performing one.
How can i solve this problem?
First a few comments, which might help clear up your understanding of async calls:
Using "timeout" to try and align async calls is a bad idea, that is not the idea about async calls. You will never know how long an async call will take, so you can never set the appropriate timeout.
I believe you are misunderstanding the usage of queue from async library you described. The documentation for the queue can be found here.
Copy pasting the documentation in here, in-case things are changed or down:
Creates a queue object with the specified concurrency. Tasks added to the queue are processed in parallel (up to the concurrency limit). If all workers are in progress, the task is queued until one becomes available. Once a worker completes a task, that task's callback is called.
The above means that the queue can simply be used to priorities the async task a given worker can perform. The different async tasks can still be finished at different times.
Potential solutions
There are a few solutions to your problem, depending on your requirements.
You can only send one async call at a time and wait for the first one to finish before sending the next one
You store the results and only display the results to the user when all calls have finished
You disregard all calls except for the latest async call
In your case I would pick solution 3 as your are searching for something. Why would you use care about the results for "a" if they are already searching for "abc" before they get the response for "a"?
This can be done by giving each request a timestamp and then sort based on the timestamp taking the latest.
SOLUTION:
Server:
exports = module.exports = function(io){
io.sockets.on('connection', function (socket) {
socket.on('result', function(data, fn) {
var filter = data.filter;
var counter = data.counter;
if(filter.length === 1 || filter.length === 5) { // TEST SYNCHRONOUSLY
setTimeout(function() {
fn({ filter: filter, counter: counter}); // return to client
}, 3000);
} else {
fn({ filter: filter, counter: counter}); // return to client
}
});
});
}
Client:
export class FilterableDataTable extends Component {
constructor(props) {
super();
this.state = {
endpoint: "http://localhost:3001",
filters: {},
counter: 0
};
this.onLazyLoad = this.onLazyLoad.bind(this);
}
onLazyLoad(event) {
var offset = event.first;
if(offset === null) {
offset = 0;
}
var filter = ''; // filter is the search character
if(event.filters.result2 != undefined) {
filter = event.filters.result2.value;
}
var returnedData = null;
this.state.counter++;
this.socket.emit('result', {
offset: offset,
limit: 20,
filter: filter,
counter: this.state.counter
}, function(data) {
returnedData = data;
console.log(returnedData);
if(returnedData.counter === this.state.counter) {
console.log('DATA: ' + JSON.stringify(returnedData));
}
}
This however does send unneeded data to the client, which in return ignores it. Somebody any idea's for further optimizing this kind of communication? For example a method to keep old data at the server and only send the latest?
I am trying to get mongodb instance synchronously. I know this is not recommended but I just experiment and wonder why this doesn't work. this.db is still undefined after 10 seconds of waiting when normally asynchronous code gets it in less than 500 milliseconds.
Repository.js:
var mongodb = require('mongodb');
var config = require('../config/config');
var mongoConfig = config.mongodb;
var mongoClient = mongodb.MongoClient;
class Repository {
constructor() {
(async () => {
this.db = await mongoClient.connect(mongoConfig.host);
})();
}
_getDb(t) {
t = t || 500;
if(!this.db && t < 10000) {
sleep(t);
t += 500;
this._getDb(t);
} else {
return this.db;
}
}
collection(collectionName) {
return this._getDb().collection(collectionName);
}
}
function sleep(ms) {
console.log('sleeping for ' + ms + ' ms');
var t = new Date().getTime();
while (t + ms >= new Date().getTime()) {}
}
module.exports = Repository;
app.js:
require('../babelize');
var Repository = require('../lib/Repository');
var collection = new Repository().collection('products');
Javascript is an event-based architecture. All code is initiated via an event from the event queue and the next event is pulled from the event queue ONLY when the code from the previous event has finished executing. This means that your main Javascript code is single threaded.
As such, when you fire an async operation, it starts up an operation and when that operation finishes, it puts an event in the event queue. That event (which will trigger the callback for the async operation) will not run until the code from the previous event finishes running and returns back to the system.
So, now to your code. You start running some code which launches an async operation. Then, you loop forever and never return back to the system. Because of that, the next event in the event queue from the completion of your async operation can NEVER run.
So, in a nutshell, you cannot spin in a loop waiting for an async operation to complete. That's incompatible with the event driven scheme that Javascript uses. You never return back to the event system to let the async completion callback ever run. So, you just have a deadlock or infinite loop.
Instead, you need to code for an async response by returning a promise or by passing in a callback that is called sometime later. And your code needs to finish executing and then let the callback get called sometime in the future. No spinning in loops in Javascript waiting for something else to run.
You can see the async coding options here: How do I return the response from an asynchronous call?