I have to do an operation that calculate for me something, but I can't use the result of it, because I always stay in a wait state, in fact in my terminal remains in execution my program until I enter ctrl+C.
I have a main in nodejs for my program where I need to use my result calculated in a module.
var myJSONClient = {
"nombre" : "<nombre_cliente>",
"intervalo" : [0,0]
};
var intervalo = gestionar.gestion(myJSONClient,vector_intervalo);
console.log("intervalo: "+intervalo); //return undefined
And this is the module
var gestion = function(myJSON,vector_intervalo) {
var dburl = 'localhost/mongoapp';
var collection = ['clientes'];
var db = require('mongojs').connect(dburl, collection );
var intervalo_final;
function cliente(nombre, intervalo){
this.nombre = nombre;
this.intervalo = intervalo;
}
var cliente1 = new cliente(myJSON.nombre,myJSON.intervalo);
db.clientes.save(cliente1, function(err, saveCliente){
if (err || !saveCliente) console.log("Client "+cliente1.nombre+" not saved Error: "+err);
else console.log("Client "+saveCliente.nombre+" saved");
intervalo_final = calculate(vector_intervalo);
console.log(intervalo_final); //here I can see the right content of the variable intervalo_final
});
console.log(intervalo_final); //this is not executed
return intervalo_final;
}
exports.gestion = gestion;
Welcome to the async world! :)
First of all, you aren't doing blocking operations in Node. Actually, networking in Node is fully asynchronous.
The part you state the console.log works it's because the callback function of the db.clientes.save call. That callback states your mongo save has finished.
What asynchronous networking means?
It means that your save will be processed sometime in the future. The script will not wait for the response to continue the code. The console.log right after the save call will be executed soon as it's reached.
As for the "wait state" of your script, that it never ends, you should take a look at this question. There's the answer.
Related
So I'm working with websockets to process data from website's API. For every new event I also send some http requests back to the website in order to obtain more data. Up untill now everything has worked fine, but now that I started using async requests to speed it up a bit things got a bit different. My code used to process one event and then move on to the next one (these events come in extremely quick - around 10 per second) but now it just seems to ignore the async (non blocking) part and move on to the next event and that way it just skips over half of the code. Note that the code works fine outside the Pusher. I'm using the 'pusher-client' module. My code looks like this:
var Request = require("request");
var requestSync = require('sync-request');
var Pusher = require('pusher-client');
var events_channel = pusher.subscribe('inventory_changes');
events_channel1.bind('listed', function(data)
{
var var2;
//Async request (to speed up the code)
function myFunction(callback){
request("url", function(error, response, body) {
if (!error && response.statusCode == 200)
{
result = JSON.stringify(JSON.parse(body));
return callback(null, result);
}
else
{
return callback(error, null);
}
});
}
myFunction(function(err, data){
if(!err)
{
var2 = data
return(data);
}
else
{
return(err);
}
});
//The part of the code below waits for the callback and the executes some code
var var1 = var2;
check();
function check()
{
if(var2 === var1)
{
setTimeout(check, 10);
return;
}
var1 = var2;
//A CHUNK OF CODE EXECUTES HERE (connected to the data from the callback)
}
});
In conclusion the code works, but not inside the pusher due to the pusher skipping the asynchronous request. How would I make the pusher wait for my async request to finish, before processing the next event (I have no idea)? If you happen to know, please let me know :)
You need to implement a queue to handle events one after another. I'm curious how it worked before, even without Pusher you'd have to implement some queue mechanism for it.
const eventsQueue = []
events_channel1.bind('listed', function(data) {
eventsQueue.push(data)
handleNewEvent()
})
let processingEvent = false
function handleNewEvent() {
if (processingEvent) return // do nothing if already processing an event
processingEvent = true
const eventData = eventsQueue.shift() // pick the first element from array
if (!eventData) return // all events are handled at the moment
... // handle event data here
processingEvent = false
handleNewEvent() // handle next event
}
Also, you should call clearTimeout method to clear your timeout when you don;t need it anymore.
And it's better to use promises or async/await instead of callbacks. Your code will be much easier to read and maintain.
I'm trying to retrieve all the child then when there's match display.
I print the value in the console and my code work well there after few second, but when I print it in the agent as a message it show not available before the response because it does not wait.
Here is my code:
function retrieveContact(agent) {
var query = admin.database().ref("/contacts").orderByKey();
query.once("value")
.then(function(snapshot) {
snapshot.forEach(function(childSnapshot) {
var key = childSnapshot.key;
var childName = childSnapshot.child('name').val();
if (agent.parameters.name == childName) {
console.log('find ' + childName);
agent.add('The email address for ' + childName + ' is ' + childSnapshot.child('email').val());
}
// console.log('testMode'+childName);
}); //// .then
}); //// .once }
SO, how can I wait my response then let the agent show the result?
How can I include the promise concept in my code ?
You don't show your entire Handler function, but if you're doing async operations (such as reading from the firebase db) you must return the Promise. This is how the Handler Dispatcher knows to wait for the Promise to complete before returning a response to the user.
In your case, it is probably as simple as
return query.once("value")
// etc
I have a REST service in Node.js with one specific request running a bunch of DB commands and other file processing that could take 10-15 seconds to run. Since I didn't want to hold up my browser request thread, I wrote a separate .js script to do the needful, called the script using child_process.spawn() in my Node.js code and immediately returned OK back to the client. This works fine, but then so does calling the same script (as a local function) by just using a simple setTimeout.
router.post("/longRequest", function(req, res) {
console.log("Started long request with id: " + req.body.id);
var longRunningFunction = function() {
// Usually runs a bunch of things that take time.
// Simulating a 10 sec delay for sample code.
setTimeout(function() {
console.log("Done processing for 10 seconds")
}, 10000);
}
// Below line used to be
// child_process.spawn('longRunningFunction.js'
setTimeout(longRunningFunction, 0);
res.json({status: "OK"})
})
So, this works for my purpose. But what's the downside ? I probably can't monitor the offline process easily as child_process.spawn which would give me a process id. But, does this cause problems in the long run ? Will it hold up Node.js processing if the 10 second processing increases to a lot more in the future ?
The actual longRunningFunction is something that reads an Excel file, parses it and does a bulk load using tedious to a MS SQL Server.
var XLSX = require('xlsx');
var FileAPI = require('file-api'), File = FileAPI.File, FileList = FileAPI.FileList, FileReader = FileAPI.FileReader;
var Connection = require('tedious').Connection;
var Request = require('tedious').Request;
var TYPES = require('tedious').TYPES;
var importFile = function() {
var file = new File(fileName);
if (file) {
var reader = new FileReader();
reader.onload = function (evt) {
var data = evt.target.result;
var workbook = XLSX.read(data, {type: 'binary'});
var ws = workbook.Sheets[workbook.SheetNames[0]];
var headerNames = XLSX.utils.sheet_to_json( ws, { header: 1 })[0];
var data = XLSX.utils.sheet_to_json(ws);
var bulkLoad = connection.newBulkLoad(tableName, function (error, rowCount) {
if (error) {
console.log("bulk upload error: " + error);
} else {
console.log('inserted %d rows', rowCount);
}
connection.close();
});
// setup your columns - always indicate whether the column is nullable
Object.keys(columnsAndDataTypes).forEach(function(columnName) {
bulkLoad.addColumn(columnName, columnsAndDataTypes[columnName].dataType, { length: columnsAndDataTypes[columnName].len, nullable: true });
})
data.forEach(function(row) {
var addRow = {}
Object.keys(columnsAndDataTypes).forEach(function(columnName) {
addRow[columnName] = row[columnName];
})
bulkLoad.addRow(addRow);
})
// execute
connection.execBulkLoad(bulkLoad);
};
reader.readAsBinaryString(file);
} else {
console.log("No file!!");
}
};
So, this works for my purpose. But what's the downside ?
If you actually have a long running task capable of blocking the event loop, then putting it on a setTimeout() is not stopping it from blocking the event loop at all. That's the downside. It's just moving the event loop blocking from right now until the next tick of the event loop. The event loop will be blocked the same amount of time either way.
If you just did res.json({status: "OK"}) before running your code, you'd get the exact same result.
If your long running code (which you describe as file and database operations) is actually blocking the event loop and it is properly written using async I/O operations, then the only way to stop blocking the event loop is to move that CPU-consuming work out of the node.js thread.
That is typically done by clustering, moving the work to worker processes or moving the work to some other server. You have to have this work done by another process or another server in order to get it out of the way of the event loop. A setTimeout() by itself won't accomplish that.
child_process.spawn() will accomplish that. So, if you have an actual event loop blocking problem to solve and the I/O is already as async optimized as possible, then moving it to a worker process is a typical node.js solution. You can communicate with that child process in a number of ways, but one possibility would be via stdin and stdout.
I'm trying to work with synchronize.js to run some functions that must be executed synchronously.
I've read something about it here and here.
To try it out, I've created the following code
var sync = require('synchronize');
var fiber = sync.fiber;
var await = sync.await;
var defer = sync.defer;
function function1(){
console.log("Executing function 1")
}
sync.fiber(function() {
console.log('Before');
var step1 = await(setTimeout(function1, 1000, defer()));
var step2 = console.log('After');
});
console.log('Outside');
Logic tells me the output should be
Before
Outside
Executing function 1
After
But what I get is
Before
Outside
Executing function 1
Somehow, step2 is not being executed and I can't figure out why.
I've never used this plugin before but try this:
var sync = require('synchronize');
var fiber = sync.fiber;
var await = sync.await;
var defer = sync.defer;
function function1(callback){
// defer expects this to be asynchronous (so it needs to execute a callback)
console.log("Executing function 1")
callback();
}
sync.fiber(function() {
console.log('Before');
var step1 = await(setTimeout(function1, 1000, defer()));
var step2 = console.log('After');
});
console.log('Outside');
Basically I think calling defer prematurely (before the callback is executed) causes it to hang.
Edit:
After reading the documentation I now realize that defer creates a callback function. That would mean it's hanging because the callback it creates is never called. Let me know if this works.
I am trying to write a node.js application, and we need to deploy it in production.
We need to make sure that node.js does not hang when there are any long running processes/operations, like querying, or the database server access.
So, i am trying to make a call to mongo or to filesystem which takes very long time to finish, so that i can verify that other node.js server is free to serve any other requests while that takes place.
Sadly, i am not able to insert a record for which mongo takes really long time to finish or to make a synch call to the file system.
Can someone tell me how to do it?
Thanks
Tuco
The trick is do a console log of the data after the block that do a call and a console.log in the callback if in the console apears first the message is actually asynchronous
Im using mongojs as driver for mongo:
collection.find({}, function(err, res) {
console.log("done")
});
console.log("sendign signal")
If its asynchronous, in the console:
sendign signal
done!
Now for the chained behavior you can make something like that
dbChain = (function() {
var chain = [], cursor = 0, busy = false;
chainin = {
push : function(aFn) {
if(!busy) {
chainin.reset();
aFn();
busy = true;
} else {
chain.push(aFn)
}
},
next : function() {
cursor++;
if(chain[cursor]) {
chain[cursor]();
} else {
chainin.reset();
}
},
reset : function() {
chain = [];
cursor = 0;
busy = false;
}
}
return chainin;
})()
and, in all the db calls you have to do:
dbChain.push(...(a function ) ...)
in all your callbacks
dbChain.next()