I have a firebase function that does lot of checks in the firebase realtime database and then returns a response.
Does the node runtime in firebase garantee that the async functions will be executed in the order they are call? or there is some sort of non fifo scheduler that executes then?
the logic of the real function is a bit complex (over 200 lines) so to avoid the extra complexity i will just use a pseudo function as example:
function checks(req,res){
let resp;
database.ref('nodeA').once('value').then(function(data) {
//do some checks and modify resp
});
database.ref('nodeB').once('value').then(function(data) {
//do some checks and modify resp
});
database.ref('nodeC').once('value').then(function(data) {
//do some checks and modify resp
res.status(200).send(resp);
});
FIRST OF ALL. I know I can make nested calls to the realtime database and garantee the execution of all checks, but my real case scenario is more complex than this and would't work for me
Is there any garantee that all checks will be executed by this sample code?
if not... how can i make a non blocking while that waits it to be ready?
like:
while(!resp.ready){
wait //how to wait without blocking the other functions
}
res.status(200).send(resp);
In the code you shared the order in which the three requests are sent to the database is in the order you specify them. The results are also guaranteed to come in that same order.
So by the time nodeC is loaded, it is guaranteed that the first two callbacks have also been invoked.
try async and await for this case,
in your code, you will send the response to the user before finishing all the validation, there is no guarantee the callback function for each promise will execute in the same order.
async function checks(req,res){
let resp;
let nodeAData=await database.ref('nodeA').once('value');
//do some checks and modify resp
let nodebData=database.ref('nodeB').once('value')
//do some checks and modify resp
.
.
.
res.status(200).send(resp);
});
Related
This is a common process for me in my previous works, so i usually have a very complex use case take for example
async function doThis(){
for (100x) {
try {
insertToDatabase()
await selectAndManipulateData()
createEmailWorker()
/** and many more **/
} catch {
logToAFile()
}
}
}
The code works, but its complicated 1 function doing all the things, the only reason i do this is because i can verify in real time if one function fails i can make sure the other function wont run so there wont be any incorrect data.
What i want to know is, what is the best architecture in defining a project structure that is not sacrificing the data integrity? (or is it already good enough?)
const doThis = async() => {
try {
for (100x) {
await insertToDatabase();
await selectAndManipulateData();
await createEmailWorker();
/** and many more **/
}
}
catch {
await logToAFile();
}
}
The best way of doing this is, you should always use await to call any function and make sure to with es6 syntax's as it gives a lot more feature. Your function should always be an async.
Always put your loop in try catch as it will give you any error in catch and it will calling function specific.
Actually, I would separate the persistence, manipulation and email jobs. Consider storing your data is a single responsibility. In addition to this, your modification and email workers should work as scheduled jobs. Once the jobs triggered, they should check if there is data related to its responsibility.
Another way is changing these scheduled jobs with triggered jobs. You can build a chain of responsibility that triggers next jobs and they would decide to work or not.
I'm using Cloud Functions for Firebase to:
Receive parameters from api.ai
Make a call to a third-party API and
Respond back to api.ai.
My call to the third-party API uses the request Node.js module and is wrapped within a function (getInfoFromApi()) in index.js.
The problem I'm having is that the execution of the secondary function call is consistently taking between 15-20 seconds. Note: The cloud function itself completes its execution consistently in the 400 ms range.
By logging simple comments to the console I can see when the function starts, when the secondary function is being called and when it receives a response from the third party, so I think I can see what's happening.
Roughly, the timings look like this:
0: cloud function initialises
400 ms: cloud function completes
16 s: getInfoFromApi() function is called (!)
17 s: third-party API returns results
My questions:
Is there an obvious reason for the delay in calling the secondary function? This doesn't seem to be caused by the cold start issue since the cloud function springs to life quickly and the delay is consistent even after repeated calls.
Is the use of the 'request' node module causing the issue? Is there a better module for creating/managing http requests from cloud functions?
You can see a simplified Gist of the index.js here: https://gist.github.com/anonymous/7e00420cf2623b33b80d88880be04f65
Here is a grab of the Firebase console showing example timings. Note: the output is slightly different from the above code as I simplified the above code to help understanding.
The getInfoFrom3rdParty() call is an asynchronous event. However, you haven't returned a promise from your function, so Functions is not waiting for the async event to complete.
It looks to me like, since you are returning undefined, the Function also assumes that it failed and retries. At some point in the retry process, the async event is probably completing before the function exits (i.e. unintentional success due to race conditions). I've seen similar outcomes in other cases where users don't return a promise or value in their functions.
I can't tell from the gist what you're trying to do--it doesn't appear to actually do anything with the third party results and probably isn't a realistic mcve of your use case. But something like this is probably what you want:
exports.getInfo = functions.https.onRequest((request, response) => {
// ....
// NOTE THE RETURN; MOST IMPORTANT PART OF THIS SAMPLE
return getInfoFromThirdParty(...).then(() => {
response.writeHead(200, {"Content-Type": "application/json"});
response.end(JSON.stringify(payload));
}).catch(e => /* write error to response */);
});
function getInfoFrom3rdParty(food) {
reqObj.body = '{"query": "'+food+'"}';
return new Promise((resolve, reject) => {
mainRequest(reqObj, function (error, response, body) {
// ....
if( error ) reject(error);
else resolve(...);
// ....
});
});
}
From my experience with cloud functions, once the "execution finish" flag completes, it will then cause a delay (from 3s up to 15s). This blocks the remaining execution and increases your total response time.
To overcome this, you could try placing a Promise to your 3rd party call, once it completes, then do the "response.send()" which ends the function.
I realize that the standard practice for promises in Mongoose is to use exec(), but the following works (or at least appears to) and I want to understand the flow. I'm not against using exec, I'm just exploring this a bit to learn.
In the following Mongoose operation:
let id:string;
SomeDocument.remove({}, (err) => { //clears collection
someDoc = new SomeDocument(data); //creates a new doc for the collection. Id is created here from what I understand.
someDoc.save( (err, result) => { doSomething(); });//doSomething gets called sometime in the future.
id = someDoc._id.toString();
}).then( (result) => {doSomethingElse(id)});//This works - but will doSomethingElse always be called after the first annonymous callback completes?
I get that doSomething() will just get called at some future point - no problem. The question is, will the first callback to the remove call complete prior to doSomethingElse in the then call being called. It seems to be, in that the id is correctly populated in doSomethingElse, but I want to make sure that isn't just a fluke of timing - i.e. I want to know if I can rely on that callback completing prior to the then. I'm using standard ES6 promises (NodeJS.Global.Promise).
The alternative is that maybe then is called after the remove operation completes, but prior to the callback completing (doesn't seem to - but I want to confirm).
Set me straight if I'm explaining this incorrectly.
Yes, as #JaromandaX explained in the comments the order of callbacks is deterministic.
However, it's still bad code, and you should not rely on this behaviour. If you are using promises, don't pass a callback to remove at all; only pass callbacks to then!
SomeDocument.remove({})
.then(() => {
const someDoc = new SomeDocument(data);
someDoc.save().then(doSomething); // doSomething will get called in the future.
return someDoc._id.toString();
//^^^^^^
})
.then(doSomethingElse); // doSomethingElse will get passed the id
doSomethingElse will get called with the result of the previous callback, which is guaranteed to have been completed for that.
I am trying to write a short mocha/chai Node test for some async process, expecting it to ignore irrelevant input. It basically looks like this (compared to the test of relevant input). The problem is how do I write the second test? It's an async process that eventually does nothing, no error/success emits...
it('should process input', function(done) {
object
.on('success', function(result) {
expect.result.to.equal("OK");
done();
})
.asyncDoSomething('relevant input');
});
it('should ignore input', function(done) {
object.asyncDoSomething('irrelevant input');
// TODO: how do I verify the async process eventually did nothing?
});
That's a good one - the only solution that comes to mind is to wait for a timeout and assume if it didn't happen in this time, then it will not happen. But this is not good design and needlessly slows down the test suite.
Have you thought about isolating the decision logic to somewhere where it could be tested synchronously and then make a test for that?
For the moment (still awaiting possibly better solutions?), I have updated the emitter to emit some sort of an 'ignored' event for all cases where it decides to ignore the input asynchronously. For testing, I check the "cause" of the ignore using:
expect(cause).to.equal(expectedCause)
I'm still learning the node.js ropes and am just trying to get my head around what I should be deferring, and what I should just be executing.
I know there are other questions relating to this subject generally, but I'm afraid without a more relatable example I'm struggling to 'get it'.
My general understanding is that if the code being executed is non-trivial, then it's probably a good idea to async it, as to avoid it holding up someone else's session. There's clearly more to it than that, and callbacks get mentioned a lot, and I'm not 100% on why you wouldn't just synch everything. I've got some ways to go.
So here's some basic code I've put together in an express.js app:
app.get('/directory', function(req, res) {
process.nextTick(function() {
Item.
find().
sort( 'date-modified' ).
exec( function ( err, items ){
if ( err ) {
return next( err );
}
res.render('directory.ejs', {
items : items
});
});
});
});
Am I right to be using process.nextTick() here? My reasoning is that as it's a database call then some actual work is having to be done, and it's the kind of thing that could slow down active sessions. Or is that wrong?
Secondly, I have a feeling that if I'm deferring the database query then it should be in a callback, and I should have the actual page rendering happening synchronously, on condition of receiving the callback response. I'm only assuming this because it seems like a more common format from some of the examples I've seen - if it's a correct assumption can anyone explain why that's the case?
Thanks!
You are using it wrong in this case, because .exec() is already asynchronous (You can tell by the fact that is accepts a callback as a parameter).
To be fair, most of what needs to be asynchronous in nodejs already is.
As for page rendering, if you require the results from the database to render the page, and those arrive asynchronously, you can't really render the page synchronously.
Generally speaking it's best practice to make everything you can asynchronous rather than relying on synchronous functions ... in most cases that would be something like readFile vs. readFileSync. In your example, you're not doing anything synchronously with i/o. The only synchronous code you have is the logic of your program (which requires CPU and thus has to be synchronous in node) but these are tiny little things by comparison.
I'm not sure what Item is, but if I had to guess what .find().sort() does is build a query string internally to the system. It does not actually run the query (talk to the DB) until .exec is called. .exec takes a callback, so it will communicate with the DB asynchronously. When that communication is done, the callback is called.
Using process.nextTick does nothing in this case. That would just delay the calling of its code until the next event loop which there is no need to do. It has no effect on synchronicity or not.
I don't really understand your second question, but if the rendering of the page depends on the result of the query, you have to defer rendering of the page until the query completes -- you are doing this by rendering in the callback. The rendering itself res.render may not be entirely synchronous either. It depends on the internal mechanism of the library that defines the render function.
In your example, next is not defined. Instead your code should probably look like:
app.get('/directory', function(req, res) {
Item.
find().
sort( 'date-modified' ).
exec(function (err, items) {
if (err) {
console.error(err);
res.status(500).end("Database error");
}
else {
res.render('directory.ejs', {
items : items
});
}
});
});
});