I wrote this node js function for google cloud function to index firebase database node entry to Algolia.
exports.indexlisting_algolia =
functions.database.ref('/Listings/{listingId}')
.onWrite((change, context) => {
const index = algolia.initIndex('Listings');
const before = change.before; // snapshot before the update
const after = change.after; // snapshot after the update
const before_data = before.val();
const after_data = after.val();
after_data.objectID = context.params.listingId;
console.log(Date.now());
console.log(context)
return index.saveObject(after_data)
.then(
() => change.after.ref.child('last_index_timestamp').set(
Date.parse(context.timestamp)));
})
the function works but it would not stop executing, it just keep repeating itself over and over again. What is wrong and how can I fix this?
By doing change.after.ref.child('last_index_timestamp').set() you are writing to the reference you are listening to in your Cloud Function. So the Cloud Function is auto-triggering itself.
You should check at the beginning of the function if it needs to be executed or not.
Most probably you would check if last_index_timestamp exists or has a specific value, by using change.before.val() and/or change.after.val().
If you want to stop the execution of the Function just return null.
See the following official sample for an example of this "technique" (lines 30 to 32) https://github.com/firebase/functions-samples/blob/952fe5d08c0a416f78def93fa337ca2bd73aedcf/message-translation/functions/index.js
A last (important) remark: you shall return the promise returned by the set() method as well as catch the potential errors, as follows:
return index.saveObject(after_data)
.then(
() => return change.after.ref.child('last_index_timestamp').set(Date.parse(context.timestamp))
)
.catch(err => {
console.log('Error:', err);
return false;
});
I would suggest that you watch the following official Video Series "Learning Cloud Functions for Firebase" (see https://firebase.google.com/docs/functions/video-series/), and in particular the three videos titled "Learn JavaScript Promises", which explain how and why we should chain and return promises in event triggered Cloud Functions.
Related
Categories:
Database:
I am trying to read the whole selected nodes (categories) and then filter through them and return them in a cloud callable function. How can I get all the data and then filter through it? When I try to log the array, it is empty.
exports.getRecipes = functions.region('europe-west1').https.onCall((data, context) => {
categories = data.categories;
eventsData = [];
for (let i = 0; i < categories.length; i++) {
admin.database().ref(categories[i]).once('value', (data) => {
eventsData.push(data.val());
});
}
console.log(eventsData);
return "hello";
});
Is there any other way getting the whole node with admin.database().ref(), without .once()?
Data is loaded from Firebase (and most modern cloud APIs) asynchronously, and while the data is being loaded the rest of your code continues to run. This is easiest to see if you add some logging to your code:
console.log("Before starting to load data")
for (let i = 0; i < categories.length; i++) {
admin.database().ref(categories[i]).once('value', (data) => {
console.log("Got data")
});
}
console.log("After starting to load data")
When you run this code, the output is:
Before starting to load data
After starting to load data
Got data
Got data
...
This is probably the order that you expected the output to be in, but it explains why your console.log(eventsData) shows an empty array: by the time you log the array, none of the data has been loaded yet and eventsData.push(data.val()) hasn't run.
The solution for this is always the same: any code that needs the data from the asynchronous call, needs to either be directly inside the callback, be called from there, or be otherwise synchronized.
Since you're loading multiple nodes we'll use Promise.all here to wait for all of those nodes to have been loaded.
exports.getRecipes = functions.region('europe-west1').https.onCall((data, context) => {
const categories = data.categories;
const eventsData = Promise.all(categories.map((category) => {
return admin.database().ref(categories[i]).once('value').then((snapshot) => {
return snapshot.val();
});
});
return eventsData;
});
Since we now return a promise, Cloud Functions will wait for that promise to resolve, and then return the resulting value to the caller.
I recommend learning more about promises and asynchronous behavior at:
The Firebase documentation on terminating functions: Sync, async, and promises.
Doug's video series on Learn JavaScript Promises (Pt.1) with HTTP Triggers in Cloud Functions
The MDN pages on Asynchronous JavaScript
Here's the workflow:
Get a https link --> write to filesystem --> read from filesystem --> Get the sha256 hash.
It works all good on my local machine running node 10.15.3 But when i initiate a lambda function on AWS, the output is null. Some problem may lie with the readable stream. Here's the code. You can run it directly on your local machine. It will output a sha256 hash as required. If you wish to run on AWS Lambda, Comment/Uncomment as marked.
//Reference: https://stackoverflow.com/questions/11944932/how-to-download-a-file-with-node-js-without-using-third-party-libraries
var https = require('https');
var fs = require('fs');
var crypto = require('crypto')
const url = "https://upload.wikimedia.org/wikipedia/commons/a/a8/TEIDE.JPG"
const dest = "/tmp/doc";
let hexData;
async function writeit(){
var file = fs.createWriteStream(dest);
return new Promise((resolve, reject) => {
var responseSent = false;
https.get(url, response => {
response.pipe(file);
file.on('finish', () =>{
file.close(() => {
if(responseSent) return;
responseSent = true;
resolve();
});
});
}).on('error', err => {
if(responseSent) return;
responseSent = true;
reject(err);
});
});
}
const readit = async () => {
await writeit();
var readandhex = fs.createReadStream(dest).pipe(crypto.createHash('sha256').setEncoding('hex'))
try {
readandhex.on('finish', function () { //MAY BE PROBLEM IS HERE.
console.log(this.read())
fs.unlink(dest, () => {});
})
}
catch (err) {
console.log(err);
return err;
}
}
const handler = async() =>{ //Comment this line to run the code on AWS Lambda
//exports.handler = async (event) => { //UNComment this line to run the code on AWS Lambda
try {
hexData = readit();
}
catch (err) {
console.log(err);
return err;
}
return hexData;
};
handler() //Comment this line to run the code on AWS Lambda
There can be multiple things that you need check.
Since, the URL you are accessing is a public one, make sure either your lambda is outside VPC or your VPC has NAT Gateway attached with internet access.
/tmp is valid temp directory for lambda, but you may need to create doc folder inside /tmp before using it.
You can check cloud-watch logs for more information on what's going if enabled.
I've seen this difference in behaviour between local and lambda before.
All async functions return promises. Async functions must be awaited. Calling an async function without awaiting it means execution continues to the next line(s), and potentially out of the calling function.
So your code:
exports.handler = async (event) => {
try {
hexData = readit();
}
catch (err) {
console.log(err);
return err;
}
return hexData;
};
readit() is defined as const readit = async () => { ... }. But your handler does not await it. Therefore hexData = readit(); assigns an unresolved promise to hexData, returns it, and the handler exits and the Lambda "completes" without the code of readit() having been executed.
The simple fix then is to await the async function: hexData = await readit();. The reason why it works locally in node is because the node process will wait for promises to resolve before exiting, even though the handler function has already returned. But since Lambda "returns" as soon as the handler returns, unresolved promises remain unresolved. (As an aside, there is no need for the writeit function to be marked async, because it doesn't await anything, and already returns a promise.)
That being said, I don't know promises well, and I barely know anything about events. So there are others things which raise warning flags for me but I'm not sure about them, maybe they're perfectly fine, but I'll raise it here just in case:
file.on('finish' and readandhex.on('finish'. These are both events, and I believe are non-blocking, so why would the handler and therefore lambda wait around for them?
In the first case, it's within a promise and resolve() is called from within the event function, so that may be fine (as I said, I don't know much about these 2 subjects so am not sure) - the important thing is that the code must block at that point until the promise is resolved. If the code can continue execution (i.e. return from writeit()) until the finish event is raised, then it won't work.
The second case is almost certainly going to be a problem because it's just saying that if x event is raised, then do y. There's no promise being awaited, so nothing to block the code, so it will happily continue to the end of the readit() function and then the handler and lambda. Again this is based on the assumption that events are non blocking (in the sense of, a declaration that you want to execute some code on some event, does not wait at that point for that event to be raised).
I would like to know if it's possible to return data before my onCall function finish.
Here is what I'm trying to do and my result data in my app is always equal to null:
exports.myFunction = functions.https.onCall((data, context) => {
if (!context.auth || !context.auth.uid) {
return {
statut: "NOK",
};
}
return Promise.all([promise1(), promise2()])
.then(results => {
const result1 = results[0].data();
return FIRESTORE.collection('MyCollection')
.add(result1)
.then(results2 => {
let queries = [];
queries.push(
function1234()
);
queries.push(
FIRESTORE.collection('MyCollection2')
.doc("123")
.set({a: 123})
);
return Promise.all(queries)
.then(r => {
return { myReturn: "AAAA" };
});
})
.then(r2 => {
console.log('0000');
return function9898();
})
})
this is my client side function:
myFunction = () => {
var myFunctionCloud = firebase.functions().httpsCallable('myFunction');
myFunctionCloud().then(function (result) {
console.log(JSON.stringify(result));
})
}
My logs :
{"data":null}
I would like to know if it's possible to return data before my onCall
function finish.
No, it is not possible since returning data in a Callable Cloud Function indicates to the Cloud Functions platform that the Callable Cloud Function is finished and that it can clean up everything from that invocation. So in many cases, your Cloud Function will be terminated before your business logic is fully executed.
In some cases, it may be possible that the Cloud Functions platform does not terminate your CF immediately, giving enough time for your business logic to be fully executed, but this is not at all guaranteed. Relying on this specific case is not recommended and actually leads to some "erratic" behaviour
One of the approach to immediately send back a result to the front-end and continue to work in the background is to call a PubSub Cloud Function (with all the relevant input in the message) and then return the value. In other words, you delegate the remaining work to another Cloud Function and you stop the current Callable one by returning a response.
Since the breaking changes in nodejs (moved to nodejs version 8), i having a serious errors and problems with my code. I have looked at google documents how to rewrite the functions but i still can't managed it.
On nodejs version 6 i wrote a function that triggers when a new item is added and then update other nodes in the realtime database
For example
// Keeps track of the length of the 'likes' child list in a separate property.
exports.countlikechange =
functions.database.ref('/likes/{postid}/{userUID}').onWrite(event => {
const collectionRef = event.data.ref.parent;
const model = event.data.val();
let genre = model.genre;
let videoID = model.videoID;
let userVideoID = model.userVideoID;
console.log("model: ",model);
console.log("genre: ",genre);
console.log("videoId: ",videoID);
console.log("userVideoID: ",userVideoID);
const countRef = collectionRef.child('likes');
// Return the promise from countRef.transaction() so our function
// waits for this async event to complete before it exits.
return countRef.transaction(current => {
if (event.data.exists() && !event.data.previous.exists()) {
const genreList = admin.database().ref(`${genre}/${videoID}/likes`).transaction(current => {
return (current || 0) + 1;
});
const userList = admin.database().ref(`users/${userVideoID}/likes`).transaction(current => {
return (current || 0) + 1;
});
const videoList = admin.database().ref(`videos/${userVideoID}/${videoID}/likes`).transaction(current => {
return (current || 0) + 1;
});
}
}).then(() => {
console.log('Counter updated.');
return null;
});
});
This func is no longer working because i have update nodejs to version 8.
On google documents the arguments changed, for example:
exports.makeUppercase = functions.database.ref('/messages/{pushId}/original')
.onWrite((change, context) => {
Also the return statment has change, it gives me error that i need to use promise.
So im kinda confused, how should i rewrite this func so when it triggers i'll update nodes in the realtime databse.
This doesn't actually have anything to do with the version of node. It has to do with the version of the firebase-functions SDK. You were using a very old pre-release version before. Since 1.0.0, the signatures have changed is a migration guide in documentation that describes the changes. In particular, read this section.
As of v 1.0 of the Firebase SDK for Cloud Functions, the event
parameter for asynchronous functions is obsolete. It has been replaced
by two new parameters: data and context.
You will need to learn the new APIs and port your code.
The requirements for return value have not changed. You are still obliged to return a promise the resolves when all the asynchronous work is complete in your function. If you are seeing a new error message about that, it's because you also upgraded your tools, and they are now checking unhandled promises for you.
I am using Dialogflow to build an Action for Google Assistant. Everything works, except the Fulfillment of my intent.
I am using the Inline Editor (Powered by Cloud Functions for Firebase) to fulfill the intent. My function in itself runs - since I can send text to the assistant from the function.
But for some reason, code execution never enters the the function that fetches data from my Collection on Firebase Firestore - although it does execute commands before and after.
Here is the code in my index.js.
'use strict';
const admin = require('firebase-admin');
const functions = require('firebase-functions');
admin.initializeApp(functions.config().firebase);
let db = admin.firestore();
const {dialogflow} = require('actions-on-google');
const app = dialogflow({debug: true});
app.intent('INTENT', (conv, {ENTITY}) => {
conv.add("Hello."); //THIS IS DISPLAYED
db.collection("COLLECTION").orderBy("FIELD", "desc").get().then(snapshot => {
conv.add("Hey!"); //THIS IS NOT DISPLAYED
snapshot.forEach(doc => {
conv.add("Hi?"); //NOR IS THIS
});
conv.add("Hmm..."); //NEITHER THIS
}).catch(error => {
conv.add('Error!'); //NOT EVEN THIS
});
conv.add("Bye."); //THIS IS DISPLAYED TOO
});
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);
Clearly, this shows that execution never really entered the db... block, and hence the function didn't even throw any error.
Here are the logs from Firebase.
Function execution started
Billing account not configured...
Warning, estimating Firebase Config based on GCLOUD_PROJECT. Initializing firebase-admin may fail
Request {...}
Headers {...}
Conversation {...}
Response {...}
Function execution took 1681 ms, finished with status code: 200
I know that the firestore function fetches data asynchronously, but there
seems no way I could execute anything even inside its .then(...) block.
I also tried returning a Promise from the .then(...) block, and using a second .then(...) with it - which again didn't work.
var fetch = db.collection("COLLECTION").orderBy("FIELD", "desc").get().then(snapshot => {
conv.add("Hey!"); //NOT DISPLAYED
var responseArr = [];
snapshot.forEach(doc => {
conv.add("Hi?"); //NOT DISPLAYED
responseArr.push(doc);
});
conv.add("Hmm..."); //NOT DISPLAYED
return Promise.resolve(responseArr);
}).then(fetch => {
conv.add("Here?"); //NOT DISPLAYED
}).catch(error => {
conv.add('Error!'); //NOT DISPLAYED
});
Finally, I also tried putting the firestore function in a separate function, like this.
function getData(){
return db.collection("COLLECTION").orderBy("FIELD", "desc").get().then(snapshot => {
snapshot.forEach(doc => {
...
});
return data; //Manipulated from above. 'data' can be a string.
}).catch(error => {
return error;
});
}
app.intent('INTENT', (conv, {ENTITY}) => {
conv.add("Hello."); //THIS IS DISPLAYED
conv.add(getData()); //THIS IS NOT DISPLAYED
conv.add("Bye."); //THIS IS DISPLAYED
});
The problem is that you're doing an asynchronous operation (the call to get()), but you're not returning a Promise from the Intent Handler itself. The library requires you to return a Promise so it knows that there is an async operation taking place.
Returning a Promise from inside the then() portion isn't enough - that doesn't return a value from the handler, it just returns a value that is passed to the next then() function or (if it was the last one) as the return value of the entire Promise chain.
In your original code, this can be done just by returning the get().then().catch() chain. Something like this as your first line:
return db.collection("COLLECTION").orderBy("FIELD", "desc").get() // etc etc
In your second example, the fetch in your then() block is not the fetch that you think it is, and only confuses matters. Structured that way, you need to return the fetch from the let assignment.
Your third example is more complicated. The line
conv.add(getData());
doesn't even seem like it would work, on the surface, because it is returning a Promise, but you can't add a promise to the conv object. You would need to rewrite that part as
return getData()
.then( data => conv.add( data ) );
But that doesn't address how the "Bye" line would work. If you actually wanted "Bye" after the data, you would have to include it as part of the then() block.
In short, when dealing with async data, you need to
Make sure you understand how Promises work and make sure all async work is done using Promises.
Add all your data inside the then() portion of a Promise
Return a Promise correctly