delete data in firestore after n time using cloud functions - node.js

I have been trying to find a way in which the documents from a collection in firestore can get deleted after a certain amount of time in my case 24h, for doing this I realized that I have to use firebase cloud functions so after investigating I found this post which does the work that I need to add in my project except that I am using typescript instead of javascript which is being used in that question and I cant change it to javascript because I am already using typescipt for sending notifications to users. How can I change the following code to typescript?
//index.ts
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp()
const db = admin.firestore();
export const deleteOldItems = functions.database
.ref('stories/{storyId}')
.onWrite((change, context) => {
var ref = change.after.ref.parent;
var now = Date.now();
var cutoff = now - 24 * 60 * 60 * 1000;
var oldItemsQuery = ref.orderByChild('created').endAt(cutoff);
return oldItemsQuery.once('value', function(snapshot) {
// create a map with all children that need to be removed
var updates = {};
snapshot.forEach(function(child) {
updates[child.key] = null
});
// execute all updates in one go and return the result to end the function
return ref.update(updates);
});
});
here is an image of the data that needs to be erased after 24h, created field is the date when the document was created and deleted id the date where the doc should be deleted.
EDIT: I turns out the problem is not because of using typescript but because I am using firestore instead of firebase real time database so I created this code which is supposed to make things work but it doesnt. How can I fix this?
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp()
const db = admin.firestore();
export const deleteOldItems = functions.firestore
.document('stories/{storyId}')
.onWrite(async (change, context) => {
const getData = await db.collection('stories').where('deleted', '>', Date.now()).get().then(
(snapshot) => snapshot.forEach((doc) =>
doc.ref.delete()
)
);
return getData;
});

You are mixing async/await with then and, in addition, you are not waiting that the set of calls to the delete() asynchronous method are all completed. So the Cloud Function platform may clean up the Cloud Function instance before all the work is done. You need to wait that all this asynchronous work is done before returning a Promise or a value. More details here in the doc.
The following should do the trick:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp()
const db = admin.firestore();
export const deleteOldItems = functions.firestore
.document('stories/{storyId}')
.onWrite(async (change, context) => {
const querySnapshot = await db.collection('stories').where('deleted', '>', Date.now()).get();
const promises = [];
querySnapshot.forEach((doc) => {
promises.push(doc.ref.delete());
});
return Promise.all(promises);
});
Since you want to delete a variable number of documents, you need to use Promise.all() to execute in parallel the asynchronous deletion tasks (Since the delete() method returns a Promise).
Updates following you questions in the comments below:
1/ Promise.all() returns a single Promise that resolves to an array of the results of the input promises. This returned promise will resolve when all of the promises in the array have resolved. The main interest of Promise.all() is that it returns a single Promise, therefore you can easily know when all the calls to the asynchronous delete() method (which returns a Promise) are done.
2/ The onWrite event type is triggered when a doc is created, modified or deleted. Since you used this event type in your question I kept it. If you want to schedule the execution, use a scheduled Cloud Function. The "inner code" is exactly the same.

I think that ".onWrite(async (change, context) " its only called when the document is created, you need to create a task every x time to check If there is a document to delete, then you make the query and delete de document.
You can use Cron job from you server o task scheduler, I think google cloud have its own task scheduler.
A functions can not last more than 60 seconds, after that sent an error.

Related

Firebase Cloud Function not working as expected

exports.resetDailyFinalKills = functions.pubsub
.schedule("58 16 * * *")
.onRun(async (context) => {
const players = firestore.collection("players");
const goodtimes = await players.where("final_kills", ">", 0);
goodtimes.forEach((snapshot) => {
snapshot.ref.update({final_kills: 0});
});
return null;
});
So I have this cloud function, and when I force run it nothing happens at all like it just says the function was successful but the final_kills field never gets updated. Can anyone help?
Like I obviously have a player here which has a final_kills value that is greater than 0. So why doesn't this reset that back down to zero?
Note sure if I am missing something here but:
You actually try to iterate over the Query object firebase creates when using the where() function on your collections. You actually never fetch the data from the database.
const players = firestore.collection("players");
// fetch the objects from firestore
const goodtimes = await players.where("final_kills", ">", 0).get();
// iterate over the docs you receive
goodtimes.docs.forEach((snapshot) => {
snapshot.ref.update({ final_kills: 0 });
});
Edit (regarding your comment):
Make sure you set your timezone properly after your .schedule() function:
// timezone in my case
functions.pubsub.schedule('5 11 * * *').timeZone('Europe/Berlin')
Check this list as a reference for your correct timezone.

Firestore Query slow on empty collection

Below is the start of my code for a firebase function. It gets to the "oncreate" log statement in less than 2 seconds. It takes almost 2 minutes to get to the "got snapshot" log statement. The Invitation collection does not exist, it has zero documents. Why is running a query on an empty collection take so long and how do I speed this up? Thanks in advance.
exports.register = functions.firestore.document("Users/{Email}").onCreate(
async (snap, context) => {
// see if Invitation exists, if yes get FamilyId from there
const collectionRef = admin.firestore().collection("Invitations");
functions.logger.info("oncreate", {structuredData: true});
collectionRef.where("Email", "==", snap.id)
.get().then((querySnapshot) => {
functions.logger.info("got snapshot", {structuredData: true});
if (querySnapshot.empty) {
addUser(snap);
return;
} ....
Since you're performing an asynchronous operation in the Cloud Functions code, you need to return a promise from the top level of your code so that the Cloud Functions container knows how long to keep it running.
From the code you shared, that means you need to add a return here:
return collectionRef.where("Email", "==", snap.id)
...
I recommend checking out the Firebase documentation on sync, async, and promises, which explains more about how to deal with asynchronous calls.

Firestore query sometimes doesn't contain updated data

I'm using firestore's node sdk. In my code I call function1, which updates a table in firestore. When that function ends I call function2, which runs a query to get a reference of the table. About 80% of the time it works, but sometimes the data I need, which was added to the document in function1 doesn't come back in the snapshot so an error is thrown.
I added an await keyword before the update, but that doesn't seem to make the code wait for the firestore update to complete.
I suppose I could also return the data I was going to update in function1 and pass it into function2, but that feels kind of hacky, though I suppose I'd save a little money because I wouldn't have to get 1 document anymore. I could also make it one big function, but that would make it an 100 line function.
Here's an abbreviated version of my code:
const function1 = async (tableId) => {
const firestore = admin.firestore();
const tableSnapshot = await firestore.collection('tables').doc(tableId).get();
await tableSnapshot.ref.update({ smallBlind: {seat: 1, amount: 5000} }) // the seat number and amount number wont always be 1 and 5000. Otherwise I wouldn't need to look it up in function2
}
const function2 = async (tableId) => {
const firestore = admin.firestore();
const tableSnapshot = await firestore.collection('tables').doc(tableId).get();
const tableData = tableSnapshot.data();
const smallBlind = tableSnapshot.data().smallBlind; // the smallBlind data is not there. so smallBlind is undefined
}
const startStuff = async () => {
await function1(42); // example tableId is passed in
await function2(42);
}
startStuff()
The above code has no async issues. I had a different async issue in another portion of my code, which caused my problem.

Cannot get FieldValue.increment working in Firestore emulator with Admin SDK

I am trying to get a simple atomic increase/decrease working in Firebase Cloud Functions with Firestore triggers.
To test these, I want to execute local tests with the Firestore emulator.
But I always get the error
FirebaseError: Function DocumentReference.update() called with invalid data. Unsupported field value: a custom object (found in field myfield)
My code is as simple as following
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp();
export const mytest = functions.firestore.document('my/{doc}').onUpdate(async (change, context) => {
return perform();
});
export async function perform(): Promise<FirebaseFirestore.WriteResult> {
const myDocRef = .... // this is a FirebaseFirestore.DocumentReference
const decrement = admin.firestore.FieldValue.increment(1);
return myDocRef.update({
myfield: decrement
});
}
In my tests I just call the perform function and then I receive the above error. It's definitely the FieldValue.increment that causes the error because if I just put a hardcoded number instead of the decrement in the update call, it works (updates to that hardcoded number).
I am on Firebase CLI 7.2.1 and emulator 1.6.2 which - according to Most efficient way to increment a value of everything in Firebase and https://github.com/firebase/firebase-js-sdk/issues/1799 should support FieldValue.increment in the emulator.
What am I doing wrong?
What version of firebase-admin are you using?
From your description it sounds like you're calling perform() directly from a Node.js script. I would recommend re-working your setup to more closely match what will actually happen in production: a client will connect directly to Firestore, create a document, and your function will be triggered.
Here's an example you can run that does this:
const admin = require('firebase-admin');
const functions = require('firebase-functions');
admin.initializeApp();
exports.createItem = functions.https.onRequest(async (request, response) => {
const item = await admin.firestore().collection("items").add({ addedAt: Date.now() });
response.send(`added ${item.id}\n`);
});
const counter = admin.firestore().collection("counters").doc("items");
exports.increment = functions.firestore.document('items/{itemId}').onCreate(async (change, context) => {
await counter.set({ total: admin.firestore.FieldValue.increment(1) }, { merge: true });
const current = await counter.get();
console.log(`incremented counter to ${current.data().total}`);
});
The first function, createItem, is an HTTP trigger that adds a document to the items/ collection. The second function is a Firestore trigger that increments a field in the counters/items document whenever a document is created.
When I run
curl -X POST http://localhost:5001/ryanpbrewster-test/us-central1/createItem
it creates a document and sends a response like
added LGOCapHSQtlXKIMEA8Do
This triggers the other function, which I can see in the emulator logs:
I function: Beginning execution of "increment"
> incremented counter to 6
I function: Finished "increment" in ~1s

Firebase cloud function always getting null snapshot

I'm experiencing some problems while testing my firebase cloud function in the interactive shell (firebase experimental:functions:shell). Within the onWrite event, event.data.val() returns null when I first call it, instead of returning the information I expect. Also, after that, it's not automatically called again after I change some data through a mobile app.
Here is a sample code:
"use strict";
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.syncMySql = functions.database.ref('/something/{user_token}/{something_id/}')
.onWrite(event => {
const snapshot = event.data.val();
const user_token = event.params.user_token;
const mock_id = event.params.something_id;
console.log(functions.config());
console.log('# User: ' + user_token + ',\n# Something: ' + something_id + ',\n# Snapshot: ' + snapshot + '\n Evento: ' + JSON.stringify(event));
return;
});
And here is a sample of how I'm trying to call it from firebase experimental:functions:shell:
syncMySql('', {params: {user_token: 'sOmEUseRtoKen1234-i_', something_id: '456789'}})
Any ideas on what am I doing wrong?
Also, what is this first parameter called in syncMySql? What's it's purpose?
For invoking functions via the test environment like this, you need to actually provide the data that was supposedly written to the DB yourself. event.data.val() won't read the data from an exising entitity based on the params, but will only return the information you passed in the first parameter of the function call.
For invoking onWrite and onUpdate functions you need to provide the data in the form {before: 'old_data', after: 'new_data' }, where old_data and new_data can be primitives and/or objects, i.e. anything you would write to your DB.
So your full call should be something like this:
syncMySql(
{
before: null,
after: 'something'
},
{params: {user_token: 'sOmEUseRtoKen1234-i_', something_id: '456789'}}
);
In this case event.data.val() should return 'something'.

Resources