How to delete data from Firestore with cloud functions - node.js

I'm writing a cloud functions in conjunction with google's Firestore database.
I'm trying to write recursive delete more data. I can't find the syntax for accessing and deleting data in other parts of the database.
The code I have already is below.
exports.deleteProject = functions.firestore.document('{userID}/projects/easy/{projectID}').onDelete(event => {
// Get an object representing the document prior to deletion
// e.g. {'name': 'Marie', 'age': 66}
// console.log(event)
// console.log(event.data)
console.log(event.data.previous.data())
var deletedValue = event.data.previous.data();
});
I found some info here but I don't have time to check through it atm, if I find something useful I'll amend the question.
https://firebase.google.com/docs/firestore/manage-data/delete-data?authuser=0

One can use below code to delete all the documents in a collection recursively.
This code worked perfectly for me.
Make sure you have JSON file of firebase credentials and firebase-admin installed.
const admin = require('firebase-admin');
const db = admin.firestore();
const serviceAccount = require('./PATH_TO_FIREBASE_CREDENTIALS.json');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount)
});
deleteCollection(db, COLLECTION_NAME, NUMBER_OF_RECORDS)
async function deleteCollection(db, collectionPath, batchSize) {
const collectionRef = db.collection(collectionPath);
const query = collectionRef.orderBy('__name__').limit(batchSize);
return new Promise((resolve, reject) => {
deleteQueryBatch(db, query, resolve).catch(reject);
});
}
async function deleteQueryBatch(db, query, resolve) {
const snapshot = await query.get();
const batchSize = snapshot.size;
if (batchSize === 0) {
// When there are no documents left, we are done
resolve();
return;
}
// Delete documents in a batch
const batch = db.batch();
snapshot.docs.forEach((doc) => {
batch.delete(doc.ref);
});
await batch.commit();
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(() => {
deleteQueryBatch(db, query, resolve);
});
}

The answer is that you must write a cloud function that deletes the data on its own and is trigger by the client. There isn't an efficient way to do it with client side. The method I use is I listen in the cloud function for the first delete and then fire the recursive.
Code to delete in node js:
db.collection("cities").document("DC").delete(

Related

Retrieve JSON from URL and convert it to Cloud Firestore Collection with Cloud Functions

Here is what I want to achieve : I want to get a JSON on a daily basis from a URL and convert it to a cloud firestore collection in order to be able to use it in my Flutter app. Ideally, the script would only add new data to the collection.
I saw that I can use scheduler from Firebase cloud functions to run tasks daily. That's not the problem for now.
However, I don't know how to use Firebase cloud functions properly to get data from URL and convert it to collection. Maybe that's not the point of cloud functions and I misunderstood something. So first question : Can I run classic nodeJS stuff inside cloud functions? I suppose I can
Next, I initialized a cloud function project locally, connected it to my Google account and started to write code into index.js.
const functions = require("firebase-functions");
const admin = require('firebase-admin');
const fetch = require('node-fetch');
const db = admin.firestore();
const collectionToiletRef = db.collection('mycollection');
let settings = { method: "Get" };
let url = "my-url.com"
fetch(url, settings)
.then(res => res.json())
.then((json) => {
print(json);
// TODO for each json object, add new document
});
Second question : How can I run this code to see if it works ? I saw that emulator can be used but how can I check visually my cloud firestore collection ? On this simple example, I only want to print my json to see if I can get the data correctly. Where would the printing be done ?
Maybe cloud functions is not what I need for this task. Maybe my code is bad. I don't know. Thanks for your help.
EDIT
I tried this but the call never ends. I think it's waiting for a promise that never returns or something like that.
const functions = require("firebase-functions");
const admin = require('firebase-admin');
const fetch = require('node-fetch');
admin.initializeApp();
const db = admin.firestore();
exports.tempoCF = functions
.firestore.document('/tempo/{docId}')
.onCreate(async (snap, context) => {
console.log("onCreate");
let settings = { method: "Get" };
let url = "https://opendata.paris.fr/api/records/1.0/search/?dataset=sanisettesparis&q=&rows=-1"
try {
let response = await fetch(url, settings);
let json = await response.json();
// TODO for each json object, add new document
await Promise.all(json["records"].map(toiletJsonObject => {
return db.collection('toilets').doc(toiletJsonObject["recordid"]).set({}); // Only to create documents, I will deal with the content later
}));
}
catch(error) {
console.log(error);
return null;
}
}
);
This code works and create all the documents I want but never return. However, the async (snap, context) => {} passed to onCreate is a Promise. And this promise ends when Promise.all ends. I'm missing something but I don't know why. I'm struggling a lot with async programming with Dart or JS. Not very clear in my mind.
Can I run classic nodeJS stuff inside cloud functions?
Sure! Since the fetch method returns a Promise you can very well use it in a background triggered or a scheduled Cloud Function.
How can I run this code to see if it works?
Your code will work perfectly in the emulator suite, but you will need to trigger the Cloud Function with one of the Firebase services that can run in the emulator. For example you can trigger the Cloud Function by creating a document in the Firestore emulator console.
The following Cloud Function will do the trick: just create a doc in a dummy tempo collection and the CF will add a new doc in a newDocscollection. It's up to you to adapt the fields values for this doc, I've just used the entire JSON object.
exports.tempoCF = functions
.firestore.document('/tempo/{docId}')
.onCreate((snap, context) => {
let settings = { method: "Get" };
let url = "https://..."
return fetch(url, settings)
.then(res => res.json())
.then((json) => {
console.log(json);
// TODO for each json object, add new document
return admin.firestore().collection('newDocs').add(json);
})
.catch(error => {
console.log(error);
return null;
});
});
You could also deploy your Cloud Function to the Firebase backend, and if you want to schedule it, just change the code as follows (change the trigger):
exports.scheduledFunction = functions.pubsub.schedule('every 5 minutes').onRun((context) => {
let settings = { method: "Get" };
let url = "https://..."
return fetch(url, settings)
.then(res => res.json())
.then((json) => {
console.log(json);
// TODO for each json object, add new document
return admin.firestore().collection('newDocs').add(json);
})
.catch(error => {
console.log(error);
return null;
});
});
Edit following your edit:
The following code does work correctly in the emulator, creating docs in the toilets collection.
exports.tempoCF = functions.firestore
.document('/tempo/{docId}')
.onCreate(async (snap, context) => {
console.log('onCreate');
let settings = { method: 'Get' };
let url =
'https://opendata.paris.fr/api/records/1.0/search/?dataset=sanisettesparis&q=&rows=-1';
try {
let response = await fetch(url, settings);
let json = await response.json();
return Promise.all( // Here we return the promise returned by Promise.all(), so the life cycle of the CF is correctly managed
json['records'].map((toiletJsonObject) => {
admin
.firestore()
.collection('toilets')
.doc(toiletJsonObject['recordid'])
.set({ adresse: toiletJsonObject.fields.adresse });
})
);
} catch (error) {
console.log(error);
return null;
}
});

add a subcollection under each document that resulted from geofirestore query

I queried 'users/userid/pros' firestore collection using cloud functions geofirestore, and I get a few specific documents('users/userid/pros/proid') from the query. Now, I want to add a new 'notifs' collection subsequently under each of these specific documents I get from the query. Here is my code to implement that functionality.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
var GeoFirestore = require('geofirestore').GeoFirestore;
admin.initializeApp();
const firestore = admin.firestore();
const geofirestore = new GeoFirestore(firestore);
exports.sendNotification = functions.firestore
.document("users/{userId}/clients/{client}")
.onCreate(async snapshot => {
const clientfield = snapshot.data().field;
const clientgeopoint = snapshot.data().g.geopoint;
const geocollection = geofirestore.collectionGroup('pros');
const query = geocollection.near({center: clientgeopoint, radius:
10}).where('field', '==', clientfield);
await query.get().then( querySnapshot => {
querySnapshot.forEach(async doc => {
await doc.ref.collection('notifs').add({
'field': clientfield, ... });
});
}).catch ((error) =>
console.log(error)
);
})
But this code gives me an error 'TypeError: Cannot read property 'collection' of undefined' on cloud functions console. How can I fix my code in order to add 'notifs' collection right under each of these specific documents like in the picture? Thanks.
I think you're trying to fetch the querysnapshot data wrong. From Muthu's reply it should look like this
let querySnapshot = await admin.firestore().collection('users').doc(userid).collection('pros').get();
querySnapshot.then(querySnapshot => {
querySnapshot.forEach(doc => {
// frame your data here
await doc.ref.collection('notifs').add({ ... });
});
});
If you are going to get the data from your snapshot you need to add the .then() statement after .get to properly reference the query.
DocumentSnapshot carries their reference in a field; For Node.JS - ref and for Dart - reference. You shall use this to perform any action within that document. Assuming the code was written in NodeJS, for creating sub-collection,
query.get().then(querySnapshot => {
querySnapshot.forEach(doc => {
// frame your data here
await doc.ref.collection('notifs').add({ ... });
})
});

loop through documents in a collection to get fields

I have a collection called users in firebase firestore. Each document in the collection users is a user registered in the app. Each document has a field called token_ids. How can I loop through all the documents to get the values in the token_ids field. I am using firebase cloud functions to do so. Here is the code snippet I tried using but it did not work:
const functions = require('firebase-functions');
const admin = require ('firebase-admin');
admin.initializeApp();
//fetch all token ids of users
const tokenReference = admin.firestore().collection("users");
const tokenSnapshot = await tokenReference.get();
tokenSnapshot.forEach((doc) =>{
console.log("Token ids are:" + doc.data().token_id);
});
});
Took me a while but finally found the solution to it. Here it is. It is the first solution given by Dhruv Shah but slightly modified :
async function fetchAllTTokenIds() {
const tokenReference = admin.firestore().collection("users");
const tokenSnapshot = await tokenReference.get();
const results = [];
tokenSnapshot.forEach(doc => {
results.push(doc.data().token_id);
});
const tokenIds = await Promise.all(results);
return console.log("Here =>" +tokenIds);
}
Since Firestore operations are asynchronous, you should ideally wrap your code in an async-await block.
For example:
async function fetchAllTTokenIds() {
const tokenReference = admin.firestore().collection("users");
const tokenSnapshot = await tokenReference.get();
const results = [];
// Recommendation: use for-of loops, if you intend to execute asynchronous operations in a loop.
for(const doc of tokenSnapShot) {
results.push(doc.data().token_id);
}
const tokenIds = await Promise.all(results);
}
In this way all the tokenIds variable will be populated with an array of tokenIds.
Alternatively, you can also make all the asynchronous calls in parallel since they are independent of each other using Promise.all (Reference)
async function fetchAllTTokenIds() {
const tokenReference = admin.firestore().collection("users");
const tokenSnapshot = await tokenReference.get();
const tokenIds = await Promise.all(tokenSnapShot.map(doc => {
return doc.data()
.then(data => (data.token_id))
}))
In this case, the tokenIds variable will contain the array of all the tokenIds.
How the code snippet will be structured depends whether you're using the Firebase Admin SDK, be it as a script ran on your local machine or a httpsCallable being called by a client app. For the first case, it is written like this:
In your script file.js, after initialising app, write the following code.
exports.test_f = async function() {
try {
const tokenReference = admin.firestore().collection("users");
const tokenSnapshot = await tokenReference.get();
tokenSnapshot.forEach((doc) =>{
console.log("Token ids are:" + doc.data().token_id);
});
} catch (error) {
console.log(error);
}
}
exports.test_f();
Run this script on your command line using the command node file.js, which will print the provided output

Synchronously iterate through firestore collection

I have a firebase callable function that does some batch processing on documents in a collection.
The steps are
Copy document to a separate collection, archive it
Run http request to third party service based on data in document
If 2 was successful, delete document
I'm having trouble with forcing the code to run synchronously. I can't figure out the correct await syntax.
async function archiveOrders (myCollection: string) {
//get documents in array for iterating
const currentOrders = [];
console.log('getting current orders');
await db.collection(myCollection).get().then(querySnapshot => {
querySnapshot.forEach(doc => {
currentOrders.push(doc.data());
});
});
console.log(currentOrders);
//copy Orders
currentOrders.forEach (async (doc) => {
if (something about doc data is true ) {
let id = "";
id = doc.id.toString();
await db.collection(myCollection).doc(id).set(doc);
console.log('this was copied: ' + id, doc);
}
});
}
To solve the problem I made a separate function call which returns a promise that I can await for.
I also leveraged the QuerySnapshot which returns an array of all the documents in this QuerySnapshot. See here for usage.
// from inside cloud function
// using firebase node.js admin sdk
const current_orders = await db.collection("currentOrders").get();
for (let index = 0; index < current_orders.docs.length; index++) {
const order = current_orders.docs[index];
await archive(order);
}
async function archive(doc) {
let docData = await doc.data();
if (conditional logic....) {
try {
// await make third party api request
await db.collection("currentOrders").doc(id).delete();
}
catch (err) {
console.log(err)
}
} //end if
} //end archive
Now i'm not familiar with firebase so you will have to tell me if there is something wrong with how i access the data.
You can use await Promise.all() to wait for all promises to resolve before you continue the execution of the function, Promise.all() will fire all requests simultaneously and will not wait for one to finish before firing the next one.
Also although the syntax of async/await looks synchronous, things still happen asynchronously
async function archiveOrders(myCollection: string) {
console.log('getting current orders')
const querySnapshot = await db.collection(myCollection).get()
const currentOrders = querySnapshot.docs.map(doc => doc.data())
console.log(currentOrders)
await Promise.all(currentOrders.map((doc) => {
if (something something) {
return db.collection(myCollection).doc(doc.id.toString()).set(doc)
}
}))
console.log('copied orders')
}

Promise not waiting for firebase query to complete and gets resolved too soon

I have created a promise, which would take an array of firebase keys as input, loop them to query firebase realtime database. My issue is even after I use async await, for firebase to provide results back, promise is getting resolved quickly.
function firebaseQuery(keys){
const result = [];
return new Promise((resolve, reject) => {
keys.forEach((key) => {
const snap = app.child(key).once('value');
const snapJSON = await snap.then(snapshot => snapshot.toJSON());
result.push({ key: key, post: snapJSON });
console.log(result);
});
resolve(result);
});
}
forEach does not pause for await statements, so it won't work like this (https://codeburst.io/javascript-async-await-with-foreach-b6ba62bbf404). Better to map the keys into an array of promises, and then use Promise.all() to wait until they all resolve. Something like this (just make sure to handle your errors)
async function firebaseQuery(keys){
const result = await Promise.all(keys.map(async key => {
const snap = app.child(key).once('value');
const snapJSON = await snap.then(snapshot => snapshot.toJSON());
const returnValue = { key: key, post: snapJSON };
console.log(returnValue);
return returnValue;
}));
}

Resources