Custom generated ID in Firestore NodeJs gets overwritten sometimes - node.js

I have 2 collections. a) Products: Stored Products, b) Barcode: Stores a prefix and a counter value which collectively forms a string and is used as ID for a new product.
app.post('/saveProduct',
body('name').not().isEmpty().trim().escape(),
body('description').not().isEmpty().trim().escape(),
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
// If error detected throw error
errors.array()['location'] = "";
return res.status(400).json({ errors: errors.array() });
}else{
let productData = {};
const BarcodeCollection = db.collection('Barcodes').doc('Code');
try {
let t_res = await db.runTransaction(async t => {
// Get the barcode which is to be used for the currect Product save request
const Barcodes = await t.get(BarcodeCollection);
// Assign Request object to ProductData Object
productData = {
"name": req.body.name,
"description": req.body.description,
"barcode": Barcodes.data().prefix+Barcodes.data().nextCode
};
// Increment Barcode nextCode
await t.update(BarcodeCollection, { nextCode: FieldValue.increment() });
// Use Product Barcode as ID and Store Product Data Oject
await db.collection('Products').doc(productData.barcode).set(productData);
});
console.log('Transaction Successful:');
return res.send({"status": 200, "message": "Product Saved", "barcode": productData.barcode});
}catch(e){
console.log('Transaction failure: '+ e);
return res.send({"status": 400, "message": "Something went wrong. Please try again later"});
}
}
});
The above code is what I am using.
Issue: The code I am using works fine but sometimes when there are multiple requests made within milliseconds. It overwrites the previously entered saved Product with a new Product. For example. I just stored Product with ID ABCD1003, within milliseconds I get another request and somehow it overwrites ABCD1003 instead of creating a new barcode as ABCD1004.
If I detect whether the nextCode already exists in the system; if true then add 1 and save. There is always a chance that by the time I add 1, ABCD1004 might be used by another product and end up overwriting since requests are made within milliseconds.
How can I prevent overwriting? Note: I require the barcode to be unique if not sequential

The only scalable way to generate document IDs is to make them randomized. This is exactly why Firestore provides the add method on a collection reference. It will randomize the document ID and is virtually guaranteed to be unique. Also note that writing documents with IDs that are sequential cause performance problems with Firestore collections.
The problem is that you need to use the transaction object (t) to write the new document. Currently you are adding it normally. It doesn't matter if the write occurs within the transaction callback - it needs to participate in the transaction along with the document that maintains the count.

Related

Google Cloud Functions Firestore Limitations

I have written a function which gets a Querysnapshot within all changed Documents of the past 24 hours in Firestore. I loop through this Querysnapshot to get the relevant informations. The informations out of this docs I want to save into maps which are unique for every user. Every user generates in average 10 documents a day. So every map gets written 10 times in average. Now I'm wondering if the whole thing is scalable or will hit the 500 writes per transaction limit given in Firebase as more users will use the app.
The limitation im speaking about is documented in Google documentation.
Furthermore Im pretty sure that my code is really slow. So im thankful for every optimization.
exports.setAnalyseData = functions.pubsub
.schedule('every 24 hours')
.onRun(async (context) => {
const date = new Date().toISOString();
const convertedDate = date.split('T');
//Get documents (that could be way more than 500)
const querySnapshot = await admin.firestore().collectionGroup('exercises').where('lastModified', '>=', `${convertedDate}`).get();
//iterate through documents
querySnapshot.forEach(async (doc) => {
//some calculations
//get document to store the calculated data
const oldRefPath = doc.ref.path.split('/trainings/');
const newRefPath = `${oldRefPath[0]}/exercises/`;
const document = await getDocumentSnapshotToSave(newRefPath, doc.data().exercise);
document.forEach(async (doc) => {
//check if value exists
const getDocument = await admin.firestore().doc(`${doc.ref.path}`).collection('AnalyseData').doc(`${year}`).get();
if (getDocument && getDocument.exists) {
await document.update({
//map filled with data which gets added to the exisiting map
})
} else {
await document.set({
//set document if it is not existing
}, {
merge: true
});
await document.update({
//update document after set
})
}
})
})
})
The code you have in your question does not use a transaction on Firestore, so is not tied to the limit you quote/link.
I'd still recommend putting a limit on your query through, and processing the documents in reasonable batches (a couple of hundred being reasonable) so that you don't put an unpredictable memory load on your code.

how do i write a Firebase cloud function to update a collection based on the data in another collection.?

I have been trying to add a feature of automatic email to myself if the value in a certain field goes below certain number. my database looks like this firestore database
I want to write a cloud function that automatically adds data to the mail collection which then triggers an email using Firebase Trigger Email. The trigger I'm looking for is the price_change(which is in string format, therefore I'm trying to convert it to int inside the cloud function) and compare it and update mail collection.
my code looks like this. how do i fix it?
exports.btcMail = functions.firestore
.document('cryptos/Bitcoin/1h/{wildcard}')
.onWrite((change, context) => {
const newval = change.data();
console.log(newval)
const price = ParsInt(newval.price_change())
if (price < -200) {
admin.firestore().collection('mail').add({
to: 'someone.someone#gmail.com',
message: {
subject: 'big change in bitcoin prices!',
html: 'This is an <code>HTML</code> email body.',
},
});}
});
Since adding a document to Firestore is an asynchronous operation, you need to return the promise from that operation. Without that, Cloud Functions will likely terminate your code before it gets a chance to write the document.
exports.btcMail = functions.firestore
.document('cryptos/Bitcoin/1h/{wildcard}')
.onWrite((change, context) => {
const newval = change.data();
console.log(newval)
const price = ParsInt(newval.price_change())
if (price < -200) {
return admin.firestore().collection('mail').add({ // 👈 add return here
to: 'someone.someone#gmail.com',
message: {
subject: 'big change in bitcoin prices!',
html: 'This is an <code>HTML</code> email body.',
},
});
} else {
return null; // 👈 Add a return here to prevent paying longer than needed
}
});

is it okay if I intentionally make my Google Cloud Functions has multiple errors?

I have collection and sub-collection like this
users/{userID}/followers/{followerID}
everytime a follower document is deleted in followers sub-collection, then it will trigger this firestore trigger below to decrease the numberOfFollowers field in user document. this is triggered when a user click unfollow button
exports.onDeleteFollower = functions
.firestore.document("users/{userID}/followers/{followerID}")
.onDelete((snapshot, context) => {
// normally triggered after a user push unfollow button
// then update the user document
const userID = context.params.userID;
const updatedData = {
numberOfFollowers: admin.firestore.FieldValue.increment(-1),
};
return db.doc(`users/${userID}`).update(updatedData);
});
now I have a case like this ....
if a user deletes their account, then I will delete the user document ( users/{userID} ), but if I delete a user document, it will not automatically delete all documents inside its sub-collection, right
so after I delete the user document, I have another function to delete all documents inside the followers sub-collection.
but the problem is, the onDeleteFollower triggers function above will be executed multiple times, and it will throw error multiple times, because the user document has been deleted ( the function above will be used to a update a field in deleted user doc)
I will have this error in functions emulator
âš  functions: Error: 5 NOT_FOUND: no entity to update: app: "myApp"
path <
Element {
type: "users"
name: "u1-HuWQ5hoCQnOAwh0zRQM0nOe96K03"
}
>
âš  Your function was killed because it raised an unhandled error.
I actually can write a logic to check if a user document still exist or not. if exist then update numberOfFollowers field
but deleting a user document is very rare if compared to a user click the unfollow button, I think it is not very efficient.
I have a plan like this, I will intentionally let the errors happened. say a user has 1000 followers, then it will trigger the onDeleteFollower function above, then I will have 1000 function errors
my question is .....
is it okay if I have multiple errors in a short time like that? will Google Cloud Function terminates my function, or .... I don't know, I am worried something bad will happen that I don't know
as far as I know, cloud functions will automatically run the function again after it is killed, will my function always ready again after an error like that?
I can't let the follower update the organizer (user) document directly from the client app, because it is not safe. creating security rules to facilitate this is complicated and it seems error prone
Have you considered instead of setting/removing users/{userID}/followers/{followerID} directly, that you create a "follow request" system?
"users/{userID}/followRequests/{requestID}": { // requestID would be auto-generated
user: "{followerID}",
follow: true // true = add user as follower, false = remove user as follower
}
This then allows you to use a single onCreate trigger to update your followers list eliminating the need for your current onCreate and onDelete triggers on users/{userID}/followers/{followerID}. From this function you can implement restrictions on following other users like follow limits or denying follow requests for blocked users.
export const newFollowRequest = functions.firestore
.document('users/{userId}/followRequests/{requestId}')
.onCreate(async (snap, context) => {
const request = snap.data();
const followingUserId = request.user;
const followedUserId = context.params.userId;
const db = admin.firestore();
const userDocRef = db.doc(`users/${followedUserId}`);
const followerDocRef = userDocRef.child(`followers/${followingUserId}`);
// /users/${followingUserId}/following/${followedUserId} ?
try {
if (request.follow) {
// Example restriction: Is the user who is attempting to follow
// blocked by followedUserId?
// await assertUserIDHasNotBlockedUserID(followedUserId, followingUserId);
// following
db.update(userDocRef, {
numberOfFollowers: admin.firestore.FieldValue.increment(1),
});
db.set(followerDocRef, {
/* ... */
});
} else {
// unfollowing
db.update(userDocRef, {
numberOfFollowers: admin.firestore.FieldValue.increment(-1),
});
db.delete(followerDocRef);
}
// delete this request when successful
db.delete(snap.ref);
// commit database changes
await db.commit();
console.log(`#${followingUserId} ${request.follow ? "followed" : "unfollowed"} #${followedUserId} successfully`);
} catch (err) {
// something went wrong, update this document with a failure reason (to show on the client);
let failureReason = undefined;
switch (err.message) {
case "other user is blocked":
failureReason = "You are blocked by #otherUser";
break;
case "user is blocked":
failureReason = "You have blocked #otherUser";
break;
}
return db.ref(snap.ref)
.update({
failureReason: failureReason || "Unknown server error";
})
.then(() => {
if (failureReason) {
console.log("REQUEST REJECTED: " + failureReason);
} else {
console.error("UNEXPECTED ERROR:", err)
}
},
(err) => {
console.error("UNEXPECTED FIRESTORE ERROR:", err);
});
}
});

How to update a quantity in another document when creating a new document in the firebase firestore collection?

When I create a new document in the note collection, I want to update the quantity in the info document. What am I doing wrong?
exports.addNote = functions.region('europe-west1').firestore
.collection('users/{userId}/notes').onCreate((snap,context) => {
const uid = admin.user.uid.toString();
var t;
db.collection('users').doc('{userId}').collection('info').doc('info').get((querySnapshot) => {
querySnapshot.forEach((doc) => {
t = doc.get("countMutable").toString();
});
});
let data = {
countMutable: t+1;
};
db.collection("users").doc(uid).collection("info").doc("info").update({countMutable: data.get("countMutable")});
});
You have... a lot going on here. A few problems:
You can't trigger firestore functions on collections, you have to supply a document.
It isn't clear you're being consistent about how to treat the user id.
You aren't using promises properly (you need to chain them, and return them out of the function if you want them to execute properly).
I'm not clear about the relationship between the userId context parameter and the uid you are getting from the auth object. As far as I can tell, admin.user isn't actually part of the Admin SDK.
You risk multiple function calls doing an increment at the same time giving inconsistent results, since you aren't using a transaction or the increment operation. (Learn More Here)
The document won't be created if it doesn't already exist. Maybe this is ok?
In short, this all means you can do this a lot more simply.
This should do you though. I'm assuming that the uid you actually want is actually the one on the document that is triggering the update. If not, adjust as necessary.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.addNote = functions.firestore.document('users/{userId}/notes/{noteId}').onCreate((snap,context) => {
const uid = context.params.userId;
return db.collection("users").doc(uid).collection("info").doc("info").set({
countMutable: admin.firestore.FieldValue.increment(1)
}, { merge: true });
});
If you don't want to create the info document if it doesn't exist, and instead you want to get an error, you can use update instead of set:
return db.collection("users").doc(uid).collection("info").doc("info").update({
countMutable: admin.firestore.FieldValue.increment(1)
});

Uploading multiple files to stripe.files.create (Node-stripe)

I am trying to upload files to stripe which are submitted by the user in my frontend to verify their identity before they can sell on my platform.
Currently, the files are sent via an API request to the backend where I can upload a single file, and afterwards, I attach it to that user's account.
let file = {
data: fs.readFileSync(files.IDFront.path),
name: files.IDFront.name,
type: files.IDFront.type
}
stripe.files.create({
purpose: 'identity_document',
file
}, function(err, file) {
if(err) res.send({sucess:false, error: err})
else {
//attach to user's account
}
This works just fine, but some identity documents require pictures of the front and back, so my question is can I upload two files at once using stripe.files.create? I can't seem to find anything in Stripe's API docs which mentions this, and I don't want to use stripe.files.create twice in one function because I feel that isn't a very efficient way to write the function.
Any suggestions would be greatly appreciated
It is important to note that your documents still need to be sent to Stripe in their own calls in order to get their respective tokens.
The function below takes an object of document names and the returned stripe tokens
{
document: <stripeToken1>,
additional_document: <stripeToken2>
}
You can then iterate through these and append to the update object in one go
// create the document object template
const documentObject = {
individual: {
verification: {},
},
}
Object.entries(imageTokens).map(([_, token]) => {
const [keyToken] = Object.entries(token)
// separate key and token from object
const [documentKey, stripeTokenId] = keyToken
// convert our naming convention to stripes expected naming
const checkedKey = documentKey === 'document_front' ? 'document' :
documentKey
// append to document object
documentObject.individual.verification[checkedKey] = {
front: stripeTokenId,
}
return await stripe.accounts.update(stripeAccountId, documentObject)

Resources