mongoose difference between pre save and validate? When to use which one? - node.js

Currently I am using pre('save') to do validation:
UserSchema.pre('save', true, function(next, done) {
var self = this //in case inside a callback
var msg = helper.validation.user.username(self.username)
if (msg) {
self.invalidate('username', msg)
done(helper.getValidationError(msg))
}
else
done()
next()
})
The helper module has a function that accepts an input and returns error message.
exports.user = {
username: function(input) {
if (!input)
return 'username is required'
var min = 3
var max = 10
if (input.length < min)
return 'username min of length is ' + min
if (input.length > max)
return 'username max of length is ' + max
return null
}
}
There is another api validate to do similar things. What's the difference between them and which one should I use in what cases?

Update:
The validation is performed before the user defined hooks. You can follow this github post, where the contributor states,
not implementing this. validating first gives us a chance to stop
before continuing into user defined hooks which may include async
updates to other collections.
if we need validation to run again after we make a change from inside
a hook we can always manually run this.validate(next).
Outdated:
Yeah, there is a small difference i know.
Pre hooks are executed before validations.
There is a closed issue on github asking for validation before pre hooks, https://github.com/Automattic/mongoose/issues/400.
And also there is a reason for not having validation before pre hooks, stated in the same link by #kamholz:
Say you have two fields, foo and fooSort, both are required. fooSort is a
lowercased or otherwise transformed version of foo to be used in
sorting. Since fooSort can be automatically generated, it makes sense
to do so in a pre-save hook. But since validation runs first, it will
fail before the pre-save hook runs and has a chance to fill in the
fooSort value. It isn't a question of being able to run validation
again manually.
Again, if you want to validate something and then need the hook for post validate:
UserSchema.post('validate', function(next){
console.log("post validate called");
next();
});
So for summary, the one difference i see is,
you can use both as long as you get inputs to save in db directly without altering anything (only validation).
If you are altering anything you have to use pre save hook.

The above answer is incorrect. If you go to: https://github.com/Automattic/mongoose/issues/400 and read the responses to the topic from the mongoose contributor you'll see that he notes they are run before on purpose.
The topic is named: Validation should run after all other pre-save hooks
Mongoose contributor aheckmann states that:
right now its running before them
not implementing this. validating first gives us a chance to stop before continuing into user defined hooks which may include async updates to other collections.

Related

is changed function in sequelize like isModified in mongoose?

mongoose isModified functionIn a hook I want to confirm whether a password has changed before executing encryption process. Mongoose has a function "isModified" and I believe Sequelize's "changed" function servers the same purpose.
I cannot get the "changed" function to work. I am looking for an example of how it is used.
There is an example of how changed() works in the official documentation
const mdl = await MyModel.findOne();
mdl.myJsonField.a = 1;
console.log(mdl.changed()) => false
await mdl.save(); // this will not save anything
mdl.changed('myJsonField', true);
console.log(mdl.changed()) => ['myJsonField']
await mdl.save(); // will save
Keep in mind that changes are detected for the top-level fields and only for changes that were made since the last save call.

Avoid triggering Firebase functions by real-time database on special cases

Sometimes we use the firebase functions triggered by real-time database (onCreate/onDelete/onUpdate ...) to do some logic (like counting, etc).
My question, would it be possible to avoid this trigger in some cases. Mainly, when I would like to allow a user to import a huge JSON to firebase?
Example:
a function E triggered on the creation of a new child in /examples. Normally, users add examples one by one to /examples and function E runs to do some logic. However, I would like to allow a user (from the front-end) to import 2000 children to /examples and the logic which is done by function E is possible at import time without the need for E. Then, I do not need E to be triggered for such a case where a high number of functions could be executed. (Note: I am aware of the 1000 limit)
Update:
based on the accepted answer, submitted my answer down.
As far as I know, there is no way to disable a Cloud Function programmatically without just deleting it. However this introduces an edge case where data is added to the database while the import is taking place.
A compromise would be to signal that the data you are uploading should be post-processed. Let's say you were uploading to /examples/{pushId}, instead of attaching the database trigger to /examples/{pushId}, attach it to /examples/{pushId}/needsProcessing (or something similar). Unfortunately this has the trade-off of not being able to make use of change objects for onUpdate() and onWrite().
const result = await firebase.database.ref('/examples').push({
title: "Example 1A",
desc: "This is an example",
attachments: { /* ... */ },
class: "-MTjzAKMcJzhhtxwUbFw",
author: "johndoe1970",
needsProcessing: true
});
async function handleExampleProcessing(snapshot, context) {
// do post processing if needsProcessing is truthy
if (!snapshot.exists() || !snapshot.val()) {
console.log('No processing needed, exiting.');
return;
}
const exampleRef = admin.database().ref(change.ref.parent); // /examples/{pushId}, as admin
const data = await exampleRef.once('value');
// do something with data, like mutate it
// commit changes
return exampleRef.update({
...data,
needsProcessing: null /* delete needsProcessing value */
});
}
const functionsExampleProcessingRef = functions.database.ref("examples/{pushId}/needsProcessing");
export const handleExampleNeedingProcessingOnCreate = functionsExampleProcessingRef.onCreate(handleExampleProcessing);
// this is only needed if you ever intend on writing `needsProcessing = /* some falsy value */`, I recommend just creating and deleting it, then you can use just the above trigger.
export const handleExampleNeedingProcessingOnUpdate = functionsExampleProcessingRef.onUpdate((change, context) => handleExampleProcessing(change.after, context));
An alternative to Sam's approach is to use feature flags to determine if a Cloud Function performs its main function. I often have this in my code:
exports.onUpload = functions.database
.ref("/uploads/{uploadId}")
.onWrite((event) => {
return ifEnabled("transcribe").then(() => {
console.log("transcription is enabled: calling Cloud Speech");
...
})
});
The ifEnabled is a simple helper function that checks (also in Realtime Database) if the feature is enabled:
function ifEnabled(feature) {
console.log("Checking if feature '"+feature+"' is enabled");
return new Promise((resolve, reject) => {
admin.database().ref("/config/features")
.child(feature)
.once('value')
.then(snapshot => {
if (snapshot.val()) {
resolve(snapshot.val());
}
else {
reject("No value or 'falsy' value found");
}
});
});
}
Most of my usage of this is during talks at conferences, to enable the Cloud Functions at the right time (as a deploy takes a bit longer than we'd like for a demo). But the same approach should work to temporarily disable features during for example data import.
Okay, another solution would be
A: Add a new table in firebase like /triggers-queue where all CRUD that should fire a background function are added. In this table, we add a key for each table that should have triggers - in our example /examples table. Any key that represents a table should also have /created, /updated, and /deleted keys as follows.
/examples
.../example-id-1
/triggers-queue
.../examples
....../created
........./example-id
....../updated
........./example-id
............old-value
....../deleted
........./example-id
............old-value
Note that the old-value should be added from app (front-end, etc).
We set triggers always onCreate on
/triggers-queue/examples/created/{exampleID} (simulate onCreate)
/triggers-queue/examples/updated/{exampleID} (simulate onUpdate)
/triggers-queue/examples/deleted/{exampleID} (simulate onDelete)
The fired function can know all the necessary info to handle the logic as follows:
Operation type: from the path (either: created, updated, or deleted)
key of the object: from the path
current data: by reading the corresponding table (i.e., /examples/id)
old data: from the triggers table
Good Points:
You can import a huge data to /examples table without firing any function as we do not add to the /triggers-queue
you can fanout functions to pass the limit 1000/sec. That is by setting triggers on (as an example to fanout on-create)
/triggers-queue/examples/created0/{exampleID} and
/triggers-queue/examples/created1/{exampleID}
bad-points:
more difficult to implement
need to write more data to firebase (like old-data) from the app.
B- Another way (although not an answer for this) is to move the login in the background function to an HTTP function and call it on every crud ops.

Updating a user concurrently in mongoose using model.save() throws error

I am getting a mongoose error when I attempt to update a user field multiple times.
What I want to achieve is to update that user based on some conditions after making an API call to an external resource.
From what I observe, I am hitting both conditions at the same time in the processUser() function
hence, user.save() is getting called almost concurrently and mongoose is not happy about that throwing me this error:
MongooseError [ParallelSaveError]: Can't save() the same doc multiple times in parallel. Document: 5ea1c634c5d4455d76fa4996
I know am guilty and my code is the culprit here because I am a novice. But is there any way I can achieve my desired result without hitting this error? Thanks.
function getLikes(){
var users = [user1, user2, ...userN]
users.forEach((user) => {
processUser(user)
})
}
async function processUser(user){
var result = await makeAPICall(user.url)
// I want to update the user based on the returned value from this call
// I am updating the user using `mongoose save()`
if (result === someCondition) {
user.meta.likes += 1
user.markModified("meta.likes")
try {
await user.save()
return
} catch (error) {
console.log(error)
return
}
} else {
user.meta.likes -= 1
user.markModified("meta.likes")
try {
await user.save()
return
} catch (error) {
console.log(error)
return
}
}
}
setInterval(getLikes, 2000)
There are some issues that need to be addressed in your code.
1) processUser is an asynchronous function. Array.prototype.forEach doesn't respect asynchronous functions as documented here on MDN.
2) setInterval doesn't respect the return value of your function as documented here on MDN, therefore passing a function that returns a promise (async/await) will not behave as intended.
3) setInterval shouldn't be used with functions that could potentially take longer to run than your interval as documented here on MDN in the Usage Section near the bottom of the page.
Hitting an external api for every user every 2 seconds and reacting to the result is going to be problematic under the best of circumstances. I would start by asking myself if this is absolutely the only way to achieve my overall goal.
If it is the only way, you'll probably want to implement your solution using the recursive setTimeout() mentioned at the link in #2 above, or perhaps using an async version of setInterval() there's one on npm here

Mongoose - Post Hook - Change Doc/Model values after load & Before Returning

Solved: I can only guess that delirium got me.
sorry for not figuring it out before posting, but here's the solution for the next persons sake. The try/catch is not necessary, but it did help me get out some errors that were not being printed to console. If there is a better way to document solving my own problem, I'd appreciate the feedback.
Old Question
I have docs stored in my mongo db that I want to modify before being returned to a find() or findOne(). I want to do this without changing the values and a single hook for all find variants. Call it a masking. I'm actually normalizing a numeric value to a scale of 0 to 1. I do not want to modify the value in the doc, just change the values when the doc is returned via find, findOne. E.G. "Change Model values after load in Mongoose"
Solution
schema.post('init', function(doc){
try{
console.log(doc); // prints the doc to console, complete
doc.property = "new property value"; // changes the document value
console.log(doc); // returns the modified document
return doc;
}
catch(error){
console.log(error);
}
}
Old Notes
Other threads note usage of the post init hook. I have tried the methods in the following related topics with no success.
Mongoose - how to tap schema middleware into the 'init' event?
Change Model values after load in Mongoose (this method has changed with 5.x)
schema.post('init', function(doc){
console.log('Tell me i fired'); // works, prints to console
console.log(doc); // works, prints full model to console
doc.valueField = doc.valueField2+doc.valueField3; // fails, no doc returned, throws no errors
});
schema.post('init', function(doc, next){
console.log('Tell me i fired'); // works, prints to console
console.log(doc); // works, prints full model to console
doc.valueField1 = doc.valueField2+doc.valueField3; // fails, no doc returned, throws no errors
next();
});
I've tried adding return(doc);, doc.save() and just about every mutation of the above code I can compile in my shrinking mental capacity. I keep throwing crap at the wall and nothing is sticking. The documentation over at mongoose is poor and the behavior changed with version 5.x with the removal of Asynch call backs (dropped the next() ).
This is a bug submission on mongoose where the dropping of next() is noted.
https://github.com/Automattic/mongoose/issues/6037
release notes from the 5.x release
https://github.com/Automattic/mongoose/blob/master/migrating_to_5.md#init-hook-signatures
~ Edited, because i solved my own problem ~

Model.create retrive how many documents have been created

I am using Model.create(Array) in Mongoose.
I want to provide user a feedback about how many documents have been created and how many of them haven't (i.e. they didn't validate).
I created a callback like this
User.create(usersToImport, function(err, docs) {
console.log(err);
console.log(docs);
}
The problem is that if any document does not validate, I only receive a validation error on the single non-valid document, while I cannot retrieve any information about the inserted documents.
Is there any way to get this information?
I think, you need something like .settle() method from when.js module.
Here is an example of doing it using when.js with mongoose 3.8.x:
when = require('when');
promises = usersToImport.map(function(user) {
return User.create(user); // returns Promise
});
when.settle(promises).then(function(results) {
// results is an array, containing following elements:
// { state: 'fulfilled', value: <document> }
// { state: 'rejected', value: <error> }
});
It's possible to do it without Promises (e.g. using async module), but the code will be much more complicated.
Model.create() isn't going to return that information for you. The best option would be to roll your own version of Model.create(), which is essentially a convenience method for calling Model.save() multiple times, and collate the information yourself. A good way to get started is the code for Model.create().

Resources