How to remove mongo specific fields from result (NodeJS, Mongoose) - node.js

I want to remove all Mongo specific fields (like '_id') from query result. Is there a simple method to do this or should I remove fields manually? If yes, then which are that fields and how to do that?
I'm using NodeJS and Mongoose

You can use select() method for remove the field from your query:
Model.find({}).select("-removed_field").then (resp => {
// your code
});
You should specified the "-" before field name, to be remove this field.
If you want remove several fields - you can specified their as array:
Model.find({}).select(["-removed_field1", "-removed_field2" ... ]).then (resp => {
// your code
});
Also you can to select only specified fields, using this method without "-"
Model.find({}).select(["field1", "field2" ... ]).then (resp => {
// your code
});

If you want hide _id property you can use text argument with prefix - which will exclude this or that field from the result, for get sepecifict fields you should pass like this:
Entity.find({ ... }, 'field1 field2', function(err, entity) {
console.log(entity); // { field1: '...', field2: '...' }
});

You can specify a field to be excluded from results by using the optional 2nd parameter projection string of the find method:
Model.find({}, "-a -b").then (res => {
// objects in the res array will all have the
// 'a' and 'b' fields excluded.
});
https://mongoosejs.com/docs/api.html#model_Model.find (see projection)

you can use mongoose instance method two show specific fields from all documents
const userSchema = new mongoose.Schema({
email: {
type: String,
},
name: {
type: String,
maxlength: 128,
index: true,
trim: true,
},
});
userSchema.method({
transform() {
const transformed = {};
const fields = ['name', 'email'];
fields.forEach((field) => {
transformed[field] = this[field];
});
return transformed;
},
});
module.exports = mongoose.model('User', userSchema);

if You want to remove any specific fields like _id, You can try in two ways:
Suppose Here you try to find a user using User Model
User.find({ email: email }, { _id: 0 });
OR
const user = User.find({ email: email });
delete user._doc._id;

OP mentioned "from result", as far as I understood, it means, removing from the query result i.e. query result will contain the field, but will be removed from the query result.
A SO answer here mentions, that to modify a query result (which are immutable), we've to convert the result to Object using toObject() method (making it mutable).
To remove a field from a query result,
let immutableQueryResult = await Col.findById(idToBeSearched)
let mutableQueryResult = immutableQueryResult.toObject()
delete mutableQueryResult.fieldToBeRemoved
console.log(mutableQueryResult)
Another way of getting the mutable result is using the _doc property of the result:
let immutableQueryResult = await Col.findById(idToBeSearched)
let mutableQueryResult = immutableQueryResult._doc // _doc property holds the mutable object
delete mutableQueryResult.fieldToBeRemoved
console.log(mutableQueryResult)

Related

Only update specific values using updateOne $set in mongoose

I want to only update two values inside todo in the Mongoose model below
const userSchema = new mongoose.Schema({
name: String,
email: String,
phone: Number,
points: Number,
todo: {
}
})
The todo actually contains 11 values and I only want to update 2 values at a time which I am getting from the frontend
let {
t,
i
} = req.body.delitem;
await User.updateOne({
username: req.body.user
}, {
$set: {
todo: {
[t]: "",
[i]: ""
}
}
})
but what is happening is these two fields are getting set to blank strings but the remaining fields are also changing to NULL whereas I want the other fields to stay the same.
Use dot notation like this:
await User.updateOne({username:req.body.user},
{
"$set": {
"todo.t": "",
"todo.i": ""
}
})
Example here
If you use an object into $set you are telling mongo to replace the object by the new one.
I believe your issue is that you're using $set and passing a whole object which overwrites the whole todo with your t and i fields and you've enforced required on those other fields.
If you see the docs in mongo for $set you can instead update specific fields in subobjects. The following snippet should be more appropriate.
let {t,i} = req.body.delitem;
await User.updateOne({username:req.body.user}, {
$set: {
"todo.t": "",
"todo.i": ""
}
})

Prevent _id field from being supplied in a MongoDB query

I'm using Mongoose in NodeJS to control a MongoDB database.
I'm creating an API and for obvious security reasons, I want to prevent the auto generated document _id field from getting replaced by a manually generated one in the API request.
Schema:
{ name: String }
Creating a document:
const record = {
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
const insertRecords = async () => {
await Quiz.create(record);
};
insertRecords();
Results in the following document:
{
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
As can be seen, the _id supplied in the query, as long as it's a valid ObjectID, would replace the _id that was supposed to be auto generated by mongo.
Is there a way to check if this _id field is in the query so that I can reject the API request? The .create method triggers the pre save middleware hook which would always have the _id of the final document so I cannot depend on it to know whether the _id was in the query or it's the auto generated one.
The only option I found is to disable the _id field altogether but this does not make sense.
Solution #1 - Use .create() method with an explicit object.
It's actually easier than you think. This is self-explanatory - we only define what we want to allow. Mongoose will ignore anything that's not in the object.
const record = {
_id: '5e35517cc894c90327a34baf'
name: 'bob'
}
const insertRecords = async () => {
await Quiz.create({
name: record.name // only allow names.
});
};
insertRecords();
Solution #2 - Define a function to clear unwanted objects.
You can define a helper function to clear out unwanted fields.
const filterObj = (obj, ...allowedFields) => {
const newObject = {};
// If the current field is one of the allowed fields, keep them in the new object.
Object.keys(obj).forEach((el) => {
if (allowedFields.includes(el)) {
newObject[el] = obj[el];
}
});
return newObject;
};
How to use:
const filteredRecord = filterObj(record, 'name'); // arbitrary list of allowed fields. In this case, we'll only allow 'name'.
await Quiz.create(filteredRecord);

Mongoose overwriting data in MongoDB with default values in Subdocuments

I currently have a problem with updating data in MongoDB via mongoose. I have a nested Document of the following structure
const someSchema:Schema = new mongoose.Schema({
Title: String,
Subdocuments: [{
SomeValue: String
Position: {
X: {type: Number, default: 0},
Y: {type: Number, default: 0},
Z: {type: Number, default: 0}
}
}]
});
Now my problem is that I am updating this with findOneAndUpdateById. I have previously set the position to values other than the default. I want to update leaving the position as is by making my request without the Position as my frontend should never update it (another application does).
However the following call
const updateById = async (Id: string, NewDoc: DocClass) => {
let doc: DocClass | null = await DocumentModel.findOneAndUpdate(
{ _id: Id },
{ $set: NewDoc },
{ new: true, runValidators: true });
if (!doc) {
throw createError.documentNotFound(
{ msg: `The Document you tried to update (Id: ${Id}) does not exist` }
);
}
return doc;
}
Now this works fine if I don't send a Title for the value in the root of the schema (also if i turn on default values for that Title) but if I leave out the Position in the Subdocument it gets reset to the default values X:0, Y:0, Z:0.
Any ideas how I could fix this and don't set the default values on update?
Why don't you find the document by id, update the new values, then save it?
const updateById = async (Id: string, NewDoc: Training) => {
const doc: Training | null = await TrainingModel.findById({ _id: Id });
if (!doc) {
throw createError.documentNotFound(
{ msg: `The Document you tried to update (Id: ${Id}) does not exist` }
);
}
doc.title = NewDoc.title;
doc.subdocument.someValue = NewDoc.subdocument.someValue
await doc.save();
return doc;
}
check out the link on how to update a document with Mongoose
https://mongoosejs.com/docs/documents.html#updating
Ok after I gave this some thought over the weekend I got to the conclusion that the behaviour of mongodb was correct.
Why?
I am passing a document and a query to the database. MongoDb then searches Documents with that query. It will update all Fields for which a value was supplied. If for Title I set a new string, the Title will get replaced with that one, a number with that one and so on. Now for my Subdocument I am passing an array. And as there is no query, the correct behavioud is that that field will get set to the array. So the subdocuments are not updated but indeed initialized. Which will correctly cause the default values to be set. If I just want to update the subdocuments this is not the correct way
How to do it right
For me the ideal way is to seperate the logic and create a seperate endpoint to update the subdocuments with their own query. So to update all given subdocuments the function would look something like this
const updateSubdocumentsById= async ({ Id, Subdocuments}: { Id: string; Subdocuments: Subdocument[]; }): Promise<Subdocument[]> => {
let updatedSubdocuments:Subdocument[] = [];
for (let doc of Subdocuments){
// Create the setter
let set = {};
for (let key of Object.keys(doc)){
set[`Subdocument.$.${key}`] = doc[key];
}
// Update the subdocument
let updatedDocument: Document| null = await DocumentModel.findOneAndUpdate(
{"_id": Id, "Subdocuments._id": doc._id},
{
"$set" : set
},
{ new : true}
);
// Aggregate and return the updated Subdocuments
if(updatedDocument){
let updatedSubdocument:Subdocument = updatedTraining.Subdocuments.filter((a: Subdocument) => a._id.toString() === doc._id)[0];
if(updatedSubdocument) updatedSubdocuments.push(updatedSubdocument);
}
}
return updatedSubdocuments;
}
Been struggling with this myself all evening. Just worked out a really simple solution that as far as I can see works perfectly.
const venue = await Venue.findById(_id)
venue.name = name
venue.venueContact = venueContact
venue.address.line1 = line1 || venue.address.line1
venue.address.line2 = line2 || venue.address.line2
venue.address.city = city || venue.address.city
venue.address.county = county || venue.address.county
venue.address.postCode = postCode || venue.address.postCode
venue.address.country = country || venue.address.country
venue.save()
res.send(venue)
The result of this is any keys that don't receive a new value will just be replaced by the original values.

Incorrect Subdocument Being Updated?

I've got a Schema with an array of subdocuments, I need to update just one of them. I do a findOne with the ID of the subdocument then cut down the response to just that subdocument at position 0 in the returned array.
No matter what I do, I can only get the first subdocument in the parent document to update, even when it should be the 2nd, 3rd, etc. Only the first gets updated no matter what. As far as I can tell it should be working, but I'm not a MongoDB or Mongoose expert, so I'm obviously wrong somewhere.
var template = req.params.template;
var page = req.params.page;
console.log('Template ID: ' + template);
db.Template.findOne({'pages._id': page}, {'pages.$': 1}, function (err, tmpl) {
console.log('Matched Template ID: ' + tmpl._id);
var pagePath = tmpl.pages[0].body;
if(req.body.file) {
tmpl.pages[0].background = req.body.filename;
tmpl.save(function (err, updTmpl) {
console.log(updTmpl);
if (err) console.log(err);
});
// db.Template.findOne(tmpl._id, function (err, tpl) {
// console.log('Additional Matched ID: ' + tmpl._id);
// console.log(tpl);
// tpl.pages[tmpl.pages[0].number].background = req.body.filename;
// tpl.save(function (err, updTmpl){
// if (err) console.log(err);
// });
// });
}
In the console, all of the ID's match up properly, and even when I return the updTmpl, it's saying that it's updated the proper record, even though its actually updated the first subdocument and not the one it's saying it has.
The schema just in case:
var envelopeSchema = new Schema({
background: String,
body: String
});
var pageSchema = new Schema({
background: String,
number: Number,
body: String
});
var templateSchema = new Schema({
name: { type: String, required: true, unique: true },
envelope: [envelopeSchema],
pagecount: Number,
pages: [pageSchema]
});
templateSchema.plugin(timestamps);
module.exports = mongoose.model("Template", templateSchema);
First, if you need req.body.file to be set in order for the update to execute I would recommend checking that before you run the query.
Also, is that a typo and req.body.file is supposed to be req.body.filename? I will assume it is for the example.
Additionally, and I have not done serious testing on this, but I believe your call will be more efficient if you specify your Template._id:
var template_id = req.params.template,
page_id = req.params.page;
if(req.body.filename){
db.Template.update({_id: template_id, 'pages._id': page_id},
{ $set: {'pages.$.background': req.body.filename} },
function(err, res){
if(err){
// err
} else {
// success
}
});
} else {
// return error / missing data
}
Mongoose doesn't understand documents returned with the positional projection operator. It always updates an array of subdocuments positionally, not by id. You may be interested in looking at the actual queries that mongoose is building - use mongoose.set('debug', true).
You'll have to either get the entire array, or build your own MongoDB query and go around mongoose. I would suggest the former; if pulling the entire array is going to cause performance issues, you're probably better off making each of the subdocuments a top-level document - documents that grow without bounds become problematic (at the very least because Mongo has a hard document size limit).
I'm not familiar with mongoose but the Mongo update query might be:
db.Template.update( { "pages._id": page }, { $set: { "pages.$.body" : body } } )

How to protect the password field in Mongoose/MongoDB so it won't return in a query when I populate collections?

Suppose I have two collections/schemas. One is the Users Schema with username and password fields, then, I have a Blogs Schema that has a reference to the Users Schema in the author field. If I use Mongoose to do something like
Blogs.findOne({...}).populate("user").exec()
I will have the Blog document and the user populated too, but how do I prevent Mongoose/MongoDB from returning the password field? The password field is hashed but it shouldn't be returned.
I know I can omit the password field and return the rest of the fields in a simple query, but how do I do that with populate. Also, is there any elegant way to do this?
Also, in some situations I do need to get the password field, like when the user wants to login or change the password.
You can change the default behavior at the schema definition level using the select attribute of the field:
password: { type: String, select: false }
Then you can pull it in as needed in find and populate calls via field selection as '+password'. For example:
Users.findOne({_id: id}).select('+password').exec(...);
.populate('user' , '-password')
http://mongoosejs.com/docs/populate.html
JohnnyHKs answer using Schema options is probably the way to go here.
Also note that query.exclude() only exists in the 2.x branch.
Edit:
After trying both approaches, I found that the exclude always approach wasn't working for me for some reason using passport-local strategy, don't really know why.
So, this is what I ended up using:
Blogs.findOne({_id: id})
.populate("user", "-password -someOtherField -AnotherField")
.populate("comments.items.user")
.exec(function(error, result) {
if(error) handleError(error);
callback(error, result);
});
There's nothing wrong with the exclude always approach, it just didn't work with passport for some reason, my tests told me that in fact the password was being excluded / included when I wanted. The only problem with the include always approach is that I basically need to go through every call I do to the database and exclude the password which is a lot of work.
After a couple of great answers I found out there are two ways of doing this, the "always include and exclude sometimes" and the "always exclude and include sometimes"?
An example of both:
The include always but exclude sometimes example:
Users.find().select("-password")
or
Users.find().exclude("password")
The exclude always but include sometimes example:
Users.find().select("+password")
but you must define in the schema:
password: { type: String, select: false }
You can achieve that using the schema, for example:
const UserSchema = new Schema({/* */})
UserSchema.set('toJSON', {
transform: function(doc, ret, opt) {
delete ret['password']
return ret
}
})
const User = mongoose.model('User', UserSchema)
User.findOne() // This should return an object excluding the password field
User.find().select('-password') is the right answer. You can not add select: false on the Schema since it will not work, if you want to login.
I'm using for hiding password field in my REST JSON response
UserSchema.methods.toJSON = function() {
var obj = this.toObject(); //or var obj = this;
delete obj.password;
return obj;
}
module.exports = mongoose.model('User', UserSchema);
I found another way of doing this, by adding some settings to schema configuration.
const userSchema = new Schema({
name: {type: String, required: false, minlength: 5},
email: {type: String, required: true, minlength: 5},
phone: String,
password: String,
password_reset: String,
}, { toJSON: {
virtuals: true,
transform: function (doc, ret) {
delete ret._id;
delete ret.password;
delete ret.password_reset;
return ret;
}
}, timestamps: true });
By adding transform function to toJSON object with field name to exclude. as In docs stated:
We may need to perform a transformation of the resulting object based
on some criteria, say to remove some sensitive information or return a
custom object. In this case we set the optional transform function.
const userSchema = new mongoose.Schema(
{
email: {
type: String,
required: true,
},
password: {
type: String,
required: true,
},
},
{
toJSON: {
transform(doc, ret) {
delete ret.password;
delete ret.__v;
},
},
}
);
Blogs.findOne({ _id: id }, { "password": 0 }).populate("user").exec()
While using password: { type: String, select: false } you should keep in mind that it will also exclude password when we need it for authentication. So be prepared to handle it however you want.
Assuming your password field is "password" you can just do:
.exclude('password')
There is a more extensive example here
That is focused on comments, but it's the same principle in play.
This is the same as using a projection in the query in MongoDB and passing {"password" : 0} in the projection field. See here
router.get('/users',auth,(req,res)=>{
User.findById(req.user.id)
//skip password
.select('-password')
.then(user => {
res.json(user)
})
})
You can pass a DocumentToObjectOptions object to schema.toJSON() or schema.toObject().
See TypeScript definition from #types/mongoose
/**
* The return value of this method is used in calls to JSON.stringify(doc).
* This method accepts the same options as Document#toObject. To apply the
* options to every document of your schema by default, set your schemas
* toJSON option to the same argument.
*/
toJSON(options?: DocumentToObjectOptions): any;
/**
* Converts this document into a plain javascript object, ready for storage in MongoDB.
* Buffers are converted to instances of mongodb.Binary for proper storage.
*/
toObject(options?: DocumentToObjectOptions): any;
DocumentToObjectOptions has a transform option that runs a custom function after converting the document to a javascript object. Here you can hide or modify properties to fill your needs.
So, let's say you are using schema.toObject() and you want to hide the password path from your User schema. You should configure a general transform function that will be executed after every toObject() call.
UserSchema.set('toObject', {
transform: (doc, ret, opt) => {
delete ret.password;
return ret;
}
});
This is more a corollary to the original question, but this was the question I came across trying to solve my problem...
Namely, how to send the user back to the client in the user.save() callback without the password field.
Use case: application user updates their profile information/settings from the client (password, contact info, whatevs). You want to send the updated user information back to the client in the response, once it has successfully saved to mongoDB.
User.findById(userId, function (err, user) {
// err handling
user.propToUpdate = updateValue;
user.save(function(err) {
// err handling
/**
* convert the user document to a JavaScript object with the
* mongoose Document's toObject() method,
* then create a new object without the password property...
* easiest way is lodash's _.omit function if you're using lodash
*/
var sanitizedUser = _.omit(user.toObject(), 'password');
return res.status(201).send(sanitizedUser);
});
});
The solution is to never store plaintext passwords. You should use a package like bcrypt or password-hash.
Example usage to hash the password:
var passwordHash = require('password-hash');
var hashedPassword = passwordHash.generate('password123');
console.log(hashedPassword); // sha1$3I7HRwy7$cbfdac6008f9cab4083784cbd1874f76618d2a97
Example usage to verify the password:
var passwordHash = require('./lib/password-hash');
var hashedPassword = 'sha1$3I7HRwy7$cbfdac6008f9cab4083784cbd1874f76618d2a97';
console.log(passwordHash.verify('password123', hashedPassword)); // true
console.log(passwordHash.verify('Password0', hashedPassword)); // false
const { password, ...others } = user._doc;
and send it like this
res.status(200).json(others);
*** I have two solutions for this:
// OPT: 1
/** If you print these params, the doc and ret are the same objects
* and opt is another object with special params (only details): */
userSchema.set('toJSON', {
transform: function(doc, ret, opt) {
console.log("DOC-RET-OPT", {
doc,
ret,
opt
});
// You can remove the specific params with this structure
delete ret['password'];
delete ret['__v'];
return ret;
}
});
// REMEMBER: You cannot apply destructuring for the objects doc or ret...
// OPT: 2
/* HERE: You can apply destructuring object 'cause the result
* is toObject instance and not document from Mongoose... */
userSchema.methods.toJSON = function() {
const {
__v,
password,
...user
} = this.toObject();
return user;
};
// NOTE: The opt param has this structure:
opt: {
_calledWithOptions: {},
flattenDecimals: true,
transform: [Function: transform],
_isNested: true,
json: true,
minimize: true
}

Resources