Desired Behaviour
After updating a property value in multiple documents, access the value of that updated property for each document updated.
(for context, I am incrementing multiple users' notification counts, and then sending each updated value back to the respective user, if they are logged in, via socket.io)
What I've Tried
let collection = mongo_client.db("users").collection("users");
let filter = { "user_email": { $in: users_to_notify } };
let update = { $inc: { "new_notifications": 1 } };
let result = await collection.updateMany(filter, update);
// get the `new_notifications` value for all updated documents here
There doesn't seem to be a returnOriginal type option applicable to the updateMany() method.
https://docs.mongodb.com/manual/reference/method/db.collection.updateMany
http://mongodb.github.io/node-mongodb-native/3.2/api/Collection.html#updateMany
That is understandable, because returnOriginal only seems to make sense when updating one document, eg in the options for the findOneAndUpdate() method.
Question
If all of the above assumptions are true, what would be the best way to get the new_notifications value for all updated documents after the updateMany() method has finished?
Is it just a matter of making another database call to get the updated values?
for example - this works:
let collection = mongo_client.db("users").collection("users");
let filter = { "user_email": { $in: users_to_notify } };
let update = { $inc: { "new_notifications": 1 } };
await collection.updateMany(filter, update);
// make another database call to get updated values
var options = { projection: { user_email: 1, username: 1, new_notifications: 1 } };
let docs = collection.find(filter, options).toArray();
/*expected result:
[{
"user_email": "user_1#somedomain.com",
"username": "user_1",
"new_notifications": 17
},
{
"user_email": "user_2#somedomain.com",
"username": "user_2",
"new_notifications": 5
}]*/
// create an object where each doc's user_email is a key with the value of new_notifications
var new_notifications_object = {};
// iterate over docs and add each property and value
for (let obj of docs) {
new_notifications_object[obj.user_email] = obj.new_notifications;
}
/*expected result:
{
"user_1#somedomain.com": 17,
"user_2#somedomain.com": 3
}*/
// iterare over logged_in_users
for (let key of Object.keys(logged_in_users)) {
// get logged in user's email and socket id
let user_email = logged_in_users[key].user_email;
let socket_id = key;
// if the logged in user's email is in the users_to_notify array
if (users_to_notify.indexOf(user_email) !== -1) {
// emit to the socket's personal room
let notifications_count = new_notifications_object[user_email];
io.to(socket_id).emit('new_notification', { "notifications_count": notifications_count });
}
}
updateMany() does not return the updated documents but some counts. You have to just run another query for getting those updates.
Would suggest you do a find({}) first, then findByIdAndUpdate() in a loop of the found items.
let updatedRecord = [];
let result = await Model.find(query);
if (result.length > 0) {
result.some(async e => {
updatedRecord.push(await Model.findByIdAndUpdate(e._id, updateObj, { new: true }));
});
}
return updatedRecord;
Related
I'm trying to update a document in MongoDB using NodeJS (NextJS). My current code is:
import connect from "../../util/mongodb";
async function api(req, res) {
if (req.method === "POST") {
const { id } = req.body;
const { name } = req.body;
const { email} = req.body;
const { anything1 } = req.body;
const { anything2 } = req.body;
if (!id) {
res.status(400).json({ "error": "missing id param" });
return;
}
const { db } = await connect();
const update = await db.collection("records_collection").findOneAndUpdate(
{ id },
{
$set: {
name,
email,
anything1,
anything2
}
},
{ returnOriginal: false }
);
res.status(200).json(update);
} else {
res.status(400).json({ "error": "wrong request method" });
}
}
export default api;
Everything is working. But I would like to request only the ID as mandatory, and for the other information, leave optional.
In this code, passing the id and name for example, the other three fields (email, anything1 and anything2) will be null in the document.
It is possible to implement the update without requiring all document information and ignore when body fields are null? (As a beginner in NodeJS and MongoDB, the only way to do that that comes to my head now is to surround it all by a lot of if...)
If I've understood your question correctly you can achieve your requirement using the body object in $set stage.
If there is a field which not exists in the object, mongo will not update that field.
As an example check this example where only name field is updated and the rest of fields are not set null.
Another example with 2 fields updated and 3 fields.
You can see how only is updated the field into object in the $set.
So you only need to pass the object received into the query. Something like this:
const update = await db.collection("records_collection").findOneAndUpdate(
{ id },
{
$set: req.body
},
{ returnOriginal: false }
);
I have a node API which connects to a mongoDB through mongoose. I am creating an advanced results middleware that enabled selecting, filtering, sorting, pagination etc. based on a Brad Traversy course Node.js API Masterclass With Express & MongoDB. This is all good.
I am adapting the code from the course to be able to use the $ne (not equal) operator and I want to be able to get a model that is not equal to a nested property (user id) of the model. I am using this for an explore feature to see a list of things, but I don't want to show the user their own things. I am having trouble figuring out how to access the id property.
********************* UPDATE *********************
It seems all the documentation I've read recommends writing const injected like this:
const injected = {
'user._id': { "$ne": req.user.id }
};
but for some reason it is not working. I can query top level properties that are just a plain string value like this:
const injected = {
access: { "$ne": "public" }
};
but not a property on an object. Does anyone know why? Is it because the property I want to query is an id? I've also tried:
const injected = {
'user._id': { "$ne": mongoose.Types.ObjectId(req.user.id) }
};
which also does not work...
So the model looks like this:
{
name: 'Awesome post',
access: 'public',
user: {
_id: '2425635463456241345', // property I want to access
}
}
then the actual advanced results middleware looks like this and it's the 'injected' object where I am trying to access id. In the course brad uses this syntax to use lte (/?averageCost[lte]=10000) but I do not get any results with my ne. Can anyone help me here?
const advancedResults = (model, populate) => async (req, res, next) => {
let query;
const injected = {
access: 'public',
'user._id[ne]': req.user.id, // I don't think user._id[ne] is correct
};
}
// Copy req.query
const reqQuery = { ...req.query, ...injected };
console.log('injected: ', injected);
// Fields to exclude
const removeFields = ['select', 'sort', 'page', 'limit'];
// Loop over removeFields and delete them from reqQuery
removeFields.forEach(param => delete reqQuery[param]);
// Create query string
let queryStr = JSON.stringify(reqQuery);
// Create operators ($gt, $gte, etc)
queryStr = queryStr.replace(/\b(gt|gte|lt|lte|in|ne)\b/g, match => `$${match}`);
// Finding resource and remove version
query = model.find(JSON.parse(queryStr)).select('-__v');
// Select Fields
if (req.query.select) {
const fields = req.query.select.split(',').join(' ');
query = query.select(fields);
}
// Sort
if (req.query.sort) {
const sortBy = req.query.sort.split(',').join(' ');
query = query.sort(sortBy);
} else {
query = query.sort('-createdAt');
}
// Pagination
const page = parseInt(req.query.page, 10) || 1;
const limit = parseInt(req.query.limit, 10) || 25;
const startIndex = (page - 1) * limit;
const endIndex = page * limit;
const total = await model.countDocuments(JSON.parse(queryStr));
query = query.skip(startIndex).limit(limit);
if (populate) {
query = query.populate(populate);
}
// Executing query
const results = await query;
// Pagination result
const pagination = {};
if (endIndex < total) {
pagination.next = {
page: page + 1,
limit,
};
}
if (startIndex > 0) {
pagination.prev = {
page: page - 1,
limit,
};
}
res.advancedResults = {
success: true,
count: results.length,
pagination,
data: results,
};
next();
};
module.exports = advancedResults;
Answering your question about how to use $ne:
The use of $ne is as follows:
"field":{
"$ne": yourValue
}
Into your query should be like:
"user._id": {
"$ne": req.user.id
}
Example here
$ne operator will return all document where the field value don't match with the given value.
As you have done, to acces the nested field is necessary use the dot notation.
Also, to ensure it works, if your schema defines _id as ObjectId maybe is necessary parse req.user.id to ObjectId.
But if in your schema is a string then should works.
So try (not tested at all):
const injected = {
'user._id': { "$ne": req.user.id }
};
I currently have a problem with updating data in MongoDB via mongoose. I have a nested Document of the following structure
const someSchema:Schema = new mongoose.Schema({
Title: String,
Subdocuments: [{
SomeValue: String
Position: {
X: {type: Number, default: 0},
Y: {type: Number, default: 0},
Z: {type: Number, default: 0}
}
}]
});
Now my problem is that I am updating this with findOneAndUpdateById. I have previously set the position to values other than the default. I want to update leaving the position as is by making my request without the Position as my frontend should never update it (another application does).
However the following call
const updateById = async (Id: string, NewDoc: DocClass) => {
let doc: DocClass | null = await DocumentModel.findOneAndUpdate(
{ _id: Id },
{ $set: NewDoc },
{ new: true, runValidators: true });
if (!doc) {
throw createError.documentNotFound(
{ msg: `The Document you tried to update (Id: ${Id}) does not exist` }
);
}
return doc;
}
Now this works fine if I don't send a Title for the value in the root of the schema (also if i turn on default values for that Title) but if I leave out the Position in the Subdocument it gets reset to the default values X:0, Y:0, Z:0.
Any ideas how I could fix this and don't set the default values on update?
Why don't you find the document by id, update the new values, then save it?
const updateById = async (Id: string, NewDoc: Training) => {
const doc: Training | null = await TrainingModel.findById({ _id: Id });
if (!doc) {
throw createError.documentNotFound(
{ msg: `The Document you tried to update (Id: ${Id}) does not exist` }
);
}
doc.title = NewDoc.title;
doc.subdocument.someValue = NewDoc.subdocument.someValue
await doc.save();
return doc;
}
check out the link on how to update a document with Mongoose
https://mongoosejs.com/docs/documents.html#updating
Ok after I gave this some thought over the weekend I got to the conclusion that the behaviour of mongodb was correct.
Why?
I am passing a document and a query to the database. MongoDb then searches Documents with that query. It will update all Fields for which a value was supplied. If for Title I set a new string, the Title will get replaced with that one, a number with that one and so on. Now for my Subdocument I am passing an array. And as there is no query, the correct behavioud is that that field will get set to the array. So the subdocuments are not updated but indeed initialized. Which will correctly cause the default values to be set. If I just want to update the subdocuments this is not the correct way
How to do it right
For me the ideal way is to seperate the logic and create a seperate endpoint to update the subdocuments with their own query. So to update all given subdocuments the function would look something like this
const updateSubdocumentsById= async ({ Id, Subdocuments}: { Id: string; Subdocuments: Subdocument[]; }): Promise<Subdocument[]> => {
let updatedSubdocuments:Subdocument[] = [];
for (let doc of Subdocuments){
// Create the setter
let set = {};
for (let key of Object.keys(doc)){
set[`Subdocument.$.${key}`] = doc[key];
}
// Update the subdocument
let updatedDocument: Document| null = await DocumentModel.findOneAndUpdate(
{"_id": Id, "Subdocuments._id": doc._id},
{
"$set" : set
},
{ new : true}
);
// Aggregate and return the updated Subdocuments
if(updatedDocument){
let updatedSubdocument:Subdocument = updatedTraining.Subdocuments.filter((a: Subdocument) => a._id.toString() === doc._id)[0];
if(updatedSubdocument) updatedSubdocuments.push(updatedSubdocument);
}
}
return updatedSubdocuments;
}
Been struggling with this myself all evening. Just worked out a really simple solution that as far as I can see works perfectly.
const venue = await Venue.findById(_id)
venue.name = name
venue.venueContact = venueContact
venue.address.line1 = line1 || venue.address.line1
venue.address.line2 = line2 || venue.address.line2
venue.address.city = city || venue.address.city
venue.address.county = county || venue.address.county
venue.address.postCode = postCode || venue.address.postCode
venue.address.country = country || venue.address.country
venue.save()
res.send(venue)
The result of this is any keys that don't receive a new value will just be replaced by the original values.
Background: Im developing an app that shows analytics for inventory management.
It gets an office EXCEL file uploaded, and as the file uploads the app convert it to an array of JSONs. Then, it comapers each json object with the objects in the DB, change its quantity according to the XLS file, and add a timestamp to the stamps array which contain the changes in qunatity.
For example:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":100,
"Warehouse":4,
"stamps":[]
}
after the XLS upload, lets say we sold 10 units, it should look like that:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":90,
"Warehouse":4,
"stamps":[{"1548147562": -10}]
}
Right now i cant find the right commands for mongoDB to do it, Im developing in Node.js and Angular, Would love to read some ideas.
for (let i = 0; i < products.length; i++) {
ProductsDatabase.findOneAndUpdate(
{"_id": products[i]['id']},
//CHANGE QUANTITY AND ADD A STAMP
...
}
You would need two operations here. The first will be to get an array of documents from the db that match the ones in the JSON array. From the list you compare the 'product_quantity' keys and if there is a change, create a new array of objects with the product id and change in quantity.
The second operation will be an update which uses this new array with the change in quantity for each matching product.
Armed with this new array of updated product properties, it would be ideal to use a bulk update for this as looping through the list and sending
each update request to the server can be computationally costly.
Consider using the bulkWrite method which is on the model. This accepts an array of write operations and executes each of them of which a typical update operation
for your use case would have the following structure
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
So your operations would follow this pattern:
(async () => {
let bulkOperations = []
const ids = products.map(({ id }) => id)
const matchedProducts = await ProductDatabase.find({
'_id': { '$in': ids }
}).lean().exec()
for(let product in products) {
const [matchedProduct, ...rest] = matchedProducts.filter(p => p._id === product.id)
const { _id, product_quantity } = matchedProduct
const changeInQuantity = product.product_quantity - product_quantity
if (changeInQuantity !== 0) {
const stamps = { [(new Date()).getTime()] : changeInQuantity }
bulkOperations.push({
'updateOne': {
'filter': { _id },
'update': {
'$inc': { 'product_quantity': changeInQuantity },
'$push': { stamps }
}
}
})
}
}
const bulkResult = await ProductDatabase.bulkWrite(bulkOperations)
console.log(bulkResult)
})()
You can use mongoose's findOneAndUpdate to update the existing value of a document.
"use strict";
const ids = products.map(x => x._id);
let operations = products.map(xlProductData => {
return ProductsDatabase.find({
_id: {
$in: ids
}
}).then(products => {
return products.map(productData => {
return ProductsDatabase.findOneAndUpdate({
_id: xlProductData.id // or product._id
}, {
sku: xlProductData.sku,
product_name: xlProductData.product_name,
product_cost: xlProductData.product_cost,
product_price: xlProductData.product_price,
Warehouse: xlProductData.Warehouse,
product_quantity: productData.product_quantity - xlProductData.product_quantity,
$push: {
stamps: {
[new Date().getTime()]: -1 * xlProductData.product_quantity
}
},
updated_at: new Date()
}, {
upsert: false,
returnNewDocument: true
});
});
});
});
Promise.all(operations).then(() => {
console.log('All good');
}).catch(err => {
console.log('err ', err);
});
Assume I have db as;
{ name: "alex" , id: "1"}
I want to update the collection. I want to add "Mr." to the value in name field.
{ name: "Mr.alex" , id: "1"}
How can I do this? Should I wrote 2 query as;
db.collection("user").find({id : "1"}).toArray(function(err, result){
var name = result[0].name;
db.collection("user").updateOne({id : "1"}, {name: "Mr." + name},function(err, result){
})
})
Isn't there any better way to do this as x = x+1 in mongodb?
AFAIK there would be two queries, update operator won't take any field's value and need provided value.
However if you need to do it for all the document or large amount of document, you can write a script and use cursor forEach and for each document change the name and call db.user.save() by passing the argument user object.
Bottom line remains same.
try below:
var cursor = db.users.find({});
while (cursor.hasNext()) {
var user = cursor.next();
user.name = "Mr." + user.name;
user.save();
}
Not possible using one query. You have to iterate through the documents and save them with updated result.
Currently there is no way to reference the retrieved document in the same query, which would allow you to find and update a document within the same operation.
Thus, you will have to make multiple queries to accomplish what you are looking for:
/*
* Single Document
*/
// I'm assuming the id field is unique and can only return one document
var doc = db.collection('user').findOne({ id: '1' });
try {
db.collection('user').updateOne({ id: '1' }, { $set: { name: 'Mr. ' + doc.name }});
} catch (e) {
print(e);
}
If you want to handle multiple update operations, you can do so by using a Bulk() operations builder:
/*
* Multiple Documents
*/
var bulk = db.collection('user').initializeUnorderedBulkOp();
// .forEach is synchronous in MongoDB, as it performs I/O operations
var users = db.collection('user').find({ id: { $gt: 1 } }).forEach(function(user) {
bulk.find({ id: user.id }).update({ $set: { name: 'Mr. ' + user.name }});
});
bulk.execute();
The key to updating the collection with the existing field is to loop through the array returned from the find().toarray() cursor method and update your collection using the Bulk API which allows you to send many update operations within a single request (as a batch).
Let's see with some examples how this pens out:
a) For MongoDB server version 3.2 and above
db.collection("user").find({id : "1"}).toArray(function(err, result){
var operations = [];
result.forEach(function(doc){
operations.push({
"updateOne": {
"filter": {
"_id": doc._id,
"name": doc.name
},
"update": {
"$set": { "name": "Mr." + doc.name }
}
}
});
// Send once in 500 requests only
if (operations.length % 500 === 0 ) {
db.collection("user").bulkWrite(operations, function(err, r) {
// do something with result
}
operations = [];
}
});
// Clear remaining queue
if (operations.length > 0) {
db.collection("user").bulkWrite(operations, function(err, r) {
// do something with result
}
}
})
In the above, you initialise your operations array which would be used by the Bulk API's bulkWrite() function and holds the update operations.
The result from the find().toarray() cursor function is then iterated to create the operations array with the update objects. The operations are limited to batches of 500.
The reason of choosing a lower value than the default batch limit of 1000 is generally a controlled choice. As noted in the documentation there, MongoDB by default will send to the server in batches of 1000 operations at a time at maximum and there is no guarantee that makes sure that these default 1000 operations requests actually fit under the 16MB BSON limit.
So you would still need to be on the "safe" side and impose a lower batch size that you can only effectively manage so that it totals less than the data limit in size when sending to the server.
a) If using MongoDB v3.0 or below:
// Get the collection
var col = db.collection('user');
// Initialize the unordered Batch
var batch = col.initializeUnorderedBulkOp();
// Initialize counter
var counter = 0;
col.find({id : "1"}).toArray(function(err, result){
result.forEach(function(doc) {
batch.find({
"_id": doc._id,
"name": doc.name
}).updateOne({
"$set": { "name": "Mr. " + doc.name }
});
counter++;
if (counter % 500 === 0) {
batch.execute(function(err, r) {
// do something with result
});
// Re-initialize batch
batch = col.initializeOrderedBulkOp();
}
});
if (counter % 1000 != 0 ){
batch.execute(function(err, r) {
// do something with result
});
}
});