MongoDB: how to insert a sub-document? - node.js

I am using sub-documents in my MEAN project, to handle orders and items per order.
These are my (simplified) schemas:
var itemPerOrderSchema = new mongoose.Schema({
itemId: String,
count: Number
});
var OrderSchema = new mongoose.Schema({
customerId: String,
date: String,
items: [ itemPerOrderSchema ]
});
To insert items in itemPerOrderSchema array I currently do:
var orderId = '123';
var item = { itemId: 'xyz', itemsCount: 7 };
Order.findOne({ id: orderId }, function(err, order) {
order.items.push(item);
order.save();
});
The problem is that I obviously want one item per itemId, and this way I obtain many sub-documents per item...
One solution could be to loop through all order.items, but this is not optimal, of course (order.items could me many...).
The same problem could arise when querying order.items...
The question is: how do I insert items in itemPerOrderSchema array without having to loop through all items already inserted on the order?

If you can use an object instead of array for items, maybe you can change your schema a bit for a single-query update.
Something like this:
{
customerId: 123,
items: {
xyz: 14,
ds2: 7
}
}
So, each itemId is a key in an object, not an element of the array.
let OrderSchema = new mongoose.Schema({
customerId: String,
date: String,
items: mongoose.Schema.Types.Mixed
});
Then updating your order is super simple. Let's say you want to add 3 of items number 'xyz' to customer 123.
db.orders.update({
customerId: 123
},
{
$inc: {
'items.xyz': 3
}
},
{
upsert: true
});
Passing upsert here to create the order even if the customer doesn't have an entry.
The downsides of this:
it is that if you use aggregation framework, it is either impossible to iterate over your items, or if you have a limited, known set of itemIds, then very verbose. You could solve that one with mapReduce, which can be a little slower, depending on how many of them you have there, so YMMB.
you do not have a clean items array on the client. You could fix that with either client extracting this info (a simple let items = Object.keys(order.items).map(key => ({ key: order.items[key] })); or with a mongoose virtual field or schema.path(), but this is probably another question, already answered.

First of all, you probably need to add orderId to your itemPerOrderSchema because the combination of orderId and itemId will make the record unique.
Assuming that orderId is added to the itemPerOrderSchema, I would suggest the following implementation:
function addItemToOrder(orderId, newItem, callback) {
Order.findOne({ id: orderId }, function(err, order) {
if (err) {
return callback(err);
}
ItemPerOrder.findOne({ orderId: orderId, itemId: newItem.itemId }, function(err, existingItem) {
if (err) {
return callback(err);
}
if (!existingItem) {
// there is no such item for this order yet, adding a new one
order.items.push(newItem);
order.save(function(err) {
return callback(err);
});
}
// there is already item with itemId for this order, updating itemsCount
itemPerOrder.update(
{ id: existingItem.id },
{ $inc: { itemsCount: newItem.itemsCount }}, function(err) {
return callback(err);
}
);
});
});
}
addItemToOrder('123', { itemId: ‘1’, itemsCount: 7 }, function(err) {
if (err) {
console.log("Error", err);
}
console.log("Item successfully added to order");
});
Hope this may help.

Related

Updating values in an object inside an array with Mongoose

I have a schema nested inside another(main) schema. I'd like to increase a Number field in the first schema, however it is an array. So I'd need to access that specific object in that array, and increase a field inside it which is a number. I think what I am looking for is the $inc operator, however I couldn't seem to get it to work.
My schema's:
const chainSchema = new mongoose.Schema({
chainName: String,
streak: Number,
});
const userSchema = new mongoose.Schema({
email: String,
password: String,
googleId: String,
pomodoroStreak: Number,
chains: [chainSchema],
});
Post route:
app.post("/chainDisplay", function (req, res) {
const clickedChain = req.body.secret;
const clickedButton = req.body.submit;
if (clickedButton === "increase") {
Chain.findOneAndUpdate(
{ chainName: clickedChain },
{ $inc: { streak: 1 } },
function (err, foundChain) {
if (!err) {
res.redirect("/chain");
}
}
);
} else if (clickedButton === "decrease") {
Chain.findOneAndUpdate(
{ chainName: clickedChain },
{ $inc: { streak: -1 } },
function (err, foundChain) {
if (!err) {
res.redirect("/chain");
}
}
);
}
});
I obtain the chainName and trying to use that as a parameter to find that specific object, and increase or decrease the streak by 1. Thank you for your help in advance.
I've tried using the $inc operator alongside mongoose's findOneandUpdate method. I am wondering if I should be updating the User, rather than the Chain itself. I was expecting to increase or decrease the "streak" key by 1 when the relevant button is clicked.

Update document in MongoDB via NodeJS

So my knowledge of NodeJS and MongoDD are non-existent (just need to do a small code update for a friend) and I'm stuck.
Need to update a single document inside a collection via a unique id but can't seem to do it.
Here's the Model (I've trimmed it down and cut out all unnecessary data). I'm trying to update the field notes inside a transaction.
In short each entry in the given (an Agent) table will have a collection of multiple Transactions & Documents. I need to update a specific Transaction with the unique _id that is auto generated.
import { Schema, model } from 'mongoose';
interface Transaction {
first_name: string;
last_name: string;
type: string;
notes: string;
}
interface Agent {
org_id: number;
transactions: Array<Transaction>;
documents: Array<string>;
}
const transactionSchema = new Schema<Transaction>({
first_name: { type: String },
last_name: { type: String },
type: { type: String },
notes: String,
});
const transactionsSchema = new Schema<Agent>({
org_id: { type: Number },
transactions: [transactionSchema],
documents: [documentTypesSchema],
});
const AgentTransaction = model<Agent>(
'agent_transaction_table',
transactionsSchema
);
export default AgentTransaction;
Here's what I tried but didn't work (obviously), again I've trimmed out all unnecessary data. Just to clarify, the endpoint itself works, but the DB update does not.
import AgentTransaction from '../models/transaction'; // the above model
transaction.put('/notes', async (req, res) => {
const { org_id, transaction_id, notes } = req.body;
try {
const notesResult = await AgentTransaction.updateOne({
'transactions._id': transaction_id,
}, {
$set: {
'notes': notes
},
});
res
.status(200)
.json({ message: 'Updated', success: true, notesResult });
} catch (error) {
res.status(400).send(error);
}
});
So I figured it out. Maybe it'll help someone else as well.
const notesResult = await AgentTransaction.updateOne({
'transactions._id': { $in: [trunc2] },
}, {
$set: {
'transactions.$.notes': notes
},
});
The main issue was that the payload object needed to target the collection folder + the wildcard + the field, not just only the field.

How to save array data in nodejs using mongoose?

I am stuck to save data in mongoDb. Here data is in array and i need to insert data if mongodb does not have. Please look code:-
var contactPersonData = [{
Name: 'Mr. Bah',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'Mr. Sel',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'Mr.ATEL',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'ANISH',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'sunny ji',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'ashish',
Organization: 'Ashima Limited - Point 2'
}]
console.log('filedata', contactPersonData);
var escapeData = [];
var tempArr = [];
function saveContact(personObj, mainCallback) {
var tempC = personObj['Organization'].trim();
var insertData = {};
Contact.findOne({ companyName: tempC })
.exec(function(err, contact) {
if (err)
return mainCallback(err);
console.log('find com', contact)
if (contact) {
//document exists
mainCallback(null, insertData);
} else {
var newContact = new Contact({ companyName: tempC, createdBy: '58ae5d18ba71d4056f30f7b1' });
newContact.save(function(err, contact) {
if (err)
return mainCallback(err);
console.log('new contact', contact)
insertData.contactId = contact._id;
insertData.name = personObj['Name'];
insertData.email = personObj['Email'];
insertData.contactNumber = { number: personObj['Phone'] };
insertData.designation = personObj['Designation'];
tempArr.push(insertData);
mainCallback(null, insertData);
})
}
});
}
async.map(contactPersonData, saveContact, function(err, result) {
console.log(err)
console.log(result)
},
function(err) {
if (err)
return next(err);
res.status(200).json({ unsaved: escapeData })
})
As per above code it has to insert six document instead of one. I think that above iteration not wait to complete previous one. So, the if condition is always false and else is executed.
Your saveContact() function is fine. The reason you get 6 documents instead of 1 document is that async.map() runs you code in parallel. All the 6 requests are made in parallel, not one after the another.
From the async.map() function's documentation -
Note, that since this function applies the iteratee to each item in parallel, there is no guarantee that the iteratee functions will complete in order.
As a result before the document is created in your database all queries are already run and all the 6 queries are not able to find that document since its still in the process of creation. Therefore your saveContact() method creates all the 6 documents.
If you run your code again then no more documents will be formed because by that time your document would be formed.
You should try running your code using async.mapSeries() to process your request serially. Just replace map() with mapSeries() in your above code. This way it will wait for one request to complete and then execute another and as a result only one document will be created. More on async.mapSeries() here.
You seem to be using async.map() wrong.
First, async.map() only has 3 parameters (i.e. coll, iteratee, and callback) so why do you have 4? In your case, coll is your contactPersonData, iteratee is your saveContact function, and callback is an anonymous function.
Second, the whole point of using async.map() is to create a new array. You are not using it that way and, instead, are using it more like an async.each().
Third, you probably should loop through the elements sequentially and not in parallel. Therefore, you should use async.mapSeries() instead of async.map().
Here's how I would revise/shorten your code:
var contactPersonData = [{
Name: 'Mr. Bah',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'Mr. Sel',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'Mr.ATEL',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'ANISH',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'sunny ji',
Organization: 'Ashima Limited - Point 2'
}, {
Name: 'ashish',
Organization: 'Ashima Limited - Point 2'
}];
function saveContact(personObj, mainCallback) {
var tempC = personObj.Organization.trim();
Contact.findOne({ companyName: tempC }, function (err, contact) {
if (err)
return mainCallback(err);
console.log('found contact', contact);
// document exists, so mark it as complete and pass the old item
if (contact)
return mainCallback(null, contact);
// document does not exist, so add it
contact = new Contact({ companyName: tempC, createdBy: '58ae5d18ba71d4056f30f7b1' });
contact.save(function (err, contact) {
if (err)
return mainCallback(err);
console.log('created new contact', contact)
// mark it as complete and pass a new/transformed item
mainCallback(null, {
contactId: contact._id,
name: personObj.Name,
email: personObj.Email, // ??
contactNumber: { number: personObj.Phone }, // ??
designation: personObj.Designation // ??
});
});
});
};
async.mapSeries(contactPersonData, saveContact, function (err, contacts) {
if (err)
return next(err);
// at this point, contacts will have an array of your old and new/transformed items
console.log('transformed contacts', contacts);
res.json({ unsaved: contacts });
});
In terms of ?? comments, it means you don't have these properties in your contactPersonData and would therefore be undefined.

Mongoose subquery and append results to mainquery

I have been struggling with the questions for a months now still no solution.
Basically I have 2 mongodb database structures.
One is called Users and another is called Items.
One user can have multiple Items.
User structure is simple =
Users = [{
_id: 1,
name: "Sam",
email: "sam#gmail.com",
group: "Rangers"
},
{
_id: 2,
name: "Michael",
email: "michael#gmail.com"
group: "Muse"
},
{
_id: 3,
name: "John",
email: "john#gmail.com"
group: "Merchant"
},
.....
]
The Items structures are as follows and each item is assigned to a user.
Items = [
{
_id: 1,
user_id: 1,
item_name: "Flying Sword",
timestamp: ...
},
{
_id: 3,
user_id: 1,
item_name: "Invisible Cloak",
timestamp: ...
},
{
_id: 4,
user_id: 2,
item_name: "Iron Shield"
},
{
_id: 5,
user_id: 7,
item_name: "Splashing Gun",
timestamp: ...
},
...
]
I want to run a mongoose query that queries the user as primary object.
And upon the returning the results of the user object I want to query the all the Items objects with the filtered users and append them as subdocuments to each user objects previously queried.
For example I want to query
Users.find({group: "Muse"}, function(err, users){
I DON"T KNOW WHAT TO WRITE INSIDE
})
Basically the results should be:
[
{
_id: 4,
name: "Jack",
email: "jack#gmail.com",
group: "Muse",
items: [
{
_id: 8
name: "Magic Wand",
user_id: 4,
timestamp: ...
}
{
_id: 12
name: "Blue Potion",
user_id: 4,
timestamp: ...
},
{
_id: 18
name: "Teleportation Scroll",
user_id: 4,
timestamp: ...
}
]
}
.....
More USERS of similar structure
]
Each user will return a maximum of three items which are sorted by timestamp.
Thanks in advance, I tried so many times and failed.
This is a multiple step question. So lets list out the steps:
Get a list of user documents that match a particular group.
Get a list of item documents that are assigned to each matched user from step 1.
Assign the appropriate item documents to a new property on the corresponding user document.
This can be tackled a few ways. A first pass might be to retrieve all the user documents and then iterating over them in memory retrieving the list of item documents for each user and appending that list to the user document. If your lists are smallish this shouldn't be too much of an issue but as scale comes into play and this becomes a larger list it could become a memory hog.
NOTE: all of the following code is untested so it might have typos or the like.
Users.find({group: "Muse"}, function(err, users){
var userIDs;
if (err) {
// do error handling
return;
}
userIDs = users.map(function (user) { return user._id; });
Items.find({user_id: {$in: userIDs}}, function (err, items) {
if (err) {
// do error handling
return;
}
users.forEach(function (user) {
user.items = items.filter(function (item) {
return item.user_id === user._id;
});
});
// do something with modified users object
});
});
While this will solve the problem there are plenty of improvements that can be made to make it a bit more performant as well as "clean".
For instance, lets use promises since this involves async operations anyway. Assuming Mongoose is configured to use the native Promise object or a then/catch compliant library
Users.find({group: "Muse"}).exec().then(function(users) {
var userIDs = users.map(function(user) {
return user._id;
});
// returns a promise
return Promise.all([
// include users for the next `then`
// avoids having to store it outside the scope of the handlers
users,
Items.find({
user_id: {
$in: userIDs
}
}).exec()
]);
}).then(function(results) {
var users = results[0];
var items = results[1];
users.forEach(function(user) {
user.items = items.filter(function(item) {
return item.user_id === user._id;
});
});
return users;
}).catch(function (err) {
// do something with errors from either find
});
This makes it subjectively a bit more readable but doesn't really help since we are doing a lot of manipulation in memory. Again, this might not be a concern if the document collections are smallish. However if is, there is a tradeoff that can be made with breaking up the request for items into one-per-user. Thus only working on chunks of the item list at a time.
We will also use Bluebird's map to limit the number of concurrent requests for items.
Users.find({group: "Muse"}).exec().then(function(users) {
return bluebird.map(users, function(user) {
return Items.find({user_id: user._id}).exec().then(function (items) {
user.items = items;
return user;
});
}, {concurrency: 5});
}).then(function(users) {
// do something with users
}).catch(function(err) {
// do something with errors from either find
});
This limits the amount of in memory manipulation for items but still leaves us iterating over users in memory. That can be tackled as well by using mongoose streams but I will leave that up to you to explore on your own (there are also other questions already on SO on how to use streams).
This makes it subjectively a bit more readable but doesn't really help since we are doing a lot of manipulation in memory. Again, this might not be a concern if the document collections are smallish. However if is, there is a tradeoff that can be made with breaking up the request for items into one-per-user. Thus only working on chunks of the item list at a time.

How can I speed up a mongoDB (mongoose) batch insert with nodejs?

I have a bunch of documents in a collection I need to copy and insert into the collection, changing only the parent_id on all of them. This is taking a very very long time and maxing out my CPU. This is the current implementation I have. I only need to change the parent_id on all the documents.
// find all the documents that need to be copied
models.States.find({parent_id: id, id: { $in: progress} }).exec(function (err, states) {
if (err) {
console.log(err);
throw err;
}
var insert_arr = [];
// copy every document into an array
for (var i = 0; i < states.length; i++) {
// copy with the new id
insert_arr.push({
parent_id: new_parent_id,
id: states[i].id,
// data is a pretty big object
data: states[i].data,
})
}
// batch insert
models.States.create(insert_arr, function (err) {
if (err) {
console.log(err);
throw err;
}
});
});
Here is the schema I am using
var states_schema = new Schema({
id : { type: Number, required: true },
parent_id : { type: Number, required: true },
data : { type: Schema.Types.Mixed, required: true }
});
There must be a better way to do this that I just cannot seem to come up with. Any suggestions are more than welcome! Thanks.
In such a case there is no point to do this on application layer. Just do this in database.
db.States.find({parent_id: id, id: { $in: progress} }).forEach(function(doc){
delete doc._id;
doc.parentId = 'newParentID';
db.States.insert(doc);
})
If you really need to do this in mongoose, I see the following problem:
your return all the documents that matches your criteria, then you iterate though them and copy them into another array (modifying them), then you iterate through modified elements and copy them back. So this is at least 3 times longer then what I am doing.
P.S. If you need to save to different collection, you should change db.States.insert(doc) to db.anotherColl.insert(doc)
P.S.2 If you can not do this from the shell, I hope you can find a way to insert my query into mongoose.

Resources