I have just started working with Mongo Db using Mongoose.js library.
I have two schemas, Orders and Payments.
I want to make a simple query to get all orders with property of sum of all payments for that order.
Schemas
Order
{ name : String }
Payment
{ date : Date, order : Schema.ObjectId, sum : Number }
In mysql I would do it this way:
select *,(select SUM(sum) from payments where payments.order = orders.id group by payments.order) as sum from orders
I was looking for aggregation methods to build the query. But all examples I have found is for the single object. I have no trouble to get the sum for single order (using findOne() and aggregation). But how I do it for array of orders?
There is no way to do this with just one query in MongoDB with the schema design you have. Your schema is fine for a relational DB, but it might not be ideal for NoSQL.
With your schema you would have to do two queries:
An aggregation job on payment to sum the payments grouped by order
A a find(),or findOne(), on order to get the name of the order
An alternative schema would be to have just one collection of order documents and storing the payments as sub-documents on the order document, eg. an order would look something like this:
{
name: "Foo"
payments: [ { date: xxxx, sum: 42.00}, { date: yyyy, sum: 12.00} ]
}
Take a look at the Mongo documentation on data models for more: http://docs.mongodb.org/manual/data-modeling/#data-models
Related
Using Mongoose driver
Consider the following code :
Connecting to database
const mongoose = require("mongoose");
require("dotenv").config();
mongoose.connect(process.env.DB);
const userSchema = new mongoose.Schema({ name: String }, {collection: 'test'});
const Model = mongoose.model('test', userSchema);
Creating dummy document
async function createDocs() {
await Model.deleteMany({});
await Model.insertMany([{name: "User1"}, {name: "User2"}, {name: "User3"},{name: "User4"}])
}
createDocs();
Filtering data using Model.find()
async function findDoc () {
let doc = await Model.find({name: 'User1'});
console.log(`Using find method : ${doc}`);
}
findDoc();
Filtering data using Model.aggregate()
async function matchDoc() {
let doc = await Model.aggregate([
{
$match: {name : 'User1'}
}
])
console.log(`Using aggregate pipeline : `, doc);
}
matchDoc();
• Both the processes produce the same output
Q1) What is an aggregate pipeline and why use it?
Q2) Which method of retrieving data is faster?
I will not get too much into this as there is a lot of information online. But essentially the aggregation pipeline gives you access to a lot of strong operators - mainly used for data analysis and object restructuring, for simple get and set operations there is no use for it.
A "real life" example of when you'd want to use the aggregation pipeline is if you want to calculate an avg of a certain value in your data, obviously this is just the tip of the iceberg in terms of power that this feature allows.
find is slightly faster, the aggregation pipeline has some overhead when compared to the query language as each stage has to load the BSON documents into memory, where find doesn't. If your use case is indeed just a simple query then find is the way to go.
You are checking a smaller piece of the aggregation pipeline.
You cannot do pipeline with a single query using find
Let's say you want to find all the orders which have a product that was purchased within a duration. Here, orders and customers are two different collections, You need multiple stages.
Let's say you stored data in a different format, For ex, date as a string, integers as a decimal. If you want to convert on the fly, you can use aggregation.
If you want to use aggregation operators in an update operation from mongo 4.2+, It helps.
You can restructure data in find. Let's say I just need a few fields after aggregation from an array field.
You cannot find a particular array or object element matching a condition using simple find. elemMatch is not that powerful.
You cannot group things with simple find
And many more.
I request you to check aggregate operators and relevant examples from the documentation
Data retrieving depends on not depend on the aggregation or find. It depends on the data, hardware, and index settings.
Good morning colleagues,
I want to make a query regarding how it would be formulated and what would be recommended and the most optimal for making queries with a large number of elements.
I have an api using express that creates a new mongodb model with a unique name and then includes elements inside.
Example:
Collections
*product234 => includes elements => { _:id: ...., name: ... }, { ...}
*product512 => includes elements => { _:id: ...., name: ... }, { ...}
Each collection hosts more than 5000 elements and I want to create a search engine that returns all the results of all the collections that match the "name" that I will send in the request.
How could I perform this query using mongoose? Would it be viable and would it not bring me performance problems by having more than 200 collections and more than 5000 elements in each one?
Answer (Edit):
As i see in the comments the best solution for this for making fast queries is to create a copy of the element with only name or needed values, reference id and reference collection name into a new collection named for example "ForSearchUse" and then make the query to that collection, if complete info is needed then you can query it to the specific collection using the reference id and name of the element
I'm new to mongodb and to overall databases side of development.
I'm trying to make a product listing site where all the categories would be displayed with the number of products within that particular category and when clicked on a particular category, it would get me all the products in that category.
Some things to note are:
every product will have only one category
each category will have multiple products
I don't know how to go about this problem and tried searching it online but couldn't exactly find what I was looking for. I've also tried making the schema for this but I do not know if it's the right approach or not and this is how it looks:
const productsSchema = {
category: String,
name: String,
price: String,
description: String,
thumbnail: String,
};
Side note: I'm using MERN stack.(if its of any help)
If I've understand well your question, you can use something like this:
db.collection.aggregate([
{
"$match": {
"category": "category1"
}
},
{
"$count": "total"
}
])
With this query you will get the total $count for each category.
Example here
In your frontend you will need a call for every category.
Maybe if your DB has a lot of different categories this is not a good approach, but if the number is not large enough you can call this query a couple times and you will get the result you want.
MongoDB Documentation reference here
I would say you should have a product schema and a product category schema, where the product category schema has an array of product ids that belong to that category.
In the product schema, you could also have a pointer to the category object that a product is linked to (as opposed to just the name of the category as a string).
Maybe take a look at mongoose populate https://mongoosejs.com/docs/populate.html
Let's say there's a user collection.
The relevant information for this question from that collection is:
username: string,
teamid: Number,
totalscore: Number
Now for the final output it should list the teams, list the users under their teams using the teamid, and adding up the totalscore of each user to calculate the teamscore.
However, I can't come up with any way to do that.
I was looking into aggregation for this but eh, it's a bit too far above my hat.
The output for this api is rendered in json using express and mongoose.
You can use the following aggregation query.
Use a group stage with $push to get the player's name and $sum to get the sum of totalscore for a team.
db.coll.aggregate([
{'$group': {
'_id': '$teamid',
'totalscore': {'$sum': '$totalscore'},
'listofplayers': {'$push':'$username'}
}
}])
I'm using mongoose to create and model my document schemas.
I have a user_id attribute on a bunch of different schemas, along with a created_at date attribute on them.
I want to generate a list, ordered by created_at date for display as a feed of recent activity on the user's homepage.
How would I query several different models, and then order these items into an array that I can then pass to my ejs view?
You can sort your mongoose query using the sort function. Here are two examples:
query.sort({ field: 'asc', test: -1 });
or
Person
.find({user_id: 123456})
.sort('-created_at')
.exec(function(err, person) {
// push these results to your array and start another query on a different schema
});