I am trying to figure out the most efficient way to find and populate documents some of which have references to another collection and some don't.
Example:
var collectionA={
{
"topic":"TopicA",
"text";"TextA")
},
{
"topic":"Topic1",
"text":"Text1"}
}
}
var collectionB={
{
"topic":"TopicB",
"text":"TextB"
"reference":["id","id"]
},
{
"topic":"Topic2",
"text":"Text2"
"reference":[]
}
}
and I have a request as follows which allows me to identify the documents I want:
req.body={"topic":["TopicA","TopicB"]}
What is the most efficient way to find the relevant documents and populate them to provide a fully populated result, in the fewest number of database calls as possible:
{"topic":"TopicA","text":"TextA"},
{"topic":"TopicB","text":"TextB","reference":[{populated document}
{populated document}]},
I am trying to use something like:
collections.find({"topic": $in req.body.top}, function(err,result){
collections.populate(result,{path:'references'}, function (err,result){
//do something
});
})
Is this on the right track?
Your help is greatly appreciated.
In order to reference other documents in mongoose you need to use Population.
If you want to reference a topic by id you would need to have something like the following in your schema: { type: Schema.Types.ObjectId, ref: 'collectionA' }
Related
I'm transitioning to ArangoDB for its power as a graph database and am struggling with the simple stuff.
If I have a document...
{ _id:'1234', tags:['apple','orange'] }
how can I update it to push or pull a value from the array? I would expect the syntax to be something like this, but didn't find anything in the docs...
collection.update({ _id:'1234' },{ tags:pull('mango') })
collection.update({ _id:'1234' },{ tags:push('avocado') })
Thank you!
You achieve this with AQL. For example
FOR c in collection UPDATE c WITH { tags : PUSH(c.tags, 'avocado')} IN collection
https://docs.arangodb.com/3.3/AQL/Operations/Update.html
So, I am building a small blog-like project in Node, and I am running into an issue with orphaned database references. I have two models in separate files that reference each other.
Here are my models:
// ./models/user
Var UserSchema = mongoose.Schema({
name: String,
posts: [{type: mongoose.SchemaTypes.ObjectId, ref:'Post'}]
});
// ./models/post
var PostSchema = mongoose.Schema({
title:String,
post_body: String,
posted_by: mongoose.SchemaTypes.ObjectId
});
My question is when you delete say, a Post, how would you delete the reference in the User's post array? My thinking was I could create a middleware to run before the delete route and delete the reference in the User's post array before I actually delete the post. Would that be considered the best way to go about it? I found a post on Stack that uses a 'pre' function in the schema like this:
// ./models/post
PostSchema.pre('remove', function(next){
this.model('User').remove({posts: this._id}, next);
});
Here is the actual stack post: Automatically remove referencing objects on deletion in MongoDB . I could not get this work though. I did ,however, implement a custom middleware to delete the references, but feel it might not be best practice. Any tips/advice would be greatly appreciated. Thanks!
You don't want .remove() here but you want .update() with $pull instead:
PostSchema.pre('update',function(next) {
this.model('User').update(
{ },
{ "$pull": { "posts": this._id } },
{ "multi": true },
next
);
})
That's the correct operation to remove something from an array. The "multi" makes sure that the "post" would be removed for all User objects that reference it, thought it probably really is only one document anyway.
The .remove() method is for "removing" whole documents. The .update() method makes "changes" to documents.
Here is my Mongoose Schema:
var SchemaA = new Schema({
field1: String,
.......
fieldB : { type: Schema.Types.ObjectId, ref: 'SchemaB' }
});
var SchemaB = new Schema({
field1: String,
.......
fieldC : { type: Schema.Types.ObjectId, ref: 'SchemaC' }
});
var SchemaC = new Schema({
field1: String,
.......
.......
.......
});
While i access schemaA using find query, i want to have fields/property
of SchemaA along with SchemaB and SchemaC in the same way as we apply join operation in SQL database.
This is my approach:
SchemaA.find({})
.populate('fieldB')
.exec(function (err, result){
SchemaB.populate(result.fieldC,{path:'fieldB'},function(err, result){
.............................
});
});
The above code is working perfectly, but the problem is:
I want to have information/properties/fields of SchemaC through SchemaA, and i don't want to populate fields/properties of SchemaB.
The reason for not wanting to get the properties of SchemaB is, extra population will slows the query unnecessary.
Long story short:
I want to populate SchemaC through SchemaA without populating SchemaB.
Can you please suggest any way/approach?
As an avid mongodb fan, I suggest you use a relational database for highly relational data - that's what it's built for. You are losing all the benefits of mongodb when you have to perform 3+ queries to get a single object.
Buuuuuut, I know that comment will fall on deaf ears. Your best bet is to be as conscious as you can about performance. Your first step is to limit the fields to the minimum required. This is just good practice even with basic queries and any database engine - only get the fields you need (eg. SELECT * FROM === bad... just stop doing it!). You can also try doing lean queries to help save a lot of post-processing work mongoose does with the data. I didn't test this, but it should work...
SchemaA.find({}, 'field1 fieldB', { lean: true })
.populate({
name: 'fieldB',
select: 'fieldC',
options: { lean: true }
}).exec(function (err, result) {
// not sure how you are populating "result" in your example, as it should be an array,
// but you said your code works... so I'll let you figure out what goes here.
});
Also, a very "mongo" way of doing what you want is to save a reference in SchemaC back to SchemaA. When I say "mongo" way of doing it, you have to break away from your years of thinking about relational data queries. Do whatever it takes to perform fewer queries on the database, even if it requires two-way references and/or data duplication.
For example, if I had a Book schema and Author schema, I would likely save the authors first and last name in the Books collection, along with an _id reference to the full profile in the Authors collection. That way I can load my Books in a single query, still display the author's name, and then generate a hyperlink to the author's profile: /author/{_id}. This is known as "data denormalization", and it has been known to give people heartburn. I try and use it on data that doesn't change very often - like people's names. In the occasion that a name does change, it's trivial to write a function to update all the names in multiple places.
SchemaA.find({})
.populate({
path: "fieldB",
populate:{path:"fieldC"}
}).exec(function (err, result) {
//this is how you can get all key value pair of SchemaA, SchemaB and SchemaC
//example: result.fieldB.fieldC._id(key of SchemaC)
});
why not add a ref to SchemaC on SchemaA? there will be no way to bridge to SchemaC from SchemaA if there is no SchemaB the way you currently have it unless you populate SchemaB with no other data than a ref to SchemaC
As explained in the docs under Field Selection, you can restrict what fields are returned.
.populate('fieldB') becomes populate('fieldB', 'fieldC -_id'). The -_id is required to omit the _id field just like when using select().
I think this is not possible.Because,when a document in A referring a document in B and that document is referring another document in C, how can document in A know which document to refer from C without any help from B.
I am trying to sort my query for reddit hot algorithm. It is based on time and i can't use beforeUpdate or beforeCreate hooks. I need to sort with a calculation when im querying but i couldn't find a solution.
Let's say my model is looks like this;
var myModel = Waterline.Collection.extend({
attributes: {
up: {
type: 'integer',
},
downs: {
type: 'integer',
}
}
});
Thanks for help.
You can create any kind of function helper or custom attribute inside your Model to help you make the right query.
https://github.com/balderdashy/waterline/blob/master/README.md#custom-types
How can I query a Mongo collection using Mongoose to find all the documents that have a specific relation between two of their own properties?
For example, how can I query a characters collections to find all those characters that have their currentHitPoints value less than their maximumHitPoints value? Or all those projects that have their currentPledgedMoney less than their pledgeGoal?
I tried to something like this:
mongoose.model('Character')
.find({
player: _currentPlayer
})
.where('status.currentHitpoints').lt('status.maximumHitpoints')
.exec(callback)
but I am getting errors since the lt argument must be a Number. The same goes if I use $.status.maximumHitpoints (I was hoping Mongoose would be able to resolve it like it does when doing collection operations).
Is this something that can be done within a Query? I would expect so, but can't find out how. Otherwise I can filter the whole collection with underscore but I suspect that is going to have a negative impact on performance.
PS: I also tried using similar approaches with the find call, no dice.
MongoDB 3.6 and above supports aggregation expressions within the query language:
db.monthlyBudget.find( { $expr: { $gt: [ "$spent" , "$budget" ] } } )
https://docs.mongodb.com/manual/reference/operator/query/expr/
Thanks to Aniket's suggestion in the question's comments, I found that the same can be done with Mongoose using the following syntax:
mongoose.model('Character')
.find({
player: _currentPlayer
})
.$where('this.status.currentHitpoints < this.status.maximumHitpoints')
.exec(callback)
Notice the $where method is used instead of the where method.
EDIT: To expand on Derick's comment below, a more performance sensitive solution would be to have a boolean property inside your Mongoose schema containing the result of the comparison, and update it everytime the document is saved. This can be easily achieved through the use of Mongoose Schema Plugin, so you would have something like:
var CharacterSchema = new mongoose.Schema({
// ...
status: {
hitpoints: Number,
maxHitpoints: Number,
isInFullHealth: {type: Boolean, default: false}
}
})
.plugin(function(schema, options) {
schema.pre('save', function(next) {
this.status.isInFullHealth = (this.status.hitPoints >= this.status.maxHitpoints);
next();
})
})
mongoose.model('Character')
.find({
player: _currentPlayer, $expr: { $lt: ['$currentHitpoints', '$maximumHitpoints'] }
})
This above query means find the record which has currentHitpoints less than maximumHitpoints
Starting in MongoDB 5.0, the $eq, $lt, $lte, $gt, and $gte comparison operators placed in an $expr operator can use an index on the from collection referenced in a $lookup stage.
Example
The following operation uses $expr to find documents where the spent amount exceeds the budget:
db.monthlyBudget.find( { $expr: { $gt: [ "$spent" , "$budget" ] } } )