I have a document in the shape of
const Model = mongoose.Schema({
something1: {type:String},
someNumber1:{type:Number},
something2: {type:String},
someNumber2:{type:Number},
aFloatNumber: {type:Number}
)}
and after indexing the document like
Model.index({something1:1 , something2:1 , aFloatNumber:1})
for better performance which I hope I am doing right and please correct me if I am doing it wrong.
I am trying to query usign syntax:
const model = await Model.find({
$and:[{something1:anInput}, {something2:anotherInput}]})
.sort(aFloatNumber)
now I want to limit the returned query as it could be a very large list to improve performance, however, this limit changes based on an input. Basically I want the mongoose to keep adding someNumber1 together and stop returning after it gets larger than the input number. Something like the code bellow:
const model = await Model.find({
$and:[{something1:anInput}, {something2:anotherInput}]})
.sort(aFloatNumber)
.limit( sum(someNumber1) >= theInputNumber )
So basically my questions are:
Am I indexing the document correctly based on my query?
Does it make any difference on the performance to limit the query since it is sorting the data and I think it is going to check all the document to be able to sort it?
If it makes a huge difference on the performance, what is the correct syntax for it as I am going to make this query a lot in my application?
You're asking for skip function of mongodb which is like offset in sql
https://docs.mongodb.com/manual/reference/operator/aggregation/skip/
Related
I had a problem using query with MongoDB.
The problem was solved but I wanted to check if there was any other approach I could have taken.
At first, my model (Ad) had a property of price: {type: String}, and I tried to find by queries $gte and $lt to get ads with a price within a given range.
After reading online I figured that query operations are not working on String type properties.
Then even after changing the type to Number - price: {type: Number} - the find function didn't work properly on the price, even though on other properties which were type Number it worked as it should.
In the end, I just deleted the whole database and reupload it, and then everything worked properly (haven't changed a thing).
Has anyone had this kind of problem and solved it differently?
I'll first start by assuming you're using mongoose as the "types" you've pasted look like mongoose schema types.
You need to separate these two concepts:
The schema that represents data at the app level
The actual data in the DB.
Let's say I have this schema for a certain collection:
{ name: String }
But in the actual database there is only one document in that collection that looks like this:
{ price: 5, product_id: 1 }
Then when I query the data what do you expect to happen? do you expect mongoose to automatically generate a name for that document and delete the actual fields?
The reason it didn't "work" as you intended was that all the values were saved as string, changing the Schema does not retroactively update the database, so when you use $lt and $gte it uses string comparison which means "10" is less than "9" because that's how string comparison work.
The schema does help with newly inserted data and can cast it to the right type if supported, for that you should check the docs with what values are available.
I'm using an ajax request from the front end to load more comments to a post from the back-end which uses NodeJS and mongoose. I won't bore you with the front-end code and the route code, but here's the query code:
Post.findById(req.params.postId).populate({
path: type, //type will either contain "comments" or "answers"
populate: {
path: 'author',
model: 'User'
},
options: {
sort: sortBy, //sortyBy contains either "-date" or "-votes"
skip: parseInt(req.params.numberLoaded), //how many are already shown
limit: 25 //i only load this many new comments at a time.
}
}).exec(function(err, foundPost){
console.log("query executed"); //code takes too long to get to this line
if (err){
res.send("database error, please try again later");
} else {
res.send(foundPost[type]);
}
});
As was mentioned in the title, everything works fine, my problem is just that this is too slow, the request is taking about 1.5-2.5 seconds. surely mongoose has a method of doing this that takes less to load. I poked around the mongoose docs and stackoverflow, but didn't really find anything useful.
Using skip-and-limit approach with mongodb is slow in its nature because it normally needs to retrieve all documents, then sort them, and after that return the desired segment of the results.
What you need to do to make it faster is to define indexes on your collections.
According to MongoDB's official documents:
Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
-- https://docs.mongodb.com/manual/indexes/
Using indexes may cause increased collection size but they improve the efficiency a lot.
Indexes are commonly defined on fields which are frequently used in queries. In this case, you may want to define indexes on date and/or vote fields.
Read mongoose documentation to find out how to define indexes in your schemas:
http://mongoosejs.com/docs/guide.html#indexes
I have a DB with a couple of levels deep nested stuff, sometimes pretty big.
now i have searched the doc and google/so, but couldn't find a simple answer:
if the schema is like:
{
roomId : String,
created : Date,
teacher : String,
students : Object,
problems : Array
}
is there a way to just read the roomId of every entry?
Not return the whole thing, but just an array of the roomIds?
(usecase: i want to make a list of all saved rooms, therefore i need absolutely nothing of all the other data, just the IDs. I want to avoid that overhead)
i'm pretty sure it can be done, but couldn't find how
Yes, use a projection
Model.findOne({...}, {roomId: 1})....
Context:
I am working on a test-taking web-app, where users answer questions in an examination format.
I currently have two collections:
tests
questions
Each document in the tests collection has a questions array that contains the Mongo IDs of documents in the questions collections.
My Question...
Is it possible to (all at once / in one go): Retrieve a specific document in tests using a provided Mongo ID and then use the Mongo IDs saved in the questions array (within that document) to then pull documents from questions?
My closest guess is to use Mongoose's DBRef convention, but I can't quite understand how to use it in this context (and even if I did, I don't understand how I can retrieve multiple questions and save them under a single test).
I would appreciate any and all help with this!
P.S. The reason questions and tests are separate is so that we can randomize the order of questions when the user takes the exam in the web-app.
Go the other direction. Put the testId on the Question model:
var TestSchema = new mongoose.Schema({
name: String
});
var Test = mongoose.model("Test", TestSchema);
var QuestionSchema = new mongoose.Schema({
testId: {
type: mongoose.Schema.ObjectId,
ref: "Test"
},
text: String,
answer: String
});
QuestionSchema.index({testId: 1})
var Question = mongoose.model("Question", QuestionSchema);
You have three situations; you have a question document in memory and you want to find the test it belongs to:
Test.findOne({_id: question.testId},callback);
Or you have a question document in memory and want to find all questions that belong to the same test:
Question.find({testId: question.testId}, callback);
Or, you have a test document in memory and want to find all of it's questions:
Question.find({testId: test._id}, callback);
I see populate() pop up as an answer on any question resembling yours. I want to make sure people realize populate() isn't a SQL JOIN. From the docs:
Populated paths are no longer set to their original _id , their value is replaced with the mongoose document returned from the database by performing a separate query before returning the results.
populate() is just syntactic sugar for serializing a second find().
In my model, I have this field:
…
shows: [{
startAt: Number,
endAt: Number
}],
…
I need a query to select all objects that have a show that hasn't started yet. In other words, I want to find all models that have at least one startAt that is smaller than a given time.
Is this possible? And if so, how can I do it?
As I didn't have any test data, I couldn't test what worked. Now I do, and #joao is absolutely correct.
It seems like Mongoose doesn't care that shows is a list. It was as simple as writing
<model>.find().where('shows.startAt').gt(<constant>).exec(callback);