I am trying to implement a bus route listing application. My goal is to list buses that passes through two user given points(locations).
I am using mongodb in the backend. A simple structure of the data is like:
....
{ name: 'Bus-N', route: ["Loc1", "Loc2", "Loc3", "Loc4", "Loc5" ], startTime:'..'},
{ name: 'Bus-M', route: ["Loc2", "Loc3", "Loc4"], startTime:'..'}
....
Assumption: I have the list of all buses and for each bus I have all of it's travel routes.
User gives his start location and his destination as input.
How can I use these inputs(start location and destination) to query across this database to list all the buses passing by the two points?
For example: If the input is (Loc3, Loc4) I need the output as (Bus-N, Bus-M).
If there is a better way to implement this(which obviously there should be), please recommend the required changes.
I am new to mongodb. Please help with this, thanks in advance.
You need to use array query operators like:
$elemMatch
$and
during your .find or .findOne query. You could find more info about it in the official MongoDB documentation, (syntax for mongoose is just the same)
You also could find useful information for your case in this question which explains the difference between $all and $in operators for array fields.
Your query should be looked like this:
Model.find({ route: { $in: ["start_point", "dest_point"] } }).exec(callback)
Related
I'm using an ajax request from the front end to load more comments to a post from the back-end which uses NodeJS and mongoose. I won't bore you with the front-end code and the route code, but here's the query code:
Post.findById(req.params.postId).populate({
path: type, //type will either contain "comments" or "answers"
populate: {
path: 'author',
model: 'User'
},
options: {
sort: sortBy, //sortyBy contains either "-date" or "-votes"
skip: parseInt(req.params.numberLoaded), //how many are already shown
limit: 25 //i only load this many new comments at a time.
}
}).exec(function(err, foundPost){
console.log("query executed"); //code takes too long to get to this line
if (err){
res.send("database error, please try again later");
} else {
res.send(foundPost[type]);
}
});
As was mentioned in the title, everything works fine, my problem is just that this is too slow, the request is taking about 1.5-2.5 seconds. surely mongoose has a method of doing this that takes less to load. I poked around the mongoose docs and stackoverflow, but didn't really find anything useful.
Using skip-and-limit approach with mongodb is slow in its nature because it normally needs to retrieve all documents, then sort them, and after that return the desired segment of the results.
What you need to do to make it faster is to define indexes on your collections.
According to MongoDB's official documents:
Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
-- https://docs.mongodb.com/manual/indexes/
Using indexes may cause increased collection size but they improve the efficiency a lot.
Indexes are commonly defined on fields which are frequently used in queries. In this case, you may want to define indexes on date and/or vote fields.
Read mongoose documentation to find out how to define indexes in your schemas:
http://mongoosejs.com/docs/guide.html#indexes
TLDR
Is there a way to limit queryByExample to a collection in NodeJS?
Problem faced
I have a complex query with some optional fields (i.e. sometimes some search fields will be omitted). So I need to create a query dynamically, e.g. in JSON. QueryByExample seems to be the right tool to use here as it gives me that flexibility to pass a JSON. However my problem is that I would like to limit my search to only one collection or directory.
e.g. I was hoping for something like
searchJSON = {
title: { $word: "test" },
description: { $word: "desc" }
};
//query
db.documents.query(qb.where(
qb.collection("collectionName"),
qb.byExample(searchJSON)
)).result()...
In this case searchJSON could have been built dynamically, for example maybe sometimes title may be omitted from the search.
This doesn't work because the query builder only allows queryByExample to be the only query. But I'd instead like to built a dynamic search query which is limited to a collection or directory.
At present, I think you would have to express the query with QueryBuilder instead of Query By Example using
qb.and([
qb.collection('collectionName'),
qb.word('title', 'test'),
qb.word('description', 'desc')
])
See http://docs.marklogic.com/jsdoc/queryBuilder.html#word
That said, it should be possible for the Node.js API to relax that restriction based on the fixes in MarkLogic 9.0-2
Please file an issue on https://github.com/marklogic/node-client-api
How to select documents in a collection with a unique field in MongoDB?
I have this document schema:
{
_id: someid,
createdAt: new Date(),
message: somemessage,
eadd: eadd
}
I would like to get recent messages with unique email address. So the output will be recent messages per email address.
I don't know how to use Group by - can I use it here?
to get aggregation in Meteor, you'll have add a package. e.g. meteorhacks:aggregate
c.f. https://themeteorchef.com/tutorials/aggregations-in-mongodb
You will have to use the mongo aggregate function
db.messages.aggregate(
[
{
$group : {
_id : "$eadd"}
}
]);
Just include the fields you want to show
You can try http://grapher.cultofcoders.com/ package. It will allow you to create relationships between your mongo collections. You will use links to create them and you can filter your requests to the database, pass parameters and more.
To install it
meteor add cultofcoders:grapher
Documentation can be found here: http://grapher.cultofcoders.com/
I'm using Node.js + mongodb. I have few documents in my collection and i want to know does my collection have any document matched my condition. Of course i can simply use
myModel.find({ myField: someValue }) and check is anything comes or not. But i want to use solution like sql provides exists keyword? Help me, please
Edit: my bad. I forget to tell that "performance first".
MongoDB's $exists actually doesn't help you very much to find out if a certain document exists in your collection. It is used for example to give you all documents that have a specific field set.
MongoDB has no native support for an sql like exists. What you can use, however, is myModel.findOne({ myField: someValue }) and then check if it is null.
To enhance performance you can tell MongoDB to only load the object id via projection, like this:
myModel.findOne({ myField: someValue }, {_id: 1})
There is an exist mechanism in mongodb, I'll demonstrate a sample below.
For example below, I'm looking for records that have tomato.consensus fields and that it's empty, so I can delete them or avoid them. In case I was looking for "tomato.consensus": Dublin, I'd change Null to Dublin, to match that.
I hope this is helpful, if not fire away any questions
tomato
----consensus
db.movieDetails.updateMany({$and: [
{"tomato.consensus": {$exists: true} },
{"tomato.consensus": null} ] },
]})
I have a sample schema like this -
Comment.add({
text:String,
url:{type:String,unique:true},
username:String,
timestamp:{type:Date,default:Date}
});
Feed.add({
url:{type:String, unique:true },
username:String,
message:{type:String,required:'{PATH} is required!'},
comments:[Comment],
timestamp:{type:Date,default:Date}
});
Now, I don't want to expose the _id fields to the outside world that's why I am not sending it to the clients anywhere.
Now, I have two important properties in my comment schema (username,url)
What I want to do is update the content of the sub document that satisfies
feed.url
comment.url
comment.username
if the comment.username is same as my client value req.user.username then update the comment.text property of that record whose url was supplied by client in req.body.url variable.
One long and time consuming approach I thought is to first find the feed with the given url and then iterating over all the subdocuments to find the document which satisfies the comment.url==req.body.url and then check if the comment.username==req.user.username if so, update the comment object.
But, I think there must be an easier way of doing this?
I already tried -
db.feeds.update({"username":"harshitladdha93#gmail.com","comments.username":"harshitladdha3#gmail.com","comments.url":"test"},{$set:{"comments.$.text":"updated text 2"}})
found from http://www.tagwith.com/question_305575_how-to-find-and-update-subdocument-within-array-based-on-parent-property
but this updates even when the comments.url or comments.usernamematches other sub documents
and I also tried
db.feeds.distinct("comments._id",{"comments.url":req.body.url})
to find the _id of document associated with the url but it returns all the _id in the subdocument
First off - you should not rely on _id not being seen by the outside world in terms of security. This is a very bad idea for a multitude of reasons (primarily REST and also the fact that it's returned by default with all your queries).
Now, to address your question, what you want is the $elemMatch operator. This says that you're looking for something where the specified sub-document within an array matches multiple queries.
E.g.
db.feeds.update({
"username":"harshitladdha93#gmail.com",
comments: {
$elemMatch: {
username: "harshitladdha3#gmail.com",
url: "test"
}
}
}, {$set: {"comments.$.text":"updated text 2"}})
If you don't use $elemMatch you're saying that you're ok with the document if any of the comments match your query - i.e. if there is a comment by user "harshitladdha3#gmail.com", and separate comment has a url "test", the document will match unless you use $elemMatch