My document structure is like the following:
{ agency_key : '',
route_id: '',
direction_id: '',
stops: [
{stop_id:15,
stop_lat: '',
stop_lon: '',
loc: [-83.118972, 42, 121567]
},
{...
}
]
}
I want to find the stops which are close to a given (lat,lon) pair within a given distance from every document in the collection. I created 2d indexes. I tried $unwind, then $geoNear but it says $geoNear can only be at the first stage of the pipeline.
I tried this:
db.tm_stops.aggregate([
... {
... $geoNear: { near: {coordinates: [-82.958841, 42.370114] }, distanceField: "stops.calculated", query: { agency_key: "DDOT"},
... includeLocs: "stops.loc" }
... }
... ])
I tried the following:
db.tm_stops.find({stops:{$near:[-82.958841, 42.370114], $maxDistance: 1 } } )
It throws this error:
error: {
"$err" : "Unable to execute query: error processing query: ns=gtfs.tm_stops limit=0 skip=0\nTree: GEONEAR field=stops maxdist=1 isNearSphere=0\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query",
"code" : 17007
What should be my mongo query? I will need to execute this from a Node.js application but I want to see some result in Mongo shell first. Thanks in advance.
I finally made the query work. I am answering my own question for future reference.
Since I am using 2d index and not 2dsphere, I didn't have to use coordinates in near operator. This query works:
db.tm_stops.aggregate([
{
$geoNear:
{ near: [-82.958841, 42.370114] ,
distanceField: "stops.calculated",
query: { agency_key: "DDOT"},
includeLocs: "stops.loc" }
}
])
But I discovered, in the returned document, i can only see the calculated distance and the (lon, lat) pair used to calculate the distance. I can't really returned the whole embedded document which is used for the calculation.
If I find a way to return the whole embedded document, I'll update my answer.
Related
I need to do some arithmetic with dates with my data using MongoDB's aggregate query feature.
My main issue is getting the current date and subtracting it from a date value set on the collection.
Example record:
[
{
_id: '1',
upcomingDate: ISODate('2019-12-31T23:59:00.000Z')
}
]
My pipeline:
[
{ "$match": { ... },
{ "$project: {
timeToGo: { "$subtract": ["$upcomingDate", new Date()] }
}
}
]
Unfortunately, new Date() results in an error:
MongoError: cant $subtract a String from a Date
My initial idea was that $upcomingDate is being read as a string, alas, setting the values to [new Date(), new Date()] resulted in this error:
MongoError: cant $subtract a String from a String
I am on using Node.js to write the query and our db is WiredTiger v3.2, and am currently unable to upgrade due to a limitation with our provider MLabs.
Any help would be much appreciated.
I have a model named Rest with lots of columns and i only want to fetch few among them while applying near operator on that Rest model. this is my code
Rest
.select('rest_status rest_address rest_name rest_contact rest_photo rest_menu rest_avg_rating')
.aggregate().near({
near:[parseFloat(req.body.lng),parseFloat(req.body.lat)],
maxDistance:100000,
spherical:true,
distanceField:"dist.calculated"
})
.then(rests =>{
// const response=[];
// for(const rest of rests){
// console.log(rest);
// response.push(rest);
// }
res.send({rests,response_status});
}).catch(err => res.send(err));
when i try like this . i get an error that select is not a function. i tried changing select position like below aggregate and near but it didnt work. I'm new to this mongoose,please tell me if there is any function or way around to fetch limited columns from my model.
i forgot to mention both near and select and working fine when other one is not used and also please help me with changing the data obtain from model
You could do something like this below:
Rest.aggregate([
{
$geoNear: {
near: {
type: "Point",
coordinates: [parseFloat(req.body.lng), parseFloat(req.body.lat)]
},
maxDistance: 100000,
spherical: true,
distanceField: "dist.calculated"
}
},
{
$project: {
rest_status: 1,
rest_address: 1,
rest_name: 1,
rest_contact: 1,
rest_photo: 1,
rest_menu: 1,
rest_avg_rating: 1
}
}
//even you can add limit skip below , if u need
,{$limit:<Number>},
{ $skip: <Number>}
]
//depends on your mongo version you may need to set cursor as well.
,
{ cursor: { batchSize: <Number or keep 0> } }
)
.then()
.catch();
Note: before execute the query make sure you have added 2dsphere index to your dist.calculated field
in case of aggegate $project is used for what you want to do with .select
instead of using .near() i have used $geoNear.
if you want to use limit skip you can follow the example, other wise can remove that.
you also can add distanceMultiplier and includeLocs field depends on your requirement.
in the above, depends on the mongoDB version you may need to use cursor in aggregate.
if not you can go aheadwithout using the cursor.
Hope this helps.
if you still get any error, please comment.
{
"_id" : ObjectId("5852725660632d916c8b9a38"),
"response_log" : [
{
"campaignId" : "AA",
"created_at" : ISODate("2016-12-20T11:53:55.727Z")
},
{
"campaignId" : "AB",
"created_at" : ISODate("2016-12-20T11:55:55.727Z")
}]
}
I have a document which contains an array. I want to select all those documents that do not have response_log.created_at in last 2 hours from current time and count of response_log.created_at in last 24 is less than 3.
I am unable to figure out how to go about it. Please help
You can use the aggregation framework to filter the documents. A pipeline with $match and $redact steps will do the filtering.
Consider running the following aggregate operation where $redact allows you to proccess the logical condition with the $cond operator and uses the system variables $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition was false.
This operation is similar to having a $project pipeline that selects the fields in the collection and creates a new field that holds the result from the logical condition query and then a subsequent $match, except that $redact uses a single pipeline stage which is more efficient:
var moment = require('moment'),
last2hours = moment().subtract(2, 'hours').toDate(),
last24hours = moment().subtract(24, 'hours').toDate();
MongoClient.connect(config.database)
.then(function(db) {
return db.collection('MyCollection')
})
.then(function (collection) {
return collection.aggregate([
{ '$match': { 'response_log.created_at': { '$gt': last2hours } } },
{
'$redact': {
'$cond': [
{
'$lt': [
{
'$size': {
'$filter': {
'input': '$response_log',
'as': 'res',
'cond': {
'$lt': [
'$$res.created_at',
last24hours
]
}
}
}
},
3
]
},
'$$KEEP',
'$$PRUNE'
]
}
}
]).toArray();
})
.then(function(docs) {
console.log(docs)
})
.catch(function(err) {
throw err;
});
Explanations
In the above aggregate operation, if you execute the first $match pipeline step
collection.aggregate([
{ '$match': { 'response_log.created_at': { '$gt': last2hours } } }
])
The documents returned will be the ones that do not have "response_log.created_at" in last 2 hours from current time where the variable last2hours is created with the momentjs library using the subtract API.
The preceding pipeline with $redact will then further filter the documents from the above by using the $cond ternary operator that evaluates this logical expression that uses $size to get the count and $filter to return a filtered array with elements that match other logical condition
{
'$lt': [
{
'$size': {
'$filter': {
'input': '$response_log',
'as': 'res',
'cond': { '$lt': ['$$res.created_at', last24hours] }
}
}
},
3
]
}
to $$KEEP the document if this condition is true or $$PRUNE to "remove" the document where the evaluated condition is false.
I know that this is probably not the answer that you're looking for but this may not be the best use case for Mongo. It's easy to do that in a relational database, it's easy to do that in a database that supports map/reduce but it will not be straightforward in Mongo.
If your data looked different and you kept each log entry as a separate document that references the object (with id 5852725660632d916c8b9a38 in this case) instead of being a part of it, then you could make a simple query for the latest log entry that has that id. This is what I would do in your case if I ware to use Mongo for that (which I wouldn't).
What you can also do is keep a separate collection in Mongo, or add a new property to the object that you have here which would store the latest date of campaign added. Then it would be very easy to search for what you need.
When you are working with a database like Mongo then how your data looks like must reflect what you need to do with it, like in this case. Adding a last campaign date and updating it on every campaign added would let you search for those campaign that you need very easily.
If you want to be able to make any searches and aggregates possible then you may be better off using a relational database.
I have the following "query":
MySchema
.where('address1.loc').near({
center: {
type: 'Point',
coordinates: user.address1.loc
},
maxDistance: 10 * 1000
}).where('address2.loc').near({
center: {
type: 'Point',
coordinates: user.address2.loc
},
maxDistance: 10 * 1000
})
.exec(function(err, objects) {
console.log(err);
console.log(objects);
if(err) return eachCallback(err);
return eachCallback();
});
My schema has two addresses (one pickup- and one handover-address). So I have to use two "nears". But this doesn't seem to be possible:
{ [MongoError: Can't canonicalize query: BadValue Too many geoNear expressions] name: 'MongoError' }
What are my alternatives?
Update:
I talked to some guys from MongoDB. Although my use case for this is valid, this doesn't seem to be possible out-of-the-box. So I'm looking for a "workaround". If you need details about the use case, let me know.
And here is how I defined the location inside my "addresses":
...
loc: {type: [Number], index: '2dsphere'},
...
I had encountered the same issue and error message. But I cannot remember exactly how I could overcome. I hope we can overcome your issue!
If your schema definition like the following, please try to the below solution;
loc : {
'type': { type: String, default: 'Point' },
coordinates: [Number]
}
Could you check address.loc.coordinates stores as number on your collection?
Alternate way
Try to use $geoWithin operator
I think you can change your questioning clauses like the following. You should use $geoWithin and $centerSphere instead of $near and $maxDistance. You've tried to find records that have loc fields in 10.000 meters proximity. $centerSphere indicates radius, it uses legacy coordinate values (only [longitude, latitude]) and it can used with geoJSON field and 2dsphere index. And your $centerSphere definition must be 10/6371 for 10 km. (tutorial).
If you use a second operator with $near operator it occurs a problem on MongoDB side.
You cannot combine the $near operator, which requires a special geospatial index, with a query operator or command that uses a different type of special index. For example you cannot combine $near with the $text query.
var r = 10/6371;
var area1 = { center: address1.loc.coordinates, radius: r, unique: true }
var area2 = { center: address2.loc.coordinates, radius: r, unique: true }
MySchema
.where('address1.loc').
.within()
.circle(area1)
.where('address2.loc').
.within()
.circle(area2)
.exec(function(err, objects) {
console.log(err);
console.log(objects);
if(err) return eachCallback(err);
return eachCallback();
});
Hope this helps..
I am trying to store some geojson points in mongo db.Here is the mongoose schema I have defined.
new mongoose.Schema({
itemName:String,
loc:{ type: [Number], index: '2dsphere'}
});
Next I am trying to run a near operator to find out some points nearby.This is how I have defined the near query.
Model.where('loc').near({ center: { type:"Point",coordinates: [10,10] }, maxDistance: 20 }).exec(function(err,data){
console.log(err);
if (!err && data) {
console.log("Success");
} else {
console.log("Error");
}
});
But this is returning an error value as follows:
{ [MongoError: Can't canonicalize query: BadValue geo near accepts
just one argu ment when querying for a GeoJSON point. Extra field
found: $maxDistance: 20] name : 'MongoError' }
If i remove the maxDistance it is returning all the collection elements.
Any idea why this error is coming?Thanks in advance for any help
Mongoose is still using the 'geoNear' database command form. This is considered obsolete in all ongoing versions of MongoDB.
Use the standard query form instead, which has been integrated with the standard query engine since MongoDB 2.6 and greater versions:
Model.find({
"loc": {
"$near": {
"$geometery": {
"type": "Point",
"coordinates": [ 10,10 ],
},
"$maxDistance": 20
}
}
},function(err,docs) {
// do something here
});
It's JavaScript, a "dynamically typed language". You don't need these ridiculous function helpers that are needed for strict typed languages with no dynamic constructs for defining and Object structure.
So do what the manual (which all examples are in JSON notation, which JavaScript natively understands) tells you to do and you are always fine.