MongoDB aggregation slower with indexes - node.js

Question

I'm facing a rather unexpected behavior with an aggregation. I'm running some performance tests with a collection of ~10M documents which have the form
{
_id: ObjectId(),
location: {
type: 'Point',
coordinates: [
26.38914519633397,
-42.404163283097745
]
},
isAvailable: true,
...more properties
}
with the aggregation :
this.mongo.db.collection('users').aggregate([
{$limit: 1000000},
{$match: {
isAvailable: true,
location: {
$geoWithin: {
$centerSphere :[
position, //an array like [ 26.3, -42.4]
radius //a number between 0 and 1
]}},
}}
],{allowDiskUse: true}).toArray()
The thing is that (everything else being equal) this aggregation takes ~1,8s with indexes (on isAvailable and location) and ~1,65s without these indexes. (Average on 50 aggregationz)
What am i missing ? What am I doing wrong ?
The indexes
{"isAvailable":1}
//and
{"location":"2dsphere"}
Edit: Some cases (10 aggregations average)
(real use case has just another match stage)
1) Match only
With indexes ~11,5s found 298,845 documents
Without indexes ~9,7s found 298,845 documents
Real use case with indexes ~33,5s found 15,553 documents
Real use case without indexes ~9,3s found 15,553 documents (wtf)
2) Limit 1M and then match
With indexes ~1,45s found 29,726 documents
Without indexes ~1,4s found 29,726 documents
Real use case with indexes ~1,8s found 1,525 documents
Real use case without indexes ~1,7s found 1,525 documents
3) Match and then limit 100k (and then another match for 3 and 4)
With indexes ~3,6s found 100,000 documents
Without indexes ~3,1s found 100,000 documents
Real use case with indexes ~3,6s found 5,137 documents
Real use case without indexes ~3,1s found 5,137 documents
Edit: Some cases with .explain("executionStats") (Match only, no limit)
1) With indexes ~11,4s
{
explainVersion: '1',
queryPlanner: {
namespace: 'database.userProps',
indexFilterSet: false,
parsedQuery: { '$and': [Array] },
optimizedPipeline: true,
maxIndexedOrSolutionsReached: false,
maxIndexedAndSolutionsReached: false,
maxScansToExplodeReached: false,
winningPlan: { stage: 'FETCH', filter: [Object], inputStage: [Object] },
rejectedPlans: [ [Object] ]
},
executionStats: {
executionSuccess: true,
nReturned: 298845,
executionTimeMillis: 11744,
totalKeysExamined: 8003852,
totalDocsExamined: 8003852,
executionStages: {
stage: 'FETCH',
filter: [Object],
nReturned: 298845,
executionTimeMillisEstimate: 1166,
works: 8003853,
advanced: 298845,
needTime: 7705007,
needYield: 0,
saveState: 8021,
restoreState: 8021,
isEOF: 1,
docsExamined: 8003852,
alreadyHasObj: 0,
inputStage: {
stage: 'IXSCAN',
nReturned: 8003852,
executionTimeMillisEstimate: 184,
works: 8003853,
advanced: 8003852,
needTime: 0,
needYield: 0,
saveState: 8039,
restoreState: 8039,
isEOF: 1,
keyPattern: { isAvailable: 1 },
indexName: 'isAvailable_1',
isMultiKey: false,
multiKeyPaths: { isAvailable: [] },
isUnique: false,
isSparse: false,
isPartial: false,
indexVersion: 2,
direction: 'forward',
indexBounds: { isAvailable: [ '[true, true]' ] },
keysExamined: 8003852,
seeks: 1,
dupsTested: 0,
dupsDropped: 0
}
}
},
command: {
aggregate: 'users',
pipeline: [ [Object] ],
allowDiskUse: true,
cursor: {},
'$db': 'database'
},
serverInfo: {...},
serverParameters: {...},
ok: 1
}
2) Without indexes ~9,8s
explainVersion: '1',
queryPlanner: {
namespace: 'database.userProps',
indexFilterSet: false,
parsedQuery: { '$and': [Array] },
optimizedPipeline: true,
maxIndexedOrSolutionsReached: false,
maxIndexedAndSolutionsReached: false,
maxScansToExplodeReached: false,
winningPlan: { stage: 'COLLSCAN', filter: [Object], direction: 'forward' },
rejectedPlans: []
},
executionStats: {
executionSuccess: true,
nReturned: 298845,
executionTimeMillis: 9446,
totalKeysExamined: 0,
totalDocsExamined: 10000001,
executionStages: {
stage: 'COLLSCAN',
filter: [Object],
nReturned: 298845,
executionTimeMillisEstimate: 632,
works: 10000003,
advanced: 298845,
needTime: 9701157,
needYield: 0,
saveState: 10000,
restoreState: 10000,
isEOF: 1,
direction: 'forward',
docsExamined: 10000001
}
},
command: {
aggregate: 'users',
pipeline: [ [Object] ],
allowDiskUse: true,
cursor: {},
'$db': 'database'
},
serverInfo: {...},
serverParameters: {...},
ok: 1
}
3) With compound index ~33s
{
explainVersion: '1',
queryPlanner: {
namespace: 'database.userProps',
indexFilterSet: false,
parsedQuery: { '$and': [Array] },
optimizedPipeline: true,
maxIndexedOrSolutionsReached: false,
maxIndexedAndSolutionsReached: false,
maxScansToExplodeReached: false,
winningPlan: { stage: 'FETCH', filter: [Object], inputStage: [Object] },
rejectedPlans: []
},
executionStats: {
executionSuccess: true,
nReturned: 298845,
executionTimeMillis: 35564,
totalKeysExamined: 423980,
totalDocsExamined: 423951,
executionStages: {
stage: 'FETCH',
filter: [Object],
nReturned: 298845,
executionTimeMillisEstimate: 35281,
works: 423980,
advanced: 298845,
needTime: 125134,
needYield: 0,
saveState: 2239,
restoreState: 2239,
isEOF: 1,
docsExamined: 423951,
alreadyHasObj: 0,
inputStage: {
stage: 'IXSCAN',
nReturned: 423951,
executionTimeMillisEstimate: 468,
works: 423980,
advanced: 423951,
needTime: 28,
needYield: 0,
saveState: 2634,
restoreState: 2634,
isEOF: 1,
keyPattern: [Object],
indexName: 'isAvailable_1_location_2dsphere',
isMultiKey: false,
multiKeyPaths: [Object],
isUnique: false,
isSparse: false,
isPartial: false,
indexVersion: 2,
direction: 'forward',
indexBounds: [Object],
keysExamined: 423980,
seeks: 29,
dupsTested: 0,
dupsDropped: 0
}
}
},
command: {
aggregate: 'users',
pipeline: [ [Object] ],
allowDiskUse: true,
cursor: {},
'$db': 'database'
},
serverInfo: {...},
serverParameters: {...},
ok: 1
}

Answer 1

This overall question is very interesting. Even with the available information, there is still a bunch of nuance that is tricky to wade through. Indeed I very much agree with #Alex Blex in the comments:
inputStage: [Object] ? =) The object is the most interesting part.
The problem with the [Object] redaction is likely related to how the MongoDB Shell is configured. Perhaps gather the verbose explain outputs again after either upgrading the shell or after increasing the inspectDepth setting (see here).
It would also be helpful to know what version of the database you are using. This is included in the serverInfo field of explain output which appears to have been manually redacted up to this point.
The reason the truncated information is so important is because it contains the critical details about how the database is actually using the indexes etc. For example:
stage: 'FETCH',
filter: [Object],
...
inputStage: {
stage: 'IXSCAN',
indexName: 'isAvailable_1_location_2dsphere',
indexBounds: [Object],
...
Here we can see that the database is doing some filtering after FETCHing the data, but we don't know what that filtering is or why it wasn't able to do so during the IXSCAN phase (e.g. directly via the indexBounds or secondarily as a filter during that stage).
Indexes are best at supporting the efficient retrieval of a selective result set. The larger that result set is relative to the total collection size, the less beneficial an index will be in supporting the operation (as was also mentioned by #Alex Blex in the comments). It may very well be the case that the database is not using the index as efficiently as expected, but it could also be the case that environmental factors (including the relative size of the results) are the primary challenges here. There are situations where indexes are not helpful, and indeed part of the optimizer's job is to figure out when it may not be appropriate to use them.
We would only be able to speculate about these components without updated explain output to investigate further.
Usage of Limit
This was also touched on in the comments, but what is your intention with using limit here?
Placing it as the first stage of the aggregation pipeline changes the semantics of what you are asking for. When it is placed first, you are instructing the database to first retrieve that number of documents from the collection and then perform subsequent processing on them (such as filtering in a later $match). This may be quite different than what your intended semantics (and therefore database behavior) are. The result is that your tests and conclusions from them may not be representative of the actual problem that you are investigating.
As a general pointer, it is always a good idea to explain and test the exact operation that you intend to run with the application. In this case that likely means removing a preceding $limit and having the compound index present. Doing so will greatly improve the applicability of the investigation and the associated suggestions.