I have a collection of documents that look like this:
[
{ group_id: 1, value: 'foo' },
{ group_id: 1, value: 'bar' },
{ group_id: 1, value: 'bar' },
{ group_id: 1, value: 'bar' },
{ group_id: 2, value: 'bar' },
{ group_id: 2, value: 'foo' },
{ group_id: 2, value: 'foo' },
{ group_id: 2, value: 'foo' }
]
For each group_id I want to return the value that occurs the most. So my output should look something like this...
[
{ group_id: 1, maxValue: 'bar', maxValueCount: 3 },
{ group_id: 2, maxValue: 'foo', maxValueCount: 3 }
]
How would you do this using the Mongoose aggregate function?
Update:
This is as far as I've gotten, I just need to return the value of the maximum count now...
const records = await Record.aggregate([
{
$group: {
_id: {
id: '$group_id',
value: '$value'
},
count: { $sum: 1 }
}
}
])
You may achieve your goal by
grouping by group_id/value (and counting the occurrences of value)
then grouping by group_id and pushing all the found values within with their count
finally keeping from the array the value having the max count
data=[
{ group_id: 1, value: 'foo' },
{ group_id: 1, value: 'bar' },
{ group_id: 1, value: 'bar' },
{ group_id: 1, value: 'bar' },
{ group_id: 1, value: 'bar' },//one more added
{ group_id: 2, value: 'bar' },
{ group_id: 2, value: 'foo' },
{ group_id: 2, value: 'foo' },
{ group_id: 2, value: 'foo' }
]
db.products.remove({})
db.products.insert(data)
const stages = [
{
$group: {
_id: {
group_id: '$group_id',
value: '$value'
},
n: { $sum: 1 }
}
},
{
$group: {
_id: '$_id.group_id',
values: {
$push: {
value: '$_id.value',
n: '$n'
}
}
}
},
{
$project: {
group_id:1,
best: {
$reduce: {
input: '$values',
initialValue: { n: 0, value: ''},
in: {
$cond: [
{
$lt: ['$$value.n', '$$this.n']
},
'$$this',
'$$value'
]
}
}
}
},
},
{
$project: {
group_id: 1,
value: '$best.value',
maxValue: '$best.n'
}
}
]
printjson(db.products.aggregate(stages).toArray())
playground
Related
I have multiple documents in a collection like this
[
{
_id: 123,
data: 1,
details: [
{
item: "a",
day: 1
},
{
item: "a",
day: 2
},
{
item: "a",
day: 3
},
{
item: "a",
day: 4
}
],
someMoreField: "xyz"
}
]
Now I want document with _id: 123 and details field should only contain day within range of 1 to 3. So the result will be like below.
{
_id: 123,
data: 1,
details: [
{
item: 'a',
day: 1,
},
{
item: 'a',
day: 2,
},
{
item: 'a',
day: 3,
},
],
someMoreField: 'xyz',
};
I tried to do this by aggregate query as:
db.collectionaggregate([
{
$match: {
_id: id,
'details.day': { $gt: 1, $lte: 3 },
},
},
{
$project: {
_id: 1,
details: {
$filter: {
input: '$details',
as: 'value',
cond: {
$and: [
{ $gt: ['$$value.date', 1] },
{ $lt: ['$$value.date', 3] },
],
},
},
},
},
},
])
But this gives me empty result. Could someone please guide me through this?
You are very close, you just need to change the $gt to $gte and $lt to $lte.
Another minor syntax error is you're accessing $$value.date but the schema you provided does not have that field, it seems you need to change it to $$value.day, like so:
db.collection.aggregate([
{
$match: {
_id: 123,
"details.day": {
$gt: 1,
$lte: 3
}
}
},
{
$project: {
_id: 1,
details: {
$filter: {
input: "$details",
as: "value",
cond: {
$and: [
{
$gte: [
"$$value.day",
1
]
},
{
$lte: [
"$$value.day",
3
]
},
],
},
},
},
},
},
])
Mongo Playground
I have data object that contains array. And I have providers table.
if array's Id should equals to provider's table id Id == id
if id is repeated take the repeated count as membersCounts else membersCounts = 0
Add the membersCounts with data object
data object
const data = {
Milk: [
{
Id: 1,
name: 'a'
},
{
Id: 2,
name: 'b'
},
{
Id: 3,
name: 'c'
},
{
Id: 4,
name: 'd'
},
{
Id: 52,
name: 'e'
}
],
Grocery: [
{
Id: 8,
name: '2a'
},
{
Id: 22,
name: '2b'
},
{
Id: 32,
name: '2c'
},
{
Id: 42,
name: '2d'
}
]
}
providers table
const providers = [
{
id: 1,
status: 'active'
},
{
id: 1,
status: 'active'
},
{
id: 1,
status: 'active'
},
{
id: 1,
status: 'active'
},
{
id: 4,
status: 'active'
},
{
id: 2,
status: 'active'
},
{
id: 3,
status: 'active'
},
{
id: 3,
status: 'active'
},
{
id: 52,
status: 'active'
},
{
id: 1,
status: 'active'
}
]
here javascript code
this code is working good but I want perform this with mongodb queries. So that performance is good .
Is possible to do with mongodb query. I need to covert the javascript code to mongodb query.
getMembersWithVendors(data, providers) {
for (var key in data) {
var arr = data[key]
arr.forEach((element) => {
element.memberCounts = 0
element.new = true
providers.map((el) => {
if (element._id == el.vendorId) {
(element.memberCounts = element.memberCounts + 1),
(element.new = false)
}
})
})
}
return data
}
output
{ Milk:
[ { Id: 1, name: 'a', memberCounts: 5 },
{ Id: 2, name: 'b', memberCounts: 1 },
{ Id: 3, name: 'c', memberCounts: 2 },
{ Id: 4, name: 'd', memberCounts: 1 },
{ Id: 52, name: 'e', memberCounts: 1 } ],
Grocery:
[ { Id: 8, name: '2a', memberCounts: 0 },
{ Id: 22, name: '2b', memberCounts: 0 },
{ Id: 32, name: '2c', memberCounts: 0 },
{ Id: 42, name: '2d', memberCounts: 0 } ] }
Thanks !!
this code is working good but I want perform this with mongodb queries. So that performance is good
That is not good idea to do all the operations in query, it may cause performance issues, because your input data is so big, but you can improve some things,
$group by query id and get count, this will return unique ids and its total counts
let providers = await db.providers.aggregate([
{
$group: {
_id: "$id",
count: { $sum: 1 }
}
}
]);
iterate loop of object's array
find from providers on the base of id
get count from filtered document
for (let key in data) {
data[key].forEach(e => {
let p = providers.find(p => p._id === e.Id);
e.memberCounts = p ? p.count : 0;
})
}
console.log(data);
Repl Playground
I have got an array of objects in MongoDB and I was moving a particular element id (i.e 1) from its position to below element having id (i.e 2). So that we can get element with id as 2 -> 1 -> 3.
const arr = [
{
id: 1,
name: 'foo'
},
{
id: 2,
name: 'bar'
},
{
id: 3,
name: 'zoo'
}
]
What I've done is used $pull and $push but it gives ConflictingUpdateOperators and I don't know how to deal with it.
updatedPlan = await Plan.findOneAndUpdate(
{ _id: req.params.id },
{
$pull: {
"arr": {
id: 1
}
},
$push: {
"arr" : {
$each: [{ id: 1, name: 'foo'}],
$position: 1
}
},
);
In MongoDB 4.2 or newer you can update a document with Aggregation Pipeline. Using simple $map on a $range of array indexes you can shuffle these indexes and use $arrayElemAt in order to build a new array:
db.col.update({ _id: req.params.id }, [
{
$set: {
arr: {
$map: {
input: { $range: [ 0, { $size: "$arr" } ] },
in: {
$let: {
vars: {
newIndex: {
$switch: {
branches: [
{ case: { "$eq": [ "$$this", 0 ] }, then: 1 },
{ case: { "$lte": [ "$$this", 1 ] }, then: { $subtract: [ "$$this", 1 ] } },
],
default: "$$this"
}
}
},
in: {
$arrayElemAt: [ "$arr", "$$newIndex" ]
}
}
}
}
}
}
}
])
I've been working on a small project that takes MQTT data from sensors and stores it in a MongoDB database. I'm working with nodeJS and mongoose. These are my schemas.
export const SensorSchema = new mongoose.Schema({
name: { type: String, required: true, unique: true },
location: { type: String, required: true },
type: { type: String, required: true },
unit: { type: String, required: true },
measurements: { type: [MeasurementSchema] }
},
{
toObject: { virtuals: true },
toJSON: { virtuals: true }
});
export const MeasurementSchema = new mongoose.Schema({
value: {type: Number, required: true},
time: {type: Date, required: true}
});
First I wrote a function that retrieves all measurements that were made in between two timestamps.
const values = Sensor.aggregate([
{ $match: Sensor.getValuesFromPath(sensorPath) },
{ $unwind: "$measurements"},
{ $match: { "measurements.time": { $gte: startTime, $lte: endTime} }},
{ $replaceRoot: { newRoot: "$measurements" } },
{ $project: { _id: 0}},
{ $sort: {time: 1}}
]).exec();
In order to draw a graph in the UI, I need to somehow sort and then limit the data that gets sent to the client. I want to send every Nth Value in a certain interval to ensure that the data somewhat resembles the course of the data.
I would prefer a solution that doesn't fetch all the data from the database.
How would I go about doing this on the db? Can I somehow access the positional index of an element after sorting it? Is $arrayElemAt or $elemMatch the solution?
Befure you run $unwind you can use $filter to apply start/end Date filtering. This will allow you to process measurements as an array. In the next step you can get every N-th element by using $range to define a list of indexes and $arrayElemAt to retrieve elements from these indexes:
const values = Sensor.aggregate([
{ $match: Sensor.getValuesFromPath(sensorPath) },
{ $addFields: {
measurements: {
$filter: {
input: "$measurements",
cond: { $and: [
{ $gte: [ "$$this.time", startTime ] },
{ $lte: [ "$$this.time", endTime ] }
]
}
}
}
} },
{ $addFields: {
measurements: {
$map: {
input: input: { $range: [ 0, { $size: "$measurements" }, N ] },
as: "index",
in: { $arrayElemAt: [ "$measurements", "$$index" ] }
}
}
} },
{ $unwind: "$measurements" },
{ $replaceRoot: { newRoot: "$measurements" } },
{ $project: { _id: 0}},
{ $sort: {time: 1}}
]).exec();
The following aggregation (i) retrieves all measurements that were made in between two timestamps, (ii) sorts by timestamp for each sensor, and (iii) gets every Nth value (specified by the variable EVERY_N).
Sample documents (with some arbitrary data for testing):
{
name: "s-1",
location: "123",
type: "456",
measurements: [ { time: 2, value: 12 }, { time: 3, value: 13 },
{ time: 4, value: 15 }, { time: 5, value: 22 },
{ time: 6, value: 34 }, { time: 7, value: 9 },
{ time: 8, value: 5 }, { time: 9, value: 1 },
]
},
{
name: "s-2",
location: "789",
type: "900",
measurements: [ { time: 1, value: 31 }, { time: 3, value: 32 },
{ time: 4, value: 35 }, { time: 6, value: 39 },
{ time: 7, value: 6}, { time: 8, value: 70 },
{ time: 9, value: 74 }, { time: 10, value: 82 }
]
}
The aggregation:
var startTime = 3, endTime = 10
var EVERY_N = 2 // value can be 3, etc.
db.collection.aggregate( [
{
$unwind: "$measurements"
},
{
$match: {
"measurements.time": { $gte: startTime, $lte: endTime }
}
},
{
$sort: { name: 1, "measurements.time": 1 }
},
{
$group: {
_id: "$name",
measurements: { $push: "$measurements" },
doc: { $first: "$$ROOT" }
}
},
{
$addFields: {
"doc.measurements": "$measurements"
}
},
{
$replaceRoot: { newRoot: "$doc" }
},
{
$addFields: {
measurements: {
$reduce: {
input: { $range: [ 0, { $size: "$measurements" } ] },
initialValue: [ ],
in: { $cond: [ { $eq: [ { $mod: [ "$$this", EVERY_N ] }, 0 ] },
{ $concatArrays: [ "$$value", [ { $arrayElemAt: [ "$measurements", "$$this" ] } ] ] },
"$$value"
]
}
}
}
}
}
] )
how you doing?
I have a trouble making a aggregation in my project, my aggregation result is different in Robo3T and Node.
db.getCollection('companies').aggregate([
{ '$match': { _id: { '$eq': ObjectId("5e30a4fe11e6e80d7fb544a4")} } },
{ $unwind: '$jobVacancies' },
{
$project: {
jobVacancies: {
_id: 1,
name: 1,
city: 1,
openingDate: 1,
closingDate: 1,
createdAt: 1,
quantity: 1,
steps: {
$filter: {
input: '$jobVacancies.steps',
as: 'step',
cond: {
$and: [
{ $eq: ['$$step.order', 0] },
{ $ne: ['$$step.users', undefined] },
{ $ne: ['$$step.users', null] },
{ $ne: ['$$step.users', []] },
],
},
},
},
},
},
},
{ $match: { 'jobVacancies.steps': { $ne: [] } } },
])
In Robo3T this is returning 1 object, but in Node (the same aggregation) is resulting 6 objects. Can you help me? Thank you
EDIT
Nodejs:
The first match create the ObjectId match for company in context of GraphQL based on my filter.
const companies = await this.MongoClient.db
.collection('companies')
.aggregate([
{
$match: await this.getFilterObject(
filters.filter(f => !f.field.includes('$$jobVacancy') && !f.field.includes('StepOrder')),
),
},
{ $unwind: '$jobVacancies' },
{
$project: {
jobVacancies: {
_id: 1,
name: 1,
city: 1,
openingDate: 1,
closingDate: 1,
createdAt: 1,
quantity: 1,
steps: {
$filter: {
input: '$jobVacancies.steps',
as: 'step',
cond: {
$and: [
{ $eq: ['$$step.order', order] },
{ $ne: ['$$step.users', undefined] },
{ $ne: ['$$step.users', null] },
{ $ne: ['$$step.users', []] },
],
},
},
},
},
},
},
{ $match: { 'jobVacancies.steps': { $ne: [] } } },
])
.toArray();
EDIT 3
This is the result of console.dir (with {depth:null}) of the pipeline
[
{
'$match': {
_id: {
'$eq': ObjectID {
_bsontype: 'ObjectID',
id: Buffer [Uint8Array] [
94, 48, 164, 254, 17,
230, 232, 13, 127, 181,
68, 164
]
}
}
}
},
{ '$unwind': '$jobVacancies' },
{
'$project': {
jobVacancies: {
_id: 1,
name: 1,
city: 1,
openingDate: 1,
closingDate: 1,
createdAt: 1,
quantity: 1,
steps: {
'$filter': {
input: '$jobVacancies.steps',
as: 'step',
cond: {
'$and': [
{ '$eq': [ '$$step.order', 0 ] },
{ '$ne': [ '$$step.users', undefined ] },
{ '$ne': [ '$$step.users', null ] },
{ '$ne': [ '$$step.users', [] ] }
]
}
}
}
}
}
},
{ '$match': { 'jobVacancies.steps': { '$ne': [] } } }
]
I think i found the solution, the document is created with properties:
jobVacancies: {
steps: {
users: []
}
}
But sometimes users array is undefined in mongodb, so I verify with
{ '$ne': [ '$$step.users', undefined ] }
I think JS undefined is different then mongodb undefined, so I initialized all steps with an empty array of users, and removed this verification and worked! –