I want to pull multiple objects from array.
Here is my sample collection:
Users
{
"_id": "wef324DGSshf",
"userTypes": [
{
"type": "students",
"users": [
{
"name": "John",
"age": 20
},
{
"name": "Mike",
"age": 20
},
{
"name": "Henry",
"age": 30
},
{
"name": "Henry",
"age": 40
}
]
}
]
}
I need to pull those objects where:
type: "students" and ages: [20,40]
So I have these 2 inputs: type & ages
Expected Response:
{
"_id": "wef324DGSshf",
"userTypes": [
{
"type": "students",
"users": [
{
"name": "Henry",
"age": 30
}
]
}
]
}
I have tried this query so far but it is not working:
Users.update({
"userTypes.type": "students",
"userTypes.users.age": {$in: [20, 40]},
},
{
$pull: {
"userTypes": {
"userTypes.users.$.age": {$in: [20, 40]}
}
}
});
Can anyone help me what I am doing wrong here?
Use an arrayFilters to specify the filtering for "type": "students" and normally perform $pull on age
db.collection.update({},
{
"$pull": {
"userTypes.$[ut].users": {
"age": {
$in: [
20,
40
]
}
}
}
},
{
arrayFilters: [
{
"ut.type": "students"
}
],
multi: true
})
Mongo Playground
Explanation: Check out the official doc about arrayFilters. You can think of the entries in arrayFilters as predicates. For a variable ut, it needs to have type: students. Let's go back to the $pull part. The predicate is applied to userTypes. That means for an entry in userTypes, ut, it needs to fit in the predicate of type: students. At the same time, we are $pulling the entries that age is in [20, 40].
Related
In my client I have a form that is sent and stored in Mongo. Made an aggregation to get the name of the people that selected a same place, date and time. Now I would like to create a Mongo document containing all matches as collections so whenever there is a match in place, date and time of people you can get it in a collection. This is what I have so far:
router.get('/match', async (req, res) => {
const matchs = await Forms.aggregate([
{
$group: {
_id: { Date: "$date", Time: "$time", Place: "$place" },
Data: { $addToSet: {Name: "$firstName", Surname:"$surname"}},
count: { $sum: 1 }
}
},
{
$match: {
count: { $gte: 2}
}
},
]);
res.json(matchs)
});
This is the result that I would like to store in Mongo:
{
"_id": {
"Date": "2022-04-20",
"Time": "15:00",
"Place": "Mall"
},
"Data": [
{
"Name": "Carl",
"Surname": "Man"
},
{
"Name": "Christian",
"Surname": "Max"
}
],
"count": 2
}
{
"_id": {
"Date": "2022-04-20",
"Time": "13:00",
"Place": "Restaurant"
},
"Data": [
{
"Name": "Felix",
"Surname": "Sad"
},
{
"Name": "Liu",
"Surname": "Lam"
}
],
"count": 2
}
You can use $out as the last stage in your pipeline. In the following example, matching_collection will contain the result of your pipeline.
{ $out : "matching_collection" }
https://www.mongodb.com/docs/v4.2/reference/operator/aggregation/out/
You can also check $merge, it could be helpful as well.
I have a large collection of documents that look as follows:
{ "_id": "5a760191813a54000b8475f1", "orders": [{ "row": "3", "seat": "11" }, { "row": "3", "seat": "12" }], "product_id": "5a7628bedbcc42000aa7f614" },
{ "_id": "5a75f6f17abe45000a3ba05e", "orders": [{ "row": "3", "seat": "12" }, { "row": "3", "seat": "13" }], "product_id": "5a7628bedbcc42000aa7f614" },
{ "_id": "5a75ebdf813a54000b8475e7", "orders": [{ "row": "5", "seat": "16" }, { "row": "5", "seat": "15" }], "product_id": "5a75f711dbcc42000c459efc" }
I need to be able to find any documents where the product_id and items in the orders array are duplicates. I can't quite seem to wrap my head around accomplishing this. Any pointers?
I don't know what output you want, but this has the information about the duplicates, maybe you want to add unwind on duplicates also.
Result documents
product_id
order (that found duplicated)
duplicates (the documents that had that order as duplicate)
For your data would print
[{
"duplicates": [
"5a760191813a54000b8475f1",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "12"
},
"product_id": "5a7628bedbcc42000aa7f614"
}]
Query
(run it on your driver, MongoPlayground doesn't keep the order of fields and can show wrong results)
aggregate(
[{"$unwind" : {"path" : "$orders"}},
{
"$group" : {
"_id" : {
"orders" : "$orders",
"product_id" : "$product_id"
},
"duplicates" : {
"$push" : "$_id"
}
}
},
{"$match" : {"$expr" : {"$gt" : [ {"$size" : "$duplicates"}, 1 ]}}},
{
"$project" : {
"_id" : 0,
"order" : "$_id.orders",
"product_id" : "$_id.product_id",
"duplicates" : 1
}
}
])
Data (i added some more data)
[
{
"_id": "5a760191813a54000b8475f1",
"orders": [
{
"row": "3",
"seat": "11"
},
{
"row": "3",
"seat": "12"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75f6f17abe45000a3ba05g",
"orders": [
{
"row": "3",
"seat": "12"
},
{
"row": "3",
"seat": "13"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75f6f17abe45000a3ba05e",
"orders": [
{
"row": "3",
"seat": "12"
},
{
"row": "3",
"seat": "13"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75ebdf813a54000b8475e7",
"orders": [
{
"row": "5",
"seat": "16"
},
{
"row": "5",
"seat": "15"
}
],
"product_id": "5a75f711dbcc42000c459efc"
}
]
Results
[{
"duplicates": [
"5a75f6f17abe45000a3ba05g",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "13"
},
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"duplicates": [
"5a760191813a54000b8475f1",
"5a75f6f17abe45000a3ba05g",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "12"
},
"product_id": "5a7628bedbcc42000aa7f614"
}]
You could use below query. $unwind the orders array, $group by order row and product and collect matching ids and count. Keep the documents where count is greater than 1. $lookup to pull in the matching documents by id and $replaceRoot to flatten the documents.
db.collection.aggregate([
{
"$unwind": "$orders"
},
{
"$group": {
"_id": {
"order": "$orders",
"product_id": "$product_id"
},
"count": {
"$sum": 1
},
"doc_ids": {
"$push": "$_id"
}
}
},
{
"$match": {
"count": {
"$gt": 1
}
}
},
{
"$lookup": {
"from": "collection",
"localField": "doc_ids",
"foreignField": "_id",
"as": "documents"
}
},
{
"$unwind": "$documents"
},
{
"$replaceRoot": {
"newRoot": "$documents"
}
}
])
https://mongoplayground.net/p/YbztEGttUMx
While this can be done purely in Mongo I do not recommend it as it's very very very memory inefficient. you basically have to hold the entire collection in memory the entire time while you do certain manipulations on it.
I will however show the pipeline for this because we will use it with the second more scaleable approach.
We want to $group based on orders and product_id, however there are 2 issues standing in our way.
The orders field might not be sorted the same in all documents, because Mongo does not support "nested" sorting we have to $unwind the array, $sort it and restore the original structure. ( mind you you're sorting the entire collection here in memory ). This step which is one of the pain points of this pipeline can be skipped if you can ensure sort order is maintained in the orders array.
Mongo is inconsistent when $grouping an array of objects. full disclosure I'm not entirely sure what's going on in there but I'm guessing there are some "shortcuts" done for efficiency which affects the stability somehow. So our approach will be to convert these objects into a string (concating the "row" and "seat" together).
db.collection.aggregate([
{
"$unwind": "$orders"
},
{
$sort: {
"orders.row": 1,
"orders.seat": 1
}
},
{
$group: {
_id: "$_id",
tmpOrders: {
$push: {
$concat: [
"$orders.row",
"$orders.seat"
]
}
},
product_id: {
$first: "$product_id"
}
}
},
{
$group: {
_id: {
orders: "$tmpOrders",
product: "$product_id"
},
dupIds: {
$push: "$_id"
}
}
},
{
$match: {
"dupIds.0": {
$exists: true
}
}
},
{
$project: {
_id: 0,
dups: "$dupIds",
}
}
])
Mongo Playground
Now as I said this approach is not scaleable, and on large collections will take a very long time to run. So I recommend utilizing indexes and iterating over product_id's and executing each pipeline separately.
// wraps the native Promise, not required.
import Bluebird = require('bluebird');
// very fast with index.
const productIds = await collection.distinct('product_id')
await Bluebird.map(productIds, async (productId) => {
const dups = await collection.aggregate([
{
$match: {
product_id: productId
}
}
... same pipeline ...
])
if (dups.length) {
// logic required.
}
// can control concurrency based on db workload.
}, { concurrency: 5})
Make sure with this approach you have an index built on product_id so it will work efficiently.
I am very new to MongoDB and I need to do a somewhat complex Update operation on my collection.
I have this kind of collection:
[
{
"Id": 1,
"extension": [
{
"keyName": "Name",
"value": "Bob"
},
{
"keyAge": "Age",
"value": 20
}
]
},
{
"Id": 2,
"extension": [
{
"keyName": "Name",
"value": "Sam"
},
{
"key": "Name",
"value": "Sam"
}
]
},
{
"Id": 3,
"extension": [
{
"keyName": "Age",
"value": 25
},
{
"key": "Age",
"value": 25
}
]
},
{
"Id": 4
}
]
I would like to update any items in the extension array of all documents
so that when an item is found with a key property, to rename it keyAge.
Here is the expected result:
[
{
"Id": 1,
"extension": [
{
"keyName": "Name",
"value": "Bob"
},
{
"keyAge": "Age",
"value": 20
}
]
},
{
"Id": 2,
"extension": [
{
"keyName": "Name",
"value": "Sam"
},
{
"keyAge": "Name",
"value": "Sam"
}
]
},
{
"Id": 3,
"extension": [
{
"keyName": "Age",
"value": 25
},
{
"keyAge": "Age",
"value": 25
}
]
},
{
"Id": 4
}
]
I tried to use $rename in a similar way to this question:
MongoDB rename database field within array
but I get the same error $rename source may not be dynamic array
I think this solution might also apply to me, I tried using it but it's not updating anything on my side, so I guess I cannot understand how to apply that answer to me...
https://stackoverflow.com/a/49193743/215553
Thanks for the help!
I tried to use $rename in a similar way to this question: MongoDB rename database field within array but I get the same error $rename source may not be dynamic array
There is a note in $rename:
$rename does not work if these fields are in array elements.
You can try update with aggregation pipeline starting from MongoDB 4.2,
check condition id key field is exists
$map to iterate loop of extension array
$map to iterate loop of array that is converted from extension object to array in key-value format
$cond check condition if k is key then return keyAge otherwise return current
$arrayToObject back to convert key-value array return from above map to object original format
db.collection.update(
{ "extension.key": { $exists: true } },
[{
$set: {
extension: {
$map: {
input: "$extension",
in: {
$arrayToObject: {
$map: {
input: { $objectToArray: "$$this" },
in: {
k: {
$cond: [
{ $eq: ["$$this.k", "key"] }, // check "key" field name
"keyAge", // update new name "keyAge"
"$$this.k"
]
},
v: "$$this.v"
}
}
}
}
}
}
}
}],
{ multi: true }
)
Playground
Question : How to Rename or add new key for existing array object key?
Answer : Inside projection of mongodb query we have map property which will resolve this.
Solution Example :
{
parents: {
$map: {
input: "$parents",
as: "parent",
in: {
caaUserId: "$$parent._id",
email: "$$parent.email",
countryCode: "$$parent.countryCode",
mobile: "$$parent.mobile",
reportType : "single"
}
}
}
}
In this example if we want to rename $parent._id as caaUserId in parents array for each Element.
Then we can use map and define caaUserId like $$parent._id. This whole code will work in mongoose projection of Query.
It should return following :
{
"parents" : [
{
"caaUserId" : "62d17fa164057000149e283f",
"email" : "john.doe#hotmail.com",
"countryCode" : 91,
"mobile": 9876543210,
"reportType":"single",
},
{
"caaUserId" : "6195d50f15ae2b001293c486",
"email" : "akka.ash#hotmail.com",
"countryCode" : 91,
"mobile": 9876543211,
"reportType":"multi",
},
]
}
This is something that works in your case. Might not be the most readable though.
import json
data = json.loads(strdata)
for entry in data:
if 'extension' in entry:
for x in entry['extension']:
for k, v in x.items():
if k == 'key':
x['keyAge'] = x.pop(k)
i have a problem with aggregation framework in MongoDB (mongoose) this is the problem. i have the following database scheme.so what i want to do is count number of people who has access through Mobile only , Card only, or both. with out any order,
{
'_id': ObjectId,
'user_access_type': ['Mobile' , 'Card']
}
{
'_id': ObjectId,
'user_access_type': ['Card' , 'Mobile']
}
{
'_id': ObjectId,
'user_access_type': ['Mobile']
}
{
'_id': ObjectId,
'user_access_type': ['Card']
}
Now i am using this but it only groups by the order of the user_access_type array,
[ { "$group" : { "_id": {"User" : "$user_access_type"} , "count": {"$sum" : 1} }]
this is the output:
{
"_id": {
"User": [
"Card",
"Mobile"
]
},
"count": 1
},
{
"_id": {
"_id": "5f7dce2359aaf004985f98eb",
"User": [
"Mobile",
"Card"
]
},
"count": 1
},
{
"_id": {
"User": [
"Mobile"
]
},
"count": 1
},
{
"_id": {
"User": [
"Card"
]
},
"count": 1
},
vs what i want:
{
"_id": {
"User": [
"Card",
"Mobile" // we can say both
]
},
"count": 2 // does not depend on order
},
{
"_id": {
"User": [
"Mobile"
]
},
"count": 1
},
{
"_id": {
"User": [
"Card"
]
},
"count": 1
},
You can use other option as well using $function,
$function can allow to add javascript code, you can use sort() to sort the array
db.collection.aggregate([
{
$addFields: {
user_access_type: {
$function: {
body: function(user_access_type){
return user_access_type.sort();
},
args: ["$user_access_type"],
lang: "js"
}
}
}
},
{
$group: {
_id: "$user_access_type",
count: { $sum: 1 }
}
}
])
Second option,
If user_access_type array having always unique elements then you can use $setUnion operator on user_access_type array as self union, some how this will re-order array in same order,
db.collection.aggregate([
{
$addFields: {
user_access_type: {
$setUnion: "$user_access_type"
}
}
},
{
$group: {
_id: "$user_access_type",
count: { $sum: 1 }
}
}
])
Playground
I have a simple datastructure in mongodb:
{
_id: ObjectID,
name: 'Name',
birthday: '25.05.2001'
items: [
{
_id: ObjectID,
name: 'ItemName',
info: 'ItemInfo',
},
{
_id: ObjectID,
name: 'ItemName',
info: 'ItemInfo',
}
]
}
Now i want a query, that takes a ObjectID (_id) of an item as criteria and gives me back the object with all items in the array AND projects a new field "selected" with value true or false into a field in the result of each array item:
I tried that with this query:
{ $unwind: '$items' },
{
$project: {
selected: {
$cond: { if: { 'items._id': itemObjectID }, then: true, else: false },
},
},
},
but MongoDB gives me back an error:
MongoError: FieldPath field names may not contain '.'.
Have no clue why its not working, any help or ideas? Thank you very much!
What you are missing here is $eq aggregation operator which checks the condition for the equality.
You can try below aggregation here if you want to check for ObjectId then you need to put mongoose.Types.ObjectId(_id)
db.collection.aggregate([
{ "$unwind": "$items" },
{ "$addFields": {
"items.selected": {
"$eq": [
1111,
"$items._id"
]
}
}},
{ "$group": {
"_id": "$_id",
"name": { "$first": "$name" },
"items": {
"$push": {
"_id": "$items._id",
"selected": "$items.selected"
}
}
}}
])
Will give following output
[
{
"_id": ObjectId("5a934e000102030405000000"),
"items": [
{
"_id": 1111,
"selected": true
},
{
"_id": 2222,
"selected": false
}
],
"name": "Name"
}
]
You can check it here
#Ashish: Thank you very much for your help! Your answer helped me to build the right query for me:
db.collection.aggregate([
{
$unwind: "$items"
},
{
$project: {
"items.name": 0,
"birthday": 0
}
},
{
"$addFields": {
"items.selected": {
"$eq": [
1111,
"$items._id"
]
}
}
},
{
$group: {
_id: "$_id",
"name": {
"$first": "$name"
},
items: {
$push: "$items"
}
}
},
{
$match: {
"items._id": {
$eq: 1111
}
}
},
])
and leads to a result that looks like:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"items": [
{
"_id": 1111,
"selected": true
},
{
"_id": 2222,
"selected": false
}
],
"name": "Name"
}
]