I want to pull multiple objects from array.
Here is my sample collection:
Users
{
"_id": "wef324DGSshf",
"userTypes": [
{
"type": "students",
"users": [
{
"name": "John",
"age": 20
},
{
"name": "Mike",
"age": 20
},
{
"name": "Henry",
"age": 30
},
{
"name": "Henry",
"age": 40
}
]
}
]
}
I need to pull those objects where:
type: "students" and ages: [20,40]
So I have these 2 inputs: type & ages
Expected Response:
{
"_id": "wef324DGSshf",
"userTypes": [
{
"type": "students",
"users": [
{
"name": "Henry",
"age": 30
}
]
}
]
}
I have tried this query so far but it is not working:
Users.update({
"userTypes.type": "students",
"userTypes.users.age": {$in: [20, 40]},
},
{
$pull: {
"userTypes": {
"userTypes.users.$.age": {$in: [20, 40]}
}
}
});
Can anyone help me what I am doing wrong here?
Use an arrayFilters to specify the filtering for "type": "students" and normally perform $pull on age
db.collection.update({},
{
"$pull": {
"userTypes.$[ut].users": {
"age": {
$in: [
20,
40
]
}
}
}
},
{
arrayFilters: [
{
"ut.type": "students"
}
],
multi: true
})
Mongo Playground
Explanation: Check out the official doc about arrayFilters. You can think of the entries in arrayFilters as predicates. For a variable ut, it needs to have type: students. Let's go back to the $pull part. The predicate is applied to userTypes. That means for an entry in userTypes, ut, it needs to fit in the predicate of type: students. At the same time, we are $pulling the entries that age is in [20, 40].
{
"_id": {
"$oid": "6200c86d083ef16be6dae5b1"
},
"type": "Hair_Length",
"values": [
{
"name": "Bald / Clean Shaven",
"value": "bald",
"default": "yes"
},
{
"name": "Crew Cut",
"value": "crewcut"
},
{
"name": "Short",
"value": "Short"
},
{
"name": "Medium",
"value": "Medium"
},
{
"name": "Long",
"value": "Long"
},
{
"name": "Chin Length",
"value": "ChinLength"
},
{
"name": "Shoulder Length",
"value": "ShoulderLength"
},
{
"name": "Dreadlocks",
"value": "Dreadlocks"
}
]
}
Good day guys! if I get name and value as json object then only I will update or add to this doc if incase we get "default":"yes" in addition to name and value then we need to update and delete if default is already exist I hope you got point
appreciate it guys tq
Since you know if your new item is a default item or not, before the query, it is better to adjust the query to the relevant case before executing it. But just for the sake of the challenge, this is a solution with one query only. I advise you to break it into two queries and use only one at each case.
db.collection.aggregate([
{
$match: {type: "Hair_Length"}
},
{
$addFields: {
newValue: {"name": "short hair", "value": "shorthair", "default": "yes"},
values: {$filter: {input: "$values",
as: "item",
cond: {$ne: ["$$item.value", "shorthair"]}
}
}
}
},
{
$facet: {
newIsDefault: [
{
$project: {"values.name": 1, "values.value": 1, type: 1, newValue: 1}
}
],
newIsRegular: [
{$project: {values: 1, type: 1, newValue: 1}
}
]
}
},
{
$project: {newIsDefault: {$arrayElemAt: ["$newIsDefault", 0]},
newIsRegular: {$arrayElemAt: ["$newIsRegular", 0]}}
},
{
$replaceRoot: {
newRoot: {
$cond: [{$eq: ["$newIsDefault.newValue.default", "yes"]},
"$newIsDefault",
"$newIsRegular"
]
}
}
},
{
$project: {
values: {$setUnion: [["$newValue"], "$values"]},
type: 1
}
},
{$merge: {into: "collection"}}
])
You can see it work on the playground on one case and on another case.
The main idea is that there are two main cases: the new item is default or not. On both cases you start by filtering out from the list the element with value matches the new item if they are there. Then, if the new item is default you remove the default field from the items in the list. Only now we can insert the new item to the list.
Query:
First you add your new item using $addFields and $filter out the item if exists, then you create the documents for the two cases using $facet. one of them to the case where the new item is default, and on e for the case it is not. then you choose which one to use, using $cond, and add the new item to it using $setUnion. Last step is $merge to replace the new doc with the old one.
Using the following code I was able to map a list of dictionaries by a key
import json
values_list = [{"id" : 1, "user":"Rick", "title":"More JQ"}, {"id" : 2, "user":"Steve", "title":"Beyond"}, {"id" : 1, "user":"Rick", "title":"Winning"}]
result = {}
for data in values_list:
id = data['id']
user = data['user']
title = data['title']
if id not in result:
result[id] = {
'id' : id,
'user' : user,
'books' : {'titles' : []}
}
result[id]['books']['titles'].append(title)
print(json.dumps((list(result.values())), indent=4))
Knowing how clean is Jolt Spec and trying to separate the schema outside of the code.
Is there a way to use Jolt Spec to achieve the same result.
The Result
[
{
"id": 1,
"user": "Rick",
"books": {
"titles": [
"More JQ",
"Winning"
]
}
},
{
"id": 2,
"user": "Steve",
"books": {
"titles": [
"Beyond"
]
}
}
]
You can use three levels of consecutive specs
[
{
"operation": "shift",
"spec": {
"*": {
"*": "#(1,id).&",
"title": "#(1,id).books.&s[]"
}
}
},
{
"operation": "shift",
"spec": {
"*": ""
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"id": "ONE",
"user": "ONE"
}
}
}
]
in the first spec, the common id values are combined by "#(1,id)." expression
in the second spec, the integer keys(1,2) of the outermost objects are removed
in the last spec,only the first of the repeating elements are picked
I need to group all array object and nested objects using given array objects based on same key value inside $project in mongoose aggregate.
Need to group all the title which one have same key value and also need to group nested objects as per appropriate keys. The key value might be dynamic one like "color, size, weight, height, ram, storage, etc,."
{ $group: {
_id: null,
"dynamicFilter": { "$push": '$product' },
}},
$project: { _id: 0,
autoFilter: {
$filter: {
input: '$dynamicFilter',
as: 'filterData',
cond: {
}
}
}
}}
Note
"$dynamicFilter" has some array object string which one get from collection
My mongoose collection's sample record (product field array value is stored as string here)
_id:ObjectId("5d2f8835d5027d4a5f9b535a")
product:"[{"title":"COLOR","attribute":[{"label":"Red","isDefaul..."}{"label":"black","isDefaul..."}]}]"
_id:ObjectId("5d2f8835d5027d4a5sdf7654")
product:"[{"title":"RAM","attribute":[{"label":"8GB","isDefaul..."}]}]"
Expect result
The "title" key value might be dynamic one like "color, size, weight, height, ram, storage, etc,."
{
"autoFilter": [
{
"title": "Color",
"attribute": [
{
"label": "Gold"
},
{
"label": "rose"
},
{
"label": "black"
},
{
"label": "blue"
}
]
},
{
"title": "RAM",
"attribute": [
{
"label": "4GB"
},
{
"label": "3GB"
},
{
"label": "8GB"
}
]
},
{
"title": "Stroage",
"attribute": [
{
"label": "32GB"
},
{
"label": "64GB"
}
]
},
{
"title": "size",
"attribute": [
{
"label": "5inch"
},
{
"label": "6inch"
}
]
}
]
}
I have a single document which has user generated tags and also entries which has an array of tag IDs for each entry (or possibly none):
// Doc (with redacted items I would like to project too)
{
"_id": ObjectId("5ae5afc93e1d0d2965a4f2d7"),
"entries" : [
{
"_id" : ObjectId("5b159ebb0ed51064925dff24"),
// Desired:
// tags: {[
// "_id" : ObjectId("5b142ab7e419614016b8992d"),
// "name" : "Shit",
// "color" : "#95a5a6"
// ]}
"tags" : [
ObjectId("5b142ab7e419614016b8992d")
]
},
],
"tags" : [
{
"_id" : ObjectId("5b142608e419614016b89925"),
"name" : "Outdated",
"color" : "#3498db"
},
{
"_id" : ObjectId("5b142ab7e419614016b8992d"),
"name" : "Shit",
"color" : "#95a5a6"
},
],
}
How can I "fill up" the tag array for each entry with the corresponding value in the tags array? I tried $lookup and aggregate but it was too complicated to get right.
From the looks of your actual data, there is no need to populate() or $lookup here since the data you want to "join" is not only in the same collection but it's actually in the same document. What you want here instead is $map or even Array.map() to simply take values in one array of the document and merge them into the other.
Aggregate $map transform
The basic case of what you need to do here is $map to transform the each array in the output. These are "entries" and within each "entry" transforming the "tags" by matching values to those within the "tags" array of the parent document:
Project.aggregate([
{ "$project": {
"entries": {
"$map": {
"input": "$entries",
"as": "e",
"in": {
"someField": "$$e.someField",
"otherField": "$$e.otherField",
"tags": {
"$map": {
"input": "$$e.tags",
"as": "t",
"in": {
"$arrayElemAt": [
"$tags",
{ "$indexOfArray": [ "$tags._id", "$$t" ] }
]
}
}
}
}
}
}
}}
])
Note there the "someField" and "otherField" as placeholders for fields which "might" be present at that level within each "entry" document of the array. The only catch with $map is that what is specified within the "in" argument is the only output you actually get, so there is a need to explicitly name every single potential field that would be in your "variable keys" structure, and including the "tags".
The counter to this in modern releases since MongoDB 3.6 is to use $mergeObjects instead which allows a "merge" of the "re-mapped" inner array of "tags" into the "entry" document of each array member:
Project.aggregate([
{ "$project": {
"entries": {
"$map": {
"input": "$entries",
"as": "e",
"in": {
"$mergeObjects": [
"$$e",
{ "tags": {
"$map": {
"input": "$$e.tags",
"as": "t",
"in": {
"$arrayElemAt": [
"$tags",
{ "$indexOfArray": [ "$tags._id", "$$t" ] }
]
}
}
}}
]
}
}
}
}}
])
As for the actual $map on the "inner" array of "tags", here you can use the $indexOfArray operator to do a comparison with the "root level" field of "tags" based on where the _id property matches the value of the current entry of this "inner" array. With that "index" returned, the $arrayElemAt operator then "extracts" the actual array entry from that matched "index" position, and transplants the current array entry in the $map with that element.
The only point of care here is in the case where the two arrays in fact do not have matching entries for some reason. If you have already taken care of this, then the code here is fine. If there is a mismatch you might instead need to $filter to match the elements and take the $arrayElemAt at index 0 instead:
"in": {
"$arrayElemAt": [
{ "$filter": {
"input": "$tags",
"cond": { "$eq": [ "$$this._id", "$$t" ] }
}},
0
]
}
The reason being that doing that allows a null where there is no match, but $indexOfArray will return -1, and that used with $arrayElemAt returns the "last" array element. And the "last" element is of course in that scenario not the "matching" result, since there was no match.
Client side transformation
So from the perspective there where you are "only" returning the "entries" content "re-mapped" and discarding the "tags" from the root of the document, the aggregation process where possible is the better option since the server only returns the elements you actually want.
If you cannot do that or otherwise really don't care if the existing "tags" element is also returned, then aggregation transformation is really not necessary here at all. In fact the "server" need not do anything, and probably "should not" considering all the data is already in the document and "additional" transforms is just adding to the document size.
So this is all actually possible to do with the result once returned to the client, and for a simple transformation of the document just the same as was demonstrated with the above aggregation pipeline examples the only code you actually need is:
let results = await Project.find().lean();
results = results.map(({ entries, tags, ...r }) =>
({
...r,
entries: entries.map(({ tags: etags, ...e }) =>
({
...e,
tags: etags.map( tid => tags.find(t => t._id.equals(tid)) )
})
),
// tags
})
);
This gives you exactly the same results and even optionally keep the tags in there by removing the comment. It's even basically "exactly the same process" of using Array.map() on each array in order to do the transformation of each one.
The syntax to "merge" is much more simple with modern JavaScript object spread operations, and overall the language is far less terse. You use Array.find() in order to "lookup" the matching content of the two arrays for tags and the only other thing to be aware of is the ObjectId.equals() method, which is needed to actually compare these two values and built in to the returned types anyway.
Of course since you are "transforming" the documents, in order to make this possible you use lean() on any mongoose operation returning the results to manipulate so the data returned is in fact plain JavaScript objects rather than Mongoose Document types bound to the schema, which is the default return.
Conclusion and Demonstration
The general lesson here is that if you are looking to "reduce data" in the returned response, then the aggregate() method is for you. If however you decide that you want the "whole" document data anyway and just want to "augment" these other array entries in the response, then just take the data back to the "client" and transform it there instead. Ideally as "frontward" as possible considering that "additions" are just adding weight to the payload response in this case.
A full demonstration listing would be:
const { Schema, Types: { ObjectId } } = mongoose = require('mongoose');
const uri = 'mongodb://localhost/test';
mongoose.Promise = global.Promise;
mongoose.set('debug', true);
const tagSchema = new Schema({
name: String,
color: String
});
const projectSchema = new Schema({
entries: [],
tags: [tagSchema]
});
const Project = mongoose.model('Project', projectSchema);
const log = data => console.log(JSON.stringify(data, undefined, 2));
(async function() {
try {
const conn = await mongoose.connect(uri);
let db = conn.connections[0].db;
let { version } = await db.command({ buildInfo: 1 });
version = parseFloat(version.match(new RegExp(/(?:(?!-).)*/))[0]);
await Promise.all(Object.entries(conn.models).map(([k,m]) => m.remove()));
await Project.insertMany(data);
let pipeline = [
{ "$project": {
"entries": {
"$map": {
"input": "$entries",
"as": "e",
"in": {
"someField": "$$e.someField",
"otherField": "$$e.otherField",
"tags": {
"$map": {
"input": "$$e.tags",
"as": "t",
"in": {
"$arrayElemAt": [
"$tags",
{ "$indexOfArray": [ "$tags._id", "$$t" ] }
]
}
}
}
}
}
}
}}
];
let other = [
{
...(({ $project: { entries: { $map: { input, as, ...o } } } }) =>
({
$project: {
entries: {
$map: {
input,
as,
in: {
"$mergeObjects": [ "$$e", { tags: o.in.tags } ]
}
}
}
}
})
)(pipeline[0])
}
];
let tests = [
{ name: 'Standard $project $map', pipeline },
...(version >= 3.6) ?
[{ name: 'With $mergeObjects', pipeline: other }] : []
];
for ( let { name, pipeline } of tests ) {
let results = await Project.aggregate(pipeline);
log({ name, results });
}
// Client Manipulation
let results = await Project.find().lean();
results = results.map(({ entries, tags, ...r }) =>
({
...r,
entries: entries.map(({ tags: etags, ...e }) =>
({
...e,
tags: etags.map( tid => tags.find(t => t._id.equals(tid)) )
})
)
})
);
log({ name: 'Client re-map', results });
mongoose.disconnect();
} catch(e) {
console.error(e)
} finally {
process.exit()
}
})();
// Data
const data =[
{
"_id": ObjectId("5ae5afc93e1d0d2965a4f2d7"),
"entries" : [
{
"_id" : ObjectId("5b159ebb0ed51064925dff24"),
"someField": "someData",
"tags" : [
ObjectId("5b142ab7e419614016b8992d")
]
},
],
"tags" : [
{
"_id" : ObjectId("5b142608e419614016b89925"),
"name" : "Outdated",
"color" : "#3498db"
},
{
"_id" : ObjectId("5b142ab7e419614016b8992d"),
"name" : "Shitake",
"color" : "#95a5a6"
},
]
},
{
"_id": ObjectId("5b1b1ad07325c4c541e8a972"),
"entries" : [
{
"_id" : ObjectId("5b1b1b267325c4c541e8a973"),
"otherField": "otherData",
"tags" : [
ObjectId("5b142608e419614016b89925"),
ObjectId("5b142ab7e419614016b8992d")
]
},
],
"tags" : [
{
"_id" : ObjectId("5b142608e419614016b89925"),
"name" : "Outdated",
"color" : "#3498db"
},
{
"_id" : ObjectId("5b142ab7e419614016b8992d"),
"name" : "Shitake",
"color" : "#95a5a6"
},
]
}
];
And this would give full output ( with the optional output from a supporting MongoDB 3.6 instance ) as:
Mongoose: projects.remove({}, {})
Mongoose: projects.insertMany([ { entries: [ { _id: 5b159ebb0ed51064925dff24, someField: 'someData', tags: [ 5b142ab7e419614016b8992d ] } ], _id: 5ae5afc93e1d0d2965a4f2d7, tags: [ { _id: 5b142608e419614016b89925, name: 'Outdated', color: '#3498db' }, { _id: 5b142ab7e419614016b8992d, name: 'Shitake', color: '#95a5a6' } ], __v: 0 }, { entries: [ { _id: 5b1b1b267325c4c541e8a973, otherField: 'otherData', tags: [ 5b142608e419614016b89925, 5b142ab7e419614016b8992d ] } ], _id: 5b1b1ad07325c4c541e8a972, tags: [ { _id: 5b142608e419614016b89925, name: 'Outdated', color: '#3498db' }, { _id: 5b142ab7e419614016b8992d, name: 'Shitake', color: '#95a5a6' } ], __v: 0 } ], {})
Mongoose: projects.aggregate([ { '$project': { entries: { '$map': { input: '$entries', as: 'e', in: { someField: '$$e.someField', otherField: '$$e.otherField', tags: { '$map': { input: '$$e.tags', as: 't', in: { '$arrayElemAt': [ '$tags', { '$indexOfArray': [Array] } ] } } } } } } } } ], {})
{
"name": "Standard $project $map",
"results": [
{
"_id": "5ae5afc93e1d0d2965a4f2d7",
"entries": [
{
"someField": "someData",
"tags": [
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
},
{
"_id": "5b1b1ad07325c4c541e8a972",
"entries": [
{
"otherField": "otherData",
"tags": [
{
"_id": "5b142608e419614016b89925",
"name": "Outdated",
"color": "#3498db"
},
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
}
]
}
Mongoose: projects.aggregate([ { '$project': { entries: { '$map': { input: '$entries', as: 'e', in: { '$mergeObjects': [ '$$e', { tags: { '$map': { input: '$$e.tags', as: 't', in: { '$arrayElemAt': [Array] } } } } ] } } } } } ], {})
{
"name": "With $mergeObjects",
"results": [
{
"_id": "5ae5afc93e1d0d2965a4f2d7",
"entries": [
{
"_id": "5b159ebb0ed51064925dff24",
"someField": "someData",
"tags": [
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
},
{
"_id": "5b1b1ad07325c4c541e8a972",
"entries": [
{
"_id": "5b1b1b267325c4c541e8a973",
"otherField": "otherData",
"tags": [
{
"_id": "5b142608e419614016b89925",
"name": "Outdated",
"color": "#3498db"
},
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
}
]
}
Mongoose: projects.find({}, { fields: {} })
{
"name": "Client re-map",
"results": [
{
"_id": "5ae5afc93e1d0d2965a4f2d7",
"__v": 0,
"entries": [
{
"_id": "5b159ebb0ed51064925dff24",
"someField": "someData",
"tags": [
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
},
{
"_id": "5b1b1ad07325c4c541e8a972",
"__v": 0,
"entries": [
{
"_id": "5b1b1b267325c4c541e8a973",
"otherField": "otherData",
"tags": [
{
"_id": "5b142608e419614016b89925",
"name": "Outdated",
"color": "#3498db"
},
{
"_id": "5b142ab7e419614016b8992d",
"name": "Shitake",
"color": "#95a5a6"
}
]
}
]
}
]
}
Note this includes some additional data to demonstrate the projection of "variable fields".