I tried something like below but dint get the expected result , This should work if the fields are not an array
// union of includes and excludes as excludesAndExcludes
getIncludesAndExcludes: (req, res)=>{
console.log('called setunion');
experienceModel.aggregate([
{ $group: {_id : {includes:"$includes", excludes:"$excludes"}}},
{ $project: { includesAndExcludes: { $setUnion: [ "$_id.includes", "$_id.excludes" ] }, _id:0 } }
], (err, data) => {
if (err) {
res.status(500).send(error);
} else {
res.json(data);
}
})
},
Most efficiently exclude the document if neither array has any element via $exists and most importantly you are going to need $ifNull to replace with a null where arrays don't exist.
experienceModel.aggregate([
// Don't include documents that have no arrays
{ "$match": {
"$or": [
{ "includes.0": { "$exists": true } },
{ "excludes.0": { "$exists": true } }
]
},
// Join the arrays and exclude the nulls
{ "$project": {
"_id": 0,
"list": {
"$setDifference": [
{ "$setUnion": [
{ "$ifNull": [ "$includes", [null] ] },
{ "$ifNull": [ "$excludes", [null] ] }
]},
[null]
]
}},
// Unwind. By earlier conditions the array must have some entries
{ "$unwind": "$list" },
// And $group on the list values as the key to produce distinct results
{ "$group": { "_id": "$list" }
],(err, data) => {
// rest of code
})
So the first $match is to filter so that at least one array must be present as a logic rule, which also possibly speeds things up. Next you join the arrays with $setUnion, being careful to replace with an array of a single element [null] using $ifNull if the field was not present. If you did not do this then any $setUnion result would be null rather than listing entries of either array. So that is quite important.
Since it is possible for the output of $setUnion to have a null item in it's list, you can remove that with $setDifference, which is the shortest form of filter you can write and works well with "sets".
All that really remains is to "de-normalize" the array in each document to single documents of each element using [$unwind][6], and we don't need new options like preserveNullAndEmptyArrays because all the logic above already took car of that. And then the final $group is simply done on those values in order to produce "unique" output, which is what the _id key in a $group statement is for.
If you want, then you can even just strip down the result from aggregate to a simple list of strings for your response using .map():
data = data.map( d => d._id );
Then all your service returns is an array of strings, and no embedded structures.
[
"Breakfast"
"Airport Pickup"
"Dinner"
"Accomodation"
]
Related
I have this document structure in the collection:
{"_id":"890138075223711744",
"guildID":"854557773990854707",
"name":"test-lab",
"game": {
"usedWords":["akşam","elma","akım"]
}
}
What is the most efficient way to get its fields except the array (it can be really large), and at the same time, see if an item exists in the array ?
I tried this:
let query = {_id: channelID}
const options = { sort: { name: 1 }, projection: { name: 1, "game.usedWords": { $elemMatch: { word}}}}
mongoClient.db(db).collection("channels").findOne(query, options);
but I got the error: "$elemMatch can not be used on nested fields"
If I've understood correctly you can use this query:
Using positional operator $ you can return only the matched word.
db.collection.find({
"game.usedWords": "akşam"
},
{
"name": 1,
"game.usedWords.$": 1
})
Example here
The output is only name and the matched word (also _id which is returned by default)
[
{
"_id": "890138075223711744",
"game": {
"usedWords": [
"akşam"
]
},
"name": "test-lab"
}
]
provided I have following documents
User
{
uuid: string,
isActive: boolean,
lastLogin: datetime,
createdOn: datetime
}
Projects
{
id: string,
users: [
{
uuid: string,
otherInfo: ...
},
{... more users}
]
}
And I want to select all users that didn't login since 2 weeks and are inactive or since 5 weeks that don't have projects.
Now, the 2 weeks is working fine but I cannot seem to figure out how to do the "5 weeks and don't have projects" part
I came up with something like below but the last part does not work because $exists obviously is not a top level operator.
Anyone ever did anything like this?
Thanks!
return await this.collection
.aggregate([
{
$match: {
$and: [
{
$expr: {
$allElementsTrue: {
$map: {
input: [`$lastLogin`, `$createdOn`],
in: { $lt: [`$$this`, twoWeeksAgo] }
}
}
}
},
{
$or: [
{
isActive: false
},
{
$and: [
{
$expr: {
$allElementsTrue: {
$map: {
input: [`$lastLogin`, `$createdOn`],
in: { $lt: [`$$this`, fiveWeeksAgo] }
}
}
}
},
{
//No projects exists on this user
$exists: {
$lookup: {
from: _.get(Config, `env.collection.projects`),
let: {
currentUser: `$$ROOT`
},
pipeline: [
{
$project: {
_id: 0,
users: {
$filter: {
input: `$users`,
as: `user`,
cond: {
$eq: [`$$user.uuid`, `$currentUser.uuid`]
}
}
}
}
}
]
}
}
}
]
}
]
}
]
}
}
])
.toArray();
Not certain why you thought $expr was needed in the initial $match, but really:
const getResults = () => {
const now = Date.now();
const twoWeeksAgo = new Date(now - (1000 * 60 * 60 * 24 * 7 * 2 ));
const fiveWeeksAgo = new Date(now - (1000 * 60 * 60 * 24 * 7 * 5 ));
// as long a mongoDriverCollectionReference points to a "Collection" object
// for the "users" collection
return mongoDriverCollectionReference.aggregate([
// No $expr, since you can actually use an index. $expr cannot do that
{ "$match": {
"$or": [
// Active and "logged in"/created in the last 2 weeks
{
"isActive": true,
"$or": [
{ "lastLogin": { "$gte": twoWeeksAgo } },
{ "createdOn": { "$gte": twoWeeksAgo } }
]
},
// Also want those who...
// Not Active and "logged in"/created in the last 5 weeks
// we'll "tag" them later
{
"isActive": false,
"$or": [
{ "lastLogin": { "$gte": fiveWeeksAgo } },
{ "createdOn": { "$gte": fiveWeeksAgo } }
]
}
]
}},
// Now we do the "conditional" stuff, just to return a matching result or not
{ "$lookup": {
"from": _.get(Config, `env.collection.projects`), // there are a lot cleaner ways to register models than this
"let": {
"uuid": {
"$cond": {
"if": "$isActive", // this is boolean afterall
"then": null, // don't really want to match
"else": "$uuid" // Okay to match the 5 week results
}
}
},
"pipeline": [
// Nothing complex here as null will return nothing. Just do $in for the array
{ "$match": { "$in": [ "$$uuid", "$users.uuid" ] } },
// Don't really need the detail, so just reduce any matches to one result of [null]
{ "$group": { "_id": null } }
],
"as": "projects"
}},
// Now test if the $lookup returned something where it mattered
{ "$match": {
"$or": [
{ "active": true }, // remember we selected the active ones already
{
"projects.0": { "$exists": false } // So now we only need to know the "inactive" returned no array result.
}
]
}}
]).toArray(); // returns a Promise
};
It's pretty simple as calculated expressions via $expr are actually really bad and not what you want in a first pipeline stage. Also "not what you need" since createdOn and lastLogin really should not have been merged into an array for $allElementsTrue which would just be an AND condition, where you described logic would really mean OR. So the $or does just fine here.
So does the $or on the separation of conditions for the isActive of true/false. Again it's either "two weeks" OR "five weeks". And this certainly does not need $expr since standard inequality range matching works fine, and uses an "index".
Then you really just want to do the "conditional" things in the let for $lookup instead of your "does it exist" thinking. All you really need to know ( since the range selection of dates is actually already done ) is whether active is now true or false. Where it's active ( meaning by your logic you don't care about projects ) simply make the $$uuid used within the $match pipeline stage a null value so it will not match and the $lookup returns an empty array. Where false ( also already matching the date conditions from earlier ) then you use the actual value and "join" ( where there are projects of course ).
Then it's just a simple matter of keeping the active users, and then only testing the remaining false values for active to see if the "projects" array from the $lookup actually returned anything. If it did not, then they just don't have projects.
Probably should note here is since users is an "array" within the projects collection, you use $in for the $match condition against the single value to the array.
Note that for brevity we can use $group inside the inner pipeline to only return one result instead of possibly many matches to actual matched projects. You don't care about the content or the "count", but simply if one was returned or nothing. Again following the presented logic.
This gets you your desired results, and it does so in a manner that is efficient and actually uses indexes where available.
Also return await certainly does not do what you think it does, and in fact it's an ESLint warning message ( I suggest you enable ESLint in your project ) since it's not a smart thing to do. It does nothing really, as you would need to await getResults() ( as per the example naming ) anyway, as the await keyword is not "magic" but just a prettier way of writing then(). As well as hopefully being easier to understand, once you understand what async/await is really for syntactically that is.
I have got some troubles with a MongoDB request that I would like to execute. I am using MongoDB 3.2 with Mongoose in a node.js context. Here is the document:
{
_id: ObjectId('12345'),
name: "The name",
components: [
{
name: "Component 1",
description: "The description",
container: {
parameters: [
{
name: "Parameter 1",
value: "1"
},
// other parameters
],
// other information...
}
},
// other components
]
}
And I would like to list all the parameters name for a specific component (using component name) in the specific document (using _id) with this output:
['Parameter 1', 'Parameter 2', ...]
I have got a mongoose Schema to handle the find or distinct methods in my application. I tried many operations using $ positioning operator. Here is my attempt but return all parameters form all components:
listParameters(req, res) {
DocumentModel.distinct(
"components.container.parameters.name",
{_id: req.params.modelId, "components.name": req.params.compId},
(err, doc) => {
if (err) {
return res.status(500).send(err);
}
res.status(200).json(doc);
}
);
}
but the output is the list of parameter name but without the filter of the specific component. Can you help me to find the right request? (if possible in mongoose JS but if it is a Mongo command line, it will be very good :))
You would need to run an aggregation pipeline that uses the $arrayElemAt and $filter operators to get the desired result.
The $filter operator will filter the components array to return the element satisfying the given condition whilst the $arrayElemAt returns the document from the array at a given index position. With that document you can then project the nested parameters array elements to another array.
Combining the above you ideally want to have the following aggregate operation:
DocumentModel.aggregate([
{ "$match": { "_id": mongoose.Types.ObjectId(req.params.modelId) } },
{ "$project": {
"component": {
"$arrayElemAt": [
{
"$filter": {
"input": "$components",
"as": "el",
"cond": { "$eq": ["$$el.name", req.params.compId] }
}
}, 0
]
}
} },
{ "$project": {
"parameters": "$component.container.parameters.name"
} }
], (err, doc) => {
if (err) {
return res.status(500).send(err);
}
const result = doc.length >= 1 ? doc[0].parameters : [];
res.status(200).json(result);
})
i don`t know how to do it with mongoose but this will work for you with mongo
db.getCollection('your collaction').aggregate([ // change to youe collaction
{$match: {_id: ObjectId("5a97ff4cf832104b76d29af7")}}, //change to you id
{$unwind: '$components'},
{$match: {'components.name': 'Component 1'}}, // change to the name you want
{$unwind: '$components.container.parameters'},
{
$group: {
_id: '$_id',
params: {$push: '$components.container.parameters.name'}
}
}
]);
say I have this array property ('articles') on a Mongoose schema:
articles: [
{
kind: 'bear',
hashtag: 'foo'
},
{
kind: 'llama',
hashtag: 'baz',
},
{
kind: 'sheep',
hashtag: 'bar',
}
]
how can I use
$pull https://docs.mongodb.org/manual/reference/operator/update/pull/
to remote objects from this array by checking the value of hashtag to see if it matches a pattern?
For example, if I want to remove an object in the articles array where hashtag='foo'.
My best guess is the following, but it doesn't seem to work:
var data = {
"$pull": {
"articles": {
"$elemMatch": {
"hashtag": "foo"
}
}
}
};
Model.update({},data); //doesn't quite work
this one seems to work:
var data = {
"$pull": {
"articles": {
"hashtag": 'foo'
}
}
};
Model.update({},data); //seems to work
if you have a better solution or if you can show an alternate solution please provide an answer thank you
The $pull operator is basically a "mini query" in itself, and as such operators like $elemMatch become irrelevant as the "query" is directly performed on each array member anyway.
As such:
Model.update(
{},
{ "$pull": { "articles": { "hashtag": "foo" }},
{ "multi": true },
function(err,numAffected) {
// handle result here
}
)
So it is looking for the matching properties within the ( correct ) specified array in all array documents, and then removing them where there is a match.
The .update() also just returns the number of affected documents, and is usually used with { "multi": true } when you expect more than one document to be updated.
If you are just looking for "one" document, or expect the modified document in response, then use .findOneAndUpdate() instead:
Model.findOneAndUpdate(
{},
{ "$pull": { "articles": { "hashtag": "foo" }},
{ "new": true },
function(err,numAffected) {
// handle result here
}
)
Though not really practical without any selection criteria, since this just updates the first document found in the collection.
how can i combine match document's subdocument together as one and return it as an array of object ? i have tried $group but don't seem to work.
my query ( this return array of object in this case there are two )
User.find({
'business_details.business_location': {
$near: coords,
$maxDistance: maxDistance
},
'deal_details.deals_expired_date': {
$gte: new Date()
}
}, {
'deal_details': 1
}).limit(limit).exec(function(err, locations) {
if (err) {
return res.status(500).json(err)
}
console.log(locations)
the console.log(locations) result
// give me the result below
[{
_id: 55 c0b8c62fd875a93c8ff7ea, // first document
deal_details: [{
deals_location: '101.6833,3.1333',
deals_price: 12.12 // 1st deal
}, {
deals_location: '101.6833,3.1333',
deals_price: 34.3 // 2nd deal
}],
business_details: {}
}, {
_id: 55 a79898e0268bc40e62cd3a, // second document
deal_details: [{
deals_location: '101.6833,3.1333',
deals_price: 12.12 // 3rd deal
}, {
deals_location: '101.6833,3.1333',
deals_price: 34.78 // 4th deal
}, {
deals_location: '101.6833,3.1333',
deals_price: 34.32 // 5th deal
}],
business_details: {}
}]
what i wanted to do is to combine these both deal_details field together and return it as an array of object. It will contain 5 deals in one array of object instead of two separated array of objects.
i have try to do it in my backend (nodejs) by using concat or push, however when there's more than 2 match document i'm having problem to concat them together, is there any way to combine all match documents and return it as one ? like what i mentioned above ?
What you are probably missing here is the $unwind pipeline stage, which is what you typically use to "de-normalize" array content, particularly when your grouping operation intends to work across documents in your query result:
User.aggregate(
[
// Your basic query conditions
{ "$match": {
"business_details.business_location": {
"$near": coords,
"$maxDistance": maxDistance
},
"deal_details.deals_expired_date": {
"$gte": new Date()
}},
// Limit query results here
{ "$limit": limit },
// Unwind the array
{ "$unwind": "$deal_details" },
// Group on the common location
{ "$group": {
"_id": "$deal_details.deals_location",
"prices": {
"$push": "$deal_details.deals_price"
}
}}
],
function(err,results) {
if (err) throw err;
console.log(JSON.stringify(results,undefined,2));
}
);
Which gives output like:
{
"_id": "101.6833,3.1333",
"prices": [
12.12,
34.3,
12.12,
34.78,
34.32
]
}
Depending on how many documents actually match the grouping.
Alternately, you might want to look at the $geoNear pipeline stage, which gives a bit more control, especially when dealing with content in arrays.
Also beware that with "location" data in an array, only the "nearest" result is being considered here and not "all" of the array content. So other items in the array may not be actually "near" the queried point. That is more of a design consideration though as any query operation you do will need to consider this.
You can merge them with reduce:
locations = locations.reduce(function(prev, location){
previous = prev.concat(location.deal_details)
return previous
},[])