Find duplicate inside array without $unwind - node.js

Question

I have below users collection
[{
"_id": 1,
"adds": ["111", "222", "333", "111"]
}, {
"_id": 2,
"adds": ["555", "666", "777", "555"]
}, {
"_id": 3,
"adds": ["888", "999", "000", "888"]
}]
I need to find the duplicates inside the adds array
The expected output should be
[{
"_id": 1,
"adds": ["111"]
}, {
"_id": 2,
"adds": [ "555"]
}, {
"_id": 3,
"adds": ["888"]
}]
I have tried using many operators $setUnion, $setDifference but none of the did the trick.
Please help!!!

Answer 1

You can use $range to generate arrays of numbers from 1 to n where n is the $size of adds. Then you can "loop" through that numbers and check if adds at index ($arrayElemAt) exists somewhere before index if yes then it should be considered as a duplicate. You can use $indexOfArray to check if element exists in array specifying 0 and index as search range.
Then you just need to use $project and $map to replace indexes with actual elements. You can also add $setUnion to avoid duplicated duplicates in final result set.
db.users.aggregate([
{
$addFields: {
duplicates: {
$filter: {
input: { $range: [ 1, { $size: "$adds" } ] },
as: "index",
cond: {
$ne: [ { $indexOfArray: [ "$adds", { $arrayElemAt: [ "$adds", "$$index" ] }, 0, "$$index" ] }, -1 ]
}
}
}
}
},
{
$project: {
_id: 1,
adds: {
$setUnion: [ { $map: { input: "$duplicates", as: "d", in: { $arrayElemAt: [ "$adds", "$$d" ] } } }, [] ]
}
}
}
])
Prints:
{ "_id" : 1, "adds" : [ "111" ] }
{ "_id" : 2, "adds" : [ "555" ] }
{ "_id" : 3, "adds" : [ "888" ] }

Answer 2

Here is another version that you might want to compare in terms of performance:
db.users.aggregate({
$project:{
"adds":{
$reduce:{
"input":{$range:[0,{$size:"$adds"}]}, // loop variable from 0 to max. index of $adds array
//"input":{$range:[0,{$subtract:[{$size:"$adds"},1]}]}, // this would be enough but looks more complicated
"initialValue":[],
"in":{
$let:{
"vars":{
"curr": { $arrayElemAt: [ "$adds", "$$this"] } // the element we're looking at
},
"in":{
// if there is another identical element after the current one then we have a duplicate
$cond:[
{$ne:[{$indexOfArray:["$adds","$$curr",{$add:["$$this",1]}]},-1]},
{$setUnion:["$$value",["$$curr"]]}, // combine duplicates found so far with new duplicate
"$$value" // continue with current value
]
}
}
}
}
}
}
})
The logic is based on a loop variable which we get through the $range operator. This loop variable allows for sequential access of the adds array. For every item that we look at, we check if there is another identical one after the current index. If yes, we have a duplicate, otherwise not.

Answer 3

You can try below aggregation. The idea is to collect the distinct values and iterate over values and check if the value is present in adds array; if present keep the value else ignore the value.
db.users.aggregate({
"$project":{
"adds":{
"$reduce":{
"input":{"$setUnion":["$adds",[]]},
"initialValue":[],
"in":{
"$concatArrays":[
"$$value",
{"$let":{
"vars":{
"match":{
"$filter":{"input":"$adds","as":"a","cond":{"$eq":["$$a","$$this"]}}
}},
"in":{
"$cond":[{"$gt":[{"$size":"$$match"},1]},["$$this"],[]]
}
}}
]
}
}
}
}
})