I'm trying to figure out what the most efficient way of doing this, preferably in one query. If I have an array...
[
{
name: "Eric",
priority: 1
{
[
If I append another object using $addToSet, how can I increment the counter? For example, if I add John...
[
{
name: "Eric",
priority: 1
},
{
name: "John",
priority: 2
{
]
I was thinking of something like this...
let room = await Room.findOneAndUpdate(
{ room_id: data.room_id },
{
$addToSet: {
guest_list: {
name: data.name
}
},
"guest_list.$.priority": guest_list.length
},
{ new: true }
);
I know this doesn't work, but it would be nice if I can get the length of the array and use that in the priority attribute as so...
"guest_list.$.priority": guest_list.length
What is the best way of doing this?
Related
I have recently shifted to MongoDB and Mongoose with Node.js. And I am wrapping my head around it all coming from SQL.
I have a collection where documents have a similar structure to the following:
{
name: String
rank: Number
}
Sometimes the name might be the same, but the rank will always be different.
I would like to remove all duplicates of name, but retain the object that has the LOWEST rank.
For instance, if my collection looked like this:
{
name: "name1"
rank: 3
},
{
name: "name1"
rank: 4
},
{
name: "name1"
rank: 2
}
I would like to remove all objects where name is the same except for:
{
name: "name1"
rank: 2
}
Is this possible to do with mongoose?
Here is my approach:
const found = await db.collection.aggregate([
{
$group: {
_id: "$name",
minRank: {
$min: "$rank"
}
}
},
])
await db.collection.deleteMany({
$or: found.map(item => ({
name: item._id,
rank: { $ne: item.minRank }
}))
})
Explanation:
From my point of view your solution would result in many unnecessary calls being made, which would result in a terrible time of execution. My solution exactly contains two steps:
find for each document's property name the corresponding lowest rank available.
delete each document, where the name is equal to one of those names and the rank is not equal to the actual lowest rank found.
Additional notes:
If not already done, you should probably define an index on the name property of your schema for performance reasons.
Okay, I figured it out using aggregate:
const duplicates = await collectionName.aggregate([
{
$group: {
_id: "$name",
dups: { $addToSet: "$_id" },
count: { $sum: 1 }
}
},
{
$match: {
count: { $gt: 1 }
}
}
]);
duplicates.forEach(async (item) => {
const duplicate_names = item.dups;
const duplicate_name = await collectionName.find({ _id: { $in: duplicate_names } }).sort({ rank: 1 });
duplicate_name.shift();
duplicate_name.forEach(async (item) => {
await collectionName.deleteOne({ _id: item._id });
});
});
Its my first time trying prisma and am stuck. So I have "products" and "filters" model.
I want the following query to work. The idea is, I want to fetch the products with dynamic matching query params (name and value). The product query parameters come dynamically from the frontend.
const products = await prisma.product.findMany({
where: {
categoryName,
subCategoryName,
filters: {
some: {
AND: [
{
name: "RAM",
value: "32GB",
},
{
name: "Storage",
value: "1TB",
},
],
},
},
},
include: {
images: true,
},
});
If there's only one parameter, like
{
name:"RAM",
value:"32GB"
}
the query returns appropriate products, but if there are more that one query params (like in the original code above), it returns empty array.
my product schema looks like this, simplified,
name String
filters Filter[]
my filter schema looks like this, simplified
name String
value String?
product Product? #relation(fields: [productId], references:[id])
productId Int?
Thank you very much
I've found the solution here
https://github.com/prisma/prisma/discussions/8216#discussioncomment-992302
It should be like this instead apparently.
await prisma.product.findMany({
where: {
AND: [
{ price: 21.99 },
{ filters: { some: { name: 'ram', value: '8GB' } } },
{ filters: { some: { name: 'storage', value: '256GB' } } },
],
},
})
I am trying to perform a sum in my search query, on (number) nested fields (array of object)
My data represent a list of object, and for every object I have a nested list of start/end timestamp that represent every visits
my data looks like that :
[
{
title: '',
...,
visits: //nested
[
{
start: 1617700260000,
end: 1617700280000
},
{
start: 1617700260001,
end: 1617700280001
}, ...
]
},
}
]
If I use a aggregate the query works but the aggregate is done on the total and not on every document.
I would like to have a new field that contains the subtract of every visists.end - visits.start to have the total amount
so I tried :
await client.search({
index: 'myIndex',
body: {
script_fields: {
total_time: {
script: {
source: "doc['visits']['end'].value - doc['visits']['start'].value",
},
},
},
},
});
But I continue to have errors and I can't manage to do it,
ResponseError: search_phase_execution_exception
without more information
I imagine that the script must handle the fact that the nested object is an array, but I can't manage to go further that this error
IF I replace the source of the script by just returning a flat value, it's ok
Do you have any idea ?
I managed to make it work like that :
await client.search({
index: 'myIndex',
body: {
_source: true, // to see the srouce object in the response
script_fields: {
total_time: {
script: {
lang: 'painless',
source:
"def total = 0; for (def i = 0; i < params['_source']['visits'].size(); i++) { total += params['_source']['visits'][i].end - params['_source']['visits'][i].start; } return total;",
},
},
},
},
});
I have the following query:
Documents.find({
$and: [
{
user_id: {$nin:
myUserId
}
},
{ date: { $gte: dateMax, $lt: dateMin } },
{documentTags: {$all: tags}}
],
})
What I'm trying to do is make the documentTags portion of the query optional. I have tried building the query as follows:
let tags = " ";
if (req.body.tags) {
tags = {videoTags: {$all: req.body.tags}};
}
let query = {
$and: [
{
user_id: {$nin:
myUserId
}
},
{ date: { $gte: dateMax, $lt: dateMin } },
tags
],
}
and then Document.find(query). The problem is no matter how I modify tags (whether undefined, as whitespace, or otherwise) I get various errors like $or/$and/$nor entries need to be full objects and TypeError: Cannot read property 'hasOwnProperty' of undefined.
Is there a way to build an optional requirement into the query?
I tried the option below and the query is just returning everything that matches the other fields. For some reason it isn't filtering by tags. I did a console.log(queryArr) and console.log(query) get the following respectively:
[
{ user_id: { '$nin': [Array] } },
{
date: {
'$gte': 1985-01-01T00:00:00.000Z,
'$lt': 2020-01-01T00:00:00.000Z
}
},
push: { documentTags: { '$all': [Array] } }
]
console.log(query)
{
'$and': [
{ user_id: [Object] },
{ date: [Object] },
push: { documentTags: [Object] }
]
}
You are almost there. Instead you could construct the object outside the query and just put the constructed query in $and when done..
let queryArr = [
{
user_id: {$nin: myUserId}
},
{ date: { $gte: dateMax, $lt: dateMin } }
];
if (req.body.tags) {
queryArr.push({videoTags: {$all: req.body.tags}});
}
let query = {
$and: queryArr
}
Now you can control the query by just pushing object into the query Array itself.
I figured out why it wasn't working. Basically, when you do myVar.push it creates a key-value pair such as [1,2,3,push:value]. This would work if you needed to append a k-v pair in that format, but you'll have difficulty using it in a query like mine. The right way for me turned out to be to use concact which appends the array with just the value that you set, rather than a k-v pair.
if (req.body.tags){
queryArgs = queryArgs.concat({documentTags: {$all: tags}});
}
let query = {
$and: queryArgs
}
I have documents that looks something like that, with a unique index on bars.name:
{ name: 'foo', bars: [ { name: 'qux', somefield: 1 } ] }
. I want to either update the sub-document where { name: 'foo', 'bars.name': 'qux' } and $set: { 'bars.$.somefield': 2 }, or create a new sub-document with { name: 'qux', somefield: 2 } under { name: 'foo' }.
Is it possible to do this using a single query with upsert, or will I have to issue two separate ones?
Related: 'upsert' in an embedded document (suggests to change the schema to have the sub-document identifier as the key, but this is from two years ago and I'm wondering if there are better solutions now.)
No there isn't really a better solution to this, so perhaps with an explanation.
Suppose you have a document in place that has the structure as you show:
{
"name": "foo",
"bars": [{
"name": "qux",
"somefield": 1
}]
}
If you do an update like this
db.foo.update(
{ "name": "foo", "bars.name": "qux" },
{ "$set": { "bars.$.somefield": 2 } },
{ "upsert": true }
)
Then all is fine because matching document was found. But if you change the value of "bars.name":
db.foo.update(
{ "name": "foo", "bars.name": "xyz" },
{ "$set": { "bars.$.somefield": 2 } },
{ "upsert": true }
)
Then you will get a failure. The only thing that has really changed here is that in MongoDB 2.6 and above the error is a little more succinct:
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 16836,
"errmsg" : "The positional operator did not find the match needed from the query. Unexpanded update: bars.$.somefield"
}
})
That is better in some ways, but you really do not want to "upsert" anyway. What you want to do is add the element to the array where the "name" does not currently exist.
So what you really want is the "result" from the update attempt without the "upsert" flag to see if any documents were affected:
db.foo.update(
{ "name": "foo", "bars.name": "xyz" },
{ "$set": { "bars.$.somefield": 2 } }
)
Yielding in response:
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })
So when the modified documents are 0 then you know you want to issue the following update:
db.foo.update(
{ "name": "foo" },
{ "$push": { "bars": {
"name": "xyz",
"somefield": 2
}}
)
There really is no other way to do exactly what you want. As the additions to the array are not strictly a "set" type of operation, you cannot use $addToSet combined with the "bulk update" functionality there, so that you can "cascade" your update requests.
In this case it seems like you need to check the result, or otherwise accept reading the whole document and checking whether to update or insert a new array element in code.
if you dont mind changing the schema a bit and having a structure like so:
{ "name": "foo", "bars": { "qux": { "somefield": 1 },
"xyz": { "somefield": 2 },
}
}
You can perform your operations in one go.
Reiterating 'upsert' in an embedded document for completeness
I was digging for the same feature, and found that in version 4.2 or above, MongoDB provides a new feature called Update with aggregation pipeline.
This feature, if used with some other techniques, makes possible to achieve an upsert subdocument operation with a single query.
It's a very verbose query, but I believe if you know that you won't have too many records on the subCollection, it's viable. Here's an example on how to achieve this:
const documentQuery = { _id: '123' }
const subDocumentToUpsert = { name: 'xyz', id: '1' }
collection.update(documentQuery, [
{
$set: {
sub_documents: {
$cond: {
if: { $not: ['$sub_documents'] },
then: [subDocumentToUpsert],
else: {
$cond: {
if: { $in: [subDocumentToUpsert.id, '$sub_documents.id'] },
then: {
$map: {
input: '$sub_documents',
as: 'sub_document',
in: {
$cond: {
if: { $eq: ['$$sub_document.id', subDocumentToUpsert.id] },
then: subDocumentToUpsert,
else: '$$sub_document',
},
},
},
},
else: { $concatArrays: ['$sub_documents', [subDocumentToUpsert]] },
},
},
},
},
},
},
])
There's a way to do it in two queries - but it will still work in a bulkWrite.
This is relevant because in my case not being able to batch it is the biggest hangup. With this solution, you don't need to collect the result of the first query, which allows you to do bulk operations if you need to.
Here are the two successive queries to run for your example:
// Update subdocument if existing
collection.updateMany({
name: 'foo', 'bars.name': 'qux'
}, {
$set: {
'bars.$.somefield': 2
}
})
// Insert subdocument otherwise
collection.updateMany({
name: 'foo', $not: {'bars.name': 'qux' }
}, {
$push: {
bars: {
somefield: 2, name: 'qux'
}
}
})
This also has the added benefit of not having corrupted data / race conditions if multiple applications are writing to the database concurrently. You won't risk ending up with two bars: {somefield: 2, name: 'qux'} subdocuments in your document if two applications run the same queries at the same time.