I have two tables, one for messages to be logged and one that is for each member of the group. When a message is upvoted, I want to add an entry for the message and push each reaction on that message to an array.
Every time a reaction is added I want to update the member table to reflect the sum of all of the reaction.value fields.
This is the code I have written to do so. When this runs from a sandbox I made in VisualStudio using a MongoDB add on it runs fine, however when ran using my app, only the message document is added and without any error it appears to skip the aggregation to the member document.
Here is the code I am using to insert the documents to the database and to aggregate once the document is inserted:
await mongodb.collection("Messages").updateOne({ _id: reaction.message.id.toString()},
{
$set:{
authorid: reaction.message.author.id,
author: reaction.message.author.username
},
$push: {
reactions: {
reauth: reAuth,
reaction: reaction.emoji.name,
removed: false,
value: actualKarmaDB,
}
}
}, {safe: true, "upsert": true})
await mongodb.collection("Messages").aggregate([
{
$match: {
_id: reaction.message.id
}
},
{
$project: {
_id: "$authorid",
username: "$author",
messageKarma: {$sum: "$reactions.value"},
}
},
{ $merge: {
into: "Members",
on: "_id",
whenMatched: "replace",
whenNotMatched: "insert"
}
}])
Also here is a look at what the insertion into “Messages” looks like:
In this case the answer was due to mongoose not supporting merge for aggregation. Had to use $out.
Additionally this is a known issue with Mongoose Node.js, see here:
Mongodb node.js $out with aggregation only working if calling toArray()
Related
I have mongoose schema that looks something like this:
{
_id: someId,
name: 'mike',
keys: {
apiKey: 'fsddsfdsfdsffds',
secretKey: 'sddfsfdsfdsfdsds'
}
}
I don't want to send back to the front the keys of course, but I want some indication, like:
{
_id: someId,
name: 'mike',
hasKeys: true
}
There is built in way to create 'field' on the way based on other fields, or do I need every time fetch the whole document, check if keys is not empty and set object property based on that?
For Mongo version 4.2+ What you're looking for is called pipelined updates, it let's you use a (restricted) aggregate pipeline as your update allowing the usage of existing field values.
Here is a toy example with your data:
db.collection.updateOne(
{ _id: someId },
[
{
"$set": {
"hasKeys": {
$cond: [
{
$ifNull: [
"$keys",
false
]
},
true,
false
]
}
}
},
])
Mongo Playground
For older Mongo versions you have to do it in code.
If you don't want to update the actual document but just populate this field when you fetch it you can use the same aggregation to fetch the document
you can use $project in mongoose aggregation like this.
$project: { hasKeys: { $cond: [{ $eq: ['$keys', null] }, false, true]}}
Actually, in the database I got a job that I request with a GET route:
So when I populate candidates I got this response format :
My problem here is I don't need that "id" object, I just need a "selected_candidates" array with users inside as objects. Actually it's an object, in another object that is in an Array.
Here the code from my controller (the populate is in the jobsService):
If I change the data format of the job like that way:
...It is working great (with a path: "candidates_selected") like expected BUT I don't have that "status" string (Normal because I don't have it anymore in the DataBase. Because of ObjectId):
I would like a solution to have them both, but maybe it's the limit of noSQL?
A solution without populate but with a Loop (I don't think it's a good idea):
I think there is no convenience way to achieve it. However you may try the aggregate framework from the native MongoDB driver.
Let your Mongoose schemas be ASchema and BSchema
const result = await ASchema.aggregate([
{$addFields: {org_doc: '$$ROOT'}}, // save original document to retrieve later
{$unwind: '$candidates_selected'},
{
$lookup: {
from: BSchema.collection.name,
let: {
selected_id: '$candidates_selected.id',
status: '$candidates_selected.status',
},
pipeline: [
{
$match: {$expr: {$eq: ['$$selected_id', '$_id']}}, // find candidate by id
},
{
$addFields: {status: '$$status'} // attach status
}
],
as: 'found_candidate'
}
},
{
$group: { // regroup the very first $unwind stage
_id: '$_id',
org_doc: {$first: '$org_doc'},
found_candidates: {
$push: {$arrayElemAt: ['$found_candidate', 0]} // result of $lookup is an array, concat them to reform the original array
}
}
},
{
$addFields: {'org_doc.candidates_selected': '$found_candidates'} // attach found_candidates to the saved original document
},
{
$replaceRoot: {newRoot: '$org_doc'} // recover the original document
}
])
When trying to update the date inside the MongoDB document, nothing is updated.
To update, I send a request to NodeJS like this:
collection.updateOne(
{ _id: ObjectId(...) },
{ $set: { 'field1.field2.0.date': new Date('2020-02-01T00:00:00Z') } },
callback
)
The task uses the npm package mongodb#2.3.26 and MongoDB 3.6.20.
I tried to send the '2020-02-01T00:00:00Z' as a value - it is not updated. I tried to update the database version - it didn't help. I tried to update the driver version - it didn't help.
It is interesting that such a request goes through Robo3T and updates the value correctly.
Could you please tell me how to update the date inside the document for MongoDB?
UPD: Structure of document
const document = {
field1: {
exp: 1,
field2: [
{
name: "test",
date: 2019-10-01 00:00:00.000Z
}
]
},
settings: {
sharedTill: 2022-10-01 00:00:00.000Z
},
updatedBy: 'Vadim Prokopchuk'
}
UPD2: Added Response from MongoDB
MongoDB returned data and does not return an error.
the syntax you're using isn't correct.
Refer the query below, it would work
collection.updateOne(
{ _id: ObjectId(...) , 'field1.field2.name':"test"},
{ $set: { 'field1.field2.$.date': new Date('2020-02-01T00:00:00Z') } },
callback
)
Here, we are finding the element of the array which matches our condition and then updating the date of that particular element of the array.
Update:
Problem here was that the native MongoDB driver needed the objectID in a different format than Mongoose. Instead of {_id: story_id} I needed to do, {_id: new mongoose.Types.ObjectId(story_id)}. The reason it returned just the two fields was that it was creating a new document with {_id: story_id} rather than updating the document which was {_id: {$oid: story_id}}. However, my original reason for doing this with the native driver vs. mongoose turned out to not work. Even the native driver does not accept positive $slice values. So I'm still looking for a way to insert an object to the beginning of an array using Mongoose or the native driver accessed through Mongoose (which does not support $position or positive $slice values).
When I run the query below, the returned results only include _id, messages and participants. I would like to get the full story record back rather than just the updated fields. Is there a way to do this? Based on Node MongoDB Native driver documentation (http://mongodb.github.io/node-mongodb-native/api-generated/collection.html#findandmodify) the "fields" parameter from the mongo console is not implemented in the findAndModify driver. I want to avoid having to do a second query to get the just updated record.
Story.collection.findAndModify(
{"_id": story_id},
{"_id": "asc"},
{
"$push": {
"messages": {
"$each": [message],
"$sort": { "_id": -1},
"$slice": 500
},
},
"$addToSet": {"participants": user_id},
},
{new: true},
{fields: {"feeling":1, "description":1, "image_height":1, "image_width":1, "storyteller":1, "image_url":1, "participants":1}}, // -> this line is ignored by MongoDB native driver
{upsert: true}
)
You can insert elements at the beginning of an array field with a $push that uses the $each and $position modifiers (which is supported by Mongoose):
Story.findByIdAndUpdate(story_id, {
$push: {
messages: {
$each: [message],
$position: 0
},
},
$addToSet: {participants: user_id},
}, {
new: true,
upsert: true
},
function (err, newDoc) { ... }
);
Let's say I have some Schema which has a virtual field like this
var schema = new mongoose.Schema(
{
name: { type: String }
},
{
toObject: { virtuals: true },
toJSON: { virtuals: true }
});
schema.virtual("name_length").get(function(){
return this.name.length;
});
In a query is it possible to sort the results by the virtual field? Something like
schema.find().sort("name_length").limit(5).exec(function(docs){ ... });
When I try this, the results are simple not sorted...
You won't be able to sort by a virtual field because they are not stored to the database.
Virtual attributes are attributes that are convenient to have around
but that do not get persisted to mongodb.
http://mongoosejs.com/docs/2.7.x/docs/virtuals.html
Virtuals defined in the Schema are not injected into the generated MongoDB queries. The functions defined are simply run for each document at the appropriate moments, once they have already been retrieved from the database.
In order to reach what you're trying to achieve, you'll also need to define the virtual field within the MongoDB query. For example, in the $project stage of an aggregation.
There are, however, a few things to keep in mind when sorting by virtual fields:
projected documents are only available in memory, so it would come with a huge performance cost if we just add a field and have the entire documents of the search results in memory before sorting
because of the above, indexes will not be used at all when sorting
Here's a general example on how to sort by virtual fields while keeping a relatively good performance:
Imagine you have a collection of teams and each team contains an array of players directly stored into the document. Now, the requirement asks for us to sort those teams by the ranking of the favoredPlayer where the favoredPlayer is basically a virtual property containing the most relevant player of the team under certain criteria (in this example we only want to consider offense and defense players). Also, the aforementioned criteria depend on the users' choices and can, therefore, not be persisted into the document.
To top it off, our "team" document is pretty large, so in order to mitigate the performance hit of sorting in-memory, we project only the fields we need for sorting and then restore the original document after limiting the results.
The query:
[
// find all teams from germany
{ '$match': { country: 'de' } },
// project only the sort-relevant fields
// and add the virtual favoredPlayer field to each team
{ '$project': {
rank: 1,
'favoredPlayer': {
'$arrayElemAt': [
{
// keep only players that match our criteria
$filter: {
input: '$players',
as: 'p',
cond: { $in: ['$$p.position', ['offense', 'defense']] },
},
},
// take first of the filtered players since players are already sorted by relevance in our db
0,
],
},
}},
// sort teams by the ranking of the favoredPlayer
{ '$sort': { 'favoredPlayer.ranking': -1, rank: -1 } },
{ '$limit': 10 },
// $lookup, $unwind, and $replaceRoot are in order to restore the original database document
{ '$lookup': { from: 'teams', localField: '_id', foreignField: '_id', as: 'subdoc' } },
{ '$unwind': { path: '$subdoc' } },
{ '$replaceRoot': { newRoot: '$subdoc' } },
];
For the example you gave above, the code could look something like the following:
var schema = new mongoose.Schema(
{ name: { type: String } },
{
toObject: { virtuals: true },
toJSON: { virtuals: true },
});
schema.virtual('name_length').get(function () {
return this.name.length;
});
const MyModel = mongoose.model('Thing', schema);
MyModel
.aggregate()
.project({
'name_length': {
'$strLenCP': '$name',
},
})
.sort({ 'name_length': -1 })
.exec(function(err, docs) {
console.log(docs);
});