most efficient way to add a calculated field mongoose - node.js

Question

Hi I have a fairly large dataset 55K records. I am calculating the moving average for those. Which is the most efficient way to store these results again?
Currently I am doing this. Which leads to extreme amount of records being written one by one. Is there a way for me to write the whole list back in one call if I add the calculated value to the records array?
updated the code with the bulk update. It however fails to update the "MA16" record. Even though I know for sure that there is valid data in the "list" array.
It seems to match the documents but won't update. It will yield a MA16 field in the database that is always null.
Logged output.
deletedCount:0
insertedCount:0
insertedIds:Object {}
matchedCount:150
modifiedCount:0
n:0
nInserted:0
nMatched:150
nModified:0
nRemoved:0
nUpserted:0
ok:1
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
});
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
console.log("Insert result", result);
}).catch(err=>{
//catch the error here
})

Answer 1

You can use BulkWrite to achieve what you want.
Try this:
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
})
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
//check the result of bulk update here
}).catch(err=>{
//catch the error here
})
You can use updateOne operator of bulkWrite.
From official MongoDB docs, bulkWrite has following options:
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
Please read MongoDB bulkWrite documentation for more info.
I hope this helps you out.