I am fetching data from an external api and I want to insert multiple records if they are not already there . I am creating an array of objects like this :
data=[
{
match_id: 212167781,
player1Id: 129753806,
player2Id: 129753811,
player1Details: '5f070c8f79f62c00042b0cd8',
player2Details: '5f3267fe6a0f891b3604b999'
},
{
match_id: 212167782,
player1Id: 129753799,
player2Id: null,
player1Details: '5f070ade79f62c00042b0cd6'
}
]
Now for instance if the second record is not already present then I want it to be inserted and I have tried upsert but nothing happens . Please help .
You should be able to do this using a simple loop and Model.findOneAndUpdate in combination with the upsert option:
async function upsertMany(data) {
for(const entry of data) {
await YourModel.findOneAndUpdate(entry, entry, { upsert: true });
}
}
// call this with your data-array
await upsertMany(data);
Related
I'm trying to create multiple documents based on a filter: if document not found => create it.
After searching a bit I found that the correct way to do so is by using updateMany and setting upsert: true (docs).
This made somewhat sense from the documentation's example but as I understand it the filter modifier would be used for the newly-created document. As in the example:
try {
db.inspectors.updateMany(
{ "Sector" : { $gt : 4 }, "inspector" : "R. Coltrane" },
{ $set: { "Patrolling" : false } },
{ upsert: true }
);
} catch (e) {
print(e);
}
"Inspector" : "R. Coltrane" would be inserted to the newly-created document.
But what if my setOnInsert modifier contains the same field as the one in the filter?
What my code looks like:
//first find the already-created tags
await tagModel.find({"tagName": tags}).select('tagName -_id').exec()
.then(async (result: Tag[])=>{
//create the new tags
const newTags = tags.map((tag: any)=>new tagModel({tagName: tag}));
//now insert only the new tags, filtering out the already-created tags ("result")
await tagModel.updateMany(
{"tagName": result},
{$setOnInsert: newTags} ,
{upsert: true},
(err:any, res:any)=>{
...
At first, result is an empty Array ([]). What is created in my MongoDB database is a new Tag document, but its tagName is the result object. Meaning, it looks like:
{
"_id": {
"$oid": "61659c92c6267fe11963b236"
},
"tagName": {
"$in": []
}
}
So essentially my question is, what am I suppose to do in this case where my update modifier should replace my filter query? Perhaps it's just something bad in my code that makes the updateMany function to malfunction? Or should I replace it with a different function?
I am executing a mongoose Model.findById() function in order to return a single instance by utilizing an express route.
Model.findById(modelid)
.then(instance => {
if(instance.isOwnedBy(user)) {
return instance.update({$push: {days: req.params.dayid}}, {new: true})
.then(foo => res.send(foo))
} else {
res.status(401).send("Unauthorized")
}
})
the above code returns an object containing the opTime, electionId...etc instead of returning the newly updated document instance. How am I able return the newly updated document after the instance.update() method?
If instance.isOwnedBy(user) and _id: modelid can be merged into one mongo query, it's better to use findOneAndUpdate() but in that way, if it doesn't find any document match, you can not know which part of the query causes the not found.
And because I don't know much about your models, conditions, I can not answer how to do it with findOneAndUpdate() but there is another way that modify the document and call save() method.
Example base on your code:
Model.findById(modelid)
.then(instance => {
if(instance && instance.isOwnedBy(user)) {
if(instance.days) instance.days.push(req.params.dayid);
else instance.days = [req.params.dayid];
instance.save().then(function(instance){
res.send(instance);
})
} else {
res.status(401).send("Unauthorized")
}
})
Note: You should check if the instance exist or not before modify it.
Hi I have a fairly large dataset 55K records. I am calculating the moving average for those. Which is the most efficient way to store these results again?
Currently I am doing this. Which leads to extreme amount of records being written one by one. Is there a way for me to write the whole list back in one call if I add the calculated value to the records array?
updated the code with the bulk update. It however fails to update the "MA16" record. Even though I know for sure that there is valid data in the "list" array.
It seems to match the documents but won't update. It will yield a MA16 field in the database that is always null.
Logged output.
deletedCount:0
insertedCount:0
insertedIds:Object {}
matchedCount:150
modifiedCount:0
n:0
nInserted:0
nMatched:150
nModified:0
nRemoved:0
nUpserted:0
ok:1
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
});
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
console.log("Insert result", result);
}).catch(err=>{
//catch the error here
})
You can use BulkWrite to achieve what you want.
Try this:
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
})
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
//check the result of bulk update here
}).catch(err=>{
//catch the error here
})
You can use updateOne operator of bulkWrite.
From official MongoDB docs, bulkWrite has following options:
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
Please read MongoDB bulkWrite documentation for more info.
I hope this helps you out.
I now use mongooses to pull and pull subdocuments to the array, and now I want to change the contents of the detail field of that subdocument with the _id of the subdocument.
{
subDocument: [{
_id: ObjectId('123'),
detail: 'I want update this part'
}]
}
I tried to use the $set method as shown below but it did not work as expected.
Model.findByIdAndUpdate(uid, { $Set: {subDocument: {_id: _id}}});
Looking at the for statement as shown below is likely to have a bad effect on performance. So I want to avoid this method.
const data = findById(uid);
for(...) {
if(data.subDocument[i]._id==_id) {
data.subDocument[i].detail = detail
}
}
Can you tell me some mongodb queries that I can implement?
And, Is it not better to use the 'for(;;)' statement shown above than to search using mongodb's query?
This should work:
Model.findOneAndUpdate({"subdocument._id": uid},
{
$set: {
"subdocument.$.detail ": "detail here"
}
},
).exec(function(err, doc) {
//code
});
To find subdocument by id, I am using something like this :
var subDocument = data.subDocument.id(subDocumentId);
if (subDocument) {
// Do some stuff
}
else {
// No subDocument found
}
Hope it helps.
I'm trying to insert/update an array of strings in a mongodb document using some typescript code running in NodeJS.
The following code is typescript but I guess JS developers will get it w/o any problems:
export function addEvents(entityId: string,
events: string[] ,
callback: () => void) {
db.collection('events', function(error, eventCollection) {
if(error) {
console.error(error); return;
}
eventCollection.update({ _id: entityId }, { "$pushAll ":
{ events: events }},
function(error, result) {
if(error) {
console.error(error); return;
}
callback();
});
});
}
the document have the following structure:
{
_id : string
events : ["array","of","strings"]
}
I simply want to append an array strings at the end of the existing array for a specific _id.
I don't quite get if I should use update,save, $push ,$pushall etc.
Can someone explain?
If I understood correctly the problem is that pushAll does nothing or update returns error? Maybe copy-paste mistake in your example but I think you have typo here.
{ "$pushAll ": { events: events }}
It should be
{ $pushAll: { events: events }}
Your combination of update and $pushAll looks like the best choice for what you're doing here -- it's for appending an array to an existing array. $push is for adding an element to an array. save would involve getting the existing events array, appending to it, then saving the document.
The extra space in "$pushAll " needs to be removed. It may have quotes: "$pushAll".
Found the problem, I needed to pass "{ upsert = true }" as a third argument to the update function.
To achieve 'upsert' semantics in this case, you'd need to use $addToSet. If you have an array of values to add, you'd need to throw in the $each modifier. From mongo shell:
db.events.update(
{ _id: entityId },
{ $addToSet: { $each: events } }
)