I have lists of data which I want to save and if already exist update.
I can do that using loop. But Is there any other way like insertMany which only supports insert but I want to insert and update too in bulk.
You can use the bulk update feature that Mongo driver provides. Instead of invoking the transactions in a loop, you may add them to a bulk transaction and execute as a batch.
First you need to initialize the bulk operation, ordered / unordered:
var bulk = db.collection.initializeUnorderedBulkOp();
or
var bulk = db.collection.initializeOrderedBulkOp();
Then you can go on adding transactions to the bulk object.
bulk.insert( {
// attributes
} ); // insert operation
or
bulk.find( {
// query attributes
} ).update( {
$set: {
// set attributes
} } ); // update operation
In the end you need to call
bulk.execute();
I don't know if this would serve your purpose.
Please refer this link:
https://docs.mongodb.com/manual/reference/method/Bulk/
Use updateMany with option { upsert: true } which update the document if it is already exists otherwise insert the new document.
Find below example with restaurant collection
{ "_id" : 1, "name" : "Central Perk Cafe", "violations" : 3 }
{ "_id" : 2, "name" : "Rock A Feller Bar and Grill", "violations" : 2 }
{ "_id" : 3, "name" : "Empire State Sub", "violations" : 5 }
{ "_id" : 4, "name" : "Pizza Rat's Pizzaria", "violations" : 8 }
The query below update the documents with violations equal to 4 and if the document not exists insert new document.
db.restaurant.updateMany(
{ violations: 4 },
{ $set: { "name" : "Eat and Treat" } },
{ upsert: true }
);
Find more details here:
https://docs.mongodb.com/manual/reference/method/db.collection.updateMany/
Related
For example, I have some documents that look like this:
{
id: 1
name: "foo"
}
And I want to append another string to the current name field value.
I tried the following using Mongoose, but it didn't work:
Model.findOneAndUpdate({ id: 1 }, { $set: { name: +"bar" } }, ...);
Edit:
From Compatibility Changes in MongoDB 3.6:
MongoDB 3.6.1 deprecates the snapshot query option.
For MMAPv1, use hint() on the { _id: 1} index instead to prevent a cursor from returning a document more than once if an intervening write operation results in a move of the document.
For other storage engines, use hint() with { $natural : 1 } instead.
Original 2017 answer:
You can't refer to the values of the document you want to update, so you will need one query to retrieve the document and another one to update it. It looks like there's a feature request for that in OPEN state since 2016.
If you have a collection with documents that look like:
{ "_id" : ObjectId("590a4aa8ff1809c94801ecd0"), "name" : "bar" }
Using the MongoDB shell, you can do something like this:
db.test.find({ name: "bar" }).snapshot().forEach((doc) => {
doc.name = "foo-" + doc.name;
db.test.save(doc);
});
The document will be updated as expected:
{ "_id" : ObjectId("590a4aa8ff1809c94801ecd0"), "name": "foo-bar" }
Note the .snapshot() call.
This ensures that the query will not return a document multiple times because an intervening write operation moves it due to the growth in document size.
Applying this to your Mongoose example, as explained in this official example:
Cat.findById(1, (err, cat) => {
if (err) return handleError(err);
cat.name = cat.name + "bar";
cat.save((err, updatedCat) => {
if (err) return handleError(err);
...
});
});
It's worth mentioning that there's a $concat operator in the aggregation framework, but unfortunately you can't use that in an update query.
Anyway, depending on what you need to do, you can use that together with the $out operator to save the results of the aggregation to a new collection.
With that same example, you will do:
db.test.aggregate([{
$match: { name: "bar" }
}, {
$project: { name: { $concat: ["foo", "-", "$name"] }}
}, {
$out: "prefixedTest"
}]);
And a new collection prefixedTest will be created with documents that look like:
{ "_id" : ObjectId("XXX"), "name": "foo-bar" }
Just as a reference, there's another interesting question about this same topic with a few answers worth reading: Update MongoDB field using value of another field
If this is still relevant, I have a solution for MongoDB 4.2.
I had the same problem where "projectDeadline" fields of my "project" documents were Array type (["2020","12","1"])
Using Robo3T, I connected to my MongoDB Atlas DB using SRV link. Then executed the following code and it worked for me.
Initial document:
{
_id : 'kjnolqnw.KANSasdasd',
someKey : 'someValue',
projectDeadline : ['2020','12','1']
}
CLI Command:
db
.getCollection('mainData')
.find({projectDeadline: {$not: {$eq: "noDeadline"}}})
.forEach((doc) => {
var deadline = doc.projectDeadline;
var deadlineDate = new Date(deadline);
db
.mainData
.updateOne({
_id: doc._id},
{"$set":
{"projectDeadline": deadlineDate}
}
)}
);
Resulting document:
{
_id : 'kjnolqnw.KANSasdasd',
someKey : 'someValue',
projectDeadline : '2020-12-01 21:00:00.000Z'
}
I am connecting to the Yelp API using the RapidAPI module in Nodejs. I am able to request a token, connect, and request data, retrieve that data, and insert the relevant information for each result it into mongodb. Here's where it gets complicated...
Let's say I make a Yelp API request and search for bars. I get a list of bars and insert them into the database. Let's say one of these in the list is "Joe's Bar & Grill". One of the fields in my mongodb is "type" and it's an array. So now, this particular document will look something like this:
{
id: 'joes-bar-and-grill',
name: 'Joe\'s Bar & Grill',
type: ['bar']
}
But then I run another request on the Yelp API on "restaurants", and in this list "Joe's Bar & Grill" shows up again. Instead of inserting a new duplicate document into mongodb, I'd like the existing document to end up looking like this:
{
id: 'joes-bar-and-grill',
name: 'Joe\'s Bar & Grill',
type: ['bar', 'restaurant']
}
In addition to this, let's say I run another request again for "bars", and "Joe's Bar & Grill" comes up again. I don't want it to automatically insert "bar" into the type array again, if "bar" already exists in its array.
I've tried findOneAndUpdate with upsert: true and a $push of new data into the array, but I cannot get it to work at all. Does anyone have any ideas?
You can use findOneAndUpdate, combined with $addToSet (to make sure that an entry in the array only exists once) and $each (to allow passing arrays to $addToSet):
Bar.findOneAndUpdate({ id : 'joes-bar-and-grill' }, {
id : 'joes-bar-and-grill',
name : 'Joe\'s Bar & Grill',
$addToSet : { type : { $each : [ 'restaurant' ] } }
}, { upsert : true })
EDIT: now that you posted your entire code, the problem becomes more obvious.
For one, I'm not sure if the third and fourth arguments that you're passing to Location.update() make sense. As far as I know, the third should be an option object, and the fourth an async function.
Secondly, it looks like you're just ignoring any update errors.
And lastly, this isn't going to work:
for (var i = 0; i < payload.businesses.length; i++) { Location.update(...) }
Because Location.update() is asynchronous, the i variable will get clobbered (you should browse around on SO to find the explanation for that; for example, see this question).
You're going to need a library that will provide you with better async support, and preferably one that will also help limiting the number of update queries.
Once such library is async, and using it, your code would become something like this:
const async = require('async');
...
async.eachLimit(payload.businesses, 5, function(business, callback) {
Location.update({ yelpID : business.id }, {
name : business.name,
latitude : business.location.latitude,
longitude : business.location.longitude,
address1 : business.location.address1,
address2 : business.location.address2,
address3 : business.location.address3,
city : business.location.city,
state : business.location.state,
zip_code : business.location.zip_code,
country : business.location.country,
timezone : 'CST'
$addToSet : { type : 'bar' }
}, { upsert : true }, callback);
}, function(err) {
if (err) {
console.error(err);
} else {
console.log('All documents inserted');
}
});
You may use $addToSet operator
The $addToSet operator adds a value to an array unless the value is
already present, in which case $addToSet does nothing to that array.
$addToSet only ensures that there are no duplicate items added to the
set and does not affect existing duplicate elements. $addToSet does
not guarantee a particular ordering of elements in the modified set.
If the field is absent in the document to update, $addToSet creates
the array field with the specified value as its element.
If the field is not an array, the operation will fail.
The below solution assumes that on each update, you receive a single type and not an array. If the input document is an array itself, you may use robertklep's solution with $each operator
db.mycoll.update(
{ "id" : "joes-bar-and-grill" },
{
$set:{
name : 'Joe\'s Bar & Grill',
},
$addToSet : { type : 'restaurant' }
},
true, false);
I have also used $set operator.
The $set operator replaces the value of a field with the specified
value.
The $set operator expression has the following form:
{ $set: { field1: value1, ... } }
Here is the mongo shell output to explain it further :
> db.mycoll.find({ "id" : "joes-bar-and-grill" });
// NO RESULT
> db.mycoll.update(
... { "id" : "joes-bar-and-grill" },
... {
... $set:{
... name : 'Joe\'s Bar & Grill',
... },
... $addToSet : { type : 'restaurant' }
... },
... true, false);
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : ObjectId("58e719b4d543c5e30d615d59")
})
// INSERTED A NEW DOCUMENT AS IT DOES NOT EXIST
> db.mycoll.find({ "id" : "joes-bar-and-grill" }); // FINDING THE OBJECT
{ "_id" : ObjectId("58e719b4d543c5e30d615d59"), "id" : "joes-bar-and-grill", "name" : "Joe's Bar & Grill", "type" : [ "restaurant" ] }
> db.mycoll.update(
... { "id" : "joes-bar-and-grill" },
... {
... $set:{
... name : 'Joe\'s Bar & Grill',
... },
... $addToSet : { type : 'bar' }
... },
... true, false);
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
// UPDATING THE DOCUMENT WITH NEW TYPE : "bar"
> db.mycoll.findOne({ "id" : "joes-bar-and-grill" });
{
"_id" : ObjectId("58e719b4d543c5e30d615d59"),
"id" : "joes-bar-and-grill",
"name" : "Joe's Bar & Grill",
"type" : [
"restaurant",
"bar"
]
}
I am trying to count the number of models in a collection based on a property:
I have an upvote model, that has: post (objectId) and a few other properties.
First, is this good design? Posts could get many upvotes, so I didn’t want to store them in the Post model.
Regardless, I want to count the number of upvotes on posts with a specific property with the following and it’s not working. Any suggestions?
upvote.count({‘post.specialProperty’: mongoose.Types.ObjectId(“id”), function (err, count) {
console.log(count);
});
Post Schema Design
In regards to design. I would design the posts collection for documents to be structured as such:
{
"_id" : ObjectId(),
"proprerty1" : "some value",
"property2" : "some value",
"voteCount" : 1,
"votes": [
{
"voter": ObjectId()// voter Id,
other properties...
}
]
}
You will have an array that will hold objects that can contain info such as voter id and other properties.
Updating
When a posts is updated you could simply increment or decrement the voteCountaccordingly. You can increment by 1 like this:
db.posts.update(
{"_id" : postId},
{
$inc: { voteCount: 1},
$push : {
"votes" : {"voter":ObjectId, "otherproperty": "some value"}
}
}
)
The $inc modifier can be used to change the value for an existing key or to create a new key if it does not already exist. Its very useful for updating votes.
Totaling votes of particular Post Criteria
If you want to total the amount for posts fitting a certain criteria, you must use the Aggregation Framework.
You can get the total like this:
db.posts.aggregate(
[
{
$match : {property1: "some value"}
},
{
$group : {
_id : null,
totalNumberOfVotes : {$sum : "$voteCount" }
}
}
]
)
I have a straightforward mongo collection with an array of subdocuments. I'm trying to do the oft asked "upsert a subdocument in an array". I have read all questions on this topic, but can't seem to get it to work.
Data structure for game_managers:
{
"_id" : ObjectId("555cf465715ff974fb09221f"),
"game_id" : "123456789",
"players" : [
{
"request_email" : "thebigcheese#foobar.com",
"request_notes" : "I love mongo!",
"user_id" : ObjectId("551eb55f555b404d68b88063")
},
{
"request_email" : "morecowbell#example.com",
"request_notes" : "I love oysters!",
"user_id" : ObjectId("551eb55f555b404d68b88063")
}
]
}
When I try to Create / Update with the following code, it always overwrites the first element. I can't get it to even
var col = db.mongo.collection('game_managers');
// Upsert a game manager record for the game
col.update( {game_id:game.place_id}, {$setOnInsert:{game_id:game.game_id}}, { upsert: true }, function(err, result, upserted) {
// Append or update game manager record.
col.update(
{game_id:game.place_id},
{$addToSet: {"players":fields}},
function(err, result) {
next();
}
);
});
I modelled the code from this similar question however it doesn't apply to arrays of subdocuments. I do not want to $pull, and then $push a new element, as the subdocument will ultimately have timestamps and some comments[{},{},{}] subdocs on them.
My Couchdb database as a main document type that looks something like:
{
"_id" : "doc1",
"type" : "main_doc",
"title" : "the first doc"
...
}
There is another type of document that stores user information. I want users to be able to tag documents as favorites. Different users can save the same or different documents as favorites. My idea was to introduce a favorite document to track this something like:
{
"_id" : "fav1",
"type" : "favorite",
"user_id" : "user1",
"doc_id" : "doc1"
}
It's easy enough to create a view with user_id as the key to get a list of their favorite doc IDs. E.g:
function(doc) {
if (doc.type == "favorite") {
emit(doc.user_id, doc.doc_id);
}
}
However I want to list of favorites to display the user_id, doc_id and title from the document. So output something like:
{ "key" : "user1", "value" : ["doc1", "the first doc"] }
In CouchDB 0.11 (just recently released), the include_docs=true feature allows you to look up any document in your view row. For example:
function(doc) {
if(doc.type == "favorite") {
emit(doc.user_id, {_id: doc.doc_id});
}
}
When you query your view with include_docs=true, you should see JSON like this:
// ... normal stuff
rows: [
{
"key":"user1",
"value":{"_id":"doc1"},
"doc": {
"_id" : "doc1",
"type" : "main_doc",
"title" : "the first doc"
// ...
}
},
{
// another doc, etc...
}
]
If you can't use the include_docs=true feature with v0.11, then you must have all information on-hand when you emit data for your view/map.
Instead of a traditional "join" style, consider storing a list of "favoriting" users in the main_doc documents.
{
"_id" : "doc1",
"type" : "main_doc",
"title" : "the first doc",
"favorited_by": ["user1", "user2"]
// ...
}
That way when your view runs, you can emit everything based on the information in that one document.
function(doc) {
if(doc.type == "main_doc") {
for (var a in doc.favorited_by) {
emit(doc.favorited_by[a], [doc._id, doc.title]);
}
}
}