I have lists of data which I want to save and if already exist update.
I can do that using loop. But Is there any other way like insertMany which only supports insert but I want to insert and update too in bulk.
You can use the bulk update feature that Mongo driver provides. Instead of invoking the transactions in a loop, you may add them to a bulk transaction and execute as a batch.
First you need to initialize the bulk operation, ordered / unordered:
var bulk = db.collection.initializeUnorderedBulkOp();
or
var bulk = db.collection.initializeOrderedBulkOp();
Then you can go on adding transactions to the bulk object.
bulk.insert( {
// attributes
} ); // insert operation
or
bulk.find( {
// query attributes
} ).update( {
$set: {
// set attributes
} } ); // update operation
In the end you need to call
bulk.execute();
I don't know if this would serve your purpose.
Please refer this link:
https://docs.mongodb.com/manual/reference/method/Bulk/
Use updateMany with option { upsert: true } which update the document if it is already exists otherwise insert the new document.
Find below example with restaurant collection
{ "_id" : 1, "name" : "Central Perk Cafe", "violations" : 3 }
{ "_id" : 2, "name" : "Rock A Feller Bar and Grill", "violations" : 2 }
{ "_id" : 3, "name" : "Empire State Sub", "violations" : 5 }
{ "_id" : 4, "name" : "Pizza Rat's Pizzaria", "violations" : 8 }
The query below update the documents with violations equal to 4 and if the document not exists insert new document.
db.restaurant.updateMany(
{ violations: 4 },
{ $set: { "name" : "Eat and Treat" } },
{ upsert: true }
);
Find more details here:
https://docs.mongodb.com/manual/reference/method/db.collection.updateMany/
I 've got the following document.
{
"_id" : ObjectId("5d0e4232ccfb3ee9ac72a64d"),
"arraydatumobject" : {
"datum" : ISODate("2010-01-02T00:00:00.000+0000")
}
}
when i try to find the date with either $gte or $lte or even both i cant find the document.
so something like this:
db.getCollection("test").find(
{
arraydatumobject : { $elemMatch:{ datum : {$gte : new ISODate("2008-01-02"), $lte : new ISODate("2011-01-02")}}}
}
)
or this
db.getCollection("test").find(
{
arraydatumobject : {datum : {$gte : new ISODate("2008-01-02"), $lte : new ISODate("2011-01-02")}}
}
)
gets me nothing.
But when i use the following i do get a result.
db.getCollection("test").find(
{
arraydatumobject : {datum : new ISODate("2010-01-02")}
}
)
i'm getting the document that i'm searching for...
What am i missing here?
With a given document of:
{
"_id" : ObjectId("5d0e4232ccfb3ee9ac72a64d"),
"arraydatumobject" : {
"datum" : ISODate("2010-01-02T00:00:00.000+0000")
}
}
Your date range find query should be:
db.getCollection('test').find({
'arraydatumobject.datum': { // you have an object use obj.prop
$gte : new ISODate("2008-01-02"),
$lte : new ISODate("2011-01-02")
}
})
Your second example should work, but you have a pair of curly braces missing there.
Also, you should compare using node.js Dates, not ISODates
Try:
db.getCollection("test").find(
{
arraydatumobject : {datum : {$gte : new Date(2008,2,1), $lte : new Date(2011, 2, 1)}}
}
)
my database is like as follows:
{
"riderId" : ObjectId("59fac6f760c247118c16539f"),
"pickupDateTime" : ISODate("2017-11-19T07:51:57.491Z"),
"fare" : 15,
"quotes" : [],
"status" : "OPEN",
"closed" : false,
"closeBy" : null
}
if i want to fetch data of given pickupDateTime to increase half an hour pickupDateTime n here is my query:
db.getCollection('auctions').find( {
pickupDateTime:{
$gte: new Date(givenTime),
$lt: new Date(intervalPickupByTime)
}
})
but it doesn't work it fetch no records and in node i calculate the half hour difference by using moment as follows:
const intervalPickupByTime =
moment(filter.pickupDateTime).add(30,
'minutes');
I didn't understand why it not works
I am connecting to the Yelp API using the RapidAPI module in Nodejs. I am able to request a token, connect, and request data, retrieve that data, and insert the relevant information for each result it into mongodb. Here's where it gets complicated...
Let's say I make a Yelp API request and search for bars. I get a list of bars and insert them into the database. Let's say one of these in the list is "Joe's Bar & Grill". One of the fields in my mongodb is "type" and it's an array. So now, this particular document will look something like this:
{
id: 'joes-bar-and-grill',
name: 'Joe\'s Bar & Grill',
type: ['bar']
}
But then I run another request on the Yelp API on "restaurants", and in this list "Joe's Bar & Grill" shows up again. Instead of inserting a new duplicate document into mongodb, I'd like the existing document to end up looking like this:
{
id: 'joes-bar-and-grill',
name: 'Joe\'s Bar & Grill',
type: ['bar', 'restaurant']
}
In addition to this, let's say I run another request again for "bars", and "Joe's Bar & Grill" comes up again. I don't want it to automatically insert "bar" into the type array again, if "bar" already exists in its array.
I've tried findOneAndUpdate with upsert: true and a $push of new data into the array, but I cannot get it to work at all. Does anyone have any ideas?
You can use findOneAndUpdate, combined with $addToSet (to make sure that an entry in the array only exists once) and $each (to allow passing arrays to $addToSet):
Bar.findOneAndUpdate({ id : 'joes-bar-and-grill' }, {
id : 'joes-bar-and-grill',
name : 'Joe\'s Bar & Grill',
$addToSet : { type : { $each : [ 'restaurant' ] } }
}, { upsert : true })
EDIT: now that you posted your entire code, the problem becomes more obvious.
For one, I'm not sure if the third and fourth arguments that you're passing to Location.update() make sense. As far as I know, the third should be an option object, and the fourth an async function.
Secondly, it looks like you're just ignoring any update errors.
And lastly, this isn't going to work:
for (var i = 0; i < payload.businesses.length; i++) { Location.update(...) }
Because Location.update() is asynchronous, the i variable will get clobbered (you should browse around on SO to find the explanation for that; for example, see this question).
You're going to need a library that will provide you with better async support, and preferably one that will also help limiting the number of update queries.
Once such library is async, and using it, your code would become something like this:
const async = require('async');
...
async.eachLimit(payload.businesses, 5, function(business, callback) {
Location.update({ yelpID : business.id }, {
name : business.name,
latitude : business.location.latitude,
longitude : business.location.longitude,
address1 : business.location.address1,
address2 : business.location.address2,
address3 : business.location.address3,
city : business.location.city,
state : business.location.state,
zip_code : business.location.zip_code,
country : business.location.country,
timezone : 'CST'
$addToSet : { type : 'bar' }
}, { upsert : true }, callback);
}, function(err) {
if (err) {
console.error(err);
} else {
console.log('All documents inserted');
}
});
You may use $addToSet operator
The $addToSet operator adds a value to an array unless the value is
already present, in which case $addToSet does nothing to that array.
$addToSet only ensures that there are no duplicate items added to the
set and does not affect existing duplicate elements. $addToSet does
not guarantee a particular ordering of elements in the modified set.
If the field is absent in the document to update, $addToSet creates
the array field with the specified value as its element.
If the field is not an array, the operation will fail.
The below solution assumes that on each update, you receive a single type and not an array. If the input document is an array itself, you may use robertklep's solution with $each operator
db.mycoll.update(
{ "id" : "joes-bar-and-grill" },
{
$set:{
name : 'Joe\'s Bar & Grill',
},
$addToSet : { type : 'restaurant' }
},
true, false);
I have also used $set operator.
The $set operator replaces the value of a field with the specified
value.
The $set operator expression has the following form:
{ $set: { field1: value1, ... } }
Here is the mongo shell output to explain it further :
> db.mycoll.find({ "id" : "joes-bar-and-grill" });
// NO RESULT
> db.mycoll.update(
... { "id" : "joes-bar-and-grill" },
... {
... $set:{
... name : 'Joe\'s Bar & Grill',
... },
... $addToSet : { type : 'restaurant' }
... },
... true, false);
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : ObjectId("58e719b4d543c5e30d615d59")
})
// INSERTED A NEW DOCUMENT AS IT DOES NOT EXIST
> db.mycoll.find({ "id" : "joes-bar-and-grill" }); // FINDING THE OBJECT
{ "_id" : ObjectId("58e719b4d543c5e30d615d59"), "id" : "joes-bar-and-grill", "name" : "Joe's Bar & Grill", "type" : [ "restaurant" ] }
> db.mycoll.update(
... { "id" : "joes-bar-and-grill" },
... {
... $set:{
... name : 'Joe\'s Bar & Grill',
... },
... $addToSet : { type : 'bar' }
... },
... true, false);
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
// UPDATING THE DOCUMENT WITH NEW TYPE : "bar"
> db.mycoll.findOne({ "id" : "joes-bar-and-grill" });
{
"_id" : ObjectId("58e719b4d543c5e30d615d59"),
"id" : "joes-bar-and-grill",
"name" : "Joe's Bar & Grill",
"type" : [
"restaurant",
"bar"
]
}
I use two ways to retrieve documents from my collection, the first one:
db.comments.find({"nid" : "req.body.data"});
returns many doc like:
{
"nid" : 20404,
"_id" : ObjectId("5638ba331294943d3d0a092b"),
"uid" : 1937,
"posted" : ISODate("2015-11-03T13:44:19.811Z"),
"text" : "txt",
"title" : "Test nid 2",
"stars" : 3,
"__v" : 0
}
,
And for another query I need to use aggregate and the query:
var pipleline = [
{$match: {nid:req.body.data}}
];
Comments.aggregate(pipleline, function(err, rank){
if(err) {
res.send("Error", String(err));
}
res.send(rank);
});
Returns [] - empty array.
Any ideas?
You can use the built in function chaining mongoose provides. Aside from match, it also has sort, project, group, and few others I don't know off the top of my head. More info here
Comments.aggregate().match({nid:req.body.data})
.exec(function(err,rank){
if(err) {
res.send("Error", String(err));
}
res.send(rank);
});