Mongoose id function returning null - node.js

I'm trying to get a sub-document from MongoDB. I can get the parent document, which looks like:
{
"_id" : ObjectId("5550a7948be994430f7df1b4"),
"address" : "Grafton St, Dublin 2",
"coords" : [
-6.2597468,
53.3422998
],
"facilities" : [
"food",
"irish history"
],
"name" : "Laura Dunphy",
"openingTimes" : [
{
"days" : "Monday - Friday",
"times" : [
"10am",
"12pm",
"2pm"
]
},
{
"days" : "Saturday",
"times" : [
"8am",
"10am",
"12pm"
]
}
],
"rating" : 3,
"reviews" : [
{
"author" : "Simon Holmes",
"id" : ObjectId("5550c26e8c334adebc7c5dc3"),
"rating" : 5,
"timestamp" : ISODate("2013-07-15T23:00:00Z"),
"reviewText" : "Great guide, it was fun."
}
]
}
When I do:
console.log('guide.reviews is ');
console.log(guide.reviews);
I see:
guide.reviews is
[{ id: 5550c26e8c334adebc7c5dc3,
rating: 5,
timestamp: Tue Jul 16 2013 00:00:00 GMT+0100 (IST),
reviewText: 'Great guide, it was fun.',
createdOn: Fri May 15 2015 18:20:57 GMT+0100 (IST),
author: 'Simon Holmes' }]
Which is fine. But then when I try to get the review using the Mongoose id function, I always get back null:
review = guide.reviews.id('5550c26e8c334adebc7c5dc3');
console.log('review is ');
console.log(review);
With result:
review is
null
Any idea what I'm doing wrong?

The MongooseDocumentArray.id(id) method searches the document array for an _id property, but your subdocuments don't have an _id property, they instead have id. You'll either have to change it to _id, or use a plain old .filter() or similar workaround.

I may be wrong, but I've never seen that sort of API in Mongoose before.
document.id exists, but it's a function that returns the stringified version of the _id field.
In your case, you can just go through the list with a loop and select the one you want.
var review = false
for (var i = 0; i < guide.reviews.length; i++) {
var el = guide.reviews[i]
if(review._id == '5550c26e8c334adebc7c5dc3') {
review = el
}
}
If you use lodash, you can also use the indexBy method to convert the array to an hash in which all items are identified by the specified field:
var _ = require('lodash')
var reviews = _.indexBy(guide.reviews, '_id')
// Then you can do reviews['5550c26e8c334adebc7c5dc3']

Related

How do I query my collection efficiently based on timeStamps saved with new Date()?

I have a document that looks like this:
{
"_id" : ObjectId("60acdcc693407d2c7b97dd68"),
"transDate" : "2021-06-03T13:53:12.238Z",
"transAmount" : "200",
"transPayersName" : "Daniela",
"transPayersAge" : "24",
"transPayersGender" : "Female",
"__v" : 0
}
I'd like to draw your attention specifically to the transDate.
The following code is what I used when inserting the data into the collection:
var transAction = new Date().toISOString();
transAction.transDate = transDate;
transAction.transAmount = transAmount;
transAction.transPayersName = transPayersName;
transAction.transPayersAge = transPayersAge;
transAction.transPayersGender = transPayersGender;
transAction.save( (err)=>{
if (err) {
console.log('Error inserting into Database: ' +err)
}
Now, I am looking for a more efficient way to query the collection based on the transDate.
I currently query the collection this way:
db.getCollection('mydbs').find({"transDate" : { "$gte": "2021-05-03T13:53:12.238Z" }});
The code above correctly yields:
{
"_id" : ObjectId("60acdcc693407d2c7b97dd68"),
"transDate" : "Tue May 25 2021 14:17:26 GMT+0300 (East Africa Time)",
"transAmount" : "200",
"transPayersName" : "Daniela",
"transPayersAge" : "24",
"transPayersGender" : "Female",
"__v" : 0
}
I believe there is a more efficient way to query the database based on the time stored in the transDate.
Your suggestions are welcome

MongoDB Shema to support concurrent update on a document

We were working on a project with a 300 documents with currentValue field in a main collection, in order to track the history of each document of first collection. we created another collection named history with approximately 6.5 millions of documents.
For each input of system we have to add around 30 history item and update currentValue field of main collection, so, We tried computational field design pattern for currentValue, which lead us to have writeConfilict in concurrent situations (at concurrency of around 1000 requests).
Then we tried to compute currentValue field with sum (amount field) and groupBy(mainId field) on history collection which takes too long (> 3s).
Main collection docs:
{
"_id" : ObjectId(...),
"stock" : [
{
"currentAmount" : -313430.0,
"lastPrice" : -10.0,
"storage" : ObjectId("..."),
"alarmCapacity" : 12
},
{
"currentAmount" : 30,
"lastPrice" : 0,
"storage" : ObjectId("..."),
"alarmCapacity" : 12
},
.
.
.
],
"name" : "name",
}
History collection docs:
{
"_id" : ObjectId("..."),
"mainId" : ObjectId("..."),
"amount" : 5,
}
If you have any other idea to handle this situation(application or db level), I would be thankful.
UPDATE 1
The update query if I use computed pattern would be:
mainCollection.findOneAndUpdate(
{
$and: [
{ _id: id },
{ "stock.storage": fromId },
{ "stock.deletedAt": null }
],
},
{
$inc: {
"stock.$.currentAmount": -1 * amount,
}
},
{
session
}
)
And Aggregation pipeline if I want to calculate currentAmount everytime:
mainCollection.aggregate([
{
$match: {
branch: new ObjectId("...")
}
},
{
$group: {
_id: "$ingredient",
currentAmount: {
$sum: "$amount"
}
}
}])
in order to have computed field, mongo design patterns, suggested computed field,
The Computed Pattern is utilized when we have data that needs to be computed repeatedly in our application. link
like below:
// your main collection will look like this
{
"_id" : ObjectId(...),
"stock" : [
{
"currentAmount" : -313430.0,
"lastPrice" : -10.0,
"storage" : ObjectId("..."),
"alarmCapacity" : 12
},
{
"currentAmount" : 30,
"lastPrice" : 0,
"storage" : ObjectId("..."),
"alarmCapacity" : 12
},
"totalAmount": 20000 // for example
}
but for having concurrent there is a better way to solve this problem with cumulative summation, in this algorithm, we sum last documents inputs, with current input:
{
"_id" : ObjectId("..."),
"mainId" : ObjectId("..."),
"amount" : 5,
"cumulative": 15 // sum of last documents input
}

How to retrieve data from mongoDB by date? [duplicate]

I've been playing around storing tweets inside mongodb, each object looks like this:
{
"_id" : ObjectId("4c02c58de500fe1be1000005"),
"contributors" : null,
"text" : "Hello world",
"user" : {
"following" : null,
"followers_count" : 5,
"utc_offset" : null,
"location" : "",
"profile_text_color" : "000000",
"friends_count" : 11,
"profile_link_color" : "0000ff",
"verified" : false,
"protected" : false,
"url" : null,
"contributors_enabled" : false,
"created_at" : "Sun May 30 18:47:06 +0000 2010",
"geo_enabled" : false,
"profile_sidebar_border_color" : "87bc44",
"statuses_count" : 13,
"favourites_count" : 0,
"description" : "",
"notifications" : null,
"profile_background_tile" : false,
"lang" : "en",
"id" : 149978111,
"time_zone" : null,
"profile_sidebar_fill_color" : "e0ff92"
},
"geo" : null,
"coordinates" : null,
"in_reply_to_user_id" : 149183152,
"place" : null,
"created_at" : "Sun May 30 20:07:35 +0000 2010",
"source" : "web",
"in_reply_to_status_id" : {
"floatApprox" : 15061797850
},
"truncated" : false,
"favorited" : false,
"id" : {
"floatApprox" : 15061838001
}
How would I write a query which checks the created_at and finds all objects between 18:47 and 19:00? Do I need to update my documents so the dates are stored in a specific format?
Querying for a Date Range (Specific Month or Day) in the MongoDB Cookbook has a very good explanation on the matter, but below is something I tried out myself and it seems to work.
items.save({
name: "example",
created_at: ISODate("2010-04-30T00:00:00.000Z")
})
items.find({
created_at: {
$gte: ISODate("2010-04-29T00:00:00.000Z"),
$lt: ISODate("2010-05-01T00:00:00.000Z")
}
})
=> { "_id" : ObjectId("4c0791e2b9ec877893f3363b"), "name" : "example", "created_at" : "Sun May 30 2010 00:00:00 GMT+0300 (EEST)" }
Based on my experiments you will need to serialize your dates into a format that MongoDB supports, because the following gave undesired search results.
items.save({
name: "example",
created_at: "Sun May 30 18.49:00 +0000 2010"
})
items.find({
created_at: {
$gte:"Mon May 30 18:47:00 +0000 2015",
$lt: "Sun May 30 20:40:36 +0000 2010"
}
})
=> { "_id" : ObjectId("4c079123b9ec877893f33638"), "name" : "example", "created_at" : "Sun May 30 18.49:00 +0000 2010" }
In the second example no results were expected, but there was still one gotten. This is because a basic string comparison is done.
To clarify. What is important to know is that:
Yes, you have to pass a Javascript Date object.
Yes, it has to be ISODate friendly
Yes, from my experience getting this to work, you need to manipulate the date to ISO
Yes, working with dates is generally always a tedious process, and mongo is no exception
Here is a working snippet of code, where we do a little bit of date manipulation to ensure Mongo (here i am using mongoose module and want results for rows whose date attribute is less than (before) the date given as myDate param) can handle it correctly:
var inputDate = new Date(myDate.toISOString());
MyModel.find({
'date': { $lte: inputDate }
})
Python and pymongo
Finding objects between two dates in Python with pymongo in collection posts (based on the tutorial):
from_date = datetime.datetime(2010, 12, 31, 12, 30, 30, 125000)
to_date = datetime.datetime(2011, 12, 31, 12, 30, 30, 125000)
for post in posts.find({"date": {"$gte": from_date, "$lt": to_date}}):
print(post)
Where {"$gte": from_date, "$lt": to_date} specifies the range in terms of datetime.datetime types.
db.collection.find({"createdDate":{$gte:new ISODate("2017-04-14T23:59:59Z"),$lte:new ISODate("2017-04-15T23:59:59Z")}}).count();
Replace collection with name of collection you want to execute query
MongoDB actually stores the millis of a date as an int(64), as prescribed by http://bsonspec.org/#/specification
However, it can get pretty confusing when you retrieve dates as the client driver will instantiate a date object with its own local timezone. The JavaScript driver in the mongo console will certainly do this.
So, if you care about your timezones, then make sure you know what it's supposed to be when you get it back. This shouldn't matter so much for the queries, as it will still equate to the same int(64), regardless of what timezone your date object is in (I hope). But I'd definitely make queries with actual date objects (not strings) and let the driver do its thing.
Use this code to find the record between two dates using $gte and $lt:
db.CollectionName.find({"whenCreated": {
'$gte': ISODate("2018-03-06T13:10:40.294Z"),
'$lt': ISODate("2018-05-06T13:10:40.294Z")
}});
Using with Moment.js and Comparison Query Operators
var today = moment().startOf('day');
// "2018-12-05T00:00:00.00
var tomorrow = moment(today).endOf('day');
// ("2018-12-05T23:59:59.999
Example.find(
{
// find in today
created: { '$gte': today, '$lte': tomorrow }
// Or greater than 5 days
// created: { $lt: moment().add(-5, 'days') },
}), function (err, docs) { ... });
db.collection.find({$and:
[
{date_time:{$gt:ISODate("2020-06-01T00:00:00.000Z")}},
{date_time:{$lt:ISODate("2020-06-30T00:00:00.000Z")}}
]
})
##In case you are making the query directly from your application ##
db.collection.find({$and:
[
{date_time:{$gt:"2020-06-01T00:00:00.000Z"}},
{date_time:{$lt:"2020-06-30T00:00:00.000Z"}}
]
})
You can also check this out. If you are using this method, then use the parse function to get values from Mongo Database:
db.getCollection('user').find({
createdOn: {
$gt: ISODate("2020-01-01T00:00:00.000Z"),
$lt: ISODate("2020-03-01T00:00:00.000Z")
}
})
Save created_at date in ISO Date Format then use $gte and $lte.
db.connection.find({
created_at: {
$gte: ISODate("2010-05-30T18:47:00.000Z"),
$lte: ISODate("2010-05-30T19:00:00.000Z")
}
})
use $gte and $lte to find between date data's in mongodb
var tomorrowDate = moment(new Date()).add(1, 'days').format("YYYY-MM-DD");
db.collection.find({"plannedDeliveryDate":{ $gte: new Date(tomorrowDate +"T00:00:00.000Z"),$lte: new Date(tomorrowDate + "T23:59:59.999Z")}})
mongoose.model('ModelName').aggregate([
{
$match: {
userId: mongoose.Types.ObjectId(userId)
}
},
{
$project: {
dataList: {
$filter: {
input: "$dataList",
as: "item",
cond: {
$and: [
{
$gte: [ "$$item.dateTime", new Date(`2017-01-01T00:00:00.000Z`) ]
},
{
$lte: [ "$$item.dateTime", new Date(`2019-12-01T00:00:00.000Z`) ]
},
]
}
}
}
}
}
])
For those using Make (formerly Integromat) and MongoDB:
I was struggling to find the right way to query all records between two dates. In the end, all I had to do was to remove ISODate as suggested in some of the solutions here.
So the full code would be:
"created": {
"$gte": "2016-01-01T00:00:00.000Z",
"$lt": "2017-01-01T00:00:00.000Z"
}
This article helped me achieve my goal.
UPDATE
Another way to achieve the above code in Make (formerly Integromat) would be to use the parseDate function. So the code below will return the same result as the one above :
"created": {
"$gte": "{{parseDate("2016-01-01"; "YYYY-MM-DD")}}",
"$lt": "{{parseDate("2017-01-01"; "YYYY-MM-DD")}}"
}
⚠️ Be sure to wrap {{parseDate("2017-01-01"; "YYYY-MM-DD")}} between quotation marks.
Convert your dates to GMT timezone as you're stuffing them into Mongo. That way there's never a timezone issue. Then just do the math on the twitter/timezone field when you pull the data back out for presentation.
Why not convert the string to an integer of the form YYYYMMDDHHMMSS? Each increment of time would then create a larger integer, and you can filter on the integers instead of worrying about converting to ISO time.
Scala:
With joda DateTime and BSON syntax (reactivemongo):
val queryDateRangeForOneField = (start: DateTime, end: DateTime) =>
BSONDocument(
"created_at" -> BSONDocument(
"$gte" -> BSONDateTime(start.millisOfDay().withMinimumValue().getMillis),
"$lte" -> BSONDateTime(end.millisOfDay().withMaximumValue().getMillis)),
)
where millisOfDay().withMinimumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T00:00:00.000Z"
and
where millisOfDay(). withMaximumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T23:59:99.999Z"
i tried in this model as per my requirements i need to store a date when ever a object is created later i want to retrieve all the records (documents ) between two dates
in my html file
i was using the following format mm/dd/yyyy
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<script>
//jquery
$(document).ready(function(){
$("#select_date").click(function() {
$.ajax({
type: "post",
url: "xxx",
datatype: "html",
data: $("#period").serialize(),
success: function(data){
alert(data);
} ,//success
}); //event triggered
});//ajax
});//jquery
</script>
<title></title>
</head>
<body>
<form id="period" name='period'>
from <input id="selecteddate" name="selecteddate1" type="text"> to
<input id="select_date" type="button" value="selected">
</form>
</body>
</html>
in my py (python) file i converted it into "iso fomate"
in following way
date_str1 = request.POST["SelectedDate1"]
SelectedDate1 = datetime.datetime.strptime(date_str1, '%m/%d/%Y').isoformat()
and saved in my dbmongo collection with "SelectedDate" as field in my collection
to retrieve data or documents between to 2 dates i used following query
db.collection.find( "SelectedDate": {'$gte': SelectedDate1,'$lt': SelectedDate2}})

update array in mongoose which matches the condition

my schema looks like
{
qty:{
property1:{
//something
}
property2:[{
size:40,
color:"black",
enabled:"true"
}]
}
}
property 2 is array what i want to do is update those array object whose enabled is true in single query
I tried writing the following query
db.col.update({
"qty.property2.enabled" = "true"
}, {
"qty.property2.color" = "green"
}, callback)
but it is not working
error:
[main] Error: can't have . in field names [qty.pro.size]
db.col.update({"qty.property2.enabled":"true"},{$set: {'qty.property2.$.color': 'green'}}, {multi: true})
this is the way to update element inside array.
equal sign '=' cannot be used inside object
updating array is done using $
Alternative solution for multiple conditions:
db.foo.update({
_id:"i1",
replies: { $elemMatch:{
_id: "s2",
update_password: "abc"
}}
},
{
"$set" : {"replies.$.text" : "blah"}
}
);
Why
So I was looking for similar solution as this question, but in my case I needed array element to match multiple conditions and using currently provided answers resulted in changes to wrong fields.
If you need to match multiple fields, for example let say we have element like this:
{
"_id" : ObjectId("i1"),
"replies": [
{
"_id" : ObjectId("s1"),
"update_password": "abc",
"text": "some stuff"
},
{
"_id" : ObjectId("s2"),
"update_password": "abc",
"text": "some stuff"
}
]
}
Trying to do update by
db.foo.update({
_id:"i1",
"replies._id":"s2",
"replies.update_password": "abc"
},
{
"$set" : {"replies.$.text" : "blah"}
}
);
Would result in updating to field that only matches one condition, for example it would update s1 because it matches update_password condition, which is clearly wrong. I might have did something wrong, but $elemMatch solution solved any problems like that.
Suppose your documet looks like this.
{
"_id" : ObjectId("4f9808648859c65d"),
"array" : [
{"text" : "foo", "value" : 11},
{"text" : "foo", "value" : 22},
{"text" : "foobar", "value" : 33}
]
}
then your query will be
db.foo.update({"array.value" : 22}, {"$set" : {"array.$.text" : "blah"}})
where first curly brackets represents query criteria and second one sets the new value.

Calculating count and average with MongoDB aggregation

I have a simple db layout like this:
client
id
sex (male/female)
birthday (date)
client
id
sex (male/female)
birthday (date)
(...)
I'm trying to write an aggregation command that outputs how many male and female clients I've got, and I'd also like to output the average age of males and females, not sure I can do this in the same command or I need 2 separate ones?
// Count of males/females, average age
Clients.aggregate({
$project : {"sex" : 1,
"sexCount" : 1,
"birthday" : 1,
"avgAge" : 1
}
},
{
$match: {"sex": {$exists: true}}
},
{
$group: {
_id : "$sex",
sexCount : { $sum: 1 },
avgAge : { $avg: "$birthday" },
}
},
{ $sort: { _id: 1 } }
, function(err, sex_dbres) {
if (err)
throw err;
else{
(...)
}
});
With the code above I get the counts of male/female, but avgAge comes as 0. Any ideas?
Many thanks
The answer would be much simpler if you were storing age in the original document (as Dmitry posted, you could just do a straight avgAge:{$avg:"$age"} in your $group step.
Aggregation Framework is pretty nifty though and has many cool operators which allow you to compute this missing age field "on the fly".
I'm going to store each step of the aggregation in a variable so it's easier to see what's going on:
today = new Date();
// split today and bday into numerical year and numerical day-of-the-year
project1= {
"$project" : {
"sex" : 1,
"todayYear" : {
"$year" : today
},
"todayDay" : {
"$dayOfYear" : today
},
"by" : {
"$year" : "$bday"
},
"bd" : {
"$dayOfYear" : "$bday"
}
}
};
// calculate age in days by subtracting bday in days from today in days
project2 = {
"$project" : {
"sex" : 1,
"age" : {
"$subtract" : [
{
"$add" : [
{
"$multiply" : [
"$todayYear",
365
]
},
"$todayDay"
]
},
{
"$add" : [
{
"$multiply" : [
"$by",
365
]
},
"$bd"
]
}
]
}
}
};
// sum up for each sex the count and compute avg age (in days)
group = {
"$group" : {
"_id" : "$sex",
"total" : {
"$sum" : 1
},
"avgAge" : {
"$avg" : "$age"
}
}
};
// divide days by 365 to get age in years.
project3 = {
"$project" : {
"_id" : 0,
"sex" : "$_id",
"total" : 1,
"averageAge" : {
"$divide" : [
"$avgAge",
365
]
}
}
};
Now you can run the aggregation:
> db.client.find({},{_id:0})
{ "sex" : "male", "bday" : ISODate("2000-02-02T08:00:00Z") }
{ "sex" : "male", "bday" : ISODate("1987-02-02T08:00:00Z") }
{ "sex" : "female", "bday" : ISODate("1989-02-02T08:00:00Z") }
{ "sex" : "female", "bday" : ISODate("1993-11-02T08:00:00Z") }
> db.client.aggregate([ project1, project2, group, project3 ])
{
"result" : [
{
"sex" : "female",
"total" : 2,
"averageAge" : 21.34109589041096
},
{
"sex" : "male",
"total" : 2,
"averageAge" : 19.215068493150685
}
],
"ok" : 1
}
>
The reason this is not simple is currently Aggregation Framework does not support direct subtraction of dates. Please vote for https://jira.mongodb.org/browse/SERVER-6239 which is targeted for the next major release - once it's implemented it should allow subtraction of dates directly (though you will still need to convert it to appropriate granularity, years in this case probably).
The date object can't be "averaged", but numbers can. You can convert your dates to the timestamp value, and then find average from it. But still that won't be an average age, you'll need to subtract result from the current date outside of the aggregation function.
Another option is to assume that age can be calculated using only year part of the date (that is, if I was born on December 1, 2000, in today's report I'll be 12 years old, not 11). In this case you can use date operators to extract year value.
$project : {"sex" : 1,
"sexCount" : 1,
"year" : {$year: "$birthday"},
}
},
$project : {"sex" : 1,
"sexCount" : 1,
"age" : {$subtract: [2012, '$year']},
}
},

Resources