Mongoose Birthday query - node.js

I store the date of birth (dob) within a user model and I would like to check if it is the users birthday today. How do I do query that?
I have started to try, but obviously failed
// User Model
var UserSchema = new Schema({
name: String,
email: { type: String, lowercase: true },
role: {
type: String,
default: 'user'
},
hashedPassword: String,
dob: { type: Date, default: new Date() },
joinedOn: { type: Date, default: new Date() },
leftOn: { type: Date },
position: { type: String },
provider: String,
salt: String,
google: {},
github: {}
});
// Birtdhays
exports.birthdays = function(req, res, next) {
var todayStart = moment().startOf('day');
var todayEnd = moment().endOf('day');
User
.find()
.where('dob').gt(todayStart).lt(todayEnd)
.limit(3)
.sort('dob')
.select('name dob')
.exec(function(err, users){
if(err) return res.send(500, err);
res.json(200, users);
});
};

Presuming that you have birthdays stored as a date object in your documents then they probably look something like this:
{
"name": "Neil",
"dob": ISODate("1971-09-22T00:00:00Z")
}
So it's not just the "day" but the full year as well from the originally selected day of birth. This probably seems like a logical way to store a date of birth and it is a useful date object for many purposes. But how to query on that? Any date derived from the current year is not going to match that value within a range and other users data on the same day can occur in different years.
There is some JavaScript date manipulation you can do in order to deal with this though, and also some functionality of MongoDB in the aggregation framework ( or alternately using JavaScript in mapReduce ), to get data out of the date that is useful for matching.
Firstly you can look at the $dayOfyear operator and code to get that from the current date to use as a match:
var now = new Date();
var start = new Date( now.getFullYear() + "-01-01" );
var diff = now - start;
var oneDay = 1000 * 60 * 60 * 24;
var dayOfYear = Math.floor(diff / oneDay);
User.aggregate(
[
{ "$project": {
"name": 1,
"dob": 1,
"dayOfYear": { "$dayOfYear": "$dob" }
}},
{ "$match": { "dayOfYear": dayOfYear }
],
function(err,users) {
// work here
}
)
Now that's all fine, or it seems. But of course what happens when there is a leap year? All days after February 28 are moved forward one. You could account for this in other ways, but how about just using the "day" and "month" to do the match on instead:
var now = new Date();
var day = now.getDate(); // funny method name but that's what it is
var month = now.getMonth() + 1; // numbered 0-11
User.aggregate(
[
{ "$project": {
"name": 1,
"dob": 1,
"day": "$dayOfMonth",
"month": "$month"
}},
{ "$match": {
"day": day,
"month": month
}}
],
function(err,users) {
// work here
}
)
And the aggregation operators for $dayOfMonth and $month help there.
So that's all better and you can query for the "birthday" now, but there is something still not really right. Whilst the query will work, it's clearly not really efficient. Since what you are doing here is running through all of the results in the collection and manipulating the existing dates in order to extract the parts to perform a match.
Ideally you don't want to do this, and have the "query" itself target the current "birthdays" in a simple stroke to avoid doing this manipulation to get a match. This is where modelling comes in, and where you should consider adding more data to your document where queries like this are common:
{
"name": "Neil",
"dob": ISODate("1971-09-22T00:00:00Z"),
"day": 22,
"month": 9
}
Then it's easy to query on this as both "day" and "month" fields can also be indexed to further improve performance and avoid scanning the whole collection for matches:
var now = new Date();
var day = now.getDate(); // funny method name but that's what it is
var month = now.getMonth() + 1; // numbered 0-11
User.find({ "day": day, "month": month },function(err,users) {
// work here
})
So those are the considerations. Either accept the manipulation with the aggregation framework ( or possibly mapReduce ), or where you are going to frequently use such a query and/or have many items in the collection then add additional fields to your document schema that can be used as a data point directly in the query itself.

I solved it in the following way:
Added the fields birthday and brithmonth to the user model and added a pre save hook to keep them updated on any changes to the birthday:
UserSchema
.pre('save', function(next) {
this.birthmonth = this.dob.getMonth() + 1;
this.birthday = this.dob.getDate()
next();
});
Thanks Neil for the inspiration!

Related

Mongoose query does not return values when comparing string digits

For the last 9 hours I`m trying to fix one bug so I have code
let query = User.find();
if(req.query.from){
query = query.find({date: {$gte: req.query.from}});
}
if(req.query.to){
query = query.find({date: {$lte: req.query.to}});
}
const User = await query;
res.status(200).json({status:'Success', User})
And the problem is when I do for example
req.query.from = 1
req.query.to = 9
Everything works but when I do
req.query.from = 1
req.query.to = 11
It doesn't return anything.
Single digit numbers can be query but If I want to mix single digit with multi digit then doesn't work.
Can anyone help me solve this problem
Edit:
My user Schema
const userSchema= new mongoose.Schema({
date: {
type:String
},
title: {
type:String
},
description:{
type:String
},
location:{
type:String
}
});
The comparison query operators $gt, $gte, $lt and $lte do not generally work well with string digits as types in MongoDB are strict and do not auto-convert like they would do
in other systems. To compare against a number with $gt/$gte or $lt/$lte, the values in your document also need to be a number. Thus you have an option of changing
the schema design of the Date field to be an int or double.
For your case, since the date field is already of String type, you can utilise the $expr operator to convert the field within the query as it works well with aggregate operators such as $toInt or $toDouble. You would also need to parse the parameter values to integer for correct comparison with the query.
You would thus in the end require a query like this should you decide to stick with the existing schema:
const users = await User.find({
'$expr': {
'$and': [
{ '$gte': [{ '$toInt': '$date' }, 1 ] },
{ '$lte': [{ '$toInt': '$date' }, 11 ] }
]
}
})
The following query derivation returns the desired results with the existing conditions:
const toIntExpression = { '$toInt': '$date' }
const query = {
'$expr': {
'$and': []
}
}
if (req.query.from) {
const from = parseInt(req.query.from, 10)
query['$expr']['$and'].push({
'$gte': [toIntExpression, from]
})
}
if (req.query.to) {
const to = parseInt(req.query.to, 10)
query['$expr']['$and'].push({
'$lte': [toIntExpression, to]
})
}
const users = await User.find(query)

mongoose query regex to ignore middle character

I am working with mongodb and my data is stored as day:"3/6/2020" i want to query the string that matches day:"3/2020" without the middle value or omitting the 6 in day:"3/6/2020".
such as
`myModel.find({"day": {$regex: "3/2020", $options:" what option to pass here to ignore the `
` middle value"}});`
or any better way
`model.find({"name": {$regex: search, $options:" regex options"}}, (err, users)=> {`
` res.status(200).send({users});`
"3/2020" to match any record with 3 and 2020 just as this "3/2020" matches with "3/6/2020"
I guess you could use aggregation framework to convert the string date in to proper date, fetch months and years and match against that properly. Something like this
model.aggregate([{
"$addFields": {
"convertedDate": {
"$toDate": "$myDate" // mydate is your field name which has string
}
}
}, {
"$addFields": {
"month": {
"$month": "$convertedDate"
},
"year": {
"$year": "$convertedDate"
}
}
}, {
"$match": {
"month": 3, // this is your search criteria
"year": 2020 // // this is your search criteria
}
}, {
"$project": {
"month": 0, // Do not take temp added fields month and year
"year": 0 // Do not take temp added fields month and year
}
}])
This might look like a big query but I guess much better than doing string comaprision using regex. If your field is saved in date format, you could also remove the first stage in which you are doing $toDate. Hope this helps
That might help you. We match the 3, than a number wich is one or two characters wide, then at the end 2020.
3\/\d{1,2}\/2020
https://regex101.com/r/jn4zwa/1
Solved. i solved it splitting my date as in my schema
```
qDay :{
type:Number, default: new Date().getDate(Date.now)
},
qMonth :{
type:Number, default: new Date().getMonth(Date.now) + 1
},
qYear :{
type:Number, default: new Date().getFullYear(Date.now)
}```
my query
```const getByDay = async (req, res)=> {
console.log(req.body);
merchantModel.find({$and:[{qMonth : req.body.month}
,{qYear: req.body.year},{qDay:req.body.day}]}).then((record)=> {
if(record.length == 0){
res.status(404).send({msg:'no record!'});
}else{
co`enter code here`nsole.log(record);
res.status(200).send({record:record});
}
});
}```

mongodb date insert like using mongoose

I am inserting bulk records in mongodb. I am using the native DB drivers to do this, as the performance is much higher. At other points in my application, i am using mongoose. The problem I am having is that mongoose translates the date into a different format whereas mongodb native just inserts it as the number of seconds since 1970. So later queries in mongoose based off that date do not work.
Here's my mongoose schema:
var MySchema = new Schema({
name : { type: String, required: true },
updatedAt : Date
});
And my mongo db mass insert:
var newRec = {
name : entry.Name,
updatedAt : Date.now
};
newRecords.push(newRec);
MySchema.collection.insert(newRecords, function(err, newRecs) {
res.json(newRecs.ops);
});
This produces in the DB:
{
"_id": {
"$oid": "562818ecf24d540f0053a38d"
},
"name": "Cool Record",
"updatedAt": 12312423512
}
Whereas if it was run through Mongoose it would produce:
{
"_id": {
"$oid": "561fd90285b5e73f5626f74e"
},
"name": "Cool Record",
"updatedAt": {
"$date": "2015-10-20T20:01:17.553Z"
}
}
If going through mongoose, queries like this work well:
MySchemda.find({ updatedAt : { $gt: lastSynced }}).exec();
But do not work otherwise.
Date.now is a number representing milliseconds since 1970. While it conceptually represents a date, it isn't actually a Date:
var x = Date.now;
typeof x;
// "number"
You need to switch your schema to be:
var MySchema = new Schema({
name : { type: String, required: true },
updatedAt : Number
});
alternately, you can use:
var newRec = {
name: entry.Name,
updatedAt: new Date()
}
and keep your schema as it is.
You may use it like this:
Define the date type and the default value, then create a variable which can be defined as your schema use year : new Date this would help a lot to set the date.
var songSchema = new mongoose.Schema({
name : String,
year : {type : Date, default : Date.now},
singer : String,
});
var song = mongoose.model("song", songSchema);
var xyz= new song({
name : "abcd",
year :new Date, // to set the date first set new Date
singer : "aabbccdd",
});
You can use setDate , setMonth, setYear method to solve the issues.There are more methods defined under the object year .You can out further at the documentation of mongoose.
xyz.save(function(err, songs){
if(err)
winston.log("Something is wrong"+ " "+ err);
else {
songs.year.setDate(03);
songs.year.setMonth(03);
songs.year.setYear(2015);
winston.log(songs);
}
});

Insert or update object element in array [duplicate]

I'm new to MongoDB and Mongoose and I'm trying to use it to save stock ticks for daytrading analysis. So I imagined this Schema:
symbolSchema = Schema({
name:String,
code:String
});
quoteSchema = Schema({
date:{type:Date, default: now},
open:Number,
high:Number,
low:Number,
close:Number,
volume:Number
});
intradayQuotesSchema = Schema({
id_symbol:{type:Schema.Types.ObjectId, ref:"symbol"},
day:Date,
quotes:[quotesSchema]
});
From my link I receive information like this every minute:
date | symbol | open | high | low | close | volume
2015-03-09 13:23:00|AAPL|127,14|127,17|127,12|127,15|19734
I have to:
Find the ObjectId of the symbol (AAPL).
Discover if the intradayQuote document of this symbol already exists (symbol and date combination)
Discover if the minute OHLCV data of this symbol exists on the quotes array (because it could be repeated)
Update or create the document and update or create the quotes inside the array
I'm able to accomplish this task without veryfing if the quotes already exists, but this method can creates repeated entries inside quotes array:
symbol.find({"code":mySymbol}, function(err, stock) {
intradayQuote.findOneAndUpdate({
{ id_symbol:stock[0]._id, day: myDay },
{ $push: { quotes: myQuotes } },
{ upsert: true },
myCallback
});
});
I already tried:
$addToSet instead of $push, but unfortunatelly this doesn't seems to work with array of documents
{ id_symbol:stock[0]._id, day: myDay, 'quotes["date"]': myDate } on the conditions of findOneAndUpdate; but unfortunatelly if mongo doesn't find it, it creates a new document for the minute instead of appending to the quotes array.
Is there a way to get this working without using one more query (I'm already using 2)? Should I rethink my Schema to facilitate this job? Any help will be appreciated. Thanks!
Basically put an $addToSet operator cannot work for you because your data is not a true "set" by definition being a collection of "completely distinct" objects.
The other piece of logical sense here is that you would be working on the data as it arrives, either as a sinlge object or a feed. I'll presume its a feed of many items in some form and that you can use some sort of stream processor to arrive at this structure per document received:
{
"date": new Date("2015-03-09 13:23:00.000Z"),
"symbol": "AAPL",
"open": 127.14
"high": 127.17,
"low": 127.12
"close": 127.15,
"volume": 19734
}
Converting to a standard decimal format as well as a UTC date since any locale settings really should be the domain of your application once data is retrieved from the datastore of course.
I would also at least flatten out your "intraDayQuoteSchema" a little by removing the reference to the other collection and just putting the data in there. You would still need a lookup on insertion, but the overhead of the additional populate on read would seem to be more costly than the storage overhead:
intradayQuotesSchema = Schema({
symbol:{
name: String,
code: String
},
day:Date,
quotes:[quotesSchema]
});
It depends on you usage patterns, but it's likely to be more effective that way.
The rest really comes down to what is acceptable to
stream.on(function(data) {
var symbol = data.symbol,
myDay = new Date(
data.date.valueOf() -
( data.date.valueOf() % 1000 * 60 * 60 * 24 ));
delete data.symbol;
symbol.findOne({ "code": symbol },function(err,stock) {
intraDayQuote.findOneAndUpdate(
{ "symbol.code": symbol , "day": myDay },
{ "$setOnInsert": {
"symbol.name": stock.name
"quotes": [data]
}},
{ "upsert": true }
function(err,doc) {
intraDayQuote.findOneAndUpdate(
{
"symbol.code": symbol,
"day": myDay,
"quotes.date": data.date
},
{ "$set": { "quotes.$": data } },
function(err,doc) {
intraDayQuote.findOneAndUpdate(
{
"symbol.code": symbol,
"day": myDay,
"quotes.date": { "$ne": data.date }
},
{ "$push": { "quotes": data } },
function(err,doc) {
}
);
}
);
}
);
});
});
If you don't actually need the modified document in the response then you would get some benefit by implementing the Bulk Operations API here and sending all updates in this package within a single database request:
stream.on("data",function(data) {
var symbol = data.symbol,
myDay = new Date(
data.date.valueOf() -
( data.date.valueOf() % 1000 * 60 * 60 * 24 ));
delete data.symbol;
symbol.findOne({ "code": symbol },function(err,stock) {
var bulk = intraDayQuote.collection.initializeOrderedBulkOp();
bulk.find({ "symbol.code": symbol , "day": myDay })
.upsert().updateOne({
"$setOnInsert": {
"symbol.name": stock.name
"quotes": [data]
}
});
bulk.find({
"symbol.code": symbol,
"day": myDay,
"quotes.date": data.date
}).updateOne({
"$set": { "quotes.$": data }
});
bulk.find({
"symbol.code": symbol,
"day": myDay,
"quotes.date": { "$ne": data.date }
}).updateOne({
"$push": { "quotes": data }
});
bulk.execute(function(err,result) {
// maybe do something with the response
});
});
});
The point is that only one of the statements there will actually modify data, and since this is all sent in the same request there is less back and forth between the application and server.
The alternate case is that it might just be more simple in this case to have the actual data referenced in another collection. This then just becomes a simple matter of processing upserts:
intradayQuotesSchema = Schema({
symbol:{
name: String,
code: String
},
day:Date,
quotes:[{ type: Schema.Types.ObjectId, ref: "quote" }]
});
// and in the steam processor
stream.on("data",function(data) {
var symbol = data.symbol,
myDay = new Date(
data.date.valueOf() -
( data.date.valueOf() % 1000 * 60 * 60 * 24 ));
delete data.symbol;
symbol.findOne({ "code": symbol },function(err,stock) {
quote.update(
{ "date": data.date },
{ "$setOnInsert": data },
{ "upsert": true },
function(err,num,raw) {
if ( !raw.updatedExisting ) {
intraDayQuote.update(
{ "symbol.code": symbol , "day": myDay },
{
"$setOnInsert": {
"symbol.name": stock.name
},
"$addToSet": { "quotes": data }
},
{ "upsert": true },
function(err,num,raw) {
}
);
}
}
);
});
});
It really comes down to how important to you is it to have the data for quotes nested within the "day" document. The main distinction is if you want to query those documents based on the data some of those "quote" fields or otherwise live with the overhead of using .populate() to pull in the "quotes" from the other collection.
Of course if referenced and the quote data is important to your query filtering, then you can always just query that collection for the _id values that match and use an $in query on the "day" documents to only match days that contain those matched "quote" documents.
It's a big decision where it matters most which path you take based on how your application uses the data. Hopefully this should guide you on the general concepts behind doing what you want to achieve.
P.S Unless you are "sure" that your source data is always a date rounded to an exact "minute" then you probably want to employ the same kind of date rounding math as used to get the discrete "day" as well.

How to create subdocument on mongodb dynamically

I have a mongodb database with a collection as follow:
var mongoose = require('mongoose');
var journalSchema = mongoose.Schema({
title : String,
journalid: {type:String, index: { unique: true, dropDups: true }},
articleCount : type:Number, default:1,
});
module.exports = mongoose.model('Journal', journalSchema);
Now that my database is growing, I would like to have a "articles count" field per year.
I could make an array as follow years : [{articleCount : Number}] and fill it up by accessing journal.years[X] for a specific year, where X correspond to 0 for 1997, 1 for 1998, etc..
However, my data are scrapped dynamically and I would like to have a function in my express server where articleCountis increased based on the year.
For example:
function updateYear(journalid, year, callback) {
Journal.findOneAndUpdate(
{'journalid':journalid},
{$inc : {'articleCount' : 1}}, // DO SOMETHING WITH year HERE
function() {
callback();
});
}
This does increase the article count but I don't know where to include the "year"...
What would be the fastest way of doing that, knowing that I have to fetch through quite a lot of articles (10 millions +) and I would like to be able to get/update the article count for a given year efficiently.
Hope I'm clear enough, Thanks!
Make your array a set of objects with a year and count:
journalYears: [
{
year: String, // or Number
count: Number
}
]
e.g.
journalYears: [
{ year: "2014", count: 25 },
{ year: "2015", count: 15 }
]
Then for your update:
function updateYear(journalId, year, callback) {
Journal.findOneAndUpdate(
{_id: journalId, "journalYears.year": year},
{ $inc: { "journalYears.$.count": 1 } },
callback
);
}
The index of the first match from your query is saved in $. It's then used to update that specific element in your array.

Resources