Cron job that checks for date expiration - node.js

Let's take the following example:
I have two mongoDb collections named user and products. A user can add unlimited products, but each product expires in 5 days and it has to be dynamically removed from the db when it expires.
user.js
const UserSchema = mongoose.Schema(
{
username: { type: String },
email: { type: String },
timezone: { type: String },
products: [{ type: Schema.Types.ObjectId, ref: 'Products' }]
}
)
export default mongoose.model('User', UserSchema);
products.js
var RunSchema = mongoose.Schema(
{
name: { type: String },
completedAt: { type: Date }
}
)
export default mongoose.model('Run', RunSchema);
Is there another way of doing this in nodejs rather than having one cron job which runs every day and checks all documents from the products collection?
I'm thinking about a solution where a each user can have a cron job that starts once they add a product...

It is too late answer the question even though , I would like to give alternative operation with minimal operation and without Cron Job
If you want remove product document after creation of 5 day's interval then you can achieve with 2 line of code in model itself.
You have add one property i.e(expireAt) in model and pass the interval to that property with help of moment npm library
var RunSchema = mongoose.Schema(
{
name: { type: String },
completedAt: { type: Date },
expireAt: {
type: Date,
default: moment(new Date()).add("5", "days"),
expires: 432000000
}
}
)
export default mongoose.model('Run', RunSchema);
after creation of product document date and time . it will automatically remove document from collection of every 5 days of interval.

First method:
You can add a cron in NodeJs and it will execute your task every x
Example:
Add the package cron in your project.
npm i cron -S
Create a file cron.js
'use strict';
let express = require('express'),
CronJob = require('cron').CronJob,
moment = require('moment'),
mongoose = require('mongoose');
Collection = mongoose.model('Collection');
var collection = {
clean : function() {
// Each hour
new CronJob('0 * * * *', function() {
console.log("Hello I'm Cron!");
// Delete all the product expired of your db
// Your code ...
}, null, true, 'Europe/Paris');
}
}
module.exports = {
init : function() {
return (
collection.clean()
)
}
}
In your server.js, add the following code to initialize the cron
/**
* Init cron tasks
*/
var cron = require('./cron');
cron.init()
Second method:
Setting expiry time in your model, check that answer for more informations.

Related

Mongoose delete records after certain time

I have a mongoose Schema which looks like this:
const USERS_DATA = new Schema({
_id: Number,
name: String,
img: String,
date: Date,
phone: String,
article: String,
createdAt: {
type: Date,
required: true,
default: Date.now,
index: { expires: '3d' }
}
},
{
collection: "users",
_id: false,
}
);
I need to push data to this schema.
const User = mongoose.model("users", USERS_DATA);
function pushToDB() {
const newUser = new User({
name: INPUT.name,
img: INPUT.img,
date: INPUT.date,
phone: INPUT.phone,
article: INPUT.article,
});
newUser.save(function (err) {
mongoose.disconnect();
if (err) return console.log(err);
});
}
This data have to be deleted after 3 days when it was pushed to database. How to implement it in node.js? I found it really confusing and tried lots of code. Any answers are appreciated! Thanks
P.S. I use mongoDb Atlas
You should separate the process to push the data into the database from the process to delete it after 3 days. You have already the first part :).
For the second part, you can write a function deleteOldDocument. This function will query for documents in DB that are created for 3 days or more, and delete them. Then, you can run this function periodically, 1 time per day for example.
The pseudo-code, in case you need it :
async function deleteOldDocument() {
const 3DaysAgo = ...; // here you can subtract 3 days from now to obtain the value
// search for documents that are created from 3 days or more, using $lt operator
const documentToDelete = await User.find({"created_at" : {$lt : 3DaysAgo }});
// delete documents from database
.....
// recall the function after 1 days, you can change the frequence
setTimeOut(async function() {
await deleteOldDocument();
}), 86400);
}
// call deleteOldDocument to start the loop
deleteOldDocument();

Making Mongodb documents inactive after certain time

Let's say I am creating a web application that posts event markers to a map to help flash mobs find out where to meet up. In this hypothetical app and due to the nature of flash mobs, these markers would have to become inactive after a certain time interval so as to not pollute the map with events that are no longer occurring. My question is what is the appropriate way to allow this to happen in mongoDB using node.js without deleting the document?
If I have a mongoose model (Marker) like this:
const mongoose = require('mongoose');
const markerSchema = new mongoose.Schema({
name: {
type: String,
},
description: {
type: String,
trim: true,
},
createdAt: {
type: Date,
default: Date.now(),
select: false,
},
expiresAt: {
type: Date
},
active: Boolean,
location: {
type: {
type: String,
enum: ['Point'],
required: true,
},
coordinates: {
type: [Number],
required: true,
},
},
});
markerSchema.pre('save', function (next) {
this.expiresAt = this.createdAt + 60 * 60 * 1000 * 6;
next();
});
const Marker = mongoose.model('Marker', tourSchema);
module.exports = Marker;
What would be the best way to update the markers active field to false after an arbitrary amount of time, say 6 hours?
I have researched Mongodb's TTL and I have also considered using the createdAt field to query for markers that are within the time frame of active using Marker.find({ expiresAt: {$gt: Date.now() }) but I do not know if this would be the best. Is there a way to run a function or middleware that periodically checks each document and sets the active field to false if the time frame is up? Just curious how you would approach this problem so querying Marker.find({ active: true }) would provide the data I am looking for.
Setting TTL in MongoDB runs a scheduled task every 60 seconds and will remove the document once expired. If you don't want to remove the document and mark it instead.
Run a CRON/Scheduled task every minute
Update the document matching your custom TTL criteria
Use the criteria in all select clause to ignore expired documents.
You can use node-cron to achieve your goal.
For example, rub a job every minute :
cron.schedule('* * * * *', function() {
console.log('running a task every minute');
});
More details and full usage example tutorials :
https://www.digitalocean.com/community/tutorials/nodejs-cron-jobs-by-examples
https://blog.logrocket.com/task-scheduling-or-cron-jobs-in-node-using-node-cron/

Why wouldn’t mongoDB update the “Date” field of my user entry?

I created a User schema in my React App as follows:
const userSchema = new Schema(
{
profileId: String,
expirationDate: { type: Date, default: new Date() },
credits: { type: Number, default: 0 },
},
{ timestamps: { createdAt: "created_at" } }
);
When the user pays me, I want to reset/update two fields: the expirationDate and credits via a post method. Here’s the code I use on my expressjs backend server to update the database entry on MongoDB Atlas:
req.user.expirationDate = new Date(
req.user.expirationDate.setDate(req.user.expirationDate.getDate() + 30)
);
req.user.credits += 1;
const user = await req.user.save();
res.send(user);
Once the operation succeeded, I can see the field of “credits” gets updated (increased by 1). However, the “expirationDate” field remains unchanged. What’s more curious is that when I send the updated user object to my frontend server with “res.send(user)”, I can see the updated expirationDate in the console.
Successfully updated user model as designed/intended: seen from frontend console
But below is what I saw in my mongoDB:
Updated user entry in MongoDB: the Date field "expirationDate" is not updated; but, the "credits" field is.
What is going on here? How to fix it?
I was having a similar issue recently and haven't figured out the actual reason behind this, but as a workaround try telling mongoose explicitly that the expirationDate-field was changed:
req.user.expirationDate = new Date(
req.user.expirationDate.setDate(req.user.expirationDate.getDate() + 30)
);
req.user.markModified('expirationDate');
await req.user.save();
EDIT:
Just debugged it again and I think the reason behind this behaviour is your default value for expirationDate. Try passing it the Date.now function instead of immediately setting it to a new date:
expirationDate: {type: Date, default: Date.now},
This fixed it for me without having to use markModified().
Although we still have some unsolved issues on why the same set of codes works differently. I have decided not to deal with it for the moment. Here's my solution to the original problem: change the datatype from Date to String. Here's the new set of codes:
Creating User schema at the frontend:
const userSchema = new Schema(
{
profileId: String,
expirationDate: { type: String, default: new Date().toDateString() },
credits: { type: Number, default: 0 },
},
{ timestamps: { createdAt: "created_at" } }
);
Updating user at the backend to MongoDB Atlas::
d = new Date(req.user.expirationDate);
req.user.expirationDate = new Date(
d.setDate(d.getDate() + 30)
).toDateString();
req.user.credits += 1;
const user = await req.user.save();
console.log(typeof req.user.expirationDate);//Checking the datatype of "expirationDate"

How can I set correct timezone for a date field in Mongoose?

I'm using moment for generating time and date:
const moment = require('moment-timezone');
const emailModel = require('./api/models/emails');
sentTime=moment().tz('America/Los_Angeles').format();
console.log(sentTime); //console log shows correct time
emailModel.findOneAndUpdate({ _id: emailInfo._id }, {sentTime: sentTime }, { upsert: true },function (err, doc) {
if (err)
console.log(err);
});
And this is Schema that I'm using mongoose :
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const EmailSchema = new Schema({
.
.
.
sentTime: {
type: Date,
trim: true
}
.
.
.
});
Problem is:
Console log shows correct time 2020-01-07T12:23:00-08:00 BUT mongoose saved incorrect timezone in DB : 2020-01-07T20:23:01.000+00:00
Currently the default behavior of Mongodb is to: (From the docs)
MongoDB stores times in UTC by default, and will convert any local
time representations into this form.
As a solution (and rightly so) what they recommend is:
Applications that must operate or report on some unmodified local time
value may store the time zone alongside the UTC timestamp, and compute
the original local time in their application logic.
Update:
Since you are already using moment-timezone a simple way I would go about this is:
Change the EmailSchema to have a timezone field and create a Mongoose virtual field on that schema to get adjusted time.
const schemaOpts = { toJSON: { virtuals: true } };
const EmailSchema = new Schema(
{
sentTime: {
type: Date,
trim: true
},
timeZone: {
type: String
}
},
schemaOpts
);
EmailSchema.virtual("adjustedTime").get(function() {
return moment.tz(this.sentTime, this.timeZone).format();
});
//fetching data
const result = await EmailSchema.findOne({}).exec();
console.info("result::", result.toJSON());
//note that if using .lean() for performance which has a caveat on using .toJSON()
trick for this case is before save, you need to add time with date. Ex: 2021/01/02 ==> 2021/01/02 15:00:00, ofcouse hour is always equal or greater than 04:00:00. Becase without time, date will be 00:00:00 and mongo will convert it to default timezone and substract hour with 4.

Time to live in mongodb, mongoose dont work. Documents doesnt get deleted

Im using this scheme for a session in my node.js app
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
// define the schema for our user session model
var UserSessionSchema = new Schema({
sessionActivity: { type: Date, expires: '15s' }, // Expire after 15 s
user_token: { type: String, required: true }
});
module.exports = mongoose.model('UserSession', UserSessionSchema);
And I create a "session" in my app with:
...
var session = new Session();
session.user_token = profile.token;
session.save(function(save_err) {
if (save_err) {
....
} else {
// store session id in profile
profile.session_key = session._id;
profile.save(function(save_err, profile) {
if (save_err) {
...
} else {
res.json({ status: 'OK', session_id: profile.session_id });
}
});
...
The problem is that the document lives permanetly, its never expires. It should only live for 15 seconds (up to a minute). Whats wrong with my code? I have tried to set the expries: string to a number i.e 15, to a string '15s' and so on.
var UserSessionSchema = new Schema({
sessionActivity: { type: Date, expires: '15s', default: Date.now }, // Expire after 15 s
user_token: { type: String, required: true }
});
A TTL index deletes a document 'x' seconds after its value (which should be a Date or an array of Dates) has passed. The TTL is checked every minute, so it may live a little longer than your given 15 seconds.
To give the date a default value, you can use the default option in Mongoose. It accepts a function. In this case, Date() returns the current timestamp. This will set the date to the current time once.
You could also go this route:
UserSessionSchema.pre("save", function(next) {
this.sessionActivity = new Date();
next();
});
This will update the value every time you call .save() (but not .update()).
To double check the indexes that have been created in the DB you can run this command in your mongo shell db.yourdb.getIndexes(). When changing the indexes you have to manually delete it in the collection before the new one will take effect. Check here for more information Mongoose expires property not working properly

Resources