I'm using MongoDB, Nodejs, etc...
I have an Activity collection about my user activities "Add new project", "Update project", "Delete project", etc, but I don't want that activity permanent on my database, I want that to expire in 30 days after created and added do user dashboard, but I don't now how to do...Hope you can help me!
Thank you!
Here's my Activity Schema:
let mongoose = require("mongoose");
let activitySchema = new mongoose.Schema({
activity: String,
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
created: {
type: Date,
default: Date.now
}
})
module.exports = mongoose.model('Activity', activitySchema);
Here's my server side:
(this is what send all user activity to dashboard)
//Render User Activity
exports.render_user_activity = (req, res) => {
let userid = req.user._id;
User.findById(userid)
.exec((err, user) => {
if (err) {
console.log(err);
}
Activity.find({ user: user })
.exec((err, activities) => {
if (err) {
console.log(err);
}
res.send(activities);
});
});
};
I've read that I can user .createIndex but I don't know how or if it's the best solution..
Thank you!
You can for sure use TTL (Time to Live) index in MongoDB.
With this TTL you can remove any document after certain amount of time is expired or at a particular date and time in calendar.
As per your question you want to expire each document after 30 days of creation.
You already have a created field just create an index around it.
let activitySchema = new mongoose.Schema({
.
.
.
.
}, {timestamps: true});
activitySchema.index({createdAt: 1},{expireAfterSeconds: 2592000});
30 days = 2592000 seconds
Related
I have created two models in my app- one for User (_id, email, username, password) and one for Expense (_id, date, detail, amount, category). For the users, I have finished the authentication with jwt.
I want logged-in users to be able to add/remove expenses and not show their expenses to other users but I don't know how I can implement that. I am not asking for code- I would be grateful if you could roughly tell me what I need to do. Thanks!
//expense schema
const expenseSchema = new mongoose.Schema(
{
date: Date,
detail: String,
amount: Number,
category: String
}
)
//controller for adding expenses
const addExpenseController = (req, res) => {
const expense = new Expense({
"date": new Date(),
"amount": req.body.amount,
"detail": req.body.detail,
"category": "expense"
});
expense.save();
res.send('expense added');
};
You should define a ref property in the expense schema pointing at the User model (change the value of the ref attribute to equal the model name given to the users):
const expenseSchema = new mongoose.Schema(
{
...
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}
}
)
Then, on creation, specify the user by setting the value of its _id.
You can either store it in the session or pass it in the body, depending on your implementation:
const addExpenseController = async (req, res) => {
try {
const expense = new Expense({
date: new Date(),
amount: req.body.amount,
detail: req.body.detail,
category: 'expense',
user: req.session.user_id, // or req.body.user_id
});
await expense.save();
res.send('expense added');
} catch (err) {
res.send('server error');
}
};
I currently have data saving and expiring to/from a database via a mongoose schema like so:
var UserSchema = new Schema({
createdAt: { type: Date, expires: '1m' },
name: String,
email: String
});
The only problem is that the document that's saved to the database is completely removed from the database. How would I refactor the above so that the name/email address stay in the database but if the user attempts to login after their expiry date then they're greeted with a message saying 'session has expired, renew session'. (or something similar)
I'm wanting to do it this way because then if the user logs in with an expired email address the server is still able to lookup the email address and spit out a "expired session" message rather than a "not found" error which is what happens when data is deleted.
So to reiterate, how do I keep expired data in a mongo/mongoose database so the app is able to find the email address the user is attempting to login with but if their session has expired they need to renew the session?
You should use concept of Schema Reference for this. Save your expired field in another table and join your main user_table and expire_table(wxample name)
var UserSchema = new Schema({
name: String,
email: String
});
//save date by-default
//expire in 1 min as in your example
var expireSchema = new Schema({
createdAt: { type: Date, default: Date.now, expires: '1m' },
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire'}
});
var userTable = mongoose.model('user_expire', UserSchema);
var expireTable = mongoose.model('expireMe', expireSchema);
//Save new user
var newUser = new userTable({
name: 'my_name',
email: 'my_email'
});
newUser.save(function(err, result) {
console.log(result, 'saved')
var newExpire = new expireTable({
user_pk:result._id
});
//use _id of new user and save it to expire table
newExpire.save(function(err, result) {
console.log('saved relation')
})
})
Now to detect whether session has expired or not
1. on executing this code before data gets expired
expireTable.findOne()
.populate('user_pk')
.exec(function (err, result) {
if (err) throw err;
console.log(result)
if(result == null) {
console.log('session has expired, renew session')
} else {
console.log('session is active')
}
});
//output - session is active
2. on executing this code after data gets expired
expireTable.findOne()
.populate('user_pk')
.exec(function (err, result) {
if (err) throw err;
console.log(result)
if(result == null) {
console.log('session has expired, renew session')
} else {
console.log('session is active')
}
});
//output - session has expired, renew session
The accepted answer is good, but with Mongo 3.0 and above, the
createdAt: { type: Date, default: Date.now, expires: '1m' }
does not work for me.
Instead I used
var expireSchema = new Schema({
expireAt: {
type: Date,
required: true,
default: function() {
// 60 seconds from now
return new Date(Date.now() + 60000);
}
},
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire'}
});
More info is here: Custom expiry times for mongoose documents in node js
EDIT
My comment above would also require invoking a Mongo function directly rather than via Mongoose syntax. This would be something like:
db.[collection-name].createIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } )
Additionally this is for the Expire Documents at a Specific Clock Time way of doing a ttl field.
And I still can't seem to get it to work, but might be because of the erratic way that the ttl reaper runs (once every 60 secs, but could be longer...)
EDIT
My issues were due to having an earlier incorrectly configured ttl index persisting on the expireAt field, this prevented my later (correctly defined) index from working. I fixed this just by deleting any non-_id earlier indexes and then re-adding my ttl index on the expireAt field.
Use db.[collection-name].getIndexes()
and
db.[collection-name].dropIndex({ "expireAt":1 }) to clear out before re-applying.
Also one other caveat - setting a Date snapshot in the default property of the expiredAt field means that the default value will always be a fixed date - instead set this Date value dynamically each time you create an instance of expireSchema:
var expireSchema = new Schema({
expireAt: Date,
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire' }
});
expireSchema
.virtual('expiryTime')
.get(function() {
//allow for reaper running at 60 second intervals to cause expireAt.fromNow message to be 'in the past'
var expiryTime = moment(this.expireAt).fromNow();
if(String(expiryTime).indexOf("ago")){
expiryTime = "in a few seconds";
}
return expiryTime;
});
var expireModel = mongoose.model('UserExpire', expireSchema);
expireModel.collection.ensureIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } );
I have a mongoose Schema which looks like this:
const USERS_DATA = new Schema({
_id: Number,
name: String,
img: String,
date: Date,
phone: String,
article: String,
createdAt: {
type: Date,
required: true,
default: Date.now,
index: { expires: '3d' }
}
},
{
collection: "users",
_id: false,
}
);
I need to push data to this schema.
const User = mongoose.model("users", USERS_DATA);
function pushToDB() {
const newUser = new User({
name: INPUT.name,
img: INPUT.img,
date: INPUT.date,
phone: INPUT.phone,
article: INPUT.article,
});
newUser.save(function (err) {
mongoose.disconnect();
if (err) return console.log(err);
});
}
This data have to be deleted after 3 days when it was pushed to database. How to implement it in node.js? I found it really confusing and tried lots of code. Any answers are appreciated! Thanks
P.S. I use mongoDb Atlas
You should separate the process to push the data into the database from the process to delete it after 3 days. You have already the first part :).
For the second part, you can write a function deleteOldDocument. This function will query for documents in DB that are created for 3 days or more, and delete them. Then, you can run this function periodically, 1 time per day for example.
The pseudo-code, in case you need it :
async function deleteOldDocument() {
const 3DaysAgo = ...; // here you can subtract 3 days from now to obtain the value
// search for documents that are created from 3 days or more, using $lt operator
const documentToDelete = await User.find({"created_at" : {$lt : 3DaysAgo }});
// delete documents from database
.....
// recall the function after 1 days, you can change the frequence
setTimeOut(async function() {
await deleteOldDocument();
}), 86400);
}
// call deleteOldDocument to start the loop
deleteOldDocument();
Suppose i have a Schema as follows:
const UserSchema = new mongoose.Schema({
name: String,
createdAt: { type: Date, default: Date.now, expires: 3600 }
});
const User= mongoose.model('User', UserSchema);
const user = new User({name: 'Me'});
console.log(user);
setTimeout(function () {
User.update({name: 'Me'}, {name: 'You'}).exec((err, user) => console.log(user));
//update statement is incorrect, but i think you got my point
}, 2000);
So i wanted to know if updating the document will reset the document's expire time if other if change the name attribute, or the time will restart (reset) only if the createdAt value is change.
The Mongoose documentation does not have any point on this.
No, modifying a document's fields other than createdAt has no effect on when the document expires.
The document will expire when: current time >= createdAt + 3600
I currently have data saving and expiring to/from a database via a mongoose schema like so:
var UserSchema = new Schema({
createdAt: { type: Date, expires: '1m' },
name: String,
email: String
});
The only problem is that the document that's saved to the database is completely removed from the database. How would I refactor the above so that the name/email address stay in the database but if the user attempts to login after their expiry date then they're greeted with a message saying 'session has expired, renew session'. (or something similar)
I'm wanting to do it this way because then if the user logs in with an expired email address the server is still able to lookup the email address and spit out a "expired session" message rather than a "not found" error which is what happens when data is deleted.
So to reiterate, how do I keep expired data in a mongo/mongoose database so the app is able to find the email address the user is attempting to login with but if their session has expired they need to renew the session?
You should use concept of Schema Reference for this. Save your expired field in another table and join your main user_table and expire_table(wxample name)
var UserSchema = new Schema({
name: String,
email: String
});
//save date by-default
//expire in 1 min as in your example
var expireSchema = new Schema({
createdAt: { type: Date, default: Date.now, expires: '1m' },
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire'}
});
var userTable = mongoose.model('user_expire', UserSchema);
var expireTable = mongoose.model('expireMe', expireSchema);
//Save new user
var newUser = new userTable({
name: 'my_name',
email: 'my_email'
});
newUser.save(function(err, result) {
console.log(result, 'saved')
var newExpire = new expireTable({
user_pk:result._id
});
//use _id of new user and save it to expire table
newExpire.save(function(err, result) {
console.log('saved relation')
})
})
Now to detect whether session has expired or not
1. on executing this code before data gets expired
expireTable.findOne()
.populate('user_pk')
.exec(function (err, result) {
if (err) throw err;
console.log(result)
if(result == null) {
console.log('session has expired, renew session')
} else {
console.log('session is active')
}
});
//output - session is active
2. on executing this code after data gets expired
expireTable.findOne()
.populate('user_pk')
.exec(function (err, result) {
if (err) throw err;
console.log(result)
if(result == null) {
console.log('session has expired, renew session')
} else {
console.log('session is active')
}
});
//output - session has expired, renew session
The accepted answer is good, but with Mongo 3.0 and above, the
createdAt: { type: Date, default: Date.now, expires: '1m' }
does not work for me.
Instead I used
var expireSchema = new Schema({
expireAt: {
type: Date,
required: true,
default: function() {
// 60 seconds from now
return new Date(Date.now() + 60000);
}
},
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire'}
});
More info is here: Custom expiry times for mongoose documents in node js
EDIT
My comment above would also require invoking a Mongo function directly rather than via Mongoose syntax. This would be something like:
db.[collection-name].createIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } )
Additionally this is for the Expire Documents at a Specific Clock Time way of doing a ttl field.
And I still can't seem to get it to work, but might be because of the erratic way that the ttl reaper runs (once every 60 secs, but could be longer...)
EDIT
My issues were due to having an earlier incorrectly configured ttl index persisting on the expireAt field, this prevented my later (correctly defined) index from working. I fixed this just by deleting any non-_id earlier indexes and then re-adding my ttl index on the expireAt field.
Use db.[collection-name].getIndexes()
and
db.[collection-name].dropIndex({ "expireAt":1 }) to clear out before re-applying.
Also one other caveat - setting a Date snapshot in the default property of the expiredAt field means that the default value will always be a fixed date - instead set this Date value dynamically each time you create an instance of expireSchema:
var expireSchema = new Schema({
expireAt: Date,
user_pk: { type: Schema.Types.ObjectId, ref: 'user_expire' }
});
expireSchema
.virtual('expiryTime')
.get(function() {
//allow for reaper running at 60 second intervals to cause expireAt.fromNow message to be 'in the past'
var expiryTime = moment(this.expireAt).fromNow();
if(String(expiryTime).indexOf("ago")){
expiryTime = "in a few seconds";
}
return expiryTime;
});
var expireModel = mongoose.model('UserExpire', expireSchema);
expireModel.collection.ensureIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } );