Mongoose - Set TTL on MongoDB Document - node.js

I am trying to set a TTL via mongoose when a document is created in MongoDB, but I'm not having any luck with any of my attempts. Latest version of mongoose is being used in my project and from what I can tell I've tried the most common answers here on SO and elsewhere online.
My Schema
const mongoose = require('mongoose');
const jobSchema = new mongoose.Schema(
{
positionTitle: {
type: String,
},
description: {
type: String,
}
});
const Jobs = mongoose.model('job', jobSchema);
module.exports = Jobs;
I have tried adding a createdAt with expires based on this question answer:
const jobSchema = new mongoose.Schema({
positionTitle: {
type: String,
},
description: {
type: String,
},
createdAt: { type: Date, expires: 3600 },
});
Along with this option that's also in the same question to have createdAt be created automatically via timestamps:
const jobSchema = new mongoose.Schema(
{
positionTitle: {
type: String,
},
description: {
type: String,
},
},
{ timestamps: true }
);
Trying variations of the following to set an index with timestamps defined:
jobSchema.index({ createdAt: 1 }, { expires: 86400 });
jobSchema.index({ createdAt: 1 }, { expires: '1 day' });
jobSchema.index({ createdAt: 1 }, { expireAfterSeconds: 3600 });
Regardless of which option I try, the document is removed after MongoDB's 60-second cycle when a createdAt field is set on the document. Would really love to know what I'm doing wrong.

After trying all the solutions in the thread you mentioned, none of them worked. In the end this code did the trick. It involves setting the expireAt field to the actual time that you want it deleted, which makes sense really.
const Schema = mongoose.Schema;
const YourSchema = new Schema({
expireAt: {
type: Date,
default: Date.now() + 10 * 60 * 1000 // expires in 10 minutes
},
});
This is the only thing that worked, all the other solutions I tried always deleted after 1min, no matter the amount of time I added.

I've been having issues with this as well. I found this thread here https://github.com/Automattic/mongoose/issues/2459 and it worked for me. Translated into your code would look like this.
const mongoose = require('mongoose');
const jobSchema = new mongoose.Schema(
{
positionTitle: {
type: String,
},
description: {
type: String,
},
expireAt: {
type: Date,
default: Date.now,
index: { expires: '5s' }
}
});
const Jobs = mongoose.model('job', jobSchema);
module.exports = Jobs;
On the link I added, it is the very last solution. I'm not exactly sure what this is doing but here is the mongo link of what it should be doing for anyone else with this issue. https://docs.mongodb.com/manual/tutorial/expire-data/. To change the amount of time that you need the document just change the expires. It accepts '#s' and '#d' for sure. Also if you want your document to be deleted at a specific time then you can do this.
expireAt: {
type: Date,
default: new Date('July 22, 2013 14:00:00'),
index: { expires: '0s' }
}
This will delete the document 0 seconds after the specified date.

Problem in TTL, Reason behind Document does not delete after some / few seconds, how to expire document in MongoDB / Mongoose using schema. Solution expireAfterSeconds / expires / index.
NOTE: - MongoDB's data expiration task runs once a minute, so an expired doc might persist up to a minute past its expiration. This feature requires MongoDB 2.2 or later. It's up to you to set createdAt to the current time when creating docs or add a default to do it for you as suggested here.
NOTE :- Below code is working fine and the document will delete after 5 minutes.
const verficationSchema = new mongoose.Schema({
email: {
type: String,
required: true,
unique: true,
lowercase: true,
trim: true,
validate(email) {
if (!validator.isEmail(email)) {
throw new Error("Email is not valid!");
}
},
},
otp: {
type: Number,
required : true
},
expireAt : {
type: Date,
default: Date,
expires : 300 // means 300 seconds = 5 minutes
}
});
NOTE :- Upper code is working fine, But document will delete after 1 minutes, because MongoDB check expiration procedure after every 1 minutes.
const verficationSchema = new mongoose.Schema({
email: {
type: String,
required: true,
unique: true,
lowercase: true,
trim: true,
validate(email) {
if (!validator.isEmail(email)) {
throw new Error("Email is not valid!");
}
},
},
otp: {
type: Number,
required : true
},
expireAt : {
type: Date,
default: Date,
expires : 8 // means 8 seconds
}
});

Related

How to delete a particular document automatically in mongodb in nodejs

I want to delete a particular task document automatically. createdAt: {type: Date} => it will take future date and time, and duration:{type: String} => it will take time in hours. whenever the future time arrives from that time to next how much duration we insert, after completion of duration the task document will delete
const mongoose = require('mongoose')
const TaskSchema = new mongoose.Schema({
taskName: { type: String, required: true },
description: { type: String },
creator: { type: String },
duration: { type: String },
createdAt: {type: Date}
})
const Tasks = mongoose.model('Task', TaskSchema)
module.exports = Tasks```
**Please help how to approach this task**
try this
const TestSchema = new Schema({
expire_at: {type: Date, default: Date.now, expires: "your desired value"}
})
this is the solution you are looking for here

Mongo DB document is not deleting after it expires

I'm trying to set up a model so that when a table get added, it will have an expiration date, after it expires, the table will delete itself. I tried implementing it like this:
expires: '1m'
and with
expires: 10
I have a table set up like the following:
const verifySchema = new mongoose.Schema({
_userId: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'User'
},
hash: { type: String, required: true },
createdAt: { type: Date, required: true, default: Date.now, expires: '1m' }
});
The problem is, nothing happens after the minute. It doesn't get deleted in the database. Am I doing anything wrong?
How can I delete a table after a minute?
Here's a working sample using mongoose v5.5.9. It turns out the missing part is the schema entry index: { expires: '1m' } in the createdAt field.
const mongoose = require('mongoose')
// $ npm install uuid
const uuid = require('uuid')
const ObjectId = mongoose.Types.ObjectId
// Avoid deprecation warnings
mongoose.set('useNewUrlParser', true);
mongoose.set('useFindAndModify', false);
mongoose.set('useCreateIndex', true);
// Create the schema.
const verifySchema = new mongoose.Schema({
_userId: {
type: ObjectId,
required: true,
ref: 'User'
},
hash: { type: String, required: true },
createdAt: {
type: Date,
required: true,
default: Date.now,
index: { expires: '1m' }
}
},
{
collection: 'verify'
});
// Connect to mongodb.
mongoose.connect('mongodb://localhost/test').then(() => {
// Create the model
const Verify = mongoose.model('Verify', verifySchema)
// Create a model instance.
const v = new Verify({
_userId: new ObjectId(),
hash: uuid.v4()
})
// Save the model.
v.save().then(() => {
// Close the connection.
mongoose.connection.close()
})
})
You can check your indexes with MongoDB Compass, or using the shell:
> use test
> db.verify.getIndexes()
Look for the field value expireAfterSeconds which will indicate the TTL time in seconds that is set for the index. In order to change the TTL, you will need to drop the index on createdAt. In the shell, the command would be db.verify.dropIndex(<index_name>) or db.verify.dropIndexes() to drop all indexes on the collection.
For upserting documents, such as with findOneAndUpdate, you will need to pass setDefaultsOnInsert: true to the options like so:
// Connect to mongodb.
mongoose.connect('mongodb://localhost/test').then(() => {
// Create the model
const Verify = mongoose.model('Verify', verifySchema)
const _userId = new ObjectId()
const hash = uuid.v4()
const options = { upsert: true, setDefaultsOnInsert: true }
// Upsert the document.
Verify.findOneAndUpdate( { _userId }, { hash }, options).then(() => {
mongoose.connection.close()
})
})
This is necessary or else the createdAt field, which contains the TTL index, won't get added to the document.

Documents do not expire after the set time in MongoDB using Mongoose [duplicate]

This question already has answers here:
Setting expiry time for a collection in mongodb using mongoose
(9 answers)
Closed 3 years ago.
I am trying to create a TTL index in MongoDB. I read the answer on here, Answer which was very helpful.
The problem is that the documents just don't expire. Here's the code:
var AcThSchema = new mongoose.Schema({
createdAt: {
type: Date,
expires: '1m',
default: Date.now
},
key: {
type: String,
required: true,
unique: true
}
});
The weird thing that I did notice was that when I use the value 1 instead of Date.now as default for createdAt, the document does get deleted after a few seconds (probably the next time the mongo's TTL process runs)
Why is the document getting deleted for a default value of 1 but not for Date.now?
The expires should be inside index. Like this
var AcThSchema = new mongoose.Schema({
createdAt: {
type: Date,
index: {
expires: '1m'
},
default: Date.now
},
key: {
type: String,
required: true,
unique: true
}
});

Why expires is not working in mongoose model?

So this is my model and I want to do that when it's created in DB to remove it from there after one hour. But this not working. I got no errors also. It expires after few seconds.
var mongoose = require('mongoose');
var Revive = mongoose.model('Revive', {
username: {
type: String,
required: true
},
count: {
type: Number,
default: 0,
max: 4
},
expire_at : { type : Date, index : { expires : '60m' }, default: Date.now }
});
module.exports = {Revive};

How do I set a document to expire using Mongoose (Node.js)? [duplicate]

Below is the command that can be used via the mongo terminal to set an expiry time for collections (a TTL):
db.log.events.ensureIndex( { "status": 1 }, { expireAfterSeconds: 3600 } )
How do I do this from my code in Node.js using mongoose?
In Mongoose, you create a TTL index on a Date field via the expires property in the schema definition of that field:
// expire docs 3600 seconds after createdAt
new Schema({ createdAt: { type: Date, expires: 3600 }});
Note that:
MongoDB's data expiration task runs once a minute, so an expired doc might persist up to a minute past its expiration.
This feature requires MongoDB 2.2 or later.
It's up to you to set createdAt to the current time when creating docs, or add a default to do it for you as suggested here.
{ createdAt: { type: Date, expires: 3600, default: Date.now }}
this code is working for me.
may it help
let currentSchema = mongoose.Schema({
id: String,
name: String,
packageId: Number,
age: Number
}, {timestamps: true});
currentSchema.index({createdAt: 1},{expireAfterSeconds: 3600});
Providing a string to expires also works nicely with Mongoose if you do not want to deal with the expire time calculation and improve the overall readability of the schema.
For example here we are setting the expires to 2m (2 minutes) and mongoose would convert to 120 seconds for us:
var TestSchema = new mongoose.Schema({
name: String,
createdAt: { type: Date, expires: '2m', default: Date.now }
});
Mongoose would create an index in the background and auto set the expireAfterSeconds to in this case 120 seconds (specified by the 2m).
It is important to note that the TTL process runs once every 60 seconds so it is not perfectly on time always.
If you are working with Mongodb Atlas Replica Sets - try:
import * as mongoose from 'mongoose';
let currentSchema = new mongoose.Schema({
createdAt: { type: Date, expires: 10000, default: Date.now },
id: String,
name: String,
packageId: Number,
age: Number
});
currentSchema.index({"lastModifiedDate": 1 },{ expireAfterSeconds: 10000 });
new Scehma({
expireAt: {
type: Date,
expires: 11,
default: Date.now
}
)}
This is the solution that worked for me according to this in the current Mongoose docs.
There is a npm library - 'mongoose-ttl'.:
var schema = new Schema({..});
schema.plugin(ttl, { ttl: 5000 });
you can see all the options of this library:
https://www.npmjs.com/package/mongoose-ttl
const Schema = new mongoose.Schema({id: {
type: Number},
createdAt: {
type: Date, expires: '4h', index: true,
default: Date.now}});
You need to add index: true while creating you schema
9/2022 Working Solution using Mongoose 6.5.4
None of the answers here worked for me, but I was able to finally get it working using the latest version of Mongoose currently available, 6.5.4.
Say our Schema looks like this:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
myCustomTTLField is the field you want to index and have control the expiration. To achieve this, we add the following under our schema definition:
MySchema.path('myCustomTTLField').index({ expires: 60 });
The argument in MySchema.path is the name of the field you want to index for TTL. The expires option should be the number of seconds that will elapse from the Date represented in myCustomTTLField before the document is deleted. In the example above, the document will be deleted 60 seconds after whatever date is saved in myCustomTTLField. The full example:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
MySchema.path('myCustomTTLField').index({ expires: 60 });
Please let me know if this works for you, I hope this helps. Mongoose TTL has been a thorn in my side for a long time, as their docs are notoriously tough to navigate. I found this solution via a small example buried in the docs here.
IMPORTANT NOTE:
TTL is not guaranteed to happen at exactly the time specified by your date + expiration seconds. This is due to how MongoDB's background delete process works. It runs every 60 seconds, so you may theoretically wait up to 60 seconds past expected TTL before seeing your document deleted. More info on that from the MongoDB docs.
FWIW I could only get the expires feature to work on a field called expiresAt. Here's my interface, and schema for implementing this in Typescript.
import { model, Schema, Types } from 'mongoose';
export interface ISession {
sessionId: string;
userId: Types.ObjectId;
role: string;
expiresAt?: Date;
}
const sessionSchema = new Schema<ISession>({
sessionId: { type: String, required: true, indexes: { unique: true} },
userId: { type: Schema.Types.ObjectId, required: true, ref: 'users'},
role: { type: String, required: true, enum: [ 'ADMIN', 'BASIC_USER' ]},
expiresAt: { type: Date, expires: '1h', default: Date.now }
}, { versionKey: false });
Reading the Mongoose documentation it seems like all the other proposed solutions should work too. I don't know why they were not for me. You can read the official Mongoose docs on expiresAt here.

Resources