I'm using the following schema to store some user tokens which should expire after a certain amount of time (30 minutes in this case):
var Schema = require('mongoose').Schema;
var tokenSchema = mongoose.Schema({
email: { type: String, required: true, trim: true },
token: { type: String, required: true },
created: { type: Date, expires: 60*30, default: Date.now },
}, {strict: 'throw'});
module.exports = mongoose.model('tokens', tokenSchema);
Now if I start my node.js application, I can check MongoDBfor the indexes on this collection using db.getCollection('tokens').getIndexes(). This results in:
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "node-android.resettokens"
},
{
"v" : 1,
"key" : {
"created" : 1
},
"name" : "created_1",
"ns" : "node-android.resettokens",
"expireAfterSeconds" : 1800,
"background" : true,
"safe" : null
}
]
If I now shut down my node app, change the value of expires in the token schema to something different (an hour for example), restart my node app and check the collection indexes again, I end up having the same result. I noticed this behavior after setting the TTL feature to 10 seconds for the first time to test its workings, and later finding that changing the expires value still made my documents get deleted very quickly.
So my question is: is there a way to automatically overwrite the old expires index with a new one, or do I have to delete it myself? I the latter is the case, is there a way to do this in mongoose, or do I have to execute a db.getCollection('tokens').dropIndex('created_1') via the mongo shell myself?
Related
I have the following schema defined in my app (node.js w/ express):
const ArtistSchema = new mongoose.Schema({
name: {type: String, required: true},
year: {type: Number, required: true},
genres: {type: [String], required: true},
imageUrl: {type: String, required: false},
trending: {type: Boolean, required: false, default: false},
trendingDate: {type: Date, required: false}
});
and a route that is supposed to retrieve those entries, who have trending set to true:
// GET trending
router.get('/trending', (req, res) => {
artist.Artist.find({trending: true}).exec((err, trendingArtists) => {
if (err) {
console.error(err);
res.status(500).json({message: err.message});
return;
}
res.json(trendingArtists);
});
});
However, it always returns an empty array when i try to filter by trending field, even though there are items in my collection that have trending set to true. I have tried wrapping everything in single and double quotes and using 1 instead of true, but no query returns results. Filtering by other fields works just fine, not filtering at all returns all entries as expected.
The entries in mongo shell look like this:
> db.artists.find({},{name:1,trending:1}).pretty()
{
"_id" : ObjectId("5de942074a486e2c21246fb9"),
"name" : "Unwound",
"trending" : "true"
}
{
"_id" : ObjectId("5de942074a486e2c21246fba"),
"name" : "Ladytron",
"trending" : "true"
}
{
"_id" : ObjectId("5de942074a486e2c21246fbb"),
"name" : "Depeche Mode",
"trending" : "true"
}
console.loging the results in the app produces this:
[
{
genres: [ 'post-hardcore', 'noise rock', 'indie rock' ],
trending: true,
_id: 5de942074a486e2c21246fb9,
name: 'Unwound',
year: 1991,
imageUrl: '2a7f00a1f8e0ab37c647600f6abff67e.jpg',
trendingDate: 2019-12-20T18:48:53.000Z
},
{
genres: [ 'synthpop', 'electroclash', 'dream pop' ],
trending: true,
_id: 5de942074a486e2c21246fba,
name: 'Ladytron',
year: 1999,
imageUrl: 'f26cc1ae1fef371793622bd199b4bb52.jpg',
trendingDate: 2019-12-20T18:49:05.000Z
},
{
genres: [ 'synthpop', 'new wave' ],
trending: true,
_id: 5de942074a486e2c21246fbb,
name: 'Depeche Mode',
year: 1980,
imageUrl: 'e5328919dac971af86dd034781a2da71.jpg',
trendingDate: 2019-12-20T18:49:43.000Z
}
]
I am at my wits' end. What could cause filtering by a boolean field to break the query regardless of what i specify as the value (i have tried true, 'true', "true", 1, '1', as well as the falsy counterparts)?
edit: i tried a few things since:
1) filtering the results after the query was executed works fine (i.e just writing res.json(trendingArtists.filter(a => a.trending === true));), though it's obviously not the way i would like to deal with filtering my queries
2) the collection i'm querying was created and edited manually, and not via api that my app implements. If i insert a new entry using a POST request via api, that entry will be returned by the query, provided trending was set to true
3) editing existing entries with PATCH requests where i set trending to false and then back to true also works, though it messes up the trendingDate field that is set to current date each time trending is changed to true
4) if the query works for an entry, it works no matter what i put as the value for the filter, as long as it's truthy for mongoose. .find({trending: true}) works just as well as .find({trending: '1'})
I guess the problem is mongoose not recognizing my manually inserted values as truthy at all, even though theoretically they should be cast to true. My problem is solved, I guess? I do not plan to insert values manually in the future, this was just for testing, and those that few that are inserted can be fixed fairly easily, but i don't think it should matter for mongoose whether i edit the entries manually or via Model.save. It seems like a bug - should i close the question and open a bug report on their github?
I am pretty confused to why making the searched fields to be "index" is making the query "theoretically" slower.
I have a not very big collection of items (6240) and all of them have the following structure.
const SomeSchema = new mongoose.Schema({
data: String,
from: {
type: Number,
},
to: {
type: Number,
},
timeStamp: {
type: Date,
default: new Date()
}
})
SomeSchema.set('toJSON', {
getters: true,
transform: (doc, ret) => {
delete ret.from
delete ret.to
return sanitizeSensitiveProperties(ret)
}
})
export const Some = mongoose.model('Some', SomeSchema, 'somethings')
The strange thing came when after trying to improve the query I changed the schema to be
...
from: {
type: Number,
index: true
},
to: {
type: Number,
index: true
},
...
With this schema I run the following query
db.rolls.find({from: {$lte: 1560858984}, to: {$gte: 1560858984}}).explain("executionStats")
This are the results NOTE THAT THE 1st ONE is the one without index
"executionTimeMillis" : 6,
"totalKeysExamined" : 0,
"totalDocsExamined" : 6240,
"executionTimeMillis" : 15,
"totalKeysExamined" : 2895,
"totalDocsExamined" : 2895,
Does this result make any sense, or is just the mongo .explain() function messing around?
As you can see I am using the Mongoose Driver in the version ^5.5.13 and I am using Mongo in the version 4.0.5
I'm using mongoose with the following schema for ConfigItem:
var ConfigItem = new Schema({
name: {
type: String,
required: true
},
value: {
type: String,
required: false
},
date: {
type: Date,
required: false
},
user: {
type: String,
required: false
}
});
Next, I have a function to save documents according to this schema:
function createConfigItem(_name, _value, _date, _user, callback) {
var config = new ConfigItems({
name: _name,
value: _value,
date: _date,
user: _user
});
config.save(function handleSaveConfig(err) {
if(callback){
callback(err);
}
});
}
Next, I have an unit test (mocha based) which uses that function in the following way:
createConfigItem('sftpHost', '1.1.1.1', '2018-03-15 10:06:40.713', 'user1', callback);
If I run the test under mocha (see Note1) I get the following at MongoDB:
> db.configitems.find()
{ "_id" : ObjectId("5af173b1f155a4bff29f3e35"), "name" : "sftpHost", "value" : "1.1.1.1", "date" : ISODate("2018-03-15T10:06:40.713Z"), "user" : "user1", "__v" : 0 }
which is what I expect.
However, if I run using istanbul via grunt (see Note2) to get a coverage report the same invocation to createConfigItem() creates the following documenta at DB:
> db.configitems.find()
{ "_id" : ObjectId("5af176cbcffb8cc20a1fe3c2"), "name" : "sftpHost", "value" : "1.1.1.1", "date" : ISODate("2018-03-15T09:06:40.713Z"), "user" : "user1", "__v" : 0 }
Note that in this case the date field at DB is one hour shifted with regards the _date string parameter.
I was thinking it could be related with timezones in some way, but I have ensured that my process run in UTC setting process.env.TZ = 'UTC'.
I'm a bit lost... is there any known issue regarding mongoose/grunt/istanbul with regards to dates and/or timezones? Any hint about this problem, pls?
Note1: command used:
/home/fermin/.nvm/versions/node/v6.12.3/bin/node --debug-brk=50806 --expose_debug_as=v8debug /home/fermin/src/ctxmboard/node_modules/mocha/bin/_mocha --timeout 0 --ui bdd --reporter /home/fermin/.PyCharm2018.1/config/plugins/NodeJS/js/mocha-intellij/lib/mochaIntellijReporter.js --recursive /home/fermin/src/ctxmboard/test/back/unit
Note2: command used:
/home/fermin/.nvm/versions/node/v6.12.3/bin/node --debug-brk=33002 --expose_debug_as=v8debug ./node_modules/.bin/istanbul cover --root lib/ --dir site/coverage -- /home/fermin/.nvm/versions/node/v6.12.3/lib/node_modules/grunt-cli/bin/grunt test
As suggested in one of the question comments, using strict ISO8601 format, changing
'2018-03-15 10:06:40.713'
by
'2018-03-15T10:06:40.713Z'
solved the problem
you can use moment node module https://momentjs.com/docs/
var date=moment.utc().toDate();
I'm trying to use meteor-collection2 to validate my collection.
I have a service, on server side :
Meteor.methods
UserSignUpService: (options) ->
# create user
Accounts.createUser options
That I call on client side :
Meteor.call 'UserSignUpService',
email: 'my.email#domain.com'
password: 'mypassword'
profile:
name: 'me'
And this is my schema :
# user profile
UserProfile = new SimpleSchema
name:
type: String
min: 1
max: 50
optional: false
# user
User = new SimpleSchema
emails:
type: [Object]
optional: false
"emails.$.address":
type: String
regEx: SimpleSchema.RegEx.Email
"emails.$.verified"
type: Boolean
createdAt:
type: Date
profile:
type: UserProfile
optional: false
services:
type: Object
optional: true
blackbox: true
# links to user collection
Meteor.users.attachSchema User
But, when user is created, there are not all fields in my mongo collection :
{ "_id" : "ikvBq95JBLXMCSnhT", "emails" : [ { "address" : "my.email#domain.com" } ] }
Whereas it should be :
{ "_id" : "WNkjGFhNkLpRR2Jex", "createdAt" : ISODate("2015-08-06T09:00:59.887Z"), "services" : { "password" : { "bcrypt" : "$2a$10$QvMLuI.Pv0bzzii3ZZP...fHfUlU9KiYfcsC2VHBf6q1OSPM6cfTW" }, "resume" : { "loginTokens" : [ { "when" : ISODate("2015-08-06T09:01:00.002Z"), "hashedToken" : "9KyqjRVSWm0nfIlS0sqODRmddlJ5GaG3mJ4+RMItOhk=" } ] } }, "emails" : [ { "address" : "my.email#domain.com", "verified" : false } ], "profile" : { "name" : "me" } }
Any idea on this problem ?
Many Thanks !
You're not actually setting the createdAt field value, try:
createdAt:
type: Date
autoValue: function(){
if this.isInsert return new Date()
else if this.isUpsert return { $setOnInsert: new Date };
else if this.isSet this.unset();
}
You can also omit optional: false in your schema since that's the default.
Also, and I suspect this is the bigger problem, not all user keys are returned to the client by default, only profile is normally published. For example you're expecting the services key but that doesn't normally come across. Look at the document in the mongo console and see what's there. You might need to publish a more comprehensive set of keys to the client.
Schema (../models/add.js)
var addSchema = new Schema({
name: {type: String, unique: true, sparse: true},
phone: Number,
email: String,
country: Number
});
module.exports = mongoose.model('Contact', addSchema);
add-manager.js
var Add = require('../models/add.js');
var AM = {};
var mongoose = require('mongoose');
module.exports = AM;
AM.notOwned = function(country, callback)
{
Add.update({country: country}, {country: country}, {upsert: true}, function(err, res){
if (err) callback (err);
else callback(null, res);
})
}
news.js
// if country # is not in the database
AM.notOwned(country, function(error, resp){
if (error) console.log("error: "+error);
else
{
// do stuff
}
})
error:
MongoError: E11000 duplicate key error index: bot.contacts.$name_1 dup key: { : null }
After seeing the error message, I googled around and learned that when the document is created, since name isn't set, its treated as null. See Mongoose Google Group Thread The first time AM.notOwned is called it will work as there isn't any documents in the collection without a name key. AM.notOwned will then insert a document with an ID field, and a country field.
Subsequent AM.notOwned calls fails because there is already a document with no name field, so its treated as name: null, and the second AM.notOwned is called fails as the field "name" is not set and is treated as null as well; thus it is not unique.
So, following the advice of the Mongoose thread and reading the mongo docs I looked at using sparse: true. However, its still throwing the same error. Further looking into it, I thought it may be the same issue as: this, but setting schema to name: {type: String, index: {unique: true, sparse: true}} doesn't fix it either.
This S.O. question/answer leads me to believe it could be caused by the index not being correct, but I'm not quite sure how to read the the db.collection.getIndexes() from Mongo console.
db.contacts.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "bot.contacts",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"name" : 1
},
"unique" : true,
"ns" : "bot.contacts",
"name" : "name_1",
"background" : true,
"safe" : null
}
]
What can I do to resolve this error?
You need to drop the old, non-sparse index in the shell so that Mongoose can recreate it with sparse: true the next time your app runs.
> db.contacts.dropIndex('name_1')