I have a field in my MongoDB collection products called date_expired.
It's of type: date and stores the a date string.
I want to retrieve all the products and change the date_expired property in the result to number of hours left from now. How do I do this?
It's similar to getter() in Laravel...?
You could create a virtual property that will return the number of hours until expiry:
ProductSchema.virtual('hoursToExpiry').get(function() {
return (this.date_expired - Date.now()) / 3600000;
});
To access this property:
console.log('hours to expiry:', doc.hoursToExpiry)
If you want to include that property in any JSON or JS object, make sure that you set virtuals : true:
console.log('%j', doc.toJSON({ virtuals : true }));
Would consider using the aggregation framework in this case to output the transformation. You can use the $project pipeline arithmetic operators $divide and $subtract to achieve the final goal. These will enable you to carry out the arithmetic of calculating the number of hours to expiry i.e. implement the formula:
hoursToExpiry = (date_expired - timeNow)/1000*60*60 //the time units are all in milliseconds
Take for instance the following short mongo shell demo that will strive to drive home this concept:
Populate test collection:
db.test.insert([
{
"date_expired": ISODate("2016-03-27T10:55:13.069Z"),
"name": "foo"
},
{
"date_expired": ISODate("2016-06-11T20:55:13.069Z"),
"name": "bar"
},
{
"date_expired": ISODate("2016-06-11T16:17:23.069Z"),
"name": "buzz"
}
])
Aggregation Operation:
db.test.aggregate([
{
"$project": {
"name": 1,
"dateExpired": "$date_expired",
"dateNow": { "$literal": new Date() },
"hoursToExpiry": {
"$divide": [
{ "$subtract": [ "$date_expired", new Date() ] },
1000*60*60
]
}
}
}
])
Result (at the time of writing):
{
"result" : [
{
"_id" : ObjectId("575c0f6e8101b29fc93e5b9d"),
"name" : "foo",
"dateExpired" : ISODate("2016-03-27T10:55:13.069Z"),
"dateNow" : ISODate("2016-06-11T13:36:21.025Z"),
"hoursToExpiry" : -1826.685543333333
},
{
"_id" : ObjectId("575c0f6e8101b29fc93e5b9e"),
"name" : "bar",
"dateExpired" : ISODate("2016-06-11T20:55:13.069Z"),
"dateNow" : ISODate("2016-06-11T13:36:21.025Z"),
"hoursToExpiry" : 7.314456666666667
},
{
"_id" : ObjectId("575c0f6e8101b29fc93e5b9f"),
"name" : "buzz",
"dateExpired" : ISODate("2016-06-11T16:17:23.069Z"),
"dateNow" : ISODate("2016-06-11T13:36:21.025Z"),
"hoursToExpiry" : 2.683901111111111
}
],
"ok" : 1
}
With the above pipeline, you can then adopt it to your Mongoose implementation with the aggregate() method as basis of your query:
Product.aggregate([
{
"$project": {
"name": 1,
"dateExpired": "$date_expired",
"dateNow": { "$literal": new Date() },
"hoursToExpiry": {
"$divide": [
{ "$subtract": [ "$date_expired", new Date() ] },
1000*60*60
]
}
}
}
]).exec(function (err, result) {
// Handle err
console.log(result);
});
or using the more affluent API:
Product.aggregate()
.project({
"name": 1,
"dateExpired": "$date_expired",
"dateNow": { "$literal": new Date() },
"hoursToExpiry": {
"$divide": [
{ "$subtract": [ "$date_expired", new Date() ] },
1000*60*60
]
}
})
.exec(function (err, result) {
// Handle err
console.log(result);
});
Related
I would like to perform autocompletion on the name but filtered on a specific city with mongoose and nodejs.
I have a mongodb collection like this :
{
"_id" : ObjectId("6007fd9d984e2507ad452cf3"),
"name" : "John",
"city" : "A",
},
{
"_id" : ObjectId("6007ff6844d9e517e1ec0976"),
"name" : "Jack",
"city" : "B",
}
What i have done so far :
I have setup MongoDB Atlas with a Search Index (with the help of search doc)
And set up the autocomplete like that :
router.get('/search', async (request, response) => {
try {
let result = await Client.aggregate([
{
"$search": {
"autocomplete": {
"query": `${request.query.term}`,
"path": "name",
"fuzzy": {
"maxEdits": 2,
"prefixLength": 3,
},
},
},
},
{
$limit: 3
},
{
$project: {
"_id": 0,
}
}
]);
response.send(result);
} catch (e) {
response.status(500).send({message: e.message});
}
});
In front-end, with autocompleteJs :
const autoCompleteJS = new autoComplete({
data: {
src: async () => {
const query = document.querySelector("#autoComplete").value;
const source = await fetch(`${window.location.origin}/search?term=${query}`);
const data = await source.json();
return data;
},
key: ["name"],
},
trigger: {
event: ["input", "focus"],
},
searchEngine: "strict",
highlight: true,
});
So far it is working well. But I don't know how to make the autocomplete result filtered based on city. It seems that the documentation does not mention this. Do you have any leads.
Use the $where pipeline stage from the aggregation pipeline after performing your search to filter out unwanted documents. So for example,
Client.aggregate([
{
"$search": {
"autocomplete": {
"query": `${request.query.term}`,
"path": "name",
"fuzzy": {
"maxEdits": 2,
"prefixLength": 3,
},
},
},
},
{
$match: { city: 'city-name' }
},
{
$limit: 3
},
{
$project: {
"_id": 0,
}
}
]);
Use a compound operator like so which lets you have more control over your results in a performant fashion:
"$search": {
"compound" : {
"filter" : [{
"text" : { path: "city", query: "New York" }
}],
"must": [{
"autocomplete": {
"query": `${request.query.term}`,
"path": "name",
"fuzzy": {
"maxEdits": 2,
"prefixLength": 3,
},
},}
}] }
using filter will filter your results, without impacting the score. must will require that the name field will also match. Also take a look at should and mustNot in the compound docs for more options.
Suppose I have some MongoDB Event documents, each of which has a number of sessions which take place on different dates. We might represent this as:
db.events.insert([
{
_id: '5be9860fcb16d525543cafe1',
name: 'Past',
host: '5be9860fcb16d525543daff1',
sessions: [
{ date: new Date(Date.now() - 1e8 ) },
{ date: new Date(Date.now() + 1e8 ) }
]
}, {
_id: '5be9860fcb16d525543cafe2',
name: 'Future',
host: '5be9860fcb16d525543daff2',
sessions: [
{ date: new Date(Date.now() + 2e8) },
{ date: new Date(Date.now() + 3e8) }
]
}
]);
I'd like to find all Events which have not yet had their first session. So I'd like to find 'Future' but not 'Past'.
At the moment I'm using Mongoose and Express to do:
Event.aggregate([
{ $unwind: '$sessions' }, {
$group: {
_id: '$_id',
startDate: { $min: '$sessions.date' }
}
},
{ $sort:{ startDate: 1 } }, {
$match: { startDate: { $gte: new Date() } }
}
])
.then(result => Event.find({ _id: result.map(result => result._id) }))
.then(event => Event.populate(events, 'host'))
.then(events => res.json(events))
But I feel like I'm making heavy weather of this. Two hits on the database (three if you include the populate statement) and a big, complicated aggregate statement.
Is there a simpler way to do this? Ideally one which only involves one trip to the database.
You could use $reduce to fold the array and find if any of of the elements have a past session.
To illustrate this, consider running the following aggregate pipeline:
db.events.aggregate([
{ "$match": { "sessions.date": { "$gte": new Date() } } },
{ "$addFields": {
"hasPastSession": {
"$reduce": {
"input": "$sessions.date",
"initialValue": false,
"in": {
"$or" : [
"$$value",
{ "$lt": ["$$this", new Date()] }
]
}
}
}
} },
//{ "$match": { "hasPastSession": false } }
])
Based on the above sample, this will yield the following documents with the extra field
/* 1 */
{
"_id" : "5be9860fcb16d525543cafe1",
"name" : "Past",
"host" : "5be9860fcb16d525543daff1",
"sessions" : [
{
"date" : ISODate("2019-01-03T12:04:36.174Z")
},
{
"date" : ISODate("2019-01-05T19:37:56.174Z")
}
],
"hasPastSession" : true
}
/* 2 */
{
"_id" : "5be9860fcb16d525543cafe2",
"name" : "Future",
"host" : "5be9860fcb16d525543daff2",
"sessions" : [
{
"date" : ISODate("2019-01-06T23:24:36.174Z")
},
{
"date" : ISODate("2019-01-08T03:11:16.174Z")
}
],
"hasPastSession" : false
}
Armed with this aggregate pipeline, you can then leverage $expr and use the pipeline expression as your query in the find() method (or using the aggregate operation above but with the $match pipeline step at the end enabled) as
db.events.find(
{ "$expr": {
"$eq": [
false,
{ "$reduce": {
"input": "$sessions.date",
"initialValue": false,
"in": {
"$or" : [
"$$value",
{ "$lt": ["$$this", new Date()] }
]
}
} }
]
} }
)
which returns the document
{
"_id" : "5be9860fcb16d525543cafe2",
"name" : "Future",
"host" : "5be9860fcb16d525543daff2",
"sessions" : [
{
"date" : ISODate("2019-01-06T23:24:36.174Z")
},
{
"date" : ISODate("2019-01-08T03:11:16.174Z")
}
]
}
You don't need to use $unwind and $group to find the $min date from the array. You can directly use $min to extract the min date from the session array and then use $lookup to populate the host key
db.events.aggregate([
{ "$match": { "sessions.date": { "$gte": new Date() }}},
{ "$addFields": { "startDate": { "$min": "$sessions.date" }}},
{ "$match": { "startDate": { "$gte": new Date() }}},
{ "$lookup": {
"from": "host",
"localField": "host",
"foreignField": "_id",
"as": "host"
}},
{ "$unwind": "$host" }
])
Is it possible you can just reach into the sessions of each event, and pull back each event where all session dates are only in the future? Something like this? Might need tweaking..
db.getCollection("events").aggregate(
[
{$match:{'$and':
[
{'sessions.date':{'$gt': new Date()}},
{'sessions.date':{ '$not': {'$lt': new Date()}}}
]
}}
]
);
Here is my item model.
const itemSchema = new Schema({
name: String,
category: String,
occupied: [Number],
active: { type: Boolean, default: true },
});
I want to filter 'occupied' array. So I use aggregate and unwind 'occupied' field.
So I apply match query. And group by _id.
But if filtered 'occupied' array is empty, the item disappear.
Here is my code
Item.aggregate([
{ $match: {
active: true
}},
{ $unwind:
"$occupied",
},
{ $match: { $and: [
{ occupied: { $gte: 100 }},
{ occupied: { $lt: 200 }}
]}},
{ $group : {
_id: "$_id",
name: { $first: "$name"},
category: { $first: "$category"},
occupied: { $addToSet : "$occupied" }
}}
], (err, items) => {
if (err) throw err;
return res.json({ data: items });
});
Here is example data set
{
"_id" : ObjectId("59c1bced987fa30b7421a3eb"),
"name" : "printer1",
"category" : "printer",
"occupied" : [ 95, 100, 145, 200 ],
"active" : true
},
{
"_id" : ObjectId("59c2dbed992fb91b7421b1ad"),
"name" : "printer2",
"category" : "printer",
"occupied" : [ ],
"active" : true
}
The result above query
[
{
"_id" : ObjectId("59c1bced987fa30b7421a3eb"),
"name" : "printer1",
"category" : "printer",
"occupied" : [ 100, 145 ],
"active" : true
}
]
and the result I want
[
{
"_id" : ObjectId("59c1bced987fa30b7421a3eb"),
"name" : "printer1",
"category" : "printer",
"occupied" : [ 100, 145 ],
"active" : true
},
{
"_id" : ObjectId("59c2dbed992fb91b7421b1ad"),
"name" : "printer2",
"category" : "printer",
"occupied" : [ ],
"active" : true
}
]
how could I do this??
Thanks in advance.
In the simplest form, you keep it simply by not using $unwind in the first place. Your conditions applied imply that you are looking for the "unique set" of matches to specific values.
For this you instead use $filter, and a "set operator" like $setUnion to reduce the input values to a "set" in the first place:
Item.aggregate([
{ "$match": { "active": true } },
{ "$project": {
"name": 1,
"category": 1,
"occupied": {
"$filter": {
"input": { "$setUnion": [ "$occupied", []] },
"as": "o",
"cond": {
"$and": [
{ "$gte": ["$$o", 100 ] },
{ "$lt": ["$$o", 200] }
]
}
}
}
}}
], (err, items) => {
if (err) throw err;
return res.json({ data: items });
});
Both have been around since MongoDB v3, so it's pretty common practice to do things this way.
If for some reason you were still using MongoDB 2.6, then you could apply $map and $setDifference instead:
Item.aggregate([
{ "$match": { "active": true } },
{ "$project": {
"name": 1,
"category": 1,
"occupied": {
"$setDifference": [
{ "$map": {
"input": "$occupied",
"as": "o",
"in": {
"$cond": {
"if": {
"$and": [
{ "$gte": ["$$o", 100 ] },
{ "$lt": ["$$o", 200] }
]
},
"then": "$$o",
"else": false
}
}
}},
[false]
]
}
}}
], (err, items) => {
if (err) throw err;
return res.json({ data: items });
});
It's the same "unique set" result as pulling the array apart, filtering the items and putting it back together with $addToSet. The difference being that its far more efficient, and retains ( or produces ) an empty array without any issues.
Suppose I have the following query:
post.getSpecificDateRangeJobs = function(queryData, callback) {
var matchCriteria = queryData.matchCriteria;
var currentDate = new Date();
var match = { expireDate: { $gte: new Date(currentDate.getFullYear(), currentDate.getMonth(), currentDate.getDate()) } };
if (queryData.matchCriteria !== "") {
match = {
expireDate: { $gte: new Date(currentDate.getFullYear(), currentDate.getMonth(), currentDate.getDate()) },
$text: { $search: matchCriteria }
};
}
var pipeline = [
{
$match: match
},
{
$group: {
_id: null,
thirtyHourAgo: {
$sum: {
$cond: [
{
$gte: [
"$publishDate",
new Date(queryData.dateGroups.thirtyHourAgo)
]
},
1,
0
]
}
},
fourtyEightHourAgo: {
$sum: {
$cond: [
{
$gte: [
"$publishDate",
new Date(queryData.dateGroups.fourtyHourAgo)
]
},
1,
0
]
}
},
thirtyDaysAgo: {
$sum: {
$cond: [
{
$lte: [
"$publishDate",
new Date(queryData.dateGroups.oneMonthAgo)
]
},
1,
0
]
}
}
}
}
];
var postsCollection = post.getDataSource().connector.collection(
post.modelName
);
postsCollection.aggregate(pipeline, function(err, groupByRecords) {
if (err) {
return callback("err");
}
return callback(null, groupByRecords);
});
};
What i want to do is:
1- check if queryData.dateGroups.thirtyHourAgo existed and has value, then only add the relevant match clause in query (count of posts only for past 30 hour).
2- check if queryData.dateGroups.fourtyHourAgo existed, then add relevant query section (count of posts for past 30 hour, and past 48 hour ago).
3 and the same for queryData.dateGroups.oneMonthAgo (count of posts for past 30 hour, 48 hour, and past one month).
I need something like Mysql if condition to check if a variable existed and not empty then include a query clause. Is it possible to do that?
My sample data is like:
/* 1 */
{
"_id" : ObjectId("58d8bcf01caf4ebddb842855"),
"vacancyNumber" : "123213",
"position" : "dsfdasf",
"number" : 3,
"isPublished" : true,
"publishDate" : ISODate("2017-03-11T00:00:00.000Z"),
"expireDate" : ISODate("2017-05-10T00:00:00.000Z"),
"keywords" : [
"dasfdsaf",
"afdas",
"fdasf",
"dafd"
],
"deleted" : false
}
/* 2 */
{
"_id" : ObjectId("58e87ed516b51f33ded59eb3"),
"vacancyNumber" : "213123",
"position" : "Software Developer",
"number" : 4,
"isPublished" : true,
"publishDate" : ISODate("2017-04-14T00:00:00.000Z"),
"expireDate" : ISODate("2017-05-09T00:00:00.000Z"),
"keywords" : [
"adfsadf",
"dasfdsaf"
],
"deleted" : false
}
/* 3 */
{
"_id" : ObjectId("58eb5b01c21fbad780bc74b6"),
"vacancyNumber" : "2432432",
"position" : "Web Designer",
"number" : 4,
"isPublished" : true,
"publishDate" : ISODate("2017-04-09T00:00:00.000Z"),
"expireDate" : ISODate("2017-05-12T00:00:00.000Z"),
"keywords" : [
"adsaf",
"das",
"fdafdas",
"fdas"
],
"deleted" : false
}
/* 4 */
{
"_id" : ObjectId("590f04fbf97a5803636ec66b"),
"vacancyNumber" : "4354",
"position" : "Software Developer",
"number" : 5,
"isPublished" : true,
"publishDate" : ISODate("2017-05-19T00:00:00.000Z"),
"expireDate" : ISODate("2017-05-27T00:00:00.000Z"),
"keywords" : [
"PHP",
"MySql"
],
"deleted" : false
}
Suppose I have three link in my application interface:
1- 30 hour ago posts.
2- 48 hour ago posts.
3- last one month posts.
Now if user click on first link i should control to group posts only for 30 hour ago, but if user click on second link, i should prepare my query to group posts for 30 hour and also for 48 hour, and if user click on third link i should prepare for all of them.
I want something like:
var pipeline = [
{
$match: match
},
{
$group: {
_id: null,
if (myVariable) {
thirtyHourAgo: {
........
........
}
}
if (mysecondVariable) {
fortyEightHourAgo: {
........
........
}
}
You can use javascript to dynamically create json document based on your query parameters.
Your updated function will look something like
post.getSpecificDateRangeJobs = function(queryData, callback) {
var matchCriteria = queryData.matchCriteria;
var currentDate = new Date();
// match document
var match = {
"expireDate": {
"$gte": currentDate
}
};
if (matchCriteria !== "") {
match["$text"]: {
"$search": matchCriteria
}
};
// group document
var group = {
_id: null
};
// Logic to calculate hours difference between current date and publish date is less than 30 hours.
if (queryData.dateGroups.thirtyHourAgo) {
group["thirtyHourAgo"] = {
"$sum": {
"$cond": [{
"$lte": [{
"$divide": [{
"$subtract": [currentDate, "$publishDate"]
}, 1000 * 60 * 60]
}, 30]
},
1,
0
]
}
};
}
// Similarly add more grouping condition based on query params.
var postsCollection = post.getDataSource().connector.collection(
post.modelName
);
// Use aggregate builder to create aggregation pipeline.
postsCollection.aggregate()
.match(match)
.group(group)
.exec(function(err, groupByRecords) {
if (err) {
return callback("err");
}
return callback(null, groupByRecords);
});
};
As I understood, I can suggest you following general query. Modify this according to your need.
db.getCollection('vacancy')
.aggregate([{$match: { $and: [
{publishDate:{ $gte: new Date(2017, 4, 13) }} ,
{publishDate:{ $lte: new Date(2017, 4, 14) }}
]} }])
Summary:
Used match to filter out result.
We are using aggregation Pipeline so you can add more aggregate operators n the pipeline
Using $and perform a logical AND because we want to fetch some documents between a give range say 1 day, 2 days or 1 month (change date parameters according to your requirement)
I want to build online test application using mongoDB and nodeJS. And there is a feature which admin can view user test history (with date filter option).
How to do the query, if I want to display only user which the test results array contains date specified by admin.
The date filter will be based on day, month, year from scheduledAt.startTime, and I think I must use aggregate framework to achieve this.
Let's say I have users document like below:
{
"_id" : ObjectId("582a7b315c57b9164cac3295"),
"username" : "lalalala#gmail.com",
"displayName" : "lalala",
"testResults" : [
{
"applyAs" : [
"finance"
],
"scheduledAt" : {
"endTime" : ISODate("2016-11-15T16:00:00.000Z"),
"startTime" : ISODate("2016-11-15T01:00:00.000Z")
},
"results" : [
ObjectId("582a7b3e5c57b9164cac3299"),
ObjectId("582a7cc25c57b9164cac329d")
],
"_id" : ObjectId("582a7b3e5c57b9164cac3296")
},
{
.....
}
],
"password" : "andi",
}
testResults document:
{
"_id" : ObjectId("582a7cc25c57b9164cac329d"),
"testCategory" : "english",
"testVersion" : "EAX",
"testTakenTime" : ISODate("2016-11-15T03:10:58.623Z"),
"score" : 2,
"userAnswer" : [
{
"answer" : 1,
"problemId" : ObjectId("581ff74002bb1218f87f3fab")
},
{
"answer" : 0,
"problemId" : ObjectId("581ff78202bb1218f87f3fac")
},
{
"answer" : 0,
"problemId" : ObjectId("581ff7ca02bb1218f87f3fad")
}
],
"__v" : 0
}
What I have tried until now is like below. If I want to count total document, which part of my aggregation framework should I change. Because in query below, totalData is being summed per group not per whole returned document.
User
.aggregate([
{
$unwind: '$testResults'
},
{
$project: {
'_id': 1,
'displayName': 1,
'testResults': 1,
'dayOfTest': { $dayOfMonth: '$testResults.scheduledAt.startTime' },
'monthOfTest': { $month: '$testResults.scheduledAt.startTime' },
'yearOfTest': { $year: '$testResults.scheduledAt.startTime' }
}
},
{
$match: {
dayOfTest: date.getDate(),
monthOfTest: date.getMonth() + 1,
yearOfTest: date.getFullYear()
}
},
{
$group: {
_id: {id: '$_id', displayName: '$displayName'},
testResults: {
$push: '$testResults'
},
totalData: {
$sum: 1
}
}
},
])
.then(function(result) {
res.send(result);
})
.catch(function(err) {
console.error(err);
next(err);
});
You can try something like this. Added the project stage to keep the test results if any of result element matches on the date passed. Add this as the first step in the pipeline and you can add the grouping stage the way you want.
$map applies an equals comparison between the date passed and start date in each test result element and generates an array with true and false values. $anyElementTrue inspects this array and returns true only if there is atleast one true value in the array. Match stage to include only elements with matched value of true.
aggregate([{
"$project": {
"_id": 1,
"displayName":1,
"testResults": 1,
"matched": {
"$anyElementTrue": {
"$map": {
"input": "$testResults",
"as": "result",
"in": {
"$eq": [{ $dateToString: { format: "%Y-%m-%d", date: '$$result.scheduledAt.startTime' } }, '2016-11-15']
}
}
}
}
}
}, {
"$match": {
"matched": true
}
}])
Alternative Version:
Similar to the above version but this one combines both the project and match stage into one. The $cond with $redact accounts for match and when match is found it keeps the complete tree or else discards it.
aggregate([{
"$redact": {
"$cond": [{
"$anyElementTrue": {
"$map": {
"input": "$testResults",
"as": "result",
"in": {
"$eq": [{
$dateToString: {
format: "%Y-%m-%d",
date: '$$result.scheduledAt.startTime'
}
}, '2016-11-15']
}
}
}
},
"$$KEEP",
"$$PRUNE"
]
}
}])