How to write a query to find documents which are created after "Aug 1 ,2020,3:58:56 PM".
I need to find documents from Cloudant which is greater than particular date. At Cloudant date is stored as "May 18, 2020, 3:58:56 PM".
when I try to query like below:
{
"selector": {
"$and": [
{
"project.projectId": {
"$eq": 34567
}
},
{
"createdDate": {
"$gte": "Aug 1 ,2020,3:58:56 PM"
}
}
]
}
}
I am getting 2019 data also. The proper search is not working.
What is the right way to filter date?
The date in your documents is a non-comparable string. You're asking Cloudant Query to do a lexicographic string comparison. For that to work, your date strings need to be in a comparable format, like "YYYY-MM-DD HH:MM:SS". However, the better solution is to use a numeric datetime.
If you don't want to modify your documents to use a different format, create a map-reduce view that first parses the date string, and then emits a vector-valued key:
d = Date.parse(doc.createdDate);
emit([doc.projectId, d.getFullYear(), d.getMonth()+1, d.getDate()], [...]);
and you can do elegant range queries from this view.
Some useful examples and other date-related tricks here: https://blog.cloudant.com/2018/08/24/Time-sortable-document-ids.html
Related
We have a collection AbstractEvent with field 'text', which contains 1~30 Chinese characters and we want to perform LIKE match with %keyword%, with high performance(less than 0.3 second, for more 2 million records).
After a bunch of effort, we decided to use VIEW and analyzer identity to do this:
FOR i IN AbstractEventView
SEARCH ANALYZER(i.text LIKE '%keyword%', 'identity')
LIMIT 10
RETURN i.text
And here is the definition of view AbstractEventView
{
"name":"AbstractEventView",
"type":"arangosearch",
"links":{
"AbstractEvent":{
"analyzers":[
"identity"
],
"fields":{
"text":{}
}
}
}
}
However, records returned contain irrelevant ones.
The flowlling is an example:
FOR i IN AbstractEventView
SEARCH ANALYZER(i.text LIKE '%速%', 'identity')
LIMIT 10
RETURN i.text
and the result is
[
"全球经济增速虽军官下滑",
"油食用消费出现明显下滑",
"本次国家经济快速下行",
"这场所迅速爆发的情况",
"经济减速风景空间资本大规模流出",
"苜蓿草众人食品物资价格不稳定",
"荤菜价格快速走低",
"情况快速升级",
"情况快速进展",
"四季功劳增速断崖式回落后"
]
油食用消费出现明显下滑and苜蓿草众人食品物资价格不稳定 are irrelavent.
We've been struggling on this for days, can anyone help me out? Thanks.
PS:
Why we do not use FULL-TEXT index?
full-text index indexed fields by tokenized text, so that we can not get matching '货币超发' when keyword is '货',because '货币' is recgonized as a word.
Why we do not use FILTER with LIKE operator directly?
Filtering without index will cost about 1 second and it is not acceptable.
I have the database collection as such:
{
data: [
item: {
date: "2022-03-22T08:10:37.023Z"
},
item: {
date: "2022-03-22T08:11:04.023Z"
},
...
]
}
I also have variables start_date and end_date which are momentjs dates. I can compare dates with isAfter() and isBefore() methods.
How to get all items with date between start_date and end_date?
Note: date: "..." is a string and not ISODate object.
data is an array.
I can also convert the date string to momentjs date for comparison.
looks like your date is in ISO format so the moment constructor should accept it, e.g.
moment(date);
To prove it, convert it into a plain js Date object and print it
moment("2022-03-22T08:11:04.023Z").toDate()
// Tue Mar 22 2022 10:11:04 GMT+0200 (Israel Standard Time)
You may want to look at isBetween
moment(date).isBetween(start_date, end_date);
Will return true if date is between those two moments.
After that you can use something like .filter to remove the data items that are not between your range and that's about it.
I want to find a document using the date field contained in the collection
//Document Exemple: MODEL : MY_EVENT
event_time_produce: '2018-01-03T11:00:00.201Z',
event_name : 'show'
I would like to find all the documents starting with '2018-01-03T11:00:'
I want to ignore the part of Millesecobde & ISOtype ..etc
Like in SQL :
SELECT * from MY_EVENT where event_time_produce LIKE '2018-01-03T11:00:%'
I have tried this but not working ( Mongoose)
MY_EVENT.find({event_time_produce : {
$regex: new RegExp('^2018-01-03T11:00:00.*$')
}})
.exec(....)
Simple search work :
MY_EVENT.find({event_time_produce : '2018-01-03T11:00:00.201Z'})
.exec(....)
but I want to get all the documents from 03 Janury 2018 at 11:00:XX
no matter the second just verify the date, hours & minute.
Isn't it an issue on your regular expression?
On your example for SQL you have 2018-01-03T11:00:% When on mongo you got ^2018-01-03T11:00:00.*$ (2 additional zeros after 11:00:)
Update:
If your data on the actual document is ISODate instead of a regular string, better to use ranges like this:
{event_time_produce:{$gte:ISODate("2018-01-03T11:00:00.000+0000"),$lte:ISODate("2018-01-03T11:00:59.999+0000") }}
I have a mongo db with a paymentDate field stored as strings YYYY-MM-DD (I can't change how these are unfortunately), and I need to do a date range search on them.
I found a post suggesting something like this:
coll.find({paymentDate: new RegExp("^(2016-02-18|2016-06-19)", "i")});
But I can't seem to get that to work at all.
Any help is appreciated, I've hit a dead end here.
Storing dates as strings in YYYY-MM-DD format works fine in this case as the string ordering matches the date ordering so you can do:
coll.find({paymentDate: {$gte: "2016-02-18", $lte: "2016-06-19"}});
Your first objective should be changing the schema and use Date instead of String. But if it's really not possible and you can't think of a really complicated RegEx, you can do it with Javascript using $where, but it's performance will be much slower:
db.test.find({
$where: function () {
var docDate = new Date(this.paymentDate);
return (docDate >= new Date('2016-02-18') && docDate <= new Date('2016-02-19'))
}
})
My index has a string field containing a variable length random id. Obviously it shouldn't be analysed.
But I don't know much about elasticsearch especially when I created the index.
Today I tried a lot to filter documents based on the length of id, finally I got this groovy script:
doc['myfield'].values.size()
or
doc['myfield'].value.size()
both returns mysterious numbers, I think that's because of the field got analysed.
If it's really the case, is there any way to get the original length or fix the problem, without rebuild the whole index?
Use _source instead of doc. That's using the source of the document, meaning the initial indexed text:
_source['myfield'].value.size()
If possible, try to re-index the documents to:
use doc[field] on a not-analyzed version of that field
even better, find out the size of the field before you index the document and consider adding its size as a regular field in the document itself
Elasticsearch stores a string as tokenized in the data structure ( Field data cache )where we have script access to.
So assuming that your field is not not_analyzed , doc['field'].values will look like this
"In america" => [ "in" , "america" ]
Hence what you get from doc['field'].values is a array and not a string.
Now the story doesn't change even if you have a single token or have the field as not_analyzed.
"america" => [ "america" ]
Now to see the size of the first token , you can use the following request
{
"script_fields": {
"test1": {
"script": "doc['field'].values[0].size()"
}
}
}