I have a list of Rest objects. It's django model
class Rest(models.Model):
product = models.ForeignKey('Product', models.DO_NOTHING)
date = models.DateTimeField()
rest = models.FloatField()
I want to select objects from it for today's date. I do it like this. Maybe there is some more convenient and compact way?
for rest in rests_list:
if rest.date.day == datetime.now().day:
result.append(rest)
First - datetime.now().day will get you the day of the month (e.g. 18 if today is 18th March 2020), but you've said you want today's date. The below is on the assumption you want today's date, not the day of the month.
(NOTE: weAreStarDust pointed out that the field is a DateTimeField, not a DateField, which I missed and have updated the answer for)
The way you are doing it right now seems like it might be fetching all of the Rests from the database and then filter them in your application code (assuming rests_listcomes from something likeRest.objects.all()`. Generally, you want to do as much filtering as possible on the database query itself, and as little filtering as possible in the client code.
In this case, what you probably want to do is:
from datetime import date
Rest.objects.filter(date__date=date.today())
That will bring back only the records that have a date of today, which are the ones you want.
If you already have all the rests somehow, and you just want to filter to the ones from today then you can use:
filter(lambda x: x.date.date() == date.today(), rests_list)
That will return a filter object containing only the items in rests_list that have date == date.today(). If you want it as a list, rather than an iterable, you can do:
list(filter(lambda x: x.date.date() == date.today(), rests_list))
or for a tuple:
tuple(filter(lambda x: x.date.date() == date.today(), rests_list))
NOTE:
If you actually want to be storing only a date, I would suggest considering use of a DateField (although not if you want to store any timezone information).
If you want to store a DateTime, I would consider renaming the field from date to datetime, or started_at - calling the field date but having a datetime can be a bit confusing and lead to errors.
As docs says
For datetime fields, casts the value as date. Allows chaining additional field lookups. Takes a date value.
from datetime import datetime
Rest.objects.filter(date__date = datetime.now().day)
You can use the django filter for filtering and get only today's date data from your model. No need to fetch all data first and then apply loop for get today's date data. You have to write your query like ...
import datetime
Rest.objects.filter(date__date = datetime.date.today())
But be sure that timezone should be same for database server and web server
Related
I want to schedule the mailing of a Cognos report, always using the current date. So, for example: today, I want to e-mail the report with data up until the 28th, but tomorrow it should contain data up until the 29th.
How can I achieve this?
Thanks in advance!
If you're expecting code, you didn't provide enough information, but let me try...
Assuming the "date" data against which you want to filter is in a query item named [Date] which is in a query subject named [Time] which is in a namespace named [namespace], create a filter like this:
[namespace].[Time].[Date] = current_date
If you want up to the current date which includes the days leading up to it you can use what dougp posted slightly modified.
[namespace].[Time].[Date] <= current_date
to ensure the where clause is pushed down to the database, I personally like to use a macro for current_date. So the expression becomes
[namespace].[Time].[Date] <= # timestampMask ( $current_timestamp , 'yyyy-mm-dd' ) #
Greetings Stackoverflowers
I have been using (eland to insert a pandas dataframe as an elasticsearch document. The code used to make this happen is shown as follows and is strongly based on the one in the url
import eland as ed
def save_to_elastic(data_df, elastic_engine, index_name, type_overrides_dict, chunk_size):
"""
es_type_overrides={
"fechaRegistro": "date",
"fechaIncidente": "date"
}
"""
df = ed.pandas_to_eland(
pd_df=data_df,
es_client=elastic_engine,
# Where the data will live in Elasticsearch
es_dest_index=index_name,
# Type overrides for certain columns, the default is keyword
# name has been set to free text and year to a date field.
es_type_overrides=type_overrides_dict,
# If the index already exists replace it
es_if_exists="replace",
# Wait for data to be indexed before returning
es_refresh=True,
chunksize=chunk_size
)
I have used to insert the pandas dataframe inside elasticsearch as follows:
from snippets.elastic_utils import save_to_elastic, conect2elastic
es = conect2elastic(user='falconiel')
save_to_elastic(data_df=siaf_consumados_elk,
type_overrides_dict={'fechaRegistro':"date",
'fechaIncidente':"date"},
elastic_engine=es,
index_name='siaf_18032021_pc',
chunk_size=1000)
Everything works fine but once I have the document in elasticsearch 26 dates have been inserted wrongly inside elasticsearch. All my data starts in january 1 2015. But elasticsearch shows some documents with December 31 2014. I haven't been able to find an explanation for this. Why some of the rows in the pandas dataframe that have the date field correct (from 2015-01-01) were changed during loading to last day of december of previous year. I would appreciate any help or insight to correct this behavior.
My datetime columns in pandas dataframe are typed as datetime. However, I am trying to test the following conversions to address the problem. They have not been so productive by now:
I have tried using the following conversions before inserting calling the function I use to save to the dataframe in elastic:
siaf_consumados_elk.fechaRegistro = pd.to_datetime(siaf_consumados_elk.fechaRegistro).dt.tz_localize(None)
siaf_consumados_elk.fechaRegistro = pd.to_datetime(siaf_consumados_elk.fechaRegistro, utc=True)
In fact the problem is UTC. I checked some of the rows in the pandas dataframe and they were reduced almost one day. For instance, one record which was registered in 2021-01-02 GMT -5 appeared as 2021-01-01. The solution was to apply the corresponding time zone before calling the function to save the dataframe as an elastic document/index. So, considering the good observation given by Mark Walkom, this what I used before calling the function:
siaf_consumados_elk.fechaRegistro = siaf_consumados_elk.fechaRegistro.dt.tz_localize(tz='America/Guayaquil')
siaf_consumados_elk.fechaIncidente = siaf_consumados_elk.fechaIncidente.dt.tz_localize(tz='America/Guayaquil')
A list with the corresponding time zones can be found at: python time zones
This permitted to index the time corretly
I am new to couchdb, i have looked at the docs and SO posts but for some reason this simple query is still eluding me.
SELECT TOP 10 * FROM x WHERE DATE BETWEEN startdate AND enddate ORDER BY score
UPDATE: It cannot be done. This is unfortunate since to get this type
of data you have to pull back potentially millions of records (a few
fields) from couch then do either filtering, sorting or limiting
yourself to get the desired results. I am now going back to my
original solution of using _changes to capture and store elsewhere the data i do need to perform that query on.
Here is my updated view (thanks to Dominic):
emit([d.getUTCFullYear(), d.getUTCMonth() + 1, d.getUTCDate(), score], doc.name);
What I need to do is:
Always sort by score descending
Optionally filter by date range (for instance, TODAY only)
Limit by x
Update: Thanks to Dominic I am much closer - but still having an
issue.
?startkey=[2017,1,13,{}]&endkey=[2017,1,10]&descending=true&limit=10&include_docs=true
This brings back documents between the dates sorted by score
However if i want top 10 regardless of date then i only get back top 10 sorted by date (and not score)
For starters, when using complex keys in CouchDB, you can only sort from left to right. This is a common misconception, but read up on Views Collation for a more in-depth explanation. (while you're at it, read the entire Guide to Views as well since you're getting started)
If you want to be able to sort by score, but filter by date only, you can accomplish this by breaking down your timestamp to only show the degree you care about.
function (doc) {
var d = new Date(doc.date)
emit([ d.getUTCFullYear(), d.getUTCMonth() + 1, d.getUTCDate(), score ])
}
You'll end up outputting a more complex key than what you currently have, but you query it like so:
startkey=[2017,1,1]&endkey=[2017,1,1,{}]
This will pick out all the documents on 1-1-2017, and it'll be sorted by score already! (in ascending order, simply swap startkey and endkey to get descending order, no change to the view needed)
As an aside, avoid emitting the entire doc as the value in your view. It is likely more efficient to leverage the include_docs=true parameter, and leaving the value of your emit empty. (please refer to this SO question for more information)
With this exact setup, you'd need separate views in order to query by different precisions. For example, to query by month you just use the year/month and so on.
However, if you are willing/able to sort your scores in your application, you can use a single view to get all the date precision you want. For example:
function (doc) {
var d = new Date(doc.date)
emit([ d.getUTCFullYear(), d.getUTCMonth() + 1, d.getUTCDate(), d.getUTCHour(), d.getUTCMinutes(), d.getUTCSeconds(), d.getUTCMilliseconds() ])
}
With this view and the group_level parameter, you can get all the scores by year, month, date, hour, etc. As I mentioned, in this case it won't be sorted by score yet, but maybe this opens up other queries to you. (eg: what users participated this month?)
How do I save only date part with mongoose schmea. Currently my model is like this:
myDate: { type: Date}
Which saves date as "2016-02-27T00:00:00.000Z" even if I pass only: "2016-02-27".
I'm using:
dateOfBirth: { type: Object }
and saving like:
dateOfBirth: {year: 1988, month: 0, day: 8}
this gives me:
ability to search by year, month
ability to make date object:
const dateOfBirth = new Date(
user.dateOfBirth.year,
user.dateOfBirth.month,
user.dateOfBirth.day
);
which will avoid timezone shiftings, since date object will be created on device's environment
UPD
P.S. I was not writing pros and cons since my answer was about alternative way of storing date separately and it should be obvious when this approach used.
After chatting with SergiiStotskyi in comments below I decided to put pros and cons to warn devs to choose best solution.
Keep in mind this approach has very few benefit(s).
Pros (I could find only two):
separate year, month, day fields helps to filter by exact year, month, day. Like: find all records for May month grouped by Year (to compare differences for months for previous years).
in comparison with Date which sensitive to timezone, year, month, day is does not shift by timezone
not exactly benefit, date can be saved as it comes from client-side if every element date was separate too
Cons:
If needed to get records by age it will be hard to determine, will require approximation used by year field (which in my use cases fits best, since no-one cares exact numbers in my project)
For exact age filtering will require aggregate with matching by year and reducing by filtering out of range items
Need to validate leap year
Behind the scene, a mongoose date is just like a date in javascript : it stores hours, seconds etc... There is no distinction with date/datetime like in SQL. In for a penny, in for a pound. That being said, it just a matter of display.
The reason is that mongo doesn't support this kind of type. You could create an object with year/month/day properties but it would be a pain to deal with.
In plain SQL (in my case: sqlite), I would be able to query for a specific date in a DATE column as follows:
SELECT * FROM table WHERE date(dateColumn) = '2015-01-01'
This works because the date() function cuts off the time part of the DATE value.
Can I do something similar with Predicates in Core Data? Or do I have to use something where I determine the start and end of that day and then look for dates between the two date values?