I need to store resource creation date in arangodb. Furthermore, I need to filter resources by creation date:
FOR doc in Docs
FILTER doc.created < #some_date
RETURN doc
What is the best way to do that?
Possible solutions:
{
// one
created: "<iso8601-date>", // Persistent index, use DATE_TIMESTAMP()
// two
created: "<timestamp-str>", // Persistent index, use TO_NUMBER
// three
created: <timestamp-int>, // Persistent index
}
Just found this documentation. In brief: date can be stored as string in iso8601 format either as a numeric timestamp. All compare operations will work correctly and efficiently.
I'd recommend you store dates in the Unix Epoch Seconds format, as this helps with date range queries and ensures your clients can reliably format the date to a usable format.
When the date is just a number, then it's very easy to do time queries over ranges as 1 hour is just 3600 seconds.
If you need more granularity, then storing dates as Unix Epoch MilliSeconds also helps.
You can store dates earlier than 1/1/1970 with this format, they will just be negative integers.
Related
I'm making a small employee management system in which I have to play with dates in many places like attendance, leaves etc.
I use luxon for dates in my node.js project. Ex: DateTime.fromISO("2020-10-15"); gives me this in the console: 2020-10-15T00:00:00.000+05:00
but when I save it in mongodb it becomes 2020-10-14T19:00:00.000+00:00 and this is problematic because i need to query objects to check if there is an attendance marked in the same day or to get today's attendance. How can I solve this issue?
Storage
When MongoDB stores time-type data, it is first converted to UTC time and then stored in the database. The date submitted by the client is East 5, and the conversion to UTC time is minus 5 hours;
Read
MongoDB will convert the UTC date stored in the database into the East 5 zone of the client according to the time zone of the client, and then return it to the client
MongoDB only has a timestamp type, it does not have a date type. Dates are converted to times when they are stored. Which timezone the date is interpreted to be in can vary, looks like your driver uses the beginning of specified date in UTC. Ruby works the same way and you can read more about the principle here.
To "solve this issue", either only store and query by timestamps in MongoDB, converting dates to timestamps in your application, or understand how your language and driver converts dates to timestamps (note that JS Date type is actually a timestamp).
I've got an edit page with a "date" type input. I'm using node.js, express, and mongoose to pull the content of the input and save it to the database using a date field in a Mongoose model. When I log the date before it goes into the database I'm seeing:2020-03-31. When I look at it in the database I'm seeing:2020-03-31T00:00:00.000+00:00. Does that +00:00 mean it's getting a timezone assigned? Do I need to edit how my MongoDB database is storing dates?
I'm running into trouble with the time zone when I read it back out and use it to populate the input on the edit page. I'm using a Mongoose model function to format the date before it goes into the value field on the html input. Within that function I'm running the below code:
console.log(date);
// Output:
// 2020-03-31T00:00:00.000Z
console.log(date.getDate());
// Output:
// 30
Is the 'Z' coming from the "+00:00" above? I'm GMT-4, so my server is interpreting that time as March 30th at 8pm and now the date picker on the edit page is populated with March 30th instead of the 31st. If I save this, then the next time I load the page the input will read March 29th. I don't want to unintentionally decrement the date by one every time the page is loaded and saved!
In this case, I really don't care about time and just want to refer to the date. Is there a different best practice I can use here?
MongoDB stores timestamps only, it does not have a facility to store dates. Javascript similarly does not have a facility for storing dates natively contrary to what you might expect - the Date type is actually a DateTime. Mongoose also does not have a date type (it uses JS Date which is a datetime).
What this means is:
When you are converting user input to a JS Date instance, you are getting the date converted to a datetime (sounds like the time component is zero in UTC in your application).
When you are reading the date out of the database, you are reading a datetime (which is the beginning of the day of that date in UTC per your provided information, not in your local time).
I would then proceed as follows:
The DateTime you are reading is the beginning of day on your date in UTC.
If mongoose converts the time to your local time, as it appears to do, you need to either stop it from doing that or convert back to UTC.
Then retrieve the year-month-day components of the UTC datetime.
Using these components you can create a new Date instance in local time or do whatever else you wish such as rendering them.
There are various libraries available in JS for dealing with dates and time zones, but the language itself has limited date support.
I have a list of Rest objects. It's django model
class Rest(models.Model):
product = models.ForeignKey('Product', models.DO_NOTHING)
date = models.DateTimeField()
rest = models.FloatField()
I want to select objects from it for today's date. I do it like this. Maybe there is some more convenient and compact way?
for rest in rests_list:
if rest.date.day == datetime.now().day:
result.append(rest)
First - datetime.now().day will get you the day of the month (e.g. 18 if today is 18th March 2020), but you've said you want today's date. The below is on the assumption you want today's date, not the day of the month.
(NOTE: weAreStarDust pointed out that the field is a DateTimeField, not a DateField, which I missed and have updated the answer for)
The way you are doing it right now seems like it might be fetching all of the Rests from the database and then filter them in your application code (assuming rests_listcomes from something likeRest.objects.all()`. Generally, you want to do as much filtering as possible on the database query itself, and as little filtering as possible in the client code.
In this case, what you probably want to do is:
from datetime import date
Rest.objects.filter(date__date=date.today())
That will bring back only the records that have a date of today, which are the ones you want.
If you already have all the rests somehow, and you just want to filter to the ones from today then you can use:
filter(lambda x: x.date.date() == date.today(), rests_list)
That will return a filter object containing only the items in rests_list that have date == date.today(). If you want it as a list, rather than an iterable, you can do:
list(filter(lambda x: x.date.date() == date.today(), rests_list))
or for a tuple:
tuple(filter(lambda x: x.date.date() == date.today(), rests_list))
NOTE:
If you actually want to be storing only a date, I would suggest considering use of a DateField (although not if you want to store any timezone information).
If you want to store a DateTime, I would consider renaming the field from date to datetime, or started_at - calling the field date but having a datetime can be a bit confusing and lead to errors.
As docs says
For datetime fields, casts the value as date. Allows chaining additional field lookups. Takes a date value.
from datetime import datetime
Rest.objects.filter(date__date = datetime.now().day)
You can use the django filter for filtering and get only today's date data from your model. No need to fetch all data first and then apply loop for get today's date data. You have to write your query like ...
import datetime
Rest.objects.filter(date__date = datetime.date.today())
But be sure that timezone should be same for database server and web server
I have a JSON-based source I'd like to transform using ADF Mapping Data Flow. I have a string containing an epoch timestamp value that I want to transform to Datetime value to later sink it into Parquet file.
Do you know a way? Docs of this language are here.
Source file:
{
"timestamp":"1574127407",
"name":"D.A."
}
Use toTimestamp() and set the formatting you wish as 2nd parameter
toTimestamp(1574127407*1000l)
From string:
toTimestamp(toInteger(toString(byName('timestamp')))*1000l,'yyyy-MM-dd HH:mm:ss')
I have came across various epoch timestamp values which are of 13 digits i.e., they even have milliseconds detailed information.
In such case, converting to integer using 'toInteger' won't serve the purpose instead this will keep the values as NULL. So, to fix this issue, we need to convert it to Long using toLong as below:
toTimestamp(toLong(toString(created)),'yyyy-MM-dd HH:mm:ss')
In above expression, 'created' is a field whose value is 13-digit epoch timestamp, something like this created='1635359043307'.
Here, toTimestamp returns the Date Timestamp with above-mentioned date format.
FYI, you can use this site https://www.epochconverter.com/ to check epoch timestamp to human date.
How do I save only date part with mongoose schmea. Currently my model is like this:
myDate: { type: Date}
Which saves date as "2016-02-27T00:00:00.000Z" even if I pass only: "2016-02-27".
I'm using:
dateOfBirth: { type: Object }
and saving like:
dateOfBirth: {year: 1988, month: 0, day: 8}
this gives me:
ability to search by year, month
ability to make date object:
const dateOfBirth = new Date(
user.dateOfBirth.year,
user.dateOfBirth.month,
user.dateOfBirth.day
);
which will avoid timezone shiftings, since date object will be created on device's environment
UPD
P.S. I was not writing pros and cons since my answer was about alternative way of storing date separately and it should be obvious when this approach used.
After chatting with SergiiStotskyi in comments below I decided to put pros and cons to warn devs to choose best solution.
Keep in mind this approach has very few benefit(s).
Pros (I could find only two):
separate year, month, day fields helps to filter by exact year, month, day. Like: find all records for May month grouped by Year (to compare differences for months for previous years).
in comparison with Date which sensitive to timezone, year, month, day is does not shift by timezone
not exactly benefit, date can be saved as it comes from client-side if every element date was separate too
Cons:
If needed to get records by age it will be hard to determine, will require approximation used by year field (which in my use cases fits best, since no-one cares exact numbers in my project)
For exact age filtering will require aggregate with matching by year and reducing by filtering out of range items
Need to validate leap year
Behind the scene, a mongoose date is just like a date in javascript : it stores hours, seconds etc... There is no distinction with date/datetime like in SQL. In for a penny, in for a pound. That being said, it just a matter of display.
The reason is that mongo doesn't support this kind of type. You could create an object with year/month/day properties but it would be a pain to deal with.