The indexed time of the log is different from the logged time of the log.
Logged time - 2021-01-25 04:19:18,002
Indexed time - Jan 25, 2021 # 04:19:27.750
There is a 10 seconds delay when comparing both the timestamps. Is there a way to make the indexed timestamp same as the logged time timestamp.
You need to use a date filter to set the #timestamp to the timestamp field of the log.
So if you've got the time in a field called log_time, you'd use a date filter like this:
date {
match => [ "log_time", "ISO8601" ]
}
See the logstash documentation for additional details on how the date filter works.
Related
I'm making a small employee management system in which I have to play with dates in many places like attendance, leaves etc.
I use luxon for dates in my node.js project. Ex: DateTime.fromISO("2020-10-15"); gives me this in the console: 2020-10-15T00:00:00.000+05:00
but when I save it in mongodb it becomes 2020-10-14T19:00:00.000+00:00 and this is problematic because i need to query objects to check if there is an attendance marked in the same day or to get today's attendance. How can I solve this issue?
Storage
When MongoDB stores time-type data, it is first converted to UTC time and then stored in the database. The date submitted by the client is East 5, and the conversion to UTC time is minus 5 hours;
Read
MongoDB will convert the UTC date stored in the database into the East 5 zone of the client according to the time zone of the client, and then return it to the client
MongoDB only has a timestamp type, it does not have a date type. Dates are converted to times when they are stored. Which timezone the date is interpreted to be in can vary, looks like your driver uses the beginning of specified date in UTC. Ruby works the same way and you can read more about the principle here.
To "solve this issue", either only store and query by timestamps in MongoDB, converting dates to timestamps in your application, or understand how your language and driver converts dates to timestamps (note that JS Date type is actually a timestamp).
I've got an edit page with a "date" type input. I'm using node.js, express, and mongoose to pull the content of the input and save it to the database using a date field in a Mongoose model. When I log the date before it goes into the database I'm seeing:2020-03-31. When I look at it in the database I'm seeing:2020-03-31T00:00:00.000+00:00. Does that +00:00 mean it's getting a timezone assigned? Do I need to edit how my MongoDB database is storing dates?
I'm running into trouble with the time zone when I read it back out and use it to populate the input on the edit page. I'm using a Mongoose model function to format the date before it goes into the value field on the html input. Within that function I'm running the below code:
console.log(date);
// Output:
// 2020-03-31T00:00:00.000Z
console.log(date.getDate());
// Output:
// 30
Is the 'Z' coming from the "+00:00" above? I'm GMT-4, so my server is interpreting that time as March 30th at 8pm and now the date picker on the edit page is populated with March 30th instead of the 31st. If I save this, then the next time I load the page the input will read March 29th. I don't want to unintentionally decrement the date by one every time the page is loaded and saved!
In this case, I really don't care about time and just want to refer to the date. Is there a different best practice I can use here?
MongoDB stores timestamps only, it does not have a facility to store dates. Javascript similarly does not have a facility for storing dates natively contrary to what you might expect - the Date type is actually a DateTime. Mongoose also does not have a date type (it uses JS Date which is a datetime).
What this means is:
When you are converting user input to a JS Date instance, you are getting the date converted to a datetime (sounds like the time component is zero in UTC in your application).
When you are reading the date out of the database, you are reading a datetime (which is the beginning of the day of that date in UTC per your provided information, not in your local time).
I would then proceed as follows:
The DateTime you are reading is the beginning of day on your date in UTC.
If mongoose converts the time to your local time, as it appears to do, you need to either stop it from doing that or convert back to UTC.
Then retrieve the year-month-day components of the UTC datetime.
Using these components you can create a new Date instance in local time or do whatever else you wish such as rendering them.
There are various libraries available in JS for dealing with dates and time zones, but the language itself has limited date support.
I am creating an app in node js and saving date to Mongo DB, the datatype of Mongoose, I used to save is Date.
moment.locale("pt");
today= moment().startOf("day").format();
The date is printing like 2019-05-27T00:00:00-03:00
But when it is saving and I am trying to retrieve the date it is coming from mongo db like 2019-05-27T03:00:00.000Z
Now the comparison didn't work. So I am finding data based on this date and it is not working.
I am using locale for Brazil.I am keeping the user details day wise from 00:00 to next 24 hours and doing a luck draw of users. So I have huge database and want to filter data based on today date. So I need date should be saved in local timezone i.e. Brazil.
Any help would be appreciated.
I suggest moving to UTC, then changing the locale, then mutating the date to match the day's beginning and then storing as ms:
const d = moment
.utc() // Get the UTC time
.locale('pt_BR') // Change locale to Brazil
.startOf('day') // Go back in time to current date # 00:00:00
.valueOf() // Get ms value
// store d in the database
If you have some date time parameter in generic inquiry to use it in where clause, it will be sent to the sql query in local time zone. Does anybody know how to convert it to UTC date time? I need this because date time fields are stored in UTC in Acumatica database.
The easy example is GI to show events in the specified date time span.
To convert date time parameter into UTC, one can use -Now()+NowUTC() functions.
Here is an example
I am trying to store and retrieve three datetime columns on azure table storage.
Start date, End date and Last Executed date.
I have another fourth column which is called timeIntervalInMinutes.
My code execution will happen on an azure VM which might be on any zone in US.
I am going to use these values to execute some task based on current time:
start date >= current date <= end date
Current time = Last Executed Date (time) + intervalInMinutes
There are few doubts that I have:
The user is going to call this worker service on azure from their app, from any timezone US. Do I need to get their timezone in their request and store it along with the date time, which will get stored in UTC format on table storage?
If Yes, then if the user tries to retrieve this configuration information, should I convert based on UTC to local time conversion?
I need to execute my custom task based on the local time of the user. So when converting to UTC and storing it in the azure table, I need to say UTC + or - to suit the local time of the user's timezone?
Please help.
The OP wrote in a comment:
I have solved this by adding entity properties with DateTimeOffset type and created separate fields for holding UtcOffset timespan value as ticks (long type). With this I can compare the UTC format timestamp with UtcOffset values and date as well for my scheduler to work. So it's now easy for me to query against date and as well as retrieve and present the user back with his / her own specified local time.