I've written a program that reads a file containing the date (in yyyy/MM/dd format) and uses the Datastax Java Driver to read the date and add it to a cassandra table.
So for instance, if my record contains a date value of '2010/06/01', then this date value gets converted into a date object (using the SimpleDateFormat class).
However, when I view the data (containing the date) in the database, I see that the date (which in the cassandra table is a timestamp type) shows the following:
2010-06-01 00:00:00+0100
The issue here is that I don't want the timestamp to have "+0100" (to indicate that this is british summer time), rather I'd want to store the date just as "2010-06-01 00:00:00+0000".
I've done the following to my program to try and 'ignore' the timezone by doing the following:
SimpleTimeZone tz = new SimpleTimeZone(0, "Out Timezone");
TimeZone.setDefault(tz);
String dateStringFromFile = "2010/06/01";
SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd");
Date theDate = sdf.parse(dateStringFromFile);
...Now when I add debug statements to my program, I can see that the date shows "2010-06-01 00:00:00+0000" on my log file (this is right for me). However when i see the date stored in Cassandra, i still see that the date shows as
"2010-06-01 00:00:00+0100" and not "2010-06-01 00:00:00+0000".
Is there anything on the cassandra side that I would have to change or update to ignore the timezone (i.e. not put +0100 on the date and to put +0000), so that the timestamp shows as "2010-06-01 00:00:00+0000"?
Please note that I am running Cassandra 3.0.5 on a Docker VM (Centos linux), Java 8.
Any advice is appreciated.
Thanks.
This solution has been sorted out. Nothing to do with cqlshrc. Its to do with forcing the timezone as 0 (like the code in the original post) and getting the time in milliseconds and writing that to the database - (which is a timestamp type column)
Related
I am using Cassandra 3.11.13 and I have table with timestamp column. Where are my data stored in terms of +0 timezone, i.e. 2022-10-14 07:51:00.000000+0000, but I am hosting in Kazakhstan GMT+6
I want to export certain rows and certain period of time. When I am exporting into CSV, I am getting a file with timezone +0.
I tried to query like select * from table_name where primary_key = 'smth' and timestamp > '2022-10-14T06:30:00+0600' and timestamp < '2022-10-14T23:59:59+0600', but it's changed nothing.
Question is: How can I fetch timestamp with certain/correct timestamp?
The CQL timestamp data type is encoded as the number of milliseconds since Unix epoch (Jan 1, 1970 00:00 GMT) so its value is encoded in UTC timezone. Clients also display timestamps with a UTC timezone by default.
If you want the data to be displayed in your timezone, you need to configure your app or client to a specific timezone. For example, you can configure cqlsh to use a different timezone by specifying it in the cqlshrc file:
;; Display timezone
timezone = Australia/Melbourne
You can find a sample copy of cqlshrc here. Note that you will need to install the pytz Python library to use different timezones with cqlsh.
For details, see Cassandra CQL shell. Cheers!
I have a node application that receives data from MQTT, where various parameters and timestamp are sent to the broker. The server and hardware are correctly configured with local time (Spain).
When I am going to do the insertion in the database, in this case MongoDB, I format the timestamp to be able to insert it in the mongo date format.
The problem is that only the javascript Date() can insert the date format in mongo and I always get two hours less than my local time.
//Format timestamp to date with dayjs
let date = dayjs(data.channels.timestamp * 1000).format()
//Output: 2021-09-16T13:32:33+02:00
//Format date to insert in MongoDB
let timestamp_ = new Date(date);
//Output: 2021-09-16T11:32:33.000Z
The correct date is: 2021-09-16T13:32:33+02:00
I tried momentjs and I have the same problem.
It is normal to have date in timezone GMT+0. Date object assumes the date is always GMT+0. Keep it like that and store everything in GMT+0.
Manipulate your query according to the time zone, not the data stored.
I'm making a small employee management system in which I have to play with dates in many places like attendance, leaves etc.
I use luxon for dates in my node.js project. Ex: DateTime.fromISO("2020-10-15"); gives me this in the console: 2020-10-15T00:00:00.000+05:00
but when I save it in mongodb it becomes 2020-10-14T19:00:00.000+00:00 and this is problematic because i need to query objects to check if there is an attendance marked in the same day or to get today's attendance. How can I solve this issue?
Storage
When MongoDB stores time-type data, it is first converted to UTC time and then stored in the database. The date submitted by the client is East 5, and the conversion to UTC time is minus 5 hours;
Read
MongoDB will convert the UTC date stored in the database into the East 5 zone of the client according to the time zone of the client, and then return it to the client
MongoDB only has a timestamp type, it does not have a date type. Dates are converted to times when they are stored. Which timezone the date is interpreted to be in can vary, looks like your driver uses the beginning of specified date in UTC. Ruby works the same way and you can read more about the principle here.
To "solve this issue", either only store and query by timestamps in MongoDB, converting dates to timestamps in your application, or understand how your language and driver converts dates to timestamps (note that JS Date type is actually a timestamp).
I've got an edit page with a "date" type input. I'm using node.js, express, and mongoose to pull the content of the input and save it to the database using a date field in a Mongoose model. When I log the date before it goes into the database I'm seeing:2020-03-31. When I look at it in the database I'm seeing:2020-03-31T00:00:00.000+00:00. Does that +00:00 mean it's getting a timezone assigned? Do I need to edit how my MongoDB database is storing dates?
I'm running into trouble with the time zone when I read it back out and use it to populate the input on the edit page. I'm using a Mongoose model function to format the date before it goes into the value field on the html input. Within that function I'm running the below code:
console.log(date);
// Output:
// 2020-03-31T00:00:00.000Z
console.log(date.getDate());
// Output:
// 30
Is the 'Z' coming from the "+00:00" above? I'm GMT-4, so my server is interpreting that time as March 30th at 8pm and now the date picker on the edit page is populated with March 30th instead of the 31st. If I save this, then the next time I load the page the input will read March 29th. I don't want to unintentionally decrement the date by one every time the page is loaded and saved!
In this case, I really don't care about time and just want to refer to the date. Is there a different best practice I can use here?
MongoDB stores timestamps only, it does not have a facility to store dates. Javascript similarly does not have a facility for storing dates natively contrary to what you might expect - the Date type is actually a DateTime. Mongoose also does not have a date type (it uses JS Date which is a datetime).
What this means is:
When you are converting user input to a JS Date instance, you are getting the date converted to a datetime (sounds like the time component is zero in UTC in your application).
When you are reading the date out of the database, you are reading a datetime (which is the beginning of the day of that date in UTC per your provided information, not in your local time).
I would then proceed as follows:
The DateTime you are reading is the beginning of day on your date in UTC.
If mongoose converts the time to your local time, as it appears to do, you need to either stop it from doing that or convert back to UTC.
Then retrieve the year-month-day components of the UTC datetime.
Using these components you can create a new Date instance in local time or do whatever else you wish such as rendering them.
There are various libraries available in JS for dealing with dates and time zones, but the language itself has limited date support.
I need to split my timestamp to date and time separately and insert then to db columns with 'date' and 'time' cqltypes.
I was trying to insert a time value as string to Cassandra table. The time was converted to UTC (05:27:00). But while I checked table using Datastax devcenter, column was populated with value '09:37:54.935541808'. I tried to retrieve the value in spring using repository, then it was returning value as '3473746674935541808'.
How to get the correct value from table for time?
It looks like the limitation of Spring-data. In Cassandra time value is encoded as a 64-bit signed integer representing the number of nanoseconds since midnight. But I don't see the time type listed as supported in spring-data-cassandra documentation, so you may need to write your custom converter for it, as described in documentation.