Cassandra query timestamp column - cassandra

Using Cassandra 2.28, Drive 3, Sparks2.
I have a timestamp column in Cassandra I need to query it by the date portion only. If I query by date: .where("TRAN_DATE= ?", "2012-01-21" : it does not bring any result. If I include the time portion it says Invalid Date. My data (as I can read in cqlsh) is: 2012-01-21 08:01:00+0000
param: "2012-01-21" > No error but no result
param: "2012-01-21 08:01:00" > Error : Invalid Date
param: "2012-01-21 08:01:00+0000" > Error : Invalid Date
SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy/mm/dd");
TRAN_DATE = DATE_FORMAT.parse("1/19/2012");
Have used the bulk loader/SSLoader to load the table
Data in table:
tran_date | id
--------------------------+-------
2012-01-14 08:01:00+0000 | ABC
2012-01-24 08:01:00+0000 | ABC
2012-01-23 08:01:00+0000 | ALM
2012-01-29 08:01:00+0000 | ALM
2012-01-13 08:01:00+0000 | ATC
2012-01-15 08:01:00+0000 | ATI
2012-01-18 08:01:00+0000 | FKT
2012-01-05 08:01:00+0000 | NYC
2012-01-11 08:01:00+0000 | JDU
2012-01-04 08:01:00+0000 | LST
How do I solve this.
Thanks

If you insert data into timestamp column without providing timezone like this one :
INSERT INTO timestamp_test (tran_date , id ) VALUES ('2016-12-19','TMP')
Cassandra will choose coordinator timezone
If no time zone is specified, the time zone of the Cassandra coordinator node handing the write request is used. For accuracy, DataStax recommends specifying the time zone rather than relying on the time zone configured on the Cassandra nodes.
If you execute select with Datastax Driver, you Need To Convert the String date into java.util.Date and set the time zone of coordinator node, In my case it was GMT+6
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
Date date = dateFormat.parse("2012-01-21");
dateFormat.setTimeZone(TimeZone.getTimeZone("GMT+6")); //Change this time zone
Now You can query with
QueryBuilder.eq("TRAN_DATE", date)
Here is a complete demo :
try (Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").withCredentials("username", "password").build(); Session session = cluster.connect("tests")) {
session.execute("INSERT INTO test_trans(tran_date , id ) VALUES ('2016-12-19','TMP')");
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
dateFormat.setTimeZone(TimeZone.getTimeZone("GMT+6"));
Date date = dateFormat.parse("2016-12-19");
System.out.println(date);
for (Row row : session.execute(QueryBuilder.select().from("timestamp_test").where(QueryBuilder.eq("tran_date", date)))) {
System.out.println(row);
}
}
Source : https://docs.datastax.com/en/cql/3.0/cql/cql_reference/timestamp_type_r.html

Related

Fetch Last Login Details using Summarize by Time Stamp in KQL

I am trying to get last login details of user in Kusto database using KQL query language. However I am not getting exact result with below query.
GlobalID - Unique GUID Value which will be created every time user logged in
UserId - Logged in UserId value
LastSuccessFullLoginTimeStamp - Max Timestamp value
//Fetch Last Logged in userID details
let window = 2h;
Events
| where Timestamp >= ago(window)
| extend UserId = tostring(Properties.UserId)
| where UserId in ('12345','56789','24680')
//| summarize LastSuccessFullLoginTimeStamp = max(Timestamp), count() by
GlobalId,UserId
|project GlobalID,UserId,TimeStamp
But I am failed to get output as like below from above sample data. Fetch latest GlobalID based on userId and last logged in time. Where I am doing wrong? I tried with summarize, make_set but in vain.
You should use the arg_max() function:
let window = 2h;
Events
| where Timestamp >= ago(window)
| extend UserId = tostring(Properties.UserId)
| where UserId in ('12345','56789','24680')
| summarize arg_max(Timestamp, *) by UserId

How to create an Azure Kusto query to group by client OS name only (OS version removed) on App insights?

What I want
The number of page views grouped by client OS (no OS version = only OS name) and week.
What I have - Kusto query:
pageViews
|where timestamp > ago(90d)
|summarize Browser_hits = count() by Date = startofweek(timestamp), client_Browser
|sort by Date
|render timechart
The problem with this query is that the client OS name is coming with the version in it and that ends up in different versions grouping separately (see picture below).
Update
This is close to what I need but it won't work for any names, I'm just posting this as an example that helps to understand the actual question.
pageViews
|where timestamp > ago(90d)
|summarize Browser_hits = count() by Date = startofweek(timestamp), BrowserNameTrimed = substring(client_Browser,0,5)
| sort by Date
| render timechart
With the previous query I get this (kind of cheating):
would this work? (parsing the browser name out of the "browser name + browser version" combination using the parse operator):
pageViews
| where timestamp > ago(90d)
| summarize Browser_hits = count() by Date = startofweek(timestamp), client_Browser
| parse kind=regex client_Browser with client_Browser #" \d+" *
| render timechart

Search Query should contain 'AggregatedValue' and 'bin(timestamp, [roundTo])' for Metric alert type

I'm trying to create a custom metric alert based on some metrics in my Application Insights logs. Below is the query I'm using;
let start = customEvents
| where customDimensions.configName == "configName"
| where name == "name"
| extend timestamp, correlationId = tostring(customDimensions.correlationId), configName = tostring(customDimensions.configName);
let ending = customEvents
| where customDimensions.configName == configName"
| where name == "anotherName"
| where customDimensions.taskName == "taskName"
| extend timestamp, correlationId = tostring(customDimensions.correlationId), configName = tostring(customDimensions.configName), name= name, nameTimeStamp= timestamp ;
let timeDiffs = start
| join (ending) on correlationId
| extend timeDiff = nameTimeStamp- timestamp
| project timeDiff, timestamp, nameTimeStamp, name, anotherName, correlationId;
timeDiffs
| summarize AggregatedValue=avg(timeDiff) by bin(timestamp, 1m)
When I run this query in Analytics page, I get results, however when I try to create a custom metric alert, I got the error Search Query should contain 'AggregatedValue' and 'bin(timestamp, [roundTo])' for Metric alert type
The only response I found was adding AggregatedValue which I already have, I'm not sure why custom metric alert page is giving me this error.
I found what was wrong with my query. Essentially, aggregated value needs to be numeric, however AggregatedValue=avg(timeDiff) produces time value, but it was in seconds, so it was a bit hard to notice. Converting it to int solves the problem,
I have just updated last bit as follows
timeDiffs
| summarize AggregatedValue=toint(avg(timeDiff)/time(1ms)) by bin(timestamp, 5m)
This brings another challenge on Aggregate On while creating the alert as AggregatedValue is not part of the grouping that is coming after by statement.

unable to query database using timestamp

I have this table
user_authentication_token (
token_id uuid,
user_id uuid,
email text,
expiration_time timestamp,
is_sign_up boolean,
PRIMARY KEY ((token_id, user_id, email, expiration_time))
and I have this data in the table
token_id | user_id | email | expiration_time | is_sign_up
--------------------------------------+--------------------------------------+-------------------------+---------------------------------+------------
98b0456a-05b2-4aca-a6c7-6a1f382e19aa | b284d51d-efbb-4204-b342-a2486029a5c5 | manu.chadha#hotmail.com | 2018-09-01 09:40:59.634000+0000 | True
But I am unable to query it.
select * from user_authentication_token where token_id=98b0456a-05b2-4aca-a6c7-6a1f382e19aa and user_id=b284d51d-efbb-4204-b342-a2486029a5c5 and email='manu.chadha#hotmail.com' and expiration_time='2018-09-01 09:40:59.634000+0000';
Error - InvalidRequest: Error from server: code=2200 [Invalid query] message="Unable to coerce '2018-09-01 09:40:59.634000+0000' to a formatted date (long)"
What am I doing wrong?
I inserted the row using Cassandra Java Datastax driver and the following code
def insertValues(tableName:String, model:UserToken):Insert = {
QueryBuilder.insertInto(tableName).value("token_id",model.tokenId)
.value("email",model.email)
.value("user_id",model.userId)
.value("expiration_time",model.expirationTime.getMillis())
.value("is_sign_up",model.isSignUp)
.ifNotExists(); }}
Interestingly, timestamp is stored as long but the cqlsh is showing it in a readable format. Probably I need to convert it into long again but how do I do it?
Your timestamp should include nothing beyond milliseconds:
select * from user_authentication_token
where token_id=98b0456a-05b2-4aca-a6c7-6a1f382e19aa
and user_id=b284d51d-efbb-4204-b342-a2486029a5c5
and email='manu.chadha#hotmail.com'
and expiration_time='2018-09-01 09:40:59.634+0000'; // removed extra precision
Update: Added timezone offset as discovered by Manu Chadha below.

Change datetime format in mariadb

Database : Mariadb
Platform : CentOS
Need to do
Import data from a text file to table. Problem with DATETIME format.
Original date time format in test file : mmddYYYYHHiissmmm
Database default format : YYYYmmddHHiiss
LOAD DATA LOCAL
INFILE '/home/test.txt'
INTO TABLE cdr FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
(ID , APARTY , BPARTY , #T1, ENDTIME, DURATION, INTG, OUTTG, INRC, OUTRC)
set STARTTIME = STR_TO_DATE(#T1,'%m-%d-%Y %H:%i:%s:%f');
After importing the values are showing NULL.
Assuming your example of 'mmddYYYYHHiissmmm' is correct, change '%m-%d-%Y %H:%i:%s:%f' to '%m%d%Y%H%i%s%f'. Here's a test:
mysql> SELECT STR_TO_DATE('12312015235959123', '%m%d%Y%H%i%s%f');
+----------------------------------------------------+
| STR_TO_DATE('12312015235959123', '%m%d%Y%H%i%s%f') |
+----------------------------------------------------+
| 2015-12-31 23:59:59.123000 |
+----------------------------------------------------+
1 row in set (0.00 sec)
You cannot change the internal representation of DATETIME or TIMESTAMP.

Resources