Cassandra column metadata shows "DateType" instead of "timestamp" after driver upgrade - cassandra

Question

I have some Java code that performs introspection on the schema of Cassandra tables. After upgrading the Cassandra driver dependency, this code is no longer working as expected. With the old driver version, the type for a timestamp column was returned from ColumnMetadata#getType() as DataType.Name#TIMESTAMP. With the new driver, the same call returns DataType.Name#CUSTOM and CustomType#getCustomTypeClassName returning org.apache.cassandra.db.marshal.DateType.
The old driver version is com.datastax.cassandra:cassandra-driver-core:2.1.9:
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>2.1.9</version>
</dependency>
The new driver version is com.datastax.cassandra:dse-driver:1.1.2:
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>dse-driver</artifactId>
<version>1.1.2</version>
</dependency>
The cluster version is DataStax Enterprise 2.1.11.969:
cqlsh> SELECT release_version FROM system.local;
release_version
-----------------
2.1.11.969
To illustrate the problem, I created a simple console application that prints column metadata for a specified table. (See below.) When built with the old driver, the output looks like this:
# old driver
mvn -Pcassandra-driver clean package
java -jar target/cassandra-print-column-metadata-cassandra-driver.jar <address> <user> <password> <keyspace> <table>
...
ts timestamp
...
When built with the new driver, the output looks like this:
# new driver
mvn -Pdse-driver clean package
java -jar target/cassandra-print-column-metadata-dse-driver.jar <address> <user> <password> <keyspace> <table>
...
ts 'org.apache.cassandra.db.marshal.DateType'
...
So far, I have only encountered this problem with timestamp columns. I have not seen it for any other data types, though my schema does not exhaustively use all of the supported data types.
DESCRIBE TABLE shows that the column is timestamp. system.schema_columns shows that the validator is org.apache.cassandra.db.marshal.DateType.
[cqlsh 3.1.7 | Cassandra 2.1.11.969 | CQL spec 3.0.0 | Thrift protocol 19.39.0]
cqlsh:my_keyspace> DESCRIBE TABLE my_table;
CREATE TABLE my_table (
prim_addr text,
ch text,
received_on timestamp,
...
PRIMARY KEY (prim_addr, ch, received_on)
) WITH
bloom_filter_fp_chance=0.100000 AND
caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
comment='emm_ks' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.100000 AND
compaction={'sstable_size_in_mb': '160', 'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
cqlsh:system> SELECT * FROM system.schema_columns WHERE keyspace_name = 'my_keyspace' AND columnfamily_name = 'my_table' AND column_name IN ('prim_addr', 'ch', 'received_on');
keyspace_name | columnfamily_name | column_name | component_index | index_name | index_options | index_type | type | validator
---------------+-------------------+-------------+-----------------+------------+---------------+------------+----------------+------------------------------------------
my_keyspace | my_table | ch | 0 | null | null | null | clustering_key | org.apache.cassandra.db.marshal.UTF8Type
my_keyspace | my_table | prim_addr | null | null | null | null | partition_key | org.apache.cassandra.db.marshal.UTF8Type
my_keyspace | my_table | received_on | 1 | null | null | null | clustering_key | org.apache.cassandra.db.marshal.DateType
Is this a bug in the driver, an intentional change in behavior, or some kind of misconfiguration on my part?
pom.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>cnauroth</groupId>
<artifactId>cassandra-print-column-metadata</artifactId>
<version>0.0.1-SNAPSHOT</version>
<description>Console application that prints Cassandra table column metadata</description>
<name>cassandra-print-column-metadata</name>
<packaging>jar</packaging>
<properties>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
<slf4j.version>1.7.25</slf4j.version>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<mainClass>cnauroth.Main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<finalName>${project.artifactId}</finalName>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>dse-driver</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>dse-driver</artifactId>
<version>1.1.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<finalName>${project.artifactId}-dse-driver</finalName>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>cassandra-driver</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>2.1.9</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<finalName>${project.artifactId}-cassandra-driver</finalName>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
</dependencies>
</project>
Main.java
package cnauroth;
import java.util.List;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.ColumnMetadata;
import com.datastax.driver.core.Session;
class Main {
public static void main(String[] args) throws Exception {
// Skipping validation for brevity
String address = args[0];
String user = args[1];
String password = args[2];
String keyspace = args[3];
String table = args[4];
try (Cluster cluster = new Cluster.Builder()
.addContactPoints(address)
.withCredentials(user, password)
.build()) {
List<ColumnMetadata> columns =
cluster.getMetadata().getKeyspace(keyspace).getTable(table).getColumns();
for (ColumnMetadata column : columns) {
System.out.println(column);
}
}
}
}

Answer 1

It looks like the internal Cassandra type used for Timestamp changed from org.apache.cassandra.db.marshal.DateType and org.apache.cassandra.db.marshal.TimestampType between Cassandra 1.2 and 2.0 (CASSANDRA-5723). If you created the table with Cassandra 1.2 (or a DSE compatible version) DateType would be used (even if you upgraded your cluster later).
It appears that the 2.1 version of the java driver was able to account for this (source) but starting with 3.0 it does not (source). Instead, it parses it as a Custom type.
Fortunately, the driver is still able to serialize and deserialize this column as the cql timestamp type is communicated over the protocol in responses, but it's a bug that the driver parses this as the wrong type. I went ahead and created JAVA-1561 to track this.
If you were to migrate your cluster to C* 3.0+ or DSE 5.0+ I suspect the problem goes away as the schema tables reference the cql name instead of the representative Java class name (unless it is indeed a custom type).