How to properly configure CDC in Cassandra - cassandra

Question

Cassandra version: 3.11
I already enabled cdc in cassandra.yaml:
cdc_enabled: true
cdc_raw_directory: /var/lib/cassandra/data/cdc_raw
And enabled the table as well:
cqlsh> describe cycling.cyclist_name;
CREATE TABLE cycling.cyclist_name (
id uuid PRIMARY KEY,
firstname text,
lastname text
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND cdc = true <<<<<<<<<
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
After restart, Cassandra created the cdc_raw directory:
root#docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/
total 36
drwxr-xr-x 9 cassandra cassandra 4096 Jan 28 10:04 .
drwxrwxrwx 6 cassandra cassandra 4096 Jan 28 09:48 ..
drwxr-xr-x 2 cassandra cassandra 4096 Jan 28 09:48 cdc_raw
drwxr-xr-x 3 cassandra cassandra 4096 Jan 28 10:04 cycling
drwxr-xr-x 26 cassandra cassandra 4096 Jan 28 09:48 system
drwxr-xr-x 6 cassandra cassandra 4096 Jan 28 09:48 system_auth
drwxr-xr-x 5 cassandra cassandra 4096 Jan 28 09:48 system_distributed
drwxr-xr-x 12 cassandra cassandra 4096 Jan 28 09:48 system_schema
drwxr-xr-x 4 cassandra cassandra 4096 Jan 28 09:48 system_traces
I executed a little Python script to insert some data in the fresh table:
>>> for lp in range(50000):
... session.execute("INSERT INTO cycling.cyclist_name (lastname, firstname, id) VALUES (%s, %s, %s)", ["RATTO_BULK", "Rissella", uuid.uuid4()])
...
But even after this 50,000 insert script, the cdc_raw diretory still empty. Can someone explain how CDC works in Cassandra?
root#docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/cdc_raw/
total 8
drwxr-xr-x 2 cassandra cassandra 4096 Jan 28 09:48 .
drwxr-xr-x 9 cassandra cassandra 4096 Jan 28 10:04 ..

Answer 1

In Cassandra 3.11 commitlog segments are copied into cdc_raw directory when data in memtable is flushed to disk for some reason - reaching memtable limit, commit log limit, or by nodetool flush. 50k writes may not be enough to trigger this situation.
In Cassandra 4.0 (not yet released) the situation is slightly improved, and you can read data faster. If you're interested in details, you can look into presentations made at DataStax Accelerate 2019 - there were 2 or 3 presentations about CDC.