Cassandra version :- [cqlsh 5.0.1 | Cassandra 3.0.9 | CQL spec 3.4.0 | Native protocol v4]
My Table structure
CREATE TABLE test (
id1 text,
id2 text,
id3 text,
id4 text,
clinet_starttime timestamp,
avail_endtime timestamp,
starttime timestamp,
client_endtime timestamp,
code int,
status text,
total_time double,
PRIMARY KEY (id1, id2, id3, id4, client_starttime)
) WITH CLUSTERING ORDER BY (id2 ASC, id3 ASC, id4 ASC, client_starttime ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
Following query is working for me
SELECT * FROM test WHERE client_starttime<1522832400000 AND client_starttime>1522831800000 AND id1='data1' AND id2='data2' AND id3='data3' AND id4 IN ('data5','data6','data7') ALLOW FILTERING;
But when I query using starttime and avail_endTime instead of client_starttime like
SELECT * FROM test WHERE starttime<1522832400000 AND avail_endTime>1522831800000 AND id1='data1' AND id2='data2' AND id3='data3' AND id4 IN ('data5','data6','data7') ALLOW FILTERING;
getting the following error
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot restrict clustering columns by IN relations when a collection is selected by the query"
If I am not using IN operator , If I use = operator then its working fine
SELECT * FROM test WHERE starttime<1522832400000 AND avail_endTime>1522831800000 AND id1='data1' AND id2='data2' AND id3='data3' AND id4 = 'data5'ALLOW FILTERING;
but I want to retrieve data for data5, data6 and data7 same time.
This is a known Cassandra bug: CASSANDRA-12654 - it's already fixed, but will be available only in version 4.0 for which there is no release date yet defined.
Related
I have a scylla table as shown below:
cqlsh:sampleks> describe table test;
CREATE TABLE test (
client_id int,
when timestamp,
process_ids list<int>,
md text,
PRIMARY KEY (client_id, when) ) WITH CLUSTERING ORDER BY (when DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
AND comment = ''
AND compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '1', 'compaction_window_unit': 'DAYS'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 172800
AND max_index_interval = 1024
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
And I see this is how we are querying it. It's been a long time I worked on cassandra so this PER PARTITION LIMIT is new thing to me (looks like recently added). Can someone explain what does this do with some example in a layman language? I couldn't find any good doc on that which explains easily.
SELECT * FROM test WHERE client_id IN ? PER PARTITION LIMIT 1;
The PER PARTITION LIMIT clause can be helpful in a "wide partition scenario."
It returns only the first two rows in the partition.
Take this query:
aploetz#cqlsh:stackoverflow> SELECT client_id,when,md
FROM test PER PARTITION LIMIT 2 ;
Considering the PRIMARY KEY definition of (client_id,when), that query will iterate over each client_id. Cassandra will then return only the first two rows (clustered by when) from that partition, regardless of how many ocurences of when may be present.
In this case, I inserted 7 rows into your test table, using two different client_ids (2 partitions total). Using a PER PARTITION LIMIT of 2, I get 4 rows returned (2 client_id x PER PARTITION LIMIT 2) == 4 rows.
client_id | when | md
-----------+---------------------------------+-----
1 | 2020-05-06 12:00:00.000000+0000 | md1
1 | 2020-05-05 22:00:00.000000+0000 | md1
2 | 2020-05-06 19:00:00.000000+0000 | md2
2 | 2020-05-06 01:00:00.000000+0000 | md2
(4 rows)
while I am running TableHistograms getting below message:
NodeTool TableHistograms keyspace TableName
Column counts are larger than 1996099046, unable to calculate percentiles
Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 0.00 0.00 0.00 268650950 NaN
75% 0.00 0.00 0.00 3449259151 NaN
95% 0.00 0.00 0.00 25628284214 NaN
98% 0.00 0.00 0.00 44285675122 NaN
99% 0.00 0.00 0.00 44285675122 NaN
Min 0.00 0.00 0.00 105779 0
Max 0.00 0.00 0.00 442856751229223372036854776000
Cassandra version:
[cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
Replication factor 3
4 node cluster
Getting the above message in one node only
Tried repairing the table but failed with streaming error :
40328:ERROR [StreamReceiveTask:53] 2019-06-10 13:54:33,684 StreamSession.java:593 - [Stream #c9214180-8b82-11e9-90ce-399bac480141] Streaming error occurred on session with peer <IP ADDRESS>
40329-java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Unable to compute ceiling for max when histogram overflowed
40330- at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51) ~[apache-cassandra-3.11.2.jar:3.11.2]
40331- at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:373) ~[apache-cassandra-3.11.2.jar:3.11.2]
40332- at org.apache.cassandra.index.SecondaryIndexManager.buildIndexesBlocking(SecondaryIndexManager.java:383) ~[apache-cassandra-3.11.2.jar:3.11.2]
40333- at org.apache.cassandra.index.SecondaryIndexManager.buildAllIndexesBlocking(SecondaryIndexManager.java:270) ~[apache-cassandra-3.11.2.jar:3.11.2]
40334- at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:216) ~[apache-cassandra-3.11.2.jar:3.11.2]
40335- at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_144]
40336- at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
40337- at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_144]
40338- at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144]
--
0354:ERROR [Reference-Reaper:1] 2019-06-10 13:54:33,907 Ref.java:224 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State#7bd8303d) to class org.apache.cassandra.io.util.ChannelProxy$Cleanup#1084465868:PATH/talename-5b621cd0c53311e7a612ffada4e45177/mc-26405-big-Index.db was not released before the reference was garbage collected
Table description includes :
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
Any idea why it is happening? Any help or suggestion is welcome.
You cannot have 2 billion cells in a partition. Also having a secondary index on a table with a 44gb partition is going to have issues for multiple reasons. There really isn't much you can do to fix this short of dropping your index and building a new data model to migrate into. You could build a custom version of Cassandra to ignore that exception but something else will come up very soon as you are at the extreme limits of whats even theoretically possible. You are already past a point that I am surprised is running.
If the streaming error is from repairs you can ignore it while you fix your data model. If it's from bootstrapping I think you will need a custom version of Cassandra to stay running in meantime (or can just ignore the down node you are replacing). Keep in mind node failures are a serious threat to you now as bootstrapping likely will not work. When you put so much in a single partition it cannot be scaled out so there are limited options.
I am new to Cassandra so I had a few quick questions, suppose I do this:
CREATE TABLE my_keyspace.my_table (
id bigint,
year int,
datetime timestamp,
field1 int,
field2 int,
PRIMARY KEY ((id, year), datetime))
I imagine Cassandra as something like Map<PartitionKey, SortedMap<ColKey, ColVal>>,
My question is when querying for something from Cassandra using a WHERE, it will be like:
SELECT * FROM my_keyspace.my_table WHERE id = 1 AND year = 4,
This could return 2 or more records, how does this fit in with the data model of Cassandra?
If it really it a Big HashMap how come duplicate records for a partition key are allowed?
Thanks!
There is a batch of entries in the SortedMap<ColKey, ColVal> for each row, using its sorted nature.
To build on your mental model, while there is only 1 partition key for id = 1 AND year = 4 there are multiple cells:
(id, year) | ColKey | ColVal
------------------------------------------
1, 4 | datetime(1):field1 | 1 \ Row1
1, 4 | datetime(1):field2 | 2 /
1, 4 | datetime(5):field1 | 1 \
1, 4 | datetime(5):field2 | 2 / Row2
...
I have a very simple Cassandra table called campaignvariantaction with a counter column called val. I encountered a very strange problem today. For some keys, Cassandra increments the counter just fine. For others, it just doesn't do it. I'm very confused as to how this could be.
Sample output from Cassandra Shell (cqlsh) is below. Note how incrementing a counter works fine for one key (first 3 examples) and doesn't work for another (last example).
Cassandra version 2.2.7 on Ubuntu.
cqlsh:z> UPDATE campaignvariantaction SET val = val + 1 WHERE campaignvariantaction = 'campaign_variant#54408#sent' AND date = 20161118;
cqlsh:z> select * from campaignvariantaction where campaignvariantaction = 'campaign_variant#54408#sent';
campaignvariantaction | date | val
-----------------------------------+----------+-----
campaign_variant#54408#sent | 20161118 | 1
(1 rows)
cqlsh:z> UPDATE campaignvariantaction SET val = val + 1 WHERE campaignvariantaction = 'campaign_variant#54408#sent' AND date = 20161118;
cqlsh:z> select * from campaignvariantaction where campaignvariantaction = 'campaign_variant#54408#sent';
campaignvariantaction | date | val
-----------------------------------+----------+-----
campaign_variant#54408#sent | 20161118 | 2
(1 rows)
cqlsh:z> UPDATE campaignvariantaction SET val = val + 1 WHERE campaignvariantaction = 'campaign_variant#979165#sent' AND date = 20161118;
cqlsh:z> select * from campaignvariantaction where campaignvariantaction = 'campaign_variant#979165#sent';
campaignvariantaction | date | val
-----------------------+------+-----
(0 rows)
Describe output:
cqlsh:z> describe table campaignvariantaction ;
CREATE TABLE z.campaignvariantaction (
campaignvariantaction text,
date int,
val counter,
PRIMARY KEY (campaignvariantaction, date)
) WITH CLUSTERING ORDER BY (date ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
I`m using Cassandra dsc 2.1.5 with 3 nodes, and the following table description:
cqlsh> DESCRIBE TABLE mykeyspace.mytable;
CREATE TABLE mykeyspace.mytable (
a text,
b text,
c timestamp,
d timestamp,
e text,
PRIMARY KEY ((a, b), c)
) WITH CLUSTERING ORDER BY (c ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
The first and the third nodes are working fine (all have the same cassandra.yaml) but the second node started to have more and more pending compaction tasks that I can see with nodetool compactionstats -H command. The situation is so bad that my spark jobs get stuck and work only when I completely shut down the second node.
I have around 130G free on the second node...
Also, here a cfstatss state:
> nodetool cfstatss mykeyspace.mytable
Keyspace: mykeyspace
Read Count: 0
Read Latency: NaN ms.
Write Count: 54316 [1/1863] Write Latency: 0.1877597945356801 ms.
Pending Flushes: 0
Table: mytable
SSTable count: 1249
Space used (live): 1125634027755
Space used (total): 1125634027755
Space used by snapshots (total): 0
Off heap memory used (total): 1202327957
SSTable Compression Ratio: 0.11699340657338655
Number of keys (estimate): 34300801
Memtable cell count: 758856
Memtable data size: 351011415
Memtable off heap memory used: 0
Memtable switch count: 10
Local read count: 0
Local read latency: NaN ms
Local write count: 54319
Local write latency: 0.188 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 48230904
Bloom filter off heap memory used: 48220912
Index summary off heap memory used: 11161093
Compression metadata off heap memory used: 1142945952
Compacted partition minimum bytes: 925
Compacted partition maximum bytes: 52066354
Compacted partition mean bytes: 299014
Average live cells per slice (last five minutes): 0.0
Maximum live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
What can be the problem?