I consider a counter ColumnFamily. Since it holds only counters, I expect to see a large number of updates in this table.
Following http://www.datastax.com/dev/blog/when-to-use-leveled-compaction, I consider using compaction = LeveledCompactionStrategy.
Is this a good idea? If yes, I would have expected counter ColumnFamilies to have compaction=LeveledCompactionStrategy by default, which seems not to be the case.
All tables (ColumnFamilies) are set to size tiered compaction strategy by default. There is no distinction based on the type. This is due to the fact that choosing LCS requires thoughtful consideration. The url you link to is the standard for determining whether or not you should enable LCS. In general, it is not a good idea to do so unless your storage is SSDs or if your use case is write-heavy.
As is mentioned in the article, you can test the impact of LCS using write-survey mode. I would urge you to make use of this feature before making the switch.
Related
I've read that you can set compaction strategy per table in Cassandra/Scylla, as described here https://docs.scylladb.com/operating-scylla/procedures/config-change/change_compaction/
The default compaction strategy is Size-tiered compaction strategy (STCS).
But is there a way to change it somehow, in the settings, such that each table that's created uses another compaction strategy by default?
Thanks.
The compaction strategy is a sub-property of the compaction configuration of each table so you will need to use the CQL ALTER TABLE command to choose a different compaction strategy other than the default.
In almost all cases, the SizeTieredCompationStrategy (STCS) is the right choice and so it is the default. There are very limited cases where you would choose a different compaction strategy.
The most common situation where you would change it is if you have a time-series use case where TimeWindowCompactionStrategy (TWCS) is recommended. LeveledCompactionStrategy (LCS) is only valid for workloads were there is very little writes and your app is almost exclusively doing reads.
So unless you fit into these narrow use cases, STCS should be your choice of compaction strategy. Cheers!
In my use case I have data that will be updated frequently during one day after its insertion and after that will be used rarely for reads only. What would be the best compaction strategy for that? TWCS or DTCS?
TWCS was created because DTCS requires a lot of tuning and operational care in order to achieve & maintain good performance. TWCS provides a similar level of performance to DTCS and is much easier to work with, so it is definitely the one to use for 99% of cases where time-series data is involved and there will be no inserts/updates after the first window.
Take a look at CASSANDRA-9666 for the details.
Twcs is the best compaction strategy for time serious data if you have frequent updates with less number of reads .
http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
According to most articles on internet Random Partitioning(RP) is better than Ordered Partitioning(OP) cause of the data distribution.
in fact, I think, that cause of data replication even if we are using the OP the data will be well distributed ! so is the first assumption is still true ?
what about reading performance ? is OP better than RP when trying to read data between two value in the same range ?
thanks a lot
I can't answer really answer confidently for HBase (which only supports Ordered Partitioning to my knowledge), but for Cassandra I would strongly discourage the use of OrderPreservingPartitioner and ByteOrderedPartitioner unless you have a very specific use case that requires it (like if you need to do range scans across keys). It is not very common for Ordered Partitioner to be used
in fact, I think, that cause of data replication even if we are using the OP the data will be well distributed ! so is the first assumption is still true ?
Not particularly, it is much more likely for hotspots to be encountered with an Ordered Partitioner vs. a Random Partitioner. As described from the Partitioners page on the Cassandra Wiki:
globally ordering all your partitions generates hot spots: some partitions close together will get more activity than others, and the node hosting those will be overloaded relative to others. You can try to mitigate with active load balancing but this works poorly in practice; by the time you can adjust token assignments so that less hot partitions are on the overloaded node, your workload often changes enough that the hot spot is now elsewhere. Remember that preserving global order means you can't just pick and choose hot partitions to relocate, you have to relocate contiguous ranges.
There are other problems with Ordered Partitioning that are described well here:
Difficult load balancing:
More administrative overhead is required to load balance the cluster. An ordered partitioner requires administrators to manually calculate partition ranges based on their estimates of the partition key distribution. In practice, this requires actively moving node tokens around to accommodate the actual distribution of data once it is loaded.
Uneven load balancing for multiple tables:
If your application has multiple tables, chances are that those tables have different row keys and different distributions of data. An ordered partitioner that is balanced for one table may cause hot spots and uneven distribution for another table in the same cluster.
With regards to:
what about reading performance ? is OP better than RP when trying to read data between two value in the same range ?
You will definitely achieve better performance for range scans (i.e. get all data between this key and that key).
So it really comes down to the kind of queries you are making. Are range scan queries between keys vital to you? In that case HBase may be a more appropriate solution for you. If it is not as important, there are reasons to consider C* instead. I won't add much more to that as I don't want my answer to devolve into comparing the two solutions :).
We want to generated auto incremental integer key in Cassandra. This is trivial task in traditional databases but little complicated in Cassandra.
I have tried out Counter data type which can be incremented using
value=value+1
and tried LWT with
UPDATE myTable SET value=newValue IF value=oldValue.
(where newValue=oldValue+1 for auto increment)
I have been strongly warned against counter variables. I am not sure why though. Can you please help me understand pros and cons of above two approaches?
Starting with a Disclaimer,
You most likely do not want an auto-incrementing integer key in C*. More likely a UUID or TimeUUID is what you want. But if you do happen to really need an auto incrementing value read on.
State and Distributed Systems do not like to blend. Generally whenever you want to make 'really' sure of the state of your distributed system you have to check all replicas and thus sacrifice availability/partition tolerance. LWT use Paxos to allow check and set operations but to do this they require a quorum of nodes and are significantly slower than normal Cassandra operations. LWT should only be used in a small percentage of the operations used in your applications. As long as there is little contention for the counter variable and you don't need it for every write you should be ok.
Counters in C* are a very different implementation. In older versions of C* they were slightly infamous for their ability to lose track of values. Their implementation has been rewritten for vastly improved stability but it would require careful planning on the application side to guarantee unique operations. You can imagine two clients both incrementing a counter simultaneously each thinking that they had received a unique value. Because of this I think you should use LWT if you really need to make sure of uniqueness.
This would be lovely, since some cf have a limited number of keys, or queries are intensive on these keys, etc.
Thx
Edit : to be more precise, I'm interested in the partition key disk-location caching for a given table.
Edit 2 : This thread seems to give the answer http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/What-is-substituting-keys-cached-column-family-argument-td7584136.html
Cassandra now prevents any manual tuning for keys or row caching (that used to be allowed with the keys_cached or rows_cahed arguments on a per-CF basis), and instead uses its own global caching manager. This is annoying in the specific case of CFs that are not often queried, but from which we expect low latency. Obviously this is my case. Anyone from Apache/Datastax could tell me if there is any chance this might evolve? Thx.
If you have a range of keys that are being queried frequently across a Cassandra cluster, Cassandra's built-in keycache will retain those values in memory for you automatically on a per-column family basis.
The keycache is turned on by default, so no configuration changes are necessary.
Now if your question pertains to being able to have Cassandra automatically warm the keycache for the entire key range for any column families you specify, I don't believe this is possible, at least according to the Cassandra 2.0 documentation.
Your best option, if you really, really need to warm the cache yourself instead of letting it happen at runtime, would be to create your own startup application that queries your desired key ranges at startup.