Is there a performance impact when running delete statements on cassandra when row doesn't exist? I am not passing the IF EXISTS clause in my delete statement as it adds an overhead of checking. I haven't found anything online about this unique use-case.
Delete operation in Cassandra is just adding a marker called "tombstone" - it will be appended to the files with data to "hide" the previously existed data. It could have some performance impact on the read operations, if you have a lot of deletes inside partitions, etc. as tombstone usually kept for 10 days in the data file (configurable per table)
There is a very interesting blog post on deletes and tombstones - I recommend to read it.
Related
I would like to know if it is safe to delete an entire partition in cassandra in a single DeleteQuery. How is the performance in this case ? Any insights ?
Partition deletes is the best that you can do from performance standpoint because it generates only a single tombstone of special type. You can read more about different types of deletes & tombstones in the following great blog post.
It is a matter of necessity, not a question of whether it is safe or not.
If you need to delete a partition then delete the partition. If you need to delete a row then delete the row. If you need to delete a column, delete the column.
I'm guessing that you've read somewhere that tombstones are an issue in Cassandra. The problem with tombstones isn't with the tombstones themselves -- it's whether you are using Cassandra to process queues or queue-like datasets.
As a friendly note, a better question is "What problem are you trying to solve?" instead of asking an open-ended question without providing background information or context. Cheers!
From the articles I read they say that tombstones affect read performance in Cassandra. I’m reading how data is updated in Cassandra and looks like data is written with a timestamp without modifying or reading the current data.
So when a read is performed before compaction is done, filtering needs to be done to take the latest value right? If that’s the case aren’t tombstones the same thing and why do they affect performance negatively but not updates to a row?
In Cassandra, update is a mutation, like, insert and delete, and except the use case of LWTs and some of the list operations, all mutations are just append to the memtable/commit log, without reading the data on the disk. So they are very fast - no checks are performed.
Read operation, in contrast to that, need to get all versions of the data from the disk/memtable, and then create an actual version of the data based on the timestamps. And for tombstones, we need to keep them in the memory, because we may read some data from the disk that could have older timestamp, and we need to detect this.
We use a very simple key-value datamodel in Cassandra, and our partition key is in 17 SSTables. I would like to understand how read works in our concrete case.
If I undestand correctly, general Cassandra reads will need to search for the newest version of each column in the memtable and in different SSTables, until it retrieves all columns and merges them.
Since SSTables are sorted by time, and our data-model is single-column, Ideally our read operations should just hit the newest SSTable containing our partition key since this will contain the whole data.
Will our read operations hit the 17 SSTables? or just the newest one containing the searched partition key?
Cassandra will search all of them as it isn't sure which columns exist where (DML occurs at the cell level and because of that, variants can exist where reconciliation is performed). Reads are done at the partition level. However, Cassandra can filter out sstables if it knows the partition key doesn't exist in certain ones. That's why compaction is important for optimal reads - to remove the unnecessary cells.
Will our read operations hit the 17 SSTables? or just the newest one containing the searched partition key?
To add to Jim's answer, Cassandra has something called a bloom filter for this. Essentially, it's a probabilistic structure that can tell you one of two things:
The SSTable might contain the data requested.
OR
The SSTable definitely does not contain the data requested.
This should prevent Cassandra from having to scan all 17 SSTables. My advice would be to run a query with TRACING ON in cqlsh, and it'll tell you just how many SSTables it needed to look through.
I created a columnstore table in memsql and populated it with around 10 million records after which I started running several update scenarios. I noticed that the size of the data in /var/lib/memsql/leaf-3307/data/columns keeps increasing constantly and nothing there seems to be deleted. Initially the size of that folder is a couple hundred Mb but it quickly jumps to a couple of Gb after some full table updates. The "Columnstore Disk Usage" reported by memsql-ops also increases but at a very slow pace (far from what I see on disk).
This makes me think that data is never actually deleted from disk. The documentation states that running the OPTIMIZE commands should compact the row segment groups and that deleted rows would be removed:
Delete - Deleting a row in a columnstore index causes the row to be marked as deleted in the segment meta data leaving the data in place within the row segment. Segments which only contain deleted rows are removed, and the optimization process covered below will compact segments that require optimization.
Running the OPTIMIZE command didn't help. I also tried truncating the table and even dropping it but nothing helped. The data in the columns folder is still there. The only way I could find of cleaning that up is to DROP the entire database.
This doesn't seem like the desired behavior and I can't find any documentation justifying it. Can anybody explain why this is happening, if it should happen or point me to some relevant documentation?
Thanks in advance
MemSQL will keep around columnstore_window_size bytes of deleted columnstore data on disk per partition database. This is part of the implementation of columnstore replication (it keeps some old files around in case slaves are behind). If you lower the value of that system variable you'll see the disk usage drop. If your not using redundancy 2 there is no harm in lowering it.
We have a massive set of data that is written in to millions of rows in cassandra. We also have a scheduler that needs to process these records and remove them after processing them successfully.
Was wondering if Deleting the row after processing vs Marking a row with a TTL (essentially delaying its deletion).
Are there any pros / cons with Deletion vs TTL w.r.t Cassandra performance ?.
Thanks much
_DD
When using TTL the record is not removed from storage immediately, it is marked as tombstone. It gets physically removed only when the compaction occurs. Till that time the data impacts the nodes processing as it consumes the resources till the compaction happens. When you do a range query event the deleted(marked as tombstone) records are scanned by Cassandra. So using TTL to delete too many entries is considered as anti-pattern. The recommendation is to use temporary tables so that individual rows need not be removed. Just drop the entire table.
From what little information you have given here it sounds to me that you are using Cassandra as a queue which is a well known anti-pattern. You can read more about that here:
http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets
However to answer your basic question there is little difference in performance between using TTL and deletes. TTL's in C* are handled as tombstones which is the same as a delete. The major difference is that a tombstone is not written to a record who's TTL has expired until that record is read again. When a delete is called a tombstone is immediately created. Tombstones in general cause significant performance problems within C* and while there are some methods to mitigate the issues that they create having large numbers of them usually point to a poor data model or poor use case for C*. If you are really looking at using C* as a queue why not look at using something more fit for that purpose such as Redis?
Based on what I've read, TTL will probably be as fast as your fastest delete process could be. The reason for this is that TTL doesn't have to seek the data in order to mark it with a tombstone. The TTL lives on the record and when the record is read and the TTL has expired, then it is marked with a tombstone.
http://docs.datastax.com/en/cql/3.1/cql/cql_using/use_expire_c.html