I am reading Cassandra: The Definitive Guide, 3rd edition. It has the following text:
The serial consistency level can apply on reads as well. If Cassandra detects that a query is reading data that is part of an uncommitted transaction, it commits the transaction as part of the read, according to the specified serial consistency level.
Why a read is committing an uncommitted transaction and doesn't it interfere with ability of the writer to rollback?
https://community.datastax.com/questions/5769/why-a-read-is-committing-an-uncommitted-transactio.html
Committed means that a mutation (INSERT, UPDATE or DELETE) is not added to commitlog.
Uncommitted is when a mutation is still in the process of being saved to the commitlog.
In oder for the LWT to provide guarantees such as IF EXISTS or IF NOT EXISTS, It has to add any data that is not written to commitlog by another in-flight operation to commitlog.
Here Uncommitted data doesnt mean that it was a failed write. Uncommitted data is a successful data written to some node in the cluster which is not updated in the current node.
here,
it commits the transaction as part of the read
means that Cassandra will initiate a read repair and update the data in the node before sending the data back to the client.
Rollback is not in the picture here because write was successful and this concerns only the replication of data across nodes
Related
Does Cassandra acknowledges write as soon as it writes to commit log?or does it wait for the write to be written to the memtable also in order to send success to client?
Write success occurs when data is written to commitlog and memtable.
Again for multi node cluster with rf > 1 , it depends on the consistency level you set for writes.
Per DSE Architecture guide:
The write consistency level determines how many replica nodes must respond with a success acknowledgment for the write to be considered successful. Success means data was written to the commit log and the memtable.
And it makes sense, because it may not be possible to write data to the memtable, for example, if all memtables are still flushing, and there is no space left for a new write - in this case, Cassandra will return error. And writing to commit log just lowers the chance that you lose the data if machine lose power, or process crashes.
What mechanisms does Delta Lake use to ensure the atomicity, consistency, isolation, and durability of transactions initiated by user operations on a DeltaTable?
0. the DeltaLog
Deltalog = Delta Lake's transaction log.
The deltalog is a collection of ordered json files. It acts as a single source of truth giving to users access to the last version of a DeltaTable's state.
1. Atomicity
Delta Lake breaks down every operation performed by an user into commits, themselves composed of actions.
A commit is recorded in the deltalog only once each of its actions has successfully completed (else it is reverted and restarted or an error is thrown), ensuring its atomicity.
2. Consistency
The consistency of a DeltaTable is guaranteed by their strong schema checking.
3. Isolation
Concurrency of commits is managed to ensure their isolation. An optimistic concurrency control is applied:
When a commit execution starts, the thread snapshots the current deltalog.
When the commit actions have completed, the thread checks if the Deltalog has been updated by another one in the meantime:
If not it records the commit in the deltalog
Else it updates its DeltaTable view and attempts again to register the commit, after a step of reprocessing if needed.
4. Durability
Commits containing actions that mutate the DeltaTable's data need to finish their writes/deletions on underlying Parquet files (stored on the filesystem) to be considered as successfully completed, making them durable.
Further readings:
Diving Into Delta Lake: Unpacking The Transaction Log
ACID properties
I have a question regarding Cassandra batch isolation:
Our cluster consist of a single datacenter, replication factor of 3, reading and writing in LOCAL_QUORUM.
We must provide a news feed resembling an 'after' trigger, to notify clients about CRUD events of data in the DB.
We thought of performing the actual operation, and inserting an event on another table (also in another partition), within a batch. Asynchronously, some process would read events from event table and send them through an MQ.
Because we're writing to different partitions, and operation order is not necessarily maintained in a batch operation; is there a chance our event is written, and our process read it before our actual data is persisted?
Could the same happen in case our batch at last fails?
Regards,
Alejandro
From ACID properties, Cassandra can provide ACD. Therefore, don't expect Isolation in its classical sense.
Batching records will provide you with Atomicity. So it does guarantee that all or none of the records within a batch are written. However, because it doesn't guarantee Isolation, you can end up having some of the records persisted and others not (e.g. wrote to your queue table, but not master table).
Cassandra docs explain how it works:
To achieve atomicity, Cassandra first writes the serialized batch to the batchlog system table that consumes the serialized batch as blob data. When the rows in the batch have been successfully written and persisted (or hinted) the batchlog data is removed. There is a performance penalty for atomicity.
Finally, using Cassandra table as MQ is considered anti-pattern.
The questions are regarding the “CAS operations” paragraph into the article : http://www.datastax.com/dev/blog/cassandra-error-handling-done-right
a)
If the paxos phase fails, the driver will throw a WriteTimeoutException with a WriteType.CAS as retrieved with WriteTimeoutException#getWriteType(). In this situation you can’t know if the CAS operation has been applied..
How do you understand this?
I thought that If the paxos (prepare) phase fails then the coordinator will not initiate the commit phase at all?
I guess that it does not matter how the paxos phase fails (not enough replicas or replica timeouts or ..).
b)
The commit phase is then similar to regular Cassandra writes… you can simply ignore this error if you make sure to use setConsistencyLevel(ConsistencyLevel.SERIAL) on the subsequent read statements on the column that was touched by this transaction, as it will force Cassandra to commit any remaining uncommitted Paxos state before proceeding with the read
Wondering about the above with relation to writes with ConsistencyLevel.QUORUM:
If the commit phase failed because there is no quorum (unavailable nodes or timeouts) then we get back WriteTimeoutException with a WriteType of SIMPLE, right?
In this case it is not clear if the write is actually successful or not, right?
So I’m not sure what are all the possibilities from now on (recover/rollback/nothing)?
Is it saying that if I use ConsistencyLevel.QUORUM for the read operation I can see the old data version (as if the above write was not successful) for some time and after that again with QUORUM read I will see that the write is successful?
(actually I’m seen exactly this in a 3 node cluster with replication factor=3 after WriteTimeoutException (2 replica were required but only 1 acknowledged the write) – quorum read just after that returned the old data and then when i check with cqlsh I see the new data).
How this is possible?
guess:
Probably after the timeout the coordinator says that we have no quorum for the commit phase yet (and subsequent QUORUM reads get the older data version) and returns the WriteTimeoutException.type=SIMPLE to the client. And when the nodes that have timeout actually respond/commit we have a quorum in this future moment and after it all quorum reads will obtain the newer data version.
But not sure about the explanation of when you use read with SERIAL.
http://www.datastax.com/documentation/cassandra/2.0/cassandra/dml/dml_ltwt_transaction_c.html
A SERIAL consistency level allows reading the current (and possibly
uncommitted) state of data without proposing a new addition or update.
If a SERIAL read finds an uncommitted transaction in progress, it will
commit it as part of the read.
What I did not understand is - how can a read operation commit an in progress transaction? Does it mean to say - it will read it as part of the commit?
Thanks for spotting the problem in the docs. The sentence should say, "If a SERIAL read finds an uncommitted transaction in progress, Cassandra will perform a read repair as part of the commit. A read repair updates replicas with the most recent version of frequently-read data.