Is there a way in the JDBC connection string to specify ML workloads to connect to Spanner's Read-Only replicas? Spanner's infrastructure efficiently handles such Read-only transactions?
You cannot use the JDBC driver to connect directly to a read-only replica, but you can use read-only transactions and specify the read staleness of the transaction (or a single query). This will effectively allow Cloud Spanner to execute the query on the closest read-only replica. Use the SET READ_ONLY_STALENESS command for this.
Example for a single query:
-- Set the max staleness to 15 seconds.
-- Cloud Spanner will choose the most efficient timestamp to
-- execute the query. This only works in auto commit mode.
SET READ_ONLY_STALENESS='MAX_STALENESS 15s';
SELECT * FROM my_table;
Example for a read-only transaction with multiple queries:
BEGIN;
-- Mark the transaction as read-only.
SET TRANSACTION READ ONLY;
-- MAX_STALENESS can only be used in auto commit.
-- Use EXACT_STALENESS or READ_TIMESTAMP for transactions.
SET READ_ONLY_STALENESS='EXACT_STALENESS 15s';
SELECT * FROM my_table;
SELECT * FROM other_table;
-- Read-only transactions are not really committed, but this
-- marks the end of the transaction for the JDBC driver.
COMMIT;
What mechanisms does Delta Lake use to ensure the atomicity, consistency, isolation, and durability of transactions initiated by user operations on a DeltaTable?
0. the DeltaLog
Deltalog = Delta Lake's transaction log.
The deltalog is a collection of ordered json files. It acts as a single source of truth giving to users access to the last version of a DeltaTable's state.
1. Atomicity
Delta Lake breaks down every operation performed by an user into commits, themselves composed of actions.
A commit is recorded in the deltalog only once each of its actions has successfully completed (else it is reverted and restarted or an error is thrown), ensuring its atomicity.
2. Consistency
The consistency of a DeltaTable is guaranteed by their strong schema checking.
3. Isolation
Concurrency of commits is managed to ensure their isolation. An optimistic concurrency control is applied:
When a commit execution starts, the thread snapshots the current deltalog.
When the commit actions have completed, the thread checks if the Deltalog has been updated by another one in the meantime:
If not it records the commit in the deltalog
Else it updates its DeltaTable view and attempts again to register the commit, after a step of reprocessing if needed.
4. Durability
Commits containing actions that mutate the DeltaTable's data need to finish their writes/deletions on underlying Parquet files (stored on the filesystem) to be considered as successfully completed, making them durable.
Further readings:
Diving Into Delta Lake: Unpacking The Transaction Log
ACID properties
VoltDB is in memory and ACID compliant database. How we can implement transactions in this database. Please suggest.
All operations against VoltDB are 100% strict-serializable ACID transactions.
There is no BEGIN, COMMIT or ROLLBACK.
If you want a multi-statement transaction, you must create a stored procedure. The run() method of the stored procedure becomes the transactional context. It implicitly starts with BEGIN and ends with COMMIT. If an exception is thrown, it will ROLLBACK.
This is covered in the tutorial: https://docs.voltdb.com/tutorial/index.php
Chapter 5 is on procedures: https://docs.voltdb.com/tutorial/Part5.php
I am having a single partitioned java stored procedure in Voltdb. I need to get the partition id of the current partition in which the procedure is running. How do I get that within the procedure?
Rather than maintaining your own transaction counts in a table, you could call the #Statistics system procedure:
SQL> exec #Statistics PROCEDUREPROFILE 0;
This provides statistics for each procedure for each partition.
SQL> exec #Statistics TABLE 0;
This provides the count of how many records per table per partition.
I can not find the documentation for this. I know there is a consistency command in cqlsh, but there is no distinction between read and write consistency. How would I set different consistency levels for read and write?
Furthermore, there is a mention of a "default" consistency level. Where is that default set? and is it for read or write?
How would I set different consistency levels for read and write?
If you just want to change consistency level for your current session, use CONSISTENCY.
If you want to change the consistency level programamtically, use the cassandra driver for your client language.
Since the Consistency Level can be set per-statement, you can either set it on every statement, or use PreparedStatements. If you're using the Java driver, you can configure a global Consistency level for BOTH reads and writes (but not only reads and only writes).
Where is that default [consistency level] set? and is it for read or write?
If you want to set up a different consistency level for reads than writes, you'll have to do it on a per-statement basis. Use QueryOptions().setConsistencyLevel to set a global consistency level (for the Java driver). It is for BOTH reads and writes.
To set the consistency level for your current session, use the CONSISTENCY command
from the cassandra shell (CQLSH).
For example:
Set CONSISTENCY to force the majority of the nodes to respond:
CONSISTENCY QUORUM
To see your current consistency level, just run CONSISTENCY; from the shell:
ty#cqlsh> consistency;
Current consistency level is ONE.
For programming client applications, set the consistency level using an appropriate driver.
To set a per-insert consistency level using the Java driver, for example, call QueryBuilder.insertInto with setConsistencyLevel.
For example:
PreparedStatement pstmt = session.prepare(
"INSERT INTO product (sku, description) VALUES (?, ?)");
pstmt.setConsistencyLevel(ConsistencyLevel.QUORUM);
To set a global consistency level for reads AND writes using the Java driver, do something like:
QueryOptions qo = new QueryOptions().setConsistencyLevel(ConsistencyLevel.ALL);
Since the Java driver only executes CQL statements, which can be either reads or writes to Cassandra, it is not possible to globally configure the Consistency Level for only reads or only writes. To do so, since the Consistency Level can be set per-statement, you can either set it on every statement, or use PreparedStatements. More info in the Queries and Results document.
More resources on read/write consistency levels
How consistency level is configured
The different consistency levels are also explained here ^
Cassandra's Java Driver documentation
See the "Consistency levels" section in configuring consistency with the Java Driver
Other resources on consistency:
You may also want to look into your replica placement strategy and replication factor (which are other forms of consistency).
I included links below for good measure:
Background on data replication in Cassandra
How to update replica strategy and replication factor using CREATE KEYSPACE
How consistency affects performance
By default, the Consistency Level is ONE for all R/W Operations.
Setting CL is done on a per query (read or upsert) basis by adding the CONSISTENCY XXXX syntax on the query as seen here:
https://cassandra.apache.org/doc/latest/cql/dml.html#insert
and
https://cassandra.apache.org/doc/4.0/tools/cqlsh.html
The consistency level is not set anymore with a CQL USING clause. See CASSANDRA-4734 for an explanation.
Instead, with CQLSH, you should use the CONSISTENCY command: see here for details.
Note that the default consistency level changed in all DataStax drivers from ONE to LOCAL_ONE: see the python driver documentation for details - however, in spite of using the python driver behind the scenes, CQLSH still has a default consistency of ONE).
The consisenty level is no longer set using PreparedStatement. That API was changed. Now you have to use BatchStatementBuilder.
Here is sample code that works.
BatchStatementBuilder builder = BatchStatement.builder(DefaultBatchType.UNLOGGED);
builder.setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM);
PreparedStatement preparedStatement = session.prepare("INSERT INTO \"ScottKeySpace16\".\"Movie\" (title, year, plot) values (:k0, :k1, :k2)");
builder.addStatement(preparedStatement.boundStatementBuilder()
.setString("k0", "Movie44")
.setInt("k1", 2022)
.setString("k2", "This is a very funny movie")
.build());
BatchStatement batchStatement = builder.build();
session.execute(batchStatement);
Default consistency is set in cassandra.conf file . It is for read and write .
You can set consistency per query basis adding USING option in query .