I've just updated to use cassandra version 0.8.1 to study..When i look at to the commitlog folder, there is no commitlogHeader file, where is it ? Is there any new update to manage commitlog in version 0.8.1 ?
The header stored when CFs were flushed so log replay could skip replaying unnecessary mutations.
This information is now stored per-sstable instead. (https://issues.apache.org/jira/browse/CASSANDRA-2419)
Related
In Cassandra 2.1 cluster, data format is ka and post upgrade to Cassandra 3.11, I see the new sstables are written in md format. For the time-series data that is going to be expired in 3 months time, can I skip running the nodetool upgradesstables?
I validated the data reads are working fine from the older ka format sstables after upgrade. The reason I want to skip the upgrade is from other threads, I know it is going to take a lot of time for format conversion and anyway this data is going to expire in 3 months.
I don't think it's mandatory to run nodetool sstablesupgrade, Cassandra 3 will be able to work with old SSTables, but you lose a lot of advantages of Cassandra 3 (for instance, space consumption is significantly reduced). Also Datastax has a warning in their upgrade documentation:
WARNING: Failure to upgrade SSTables when required results in a significant performance impact and increased disk usage and possible data loss. Upgrading is not complete until the SSTables are upgraded.
Is there a way to read the SSTable in Cassandra? I see from the documentation that sstabledump is an enterprise version, Is it possible to get the trial version of sstabledump?
Or is there a way to read the SSTable using the existing utilities in Cassandra/bin folder?
sstabledump is also available in apache cassandra.
It can be found in tools/bin directory in cassandra 3.x
Note: sstable2json was replaced by sstabledump in 3.0
You can use sstable2json for that.
http://docs.datastax.com/en/archived/cassandra/2.2/cassandra/tools/toolsSSTable2Json.html
https://www.datastax.com/dev/blog/debugging-sstables-in-3-0-with-sstabledump
In 2.x versions of cassandra we can view sstable content with bin/sstable2json sstable.db What is proper way of checking sstable data in new version.3.x(currently 3.5).
Check out sstabledump. It has replaced sstable2json in Cassandra 3.X.
http://www.datastax.com/dev/blog/debugging-sstables-in-3-0-with-sstabledump
I know similar questions have been asked before but I think my use case is very specific for which I could not find any answer.
In Production we are using Cassandra 1.2 with ByteOrderPartitioner in a 6 Node cluster with Priam as seed management tool. We have recently upgraded all the dependencies and trying to migrate to Cassandra 3.0.2 with Murmur Partitioner and for backward compatibility we need to enable thrift on new cluster .Also we want to migrate away from Priam also.
I was able to setup new cluster but facing lot of issues during data migration. I tried 3 things:
1) Use Copy Command : Fails when number of rows is large
2) SSTable2Json : Cassandra 3.0.2 has stopped supporting SSTable2Json
3) SSTableloader: Failing I think because of different cassandra version of source and destination
java.lang.RuntimeException: Could not retrieve endpoint ranges:
at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:233)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
Caused by: InvalidRequestException(why:unconfigured table schema_columnfamilies)
at org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:37849)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1562)
at org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1547)
at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:225)
... 2 more
Right now I am kind of stuck,any help regarding this will be deeply appreciated. Please let me know if you need more details.
No, You cannot upgrade your sstables from 1.2 to 3.0.2 directly since the sstable will differ for different version. This link describes the steps for upgrading the cassandra versions. But it also does not helps for you, since you are having a change in the partitioner type.
Changing the partitioner type is not yet supported in cassandra as of
now (Link).
One of the solution I would prefer is,
Create a stand alone utility which is of cassandra 3.0.2 version to read all the data from you source cassandra and write to sstable
with the help of CQLSSTableWriter with the partition type of Murmur Partitioner (The trick is, you are writing
the sstable with the version 3.0.2, so this sstable will be easily
recognized by your new cluster). Then use SSTableLoader in your target cluster
But I am not sure about why you still require backward compatibility, while creating CQLSSTableWritter you can specify the column family schema with keyword
"WITH COMPACT STORAGE". But I didn't tried CQLSSTableWritter with "WITH COMPACT STORAGE", but without "WITH COMPACT STORAGE" I had tried, it will work for your case too.
Ok so if you try to migrate directly from 1.2 to 3.0.2, you're really looking for trouble.
The migration path should be
latest minor or 1.2
2.0 latest minor
2.1 latest minor
3.0.2
For each jump between version, read the https://github.com/apache/cassandra/blob/trunk/NEWS.txt file to know if you need special actions (upgrade sstable, ...)
Question about changing partitioner types: i want to use sstableloader to copy data from an old cluster to a newer cluster. but the old cluster is using RandomPartitioner, whereas the new one uses Murmur3Partitioner. You might ask why not using COPY command to export data to csv and import it again? Well we have huge data sets and COPY command would not work (all other node's data would aggregate to one machine).
is it possible to switch new cluster partitioner to RandomPartitioner, do data replication using sstableloader, and switch back? (i tried switching, but cassandra won't restart because of it...)
No, you can't change the partitioner. That would require Cassandra to redistribute all the data and is not supported.
You could use sstable2json (with the old yaml) and then json2sstable (with the new yaml) to convert your SSTables manually. Then you can use sstableloader.