After using UpdateEngine.applyPayload to complete the update, how to restart without switching to the updated partition
I haven't found such a solution.
Related
I want to upgrade cassandra 2.1.19 cluster to 3.11.1 without downtime.
Will 3.11.1 nodes work together with 2.1.19 nodes at the same time?
Key point will be how you connect to your cluster. You will need to try out on test systems if everything works from your application side doing the switch.
I recommend a two stop process in this case, migrate from 2.1.19 to 3.0.x - one node at atime.
For every node do the following (i said you need to test before before going to production right?):
nodetool drain - wait for finish
stop cassandra
backup your configs, the old one wont work out of the box
remove the cassandra package / tarball
read about the java and other cassandra 3.x requirements and ensure you met them
add the repo and install the 3.0.x package or tar ball
some packages start the node immediately - you may have to stop them again
make up the new config files (diff or something will be your friend, read the docs about the new options), one time only you should be able to resuse if on all the other nodes
start cassandra (did I say test this on a test system?) and wait until the node has joined the ring again nodetool status
upgrade your sstables with nodetool upgradesstables - almost always needed, dont skip this even if "something" works right now
this upgrade tends to be really slow - it's just a single thread running rewriting all your data, so i/o will be a factor here
all up and running -> go ahead to the next node and repeat
After that - upgrade 3.0.x to 3.11.x in the same fashion, add the new repo, configure for 3.11.x as for 3.0.x above and so on. But this time you can skip upgrading sstables as the format stays the same (but it wont harm if you do so).
Did I mention to do this on testing system first? One thing that will happen and may break things - older native protocols will be gone as well as rpc/ thrift.
Hope I didn't miss something ;)
I have a 3 node Cassandra cluster and I want to make some adjustments to the cassandra.yaml
My question is, how should I perform this? One node at a time or is there a way to make it happen without shutting down nodes?
Btw, I am using Cassandra 2.2 and this is a production cluster.
There are multiple approaches here:
If you edit the cassandra.yaml file, you need to restart cassandra to re-read the contents of that file. If you restart all nodes at once, your cluster will be unavailable. Restarting one node at a time is almost always safe (provided you have sane replication-factors and consistency-levels). If your cluster is configured to survive a rack or datacenter outage, then you can safely restart more nodes concurrently.
Many settings can be changed without a restart via JMX, though I don't have a documentation link handy. Changing via JMX WON'T change cassandra.yml though, so you'll need to update that also or your config will revert back to what's in the file when the node restarts.
If you're using DSE, OpsCenter's Lifecycle Manager feature makes updating configs a simple point-and-click affair (disclaimer, I'm biased as I'm an LCM dev).
After restart my cassandra node does not start anymore. Ends with following error message.
ERROR 18:39:37 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
java.lang.AssertionError: We shouldn't have got there is the base row had no associated entry
cassandra has heavy cpu usage and use 2,1 gb of memory there is be 1gb more available. I run nodetool cleanup and repair, but did not help.
I have 5 materialzied views on this table, but the amount of rows in table is under 2000, that is not much.
The cassandra runs in a docker container. The container is access able, but can not call cqlsh and my website cound not connect too
How can I fix the error? Is it possible to fix it?
I do not really fix it, but I get it run. My first container is now complitly crashed and is not start able anymore. But I had same problem with other container, that are enter able. I run apt-get update and apt-get upgrade and get cassandra work again.
It is not importand if there are any upgrades, only to run the command make cassandra call able again. Have to do it at each restart, but better as a completle crashed database.
I'm doing little experiment with a couple of Cassandra servers.
The thing is that I want to add a new datacenter to an existing cluster.
Using 1.1.10 version and following this guidelines:
http://www.datastax.com/docs/1.1/cluster_management
Everything goes well until suddenly the stream stops. Sometimes stops when copying a file and sometimes just stops without errors.
What I do is remove all the data, restart Cassandra and start the rebuild process.
Since the rebuilds doesn't finish, it is a good idea to leave the data that was transferred and execute repairs?
Thanks
I've solved this issue changing the streaming_socket_timeout_in_ms on the yaml file to 30000
My current cassandra version is 1.1.2, it is implemented with a single node cluster, i would like to upgrade it 1.2.6 with multiple nodes in the ring. is it a proper way to migrate it directly to 1.2.6 or i should follow version by version migration.
I found the upgrading steps from this link
http://fossies.org/linux/misc/apache-cassandra-1.2.6-bin.tar.gz:a/apache-cassandra-1.2.6/NEWS.txt.
There are 9 other releases available between this two versions.
I migrate a two cluster nodes from 1.1.6 to 1.2.6 without problems and without doing version by version. Anyway, you should take a closer look into:
http://www.datastax.com/documentation/cassandra/1.2/index.html?pagename=docs&version=1.2&file=index#upgrade/upgradeC_c.html#concept_ds_smb_nyr_ck
Because there are a lot of new features from version 1.2 like the partioners maybe you need to change some configurations for your cluster.
You may directly hop on to C1.2.6.
We migrated our 4-node cluster from C1.0.9 to C1.2.8 recently without any issues. This was a rolling upgrade i.e. upgrade one node at a time and after each upgrade of a node, allow the cluster to stabilize (depends upon the traffic during upgrade)
These are the steps that we followed:
Perform below on each node,
Run Disablegossip and disablethrift, such that this node is seen as DOWN by other nodes.
flush/drain the memtables, run compaction to merge SSTables
take snapshot and enable incremental backups
This stops all the other nodes/clients from writing to this node and since memtables are flushed to disk, startup times are fast as it need not walk-through commit logs.
stop Cassandra (though this node is down, cluster is available for write/read, so zero downtime)
upgrade sstables to new storage format using sstableupgrade
install/untar Cassandra 1.2.8 on the new locations
move upgraded sstables to appropriate location
merge Cassandra.yaml from previous version and current version by a manual diff (need to detail out difference)
start Cassandra
watch the startup messages to ensure the node comes up without difficulty and is shown in the ring with mixed 1.0.x/1.2.x