how to monitor/log slow running queries in Apache Cassandra 2.2.X version without using any external monitoring tools? Is there is any parameter that we can set in YAML to log slow running queries? or any other approach?
Also in CASSANDRA-12403, i see they added parameter "slow_query_log_timeout_in_ms: 500" for this purpose. Can we add this parameter in Cassandra 2.2.X version's Cassandra.YAML file? or do we need to apply this patch for 2.2.X version in order to make it work?
Its a feature in a newer version, you can upgrade or apply the patch and go off of a custom build. In 2.2.x theres no support to do it by itself.
Its a bit of a long shot but you might be able to get https://github.com/smartcat-labs/cassandra-diagnostics with https://github.com/smartcat-labs/cassandra-diagnostics/blob/dev/cassandra-diagnostics-core/COREMODULES.md#slow-query-module to work. It also only supports 2.1 and 3.0 though, I dont see 2.2 there.
Related
I am experiencing difficulties to get started using StormCrawler using the StormCrawler+ElasticSearch archetype. On the StormCrawler website, I see two versions, namely 1x and 2x. Similarly, Apache Storm comes in version 1 and 2.
Should I install StormCrawler using the version 1x or 2x?
What version of JDK does StormCrawler require? Is there a need to use Oracle JDK or can the OpenJDK be used as well?
I want to use StormCrawler to identify and process images and documents. At what place in the topology can these tasks best be added?
Update: According to the following URL (Storm Crawler with Java 11), StormCrawler 2 is advised. What StormCrawler+ElasticSearch archetype should be used when using StormCrawler 2?
SC 1.x is stable and the current version, 2.x is less tested but will be the main version at some point.
The thread mentioned in the question does not advise you to use SC2 as such, it mentions that you should use it if you need Java 11. If you are on Java 8, then you can use whichever version you want.
SC works fine with openjdk.
As for question #3, it depends what you want to do. Can you please elaborate?
We have spark configuration that uses spark.jars.ivySettings to customize jars resolution.
Spark jobs run in environment without internet access, so we want to skip maven central calls and use our repositories.
In spark 2.2.1 everything was working fine, but when we upgraded to 2.3, repositories specified in ivy settings are ignored. As the result our jobs are failing due to missing dependencies.
Specifying our repos with new spark.jars.repositories makes it visible for spark, but does not change an order (so it will always first check maven central, which we cannot allow).
Is this some bug introduced in new version? Or I'm doing something wrong here?
Ok, I found where is the problem. So apparently the way of acquiring spark.jars.ivySettings has changed in 2.3. Now system properties are used for that:
sys.props.get("spark.jars.ivySettings")
This change is not followed by documentation update, and for me it seems like a bug.
I recently found out that Cassandra 3.0.0 and PrestoDB don't play well together.
I have a lot of data loaded into Cassandra 3.0 and I wouldn't like to rebuild the whole thing. Is there a safe way to downgrade to 2.x temporarily until Presto is updated, so then I can come back to 3.0?
I know downgrading is not officially supported, but I'm wondering whether more experienced S.O. Cassandra users could point me in the right direction here. I assume the answer will be "don't try it", but who knows, maybe there's a way. Thanks in advance.
Update 2016-11-05 : Using version 0.147 and newer of PrestoDB and this issue has been solved. In the end I did not need to downgrade Cassandra in order to use PrestoDB. Thanks for your replies.
If you've started with 3.0, the only way I can think of is to export all your data then reimport it. The storage format has massively changed and 2.x can't read the 3.0 tables.
Unfortunately sstable2json was removed in 3.0, so you'll probably need to export it all manually then import into a previous version.
I need to upgrade my solr search from 4.7 version to 5.3.1 .
I am working on a linux platform.
Can you please provide me the steps that i need to follow .
Thank you!
I do not think that there is a definitive step by step guide for the upgrade that you are looking for.
I have a 4.3.x SOLR running in a production environment and I am contemplating the leap to upgrade to 5.x. However its clear that a lot has changed and that my upgrade is not going to be straight forward.
Also other priorities in my project have kept me from doing the upgrade.
So rest of the discussion is more a thought process than actual upgrade experience.
Last I researched I found the below links useful
https://support.lucidworks.com/hc/en-us/articles/203776523-How-to-upgrade-between-major-Solr-Versions
https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+4+to+Solr+5
From the Major changes link you will notice that a lot has changed ..
Most notably there are changes to the index format, SolrJ removal of deprecated API and that the deployment is now as a standalone server instead of a war file.
So I would suggest that you ask yourself the following questions ...
Is it possible to recreate the index from scratch ? How much time does it take to create your complete index ? If your index can be recreated quickly then , I would suggest that you do that using 5.x engine on a separate machine, while your production environment is served by your existing server. Then plan a complete upgrade from 4.x to 5.x by simply pointing your Production instance to the new SOLR engine. This approach will give you a clean slate to start with and a brand new index (but with existing data).
If you have a very large index (e.g. it takes several days to recreate it from scratch), then you may want to perform an upgrade of the live index. In that case I suggest that you consider the following.
The SOLR upgrade guide mentions 4.10 as a version that is 4.x (so I assume its is easy to upgrade from any 4.x to 4.10) and has some features built in to help with the move to 5.x. So first upgrade to 4.10 ensure that your index continues to work properly. Then use the guides mentioned above to upgrade to 5.x
We are trying to implement external authentication to Cassandra on DSE 4.7. Followed few of the guides where we have to extend IAuthenticator class but after doing that there is less documentation on how to integrate.
Is it more of plug and play where we extend IAuthenticator class build a jar and place it in lib(/usr/share/dse/resources/cassandra/lib) and change the yaml file accordingly or is it take a source code from Github build entire tree and then use?
If so is Datastax's Cassandra available on Github?
What do we need to do to build external authentication other that LDAP and Kerberos in DSE 4.7?
extend IAuthenticator class build a jar and place it in
lib(/usr/share/dse/resources/cassandra/lib) and change the yaml file
accordingly
^^ yes, this is the right approach.
Datastax's Cassandra available on Github?
Not exactly. You'll see the version of c* that ships with DSE in the release notes, you can check the source in the apache/cassandra github and it will match (up to and excluding the build number). The exact c* build under DSE will have some critical patches from future versions and that exact source code is not avaliable. However, the dot release in apache/cassandra is good enough for all intents and purposes.
I.E. look at https://github.com/apache/cassandra/tree/cassandra-2.1.8 for 4.7.1
As mentioned by #Mikea we need to override ISaslAwareAuthenticator and while using Cassandra in DSE need to be very sure of Cassandra version and then dig into appropriator Github repo.