I have created the custom trigger.jar and placed in the directory where Cassandra service can read the same, after performing the "nodetool realoadtriggers" it just prints "Loading new /trigger.jar" in system.log, but it's not executing the java class in the jar.
Note: Also, I have created the trigger on the table which should trigger this custom trigger jar on writing the inserting the data to the table, but it's not loading the class
The java class implements ITrigger overrides argument method. Any pointers for debugging the same will be helpful.
Cassandra verison - DSE 6.7.5
From the DSE 6.0 upgrade guide:
The org.apache.cassandra.triggers.ITrigger interface was modified from augment to augmentNonBlocking for non-blocking internal architecture. Updated trigger implementations must be provided on upgraded nodes.
So you need to change your implementation to comply.
Related
I'm using azure databricks 5.5 LTS with spark 2.4.3 and scala 2.11. Almost every request going to the databricks cluster is coming up with the following error log
ERROR RetryingHMSHandler: NoSuchObjectException(message:There is no database named global_temp)
at org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:487)
at org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
While this isn't affecting the end-result of what we're trying to do, our logs are constantly getting filled with this and isn't very pleasant to go through. I've tried turning it off by setting the following property to the driver and executor
log4j.level.org.apache.hadoop.hive.metastore.RetryingHMSHandler=OFF
only to, later on, realize the class RetryingHMSHandler actually uses slf4j logger, is there an elegant way to overcome this?
Maybe late, but faced the same issue with Databricks cluster 9.1 LTS (Apache Spark 3.1.2, Scala 2.12). Solved by using a init script that added the following two properties
log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL, publicFile
log4j.additivity.org.apache.hadoop.hive.metastore.RetryingHMSHandler=false
to driver's log4j.properties.
My goal was to remove all verbose logs from the "log4j-active.log" file that can be downloaded from a job UI. By following https://learn.microsoft.com/en-us/azure/databricks/kb/clusters/overwrite-log4j-logs, I decided to add/overwrite some property values within driver's log4j.properties (first I had a look at its content, of course).
Added that two properties, I was able to remove also RetryingHMSHandler (the only third-party log call that was still surviving)
Hope it helps ;)
I'm trying to deploy an jdbc-sink to a Helm based Kuberentes install of Spring Cloud Data Flow.
How would I go about adding JDBC jars in order to make use of the starters? I'm getting the following error when trying to deploy the app (in this case MySQL):
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under '' to com.zaxxer.hikari.HikariDataSource:
Property: driverclassname
Value: com.mysql.cj.jdbc.Driver
Origin: "driverClassName" from property source "source"
Reason: Failed to load driver class com.mysql.cj.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Would I need to extend the existing starter, and manually add the Driver as there's no way of guaranteeing which driver it should be using?
Thanks!
We ship the OSS license-friendly drivers for few databases in SCDF and the app-starters that require database access, including the jdbc applications.
For proprietary drivers, there's a procedure to patch the out-of-the-box app-starters that we maintain and ship — more details in the reference guide here.
Once when you bundle the relevant driver in the classpath, you'd produce a docker image to then use it in SCDF.
We are using embedded cassandra in our groovy test cases, we are migrating from logback to log4j2. Whenever i run the groovy test which uses cassandra it gives an exception of NoClassDefFoundError for ch/qos/logback /classic /Logger. I have excluded logback dependency from all existing cassandra dependency still its looking for logback. How should i make cassandra log using log4j2
Cassandra isn't setup or designed to run embedded so while there might be some hacks that can get you by it will be something difficult to keep working across versions.
I would recommend using ccm for your tests to run it out of jvm and it will also give you more control for interesting configurations. The java driver has a useful bridge for java applications in their tests here: CCMBridge.java
Longterm you might be able to use something CASSANDRA-14821 as there will be native connections exposed and give you a lot more control over results of queries and such.
I have installed cassandra CQL shell on my local system, I am using jmeter v3.0 for testing the queries per second(QPS) on cassandra CQL shell. I have installed "cassandra support" plugin available in "plugin manager" with jmeter.
I have created keyspace in cassandra(keyspace1), created a table(student) and added some data in CQL shell.
I have added "cassandra properties" from config elements and entered the properties in jmeter.
Here are the properties:
I have added "cassandra get" sampler.
added "view results tree" listener.
when I run it I am getting the following error:
ERROR: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException: Read a negative frame
size (-2080374784)!
I have given the "schema properties" as seen on github.
but no use. I am getting the same error.
Can anyone suggest me how to resolve this error?
ERROR: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException: Read a negative frame
size (-2080374784)!
I want to use the cassandra samplers for put, get and delete operations on the database.
It looks like the Netflix plugin you are using is somewhat deprecated because it uses the Cassandra thrift API which is deprecated (the plugin did not have a lot of recent commits in github too).
See announcements here and here
Even if you succeed in your test with this plugin, it would not be very representative of a current client use (hence load).
IMHO you should make your test with JSR223 groovy scripts (preprocessor and samplers) and use the Datastax standard java driver + CQL in your script. I did it some time ago, it works fine.
(update: documented here)
Or may be try this JMeter plugin from a Datastax guy, it seems to use CQL . I didn't tried it, but it looks fine.
HTH,
Alain
I have an application that connects to Cassandra using the Java Driver, fetches some configuration and based on the results generates and executes some PIG scripts.
Now, I am able to successfully connect to Cassandra, when jars required for PIG are not in the classpath. Similarly, I am able to launch PigServer class and execute scripts / statements using the entire DSE stack when I am not connecting to Cassandra using the java driver to retrieve the configuration.
When I use both of them I get following exception:
org.jboss.netty.channel.ChannelPipelineException: Failed to initialize a pipeline.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:181)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:570)
... 35 more
Caused by: org.jboss.netty.channel.ChannelPipelineException: Failed to initialize a pipeline.
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:208)
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
at com.datastax.driver.core.Connection.<init>(Connection.java:100)
at com.datastax.driver.core.Connection.<init>(Connection.java:51)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:376)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:207)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:170)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:87)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:576)
at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:520)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:67)
at com.datastax.driver.core.Cluster.buildFrom(Cluster.java:94)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:501)
I see others have seen similar exception, but when trying to execute Cassandra statements, from MapReduce tasks, which is not my case:
https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/FhW_8e4FyAI
http://www.datastax.com/dev/blog/the-native-cql-java-driver-goes-ga#comment-297187
Thanks!
DSE stacks connect to Cassandra through thrift API which is different from Cassandra Java Driver.
You can't use Cassandra Java driver for Pig/Hadoop before CASSANDRA-6311 is resolved.
There may be the bad security certificate/security certificate expiration issue if you are using certificate.