JMeter 3.3 connect Spark 2.2.1 error: "Cannot create PoolableConnectionFactory (Method not supported)" - apache-spark

Question

Use this list of jars, I can successfully connect SQuirrel SQL to Spark 2.2.1:
commons-logging-1.1.3.jar
hadoop-common-2.7.3.jar
hive-exec-1.2.1.spark2.jar
hive-jdbc-1.2.1.spark2.jar
hive-metastore-1.2.1.spark2.jar
http-client-1.0.4.jar
httpclient-4.5.2.jar
httpcore-4.4.4.jar
libfb303-0.9.3.jar
libthrift-0.9.3.jar
log4j-1.2.17.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
spark-hive-thriftserver_2.11-2.2.1.jar
spark-hive_2.11-2.2.1.jar
spark-network-common_2.11-2.2.1.jar
I think the above jars are more than necessary. But when trying to connect JMeter 3.3 to same Spark 2.2.1 ThriftServer with them, I got below error message
enter code here Cannot create PoolableConnectionFactory (Method not supported)
The JDBC configuration is here:
The full response at Jmeter is here:
Thread Name: test 1-1
Sample Start: 2018-04-03 13:34:43 CST
Load time: 511
Connect Time: 510
Latency: 0
Size in bytes: 62
Sent bytes:0
Headers size in bytes: 0
Body size in bytes: 62
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: null 0
Response message: java.sql.SQLException: Cannot create PoolableConnectionFactory (Method not supported)
Response headers:
SampleResult fields:
ContentType: text/plain
DataEncoding: UTF-8
I also try to use newer Hive JDBC driver 2.3.0, but it is obviously that is not worked with Spark 2.2.1 either on beeline or any others including Jmeter.
Error message when using beeline with Hive JDBC driver 2.3.0 is here:
$ beeline -u jdbc:hive2://<hostip>:10000/tpch_sf100_orc -n rxxxds
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.3.0-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apache-tez-0.9.0-bin/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.9.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://<hostip>:10000/tpch_sf100_orc
18/04/03 13:41:58 [main]: ERROR jdbc.HiveConnection: Error opening session
org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=tpch_sf100_orc})
at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[hive-exec-2.3.0.jar:2.3.0]
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) ~[hive-exec-2.3.0.jar:2.3.0]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:168) ~[hive-exec-2.3.0.jar:2.3.0]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:155) ~[hive-exec-2.3.0.jar:2.3.0]
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:680) [hive-jdbc-2.3.0.jar:2.3.0]
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:200) [hive-jdbc-2.3.0.jar:2.3.0]
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [hive-jdbc-2.3.0.jar:2.3.0]
at java.sql.DriverManager.getConnection(DriverManager.java:664) [?:1.8.0_112]
at java.sql.DriverManager.getConnection(DriverManager.java:208) [?:1.8.0_112]
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.Commands.connect(Commands.java:1641) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.Commands.connect(Commands.java:1536) [hive-beeline-2.3.0.jar:2.3.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1274) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1313) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:867) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:776) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1010) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519) [hive-beeline-2.3.0.jar:2.3.0]
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501) [hive-beeline-2.3.0.jar:2.3.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
at org.apache.hadoop.util.RunJar.run(RunJar.java:239) [hadoop-common-2.9.0.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:153) [hadoop-common-2.9.0.jar:?]
18/04/03 13:41:58 [main]: WARN jdbc.HiveConnection: Failed to connect to <hostip>:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://<hostip>:10000/tpch_sf100_orc: Could not establish connection to jdbc:hive2://<hostip>:10000/tpch_sf100_orc: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=tpch_sf100_orc}) (state=08S01,code=0)
Beeline version 2.3.0 by Apache Hive
What else can be done to connect JMeter to Spark?

Answer 1

Most probably this is due to client and server libraries mismatch, you need to use the same version as on the server.
So identify which version of hive is running on your server and download appropriate hive-jdbc jar and all the dependencies (you can retrieve them using i.e. Maven Dependency Plugin) and update the ones in JMeter Classpath with the correct versions.
It might be better to use "clean" JMeter installation for this to avoid possible jar hell so it is a good time to upgrade to JMeter 4.0

Answer 2

You could also build your own "JDBC sampler" and add/adapt just few line of code from the original to make it works. I had to do that for Hive and Phoenix support 4 years ago and it worked. It required a Timeout method that was not implemented back then.
Here are the sources:
https://github.com/csalperwyck/JMeterJDBCSamplerWithOutTimeOut