What *.jar I need to make my groovysh work with hbase 1.1.2 i am trying to run a simple script and the following imports fail -
groovy:000>
import org.apache.hadoop.hbase.client.Put
ERROR java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/io/HeapSize
at java_lang_Runnable$run.call (Unknown Source)
groovy:000>
import org.apache.hadoop.hbase.client.Result
ERROR java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/CellScannable
at java_lang_Runnable$run.call (Unknown Source)
import org.apache.hadoop.hbase.util.Bytes
Invalid import definition: 'org.apache.hadoop.hbase.util.Bytes'; reason: startup failed:
script14891462389401754287428.groovy: 1: unable to resolve class org.apache.hadoop.hbase.util.Bytes
# line 1, column 1.
import org.apache.hadoop.hbase.util.Bytes
I have loaded hbase-client.jar in my -classthpath. Just need to write a simple script that puts and increments hbase variables and execute via groovysh.
Edit 1
I get this now
groovy:000> groovy.grape.Grape.grab(group:'org.apache.hbase', module:'hbase-client', version:'1.3.0')
ERROR java.lang.RuntimeException:
Error grabbing Grapes -- [download failed: junit#junit;4.12!junit.jar, download failed: org.slf4j#slf4j-api;1.7.7!slf4j-api.jar, download failed: org.slf4j#slf4j-log4j12;1.6.1!slf4j-log4j12.jar]
groovy:000> groovy.grape.Grape.grab('org.apache.hbase:hbase-client:1.3.0')
hbase client has a lot of dependencies:
http://grepcode.com/snapshot/repo1.maven.org/maven2/org.apache.hbase/hbase-client/1.1.1/
You can't just grab one jar and stick it on the classpath, you need a whole load of them
I don't use groovysh, but you should be able to do:
:grab 'org.apache.hbase:hbase-client:1.3.0'
And that should pull down hbase-client and all of its dependencies on to your classpath
Related
I am running a job from a jar file in Azure Databricks. This jar has a dependency on azure-storage-file-share. Previously, there wasn't an issue installing this dependency using Maven within the Databricks UI. Now, I get failure with this error message:
Run result unavailable: job failed with error message Library installation failed for library due to user error for maven { coordinates: "com.azure:azure-storage-file-share:12.16.2" } Error messages: Library installation attempted on the driver node of cluster 0131-144423-r1g81134 and failed. Please refer to the following error message to fix the library or contact Databricks support. Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/io.netty_netty-transport-native-kqueue-4.1.86.Final.jar does not exist
To try to work around this, I manually installed this netty library (and several others) as well. I can see in the logs that it was able to download the jar successfully:
downloading https://maven-central.storage-download.googleapis.com/maven2/io/netty/netty-transport-native-kqueue/4.1.86.Final/netty-transport-native-kqueue-4.1.86.Final-osx-x86_64.jar ... [SUCCESSFUL ] io.netty#netty-transport-native-kqueue;4.1.86.Final!netty-transport-native-kqueue.jar (202ms)
However, it still fails. This is error message is in the same log after the one above:
23/01/31 14:50:09 WARN LibraryState: [Thread 137] Failed to install library maven;azure-storage-file-share;com.azure;12.16.2;; java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/io.netty_netty-transport-native-kqueue-4.1.86.Final.jar does not exist
I tried to read an XML file in Jupiter notebook getting the below error
import os
from os import environ
environ['PYSPARK_SUBMIT_ARGS'] = '--packages com.databricks:spark-xml_2.13:0.4.1 pyspark-shell'
df = spark.read.format('xml').option('rowTag', 'row').load('test.xml')
The error:
Py4JJavaError: An error occurred while calling o251.load.
: java.lang.ClassNotFoundException: Failed to find data source: xml. Please find packages at http://spark.apache.org/third-party-projects.html
I am trying to package the following Python code into an executable file using PyInstaller:
import pandas as pd
import teradatasql
with teradatasql.connect(host='abcdxxx', user='abcdxxx', password='abcdxxx') as connect:
query = "SHOW TABLE AdventureWorksDW.DimAccount"
df = pd.read_sql(query, connect)
print(df)
When I run the .exe file, it gives me the error:
PyInstallerImportError: Failed to load dynlib/dll
'C:\\Users\\GAX~1.P\\AppData\\Local\\Temp\\_MEI153202\\teradatasql\\teradatasql.dll'.
Most likely this dynlib/dll was not found when the application was frozen.
[9924] Failed to execute script 'hello' due to unhandled exception!
I tried to make the following changes to the .spec file:
b = [
('C:\Users\Path_to_Python\Python\Python310\Lib\site-
packages\teradatasql\teradatasql.dll', '.\\teradatasql')
]
a = Analysis(['hello.py'],
pathex=[],
binaries=b,
datas=[] # , .....
)
But it doesn't resolve the problem. How to fix this?
We provide an article explaining how to include the Teradata SQL Driver for Python into an application packaged by PyInstaller:
https://support.teradata.com/community?id=community_blog&sys_id=c327eac51b1e9c103b00bbb1cd4bcb37
I am setting Jmeter with Groovy of Cassandra DB.
However, I cannot fix these errors.
could you help me?
Response message: javax.script.ScriptException:
org.codehaus.groovy.control.MultipleCompilatiON-ERRORsException:
startup failed: General error during class generation:
java.lang.NoClassDefFoundError: Unable to load class
com.datastax.driver.core.Session due to missing dependency
org/apache/cassandra/transport/Message$Request
You need to add:
Cassandra JDBC Driver itself
All its dependencies, to wit:
asm-5.0.3.jar
asm-analysis-5.0.3.jar
asm-commons-5.0.3.jar
asm-tree-5.0.3.jar
asm-util-5.0.3.jar
guava-19.0.jar
jffi-1.2.16.jar
jffi-1.2.16-native.jar
jnr-constants-0.9.9.jar
jnr-ffi-2.1.7.jar
jnr-posix-3.0.44.jar
jnr-x86asm-1.0.2.jar
metrics-core-3.2.2.jar
netty-buffer-4.0.56.Final.jar
netty-codec-4.0.56.Final.jar
netty-common-4.0.56.Final.jar
netty-handler-4.0.56.Final.jar
netty-transport-4.0.56.Final.jar
slf4j-api-1.7.25.jar
to JMeter Classpath
So you will need to:
Download cassandra-driver-core-3.6.0.jar
Download all the aforementioned dependencies
Copy the driver and the dependencies to "lib" folder of your JMeter installation
Restart JMeter to pick the libraries up
More information: Cassandra Load Testing with Groovy
I have opened Spark-Shell. In shell we already have an variable -
spark: org.apache.spark.sql.SparkSession
I have a third party Jar which has package name starting with "spark", like -
spark.myreads.one.KafkaProducerWrapper
When I try to import above package on spark shell then 'm getting exception -
scala> import spark.myreads.one.KafkaProducerWrapper
<console>:38: error: value myreads is not a member of org.apache.spark.sql.SparkSession
import spark.myreads.one.KafkaProducerWrapper
How can I import such a package on Spark-Shell resolving above conflict.
I'm using Spark-2.0.0 , JDK-1.8 and Scala -2.11
Use _root_ as beginning part like this:
import _root_.spark.myreads.one.KafkaProducerWrapper