I want to resolve the error when I run spark-shell
I am on arch installing apache spark from the AUR
When I run the command spark-shell I get the following error
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
I also installed the slf4j from aur and it does not help either. how can run spark on arch
It seems like you have a problem regarding Java which I canno't help you with...
Did you do the correct configuration for Spark ?
cd /etc/profile.d
sudo nano apache-spark.sh
Modifiy : SPARK_HOME=/opt/apache-spark by SPARK_HOME=/opt/apache-spark/bin
Worked for me
Related
I'm new to Cassandra and Pyspark, initially I installed cassandra version 3.11.1, openjdk 1.8, pyspark 3.x and scala 1.12. I was getting a lot of errors as shown below after running my python server.
raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o33.load.
: java.lang.NoClassDefFoundError: scala/Product$class
at com.datastax.spark.connector.util.ConfigParameter.<init>(ConfigParameter.scala:7)
at com.datastax.spark.connector.rdd.ReadConf$.<init>(ReadConf.scala:33)
at com.datastax.spark.connector.rdd.ReadConf$.<clinit>(ReadConf.scala)
at org.apache.spark.sql.cassandra.DefaultSource$.<init>(DefaultSource.scala:134)
at org.apache.spark.sql.cassandra.DefaultSource$.<clinit>(DefaultSource.scala)
at org.apache.spark.sql.cassandra.DefaultSource.createRelation(DefaultSource.scala:55)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:355)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:325)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:307)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:307)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: scala.Product$class
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 23 more
I didn't know what exactly this error is but after some research I realized that the pyspark Cassandra connection is having some issues. Then I checked the versions also. During my research I saw that Cassandra versions other than 4.x is not compatible with Python3.9. I uninstalled Cassandra and tried to install cassandra4 distribution but that is throwing me another set of errors after running the command:
wget http://mirror.cogentco.com/pub/apache/cassandra/4.0-beta2/apache-cassandra-4.0-beta2-bin.tar.gz
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
cassandra : Depends: python3 (>= 3.6) but 3.5.1-3 is to be installed
Recommends: ntp but it is not going to be installed or
time-daemon
E: Unable to correct problems, you have held broken packages.
Can someone help me understand the issue? How can I install Cassandra and Pyspark along with Python3.9. Is there any version incompatibility here?
updating question based on answer
I have updated my versions on another machine:
Currently, I'm using the following versions: Pyspark 3.0.1 Cassandra:4.0 cqlsh:5.0.1 python:3.6 Scala:2.12
I tried using the connector 3.0.0 as well as 3.1.0 both are giving me errors:
UNRESOLVED DEPENDENCY: com.datastax.spark#spark-cassandra-connector_2.12;3.0.0: not found
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: com.datastax.spark#spark-cassandra-connector_2.12;3.0.0: not found]
at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1389)
at org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:308)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:871)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
.......
raise Exception("Java gateway process exited before sending its port number")
Exception: Java gateway process exited before sending its port number
Connection string used: --packages com.datastax.spark:spark-cassandra-connector_2.12:3.0.0 --conf spark.cassandra.connection.host=127.0.0.1 pyspark-shell as pyspark version is 3.0.1 now.
You are using wrong version of Cassandra connector - if you are using pyspark 3.x, the you need to get corresponding version - 3.0 or 3.1. Your version is for old versions of Spark:
pyspark --packages com.datastax.spark:spark-cassandra-connector_2.12:3.1.0
P.S. Cassandra 4.0 is also released already - it makes no sense use beta2
I have downloaded burpsuite pro version but I got this error while using this command
java -javaagent:BurpSuiteLoader_v2021.3.2.jar -noverify -jar burpsuite_pro_v2021.3.2.jar
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: burp/StartBurp has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
I am trying to install android studio in ubuntu 16.04 LTS.
I downloaded the latest release and installed jdk using
sudo apt install openjdk-9-jre-headless
After the above step I am trying to run ./studio.sh after extracting Android studio zip file but It is giving me the following error. Please help.
com.intellij.ide.plugins.PluginManager$StartupAbortedException: java.lang.reflect.InvocationTargetException
at com.intellij.ide.plugins.PluginManager$2.run(PluginManager.java:96)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.ide.plugins.PluginManager$2.run(PluginManager.java:93)
... 1 more
Caused by: java.lang.ExceptionInInitializerError
at com.intellij.openapi.application.ApplicationNamesInfo.getInstance(ApplicationNamesInfo.java:49)
at com.intellij.ui.AppUIUtil.getFrameClass(AppUIUtil.java:127)
at com.intellij.ui.AppUIUtil.updateFrameClass(AppUIUtil.java:120)
at com.intellij.idea.StartupUtil.prepareAndStart(StartupUtil.java:114)
at com.intellij.idea.MainImpl.start(MainImpl.java:34)
... 6 more
Caused by: java.lang.RuntimeException: Cannot load resource: /idea/AndroidStudioApplicationInfo.xml
at com.intellij.openapi.application.ApplicationNamesInfo.<init>(ApplicationNamesInfo.java:59)
at com.intellij.openapi.application.ApplicationNamesInfo.<init>(ApplicationNamesInfo.java:30)
at com.intellij.openapi.application.ApplicationNamesInfo$ApplicationNamesInfoHolder.<clinit>(ApplicationNamesInfo.java:43)
... 11 more
Caused by: java.io.FileNotFoundException: /idea/AndroidStudioApplicationInfo.xml
at com.intellij.openapi.util.JDOMUtil.loadDocument(JDOMUtil.java:351)
at com.intellij.openapi.application.ApplicationNamesInfo.<init>(ApplicationNamesInfo.java:55)
... 13 more
I saw similar problem on StackOverflow. Please, do not install Java 9 before previous would be officially deprecated. This version is still in sdevelopment process and might be highly unstable
Unistall openJdk-9 and install Oracle-jdk-8 following this post:
http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html
I installed Light table in 0.7.2 for linux x64(Ubuntu 14.04).
I installed plugin Groovy 0.0.7
I stored in file test.groovy
println (1..10)
When I try to evaluate it(ctrl+enter) i got
We couldn't connect. Looks like there was an issue trying to connect to the project. Here's what we got:
null
null
find: `/usr/bin/../embeddable': No such file or directory
Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObject
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more
I am new to Lighttable, how could I fix it ?
Reinstall the plugin and check where it is being installed would be the quickest route. The other alternative is to embed the plugin into the project which would take you a bit of extra time.
Here would be the two quickest suggestions:
Pull the plugin out. Then do git clean -fdx and be sure the plugin is gone. Then reinstall from scratch.
Find where the plugin is located. By default, clojure and groovy typically are located at LightTable/deploy/plugins. If it is, then this is easy, do a git pull on the directory because the .jar may not be in the same location as the plugin itself.
Try number 1 first. It is simpler and faster to accomplish.
When trying to run a basic javafx application on ubuntu linux, I am seeing the following error. The error appears while using either the commandline or netbeans to run the application.
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Can't load library: /home/venkat/.m2/repository/com/oracle/javafx/javafx/2.1.0-beta/i386/libglass.so
at com.sun.javafx.tk.quantum.QuantumToolkit.startup(QuantumToolkit.java:277)
at com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:90)
at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:163)
at com.sun.javafx.application.LauncherImpl.access$000(LauncherImpl.java:47)
at com.sun.javafx.application.LauncherImpl$1.run(LauncherImpl.java:115)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.UnsatisfiedLinkError: Can't load library: /home/venkat/.m2/repository/com/oracle/javafx/javafx/2.1.0-beta/i386/libglass.so
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1828)
at java.lang.Runtime.load0(Runtime.java:792)
at java.lang.System.load(System.java:1059)
at com.sun.glass.utils.NativeLibLoader.loadLibraryFullPath(NativeLibLoader.java:143)
at com.sun.glass.utils.NativeLibLoader.loadLibraryInternal(NativeLibLoader.java:56)
at com.sun.glass.utils.NativeLibLoader.loadLibrary(NativeLibLoader.java:31)
at com.sun.glass.ui.Application$1.run(Application.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.glass.ui.Application.loadNativeLibrary(Application.java:73)
at com.sun.glass.ui.Application.loadNativeLibrary(Application.java:85)
at com.sun.glass.ui.gtk.GtkPlatformFactory.<clinit>(GtkPlatformFactory.java:23)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:186)
at com.sun.glass.ui.PlatformFactory.getPlatformFactory(PlatformFactory.java:20)
at com.sun.glass.ui.Application.Run(Application.java:108)
at com.sun.javafx.tk.quantum.QuantumToolkit.startup(QuantumToolkit.java:267)
... 5 more
This exception is caused because maven is unable to find the native libraries on the classpath.
Netbeans
You can solve the problem by adding JVM argument inside the run profile.
-Djava.library.path=/home/venkat/Programs/javafx/2.1.0-beta/rt/lib/i386/
Image
Command line
If you face the same problem while trying to run the app on the commandline, the following exports should fix it.
export JAVAFX_HOME=/home/venkat/Programs/javafx/2.1.0-beta
export CLASSPATH=\$JAVAFX_HOME/rt/lib/jfxrt.jar
export LD_LIBRARY_PATH=/home/venkat/Programs/javafx/2.1.0-beta/rt/lib/i386/
The first two env variables above fix a classpath problem which is seen when javafx runtime cannot be found.