I am running a job from a jar file in Azure Databricks. This jar has a dependency on azure-storage-file-share. Previously, there wasn't an issue installing this dependency using Maven within the Databricks UI. Now, I get failure with this error message:
Run result unavailable: job failed with error message Library installation failed for library due to user error for maven { coordinates: "com.azure:azure-storage-file-share:12.16.2" } Error messages: Library installation attempted on the driver node of cluster 0131-144423-r1g81134 and failed. Please refer to the following error message to fix the library or contact Databricks support. Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/io.netty_netty-transport-native-kqueue-4.1.86.Final.jar does not exist
To try to work around this, I manually installed this netty library (and several others) as well. I can see in the logs that it was able to download the jar successfully:
downloading https://maven-central.storage-download.googleapis.com/maven2/io/netty/netty-transport-native-kqueue/4.1.86.Final/netty-transport-native-kqueue-4.1.86.Final-osx-x86_64.jar ... [SUCCESSFUL ] io.netty#netty-transport-native-kqueue;4.1.86.Final!netty-transport-native-kqueue.jar (202ms)
However, it still fails. This is error message is in the same log after the one above:
23/01/31 14:50:09 WARN LibraryState: [Thread 137] Failed to install library maven;azure-storage-file-share;com.azure;12.16.2;; java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/io.netty_netty-transport-native-kqueue-4.1.86.Final.jar does not exist
Related
I have upgraded Weblogic version in linux server by changing the wl_home path in the setDomainEnv.sh for 12.1.2 to 12.1.3 and restart. when restarting it gives below errors.
Appreciate if anyone can give idea about this.
java.lang.IllegalAccessError: tried to access method com.bea.logging.LogBufferHandler.bufferLogObject(Ljava/lang/Object;)V from class weblogic.logging.log4j.WLLog4jMemoryBufferAppender
java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.diagnostics.lifecycle.LoggingServerService
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.diagnostics.lifecycle.DiagnosticFoundationService errors were found
java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.diagnostics.lifecycle.DiagnosticFoundationService
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of com.oracle.injection.integration.CDIIntegrationService errors were found
java.lang.IllegalStateException: Unable to perform operation: resolve on com.oracle.injection.integration.CDIIntegrationService
Use the wllog4j.jar which is part of WLS 12.1.3 build. The one part of WLS_HOME/server/lib diretory.
If your domains-home/lib folder holds older wllog4.jar which was part of 12.1.2 then you will face this issue
Error occured while trying to build the recommendation engine using PredictionIO. Please anyone know how to solve this issue.
root#testing:~/PredictionIO/engines# pio build --verbose
[INFO] [Console$] Using command '/root/PredictionIO/sbt/sbt' at the current working directory to build.
[INFO] [Console$] If the path above is incorrect, this process will fail.
[INFO] [Console$] Uber JAR disabled. Making sure lib/pio-assembly-0.9.4.jar is absent.
[INFO] [Console$] Going to run: /root/PredictionIO/sbt/sbt package assemblyPackageDependency
[ERROR] [Console$] Error: Invalid or corrupt jarfile /root/PredictionIO/sbt/sbt-launch-0.13.7.jar
[ERROR] [Console$] Return code of previous step is 1. Aborting.
For me help to download this file
https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.7/sbt-launch.jar
Rename downloaded file to sbt-launch-0.13.7.jar and replace previous file in PredictionIO/sbt/
We are running Linux RHEL v6.4 where we have Jube 2.0.33 installed. Jube is running with below env vars set
export https_proxy=
export http_proxy=
export DOCKER_REGISTRY=
export KUBERNETES_MASTER=
export FABRIC8_CONSOLE=
Now, we are using below command after setting above env vars to build simple project (contained at quickstarts-master\quickstarts\java\simple-mainclass - from Github samples)
mvn clean install jube:build fabric8:json fabric8:deploy
The build becomes successful, however when we go to jube fabric8 hawtio console and try to run application... we don't see pods running... instead we see below stacktrace printed on jube server console.
Stack trace says the built image is trying to be uploaded to external maven repos which we don't want.... we tried setting "upload" xml element in maven to "false" too.... however still no luck, we end up seeing same stack trace.
ERROR: Failed to create pod: quickstart-camel-servlet-8. java.io.IOException: Failed to resolve mvn:io.fabric8.jube.images.fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/zip/image. org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:zip:image:2.2-SNAPSHOT (runtime)
java.io.IOException: Failed to resolve mvn:io.fabric8.jube.images.fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/zip/image. org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:zip:image:2.2-SNAPSHOT (runtime)
Caused by: org.sonatype.aether.collection.DependencyCollectionException: Failed to collect dependencies for io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:zip:image:2.2-SNAPSHOT (runtime)
Caused by: org.sonatype.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:zip:image:2.2-SNAPSHOT
Caused by: org.sonatype.aether.resolution.ArtifactResolutionException: Could not transfer artifact io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:pom:2.2-SNAPSHOT from/to central (http://repo1.maven.org/maven2): Error transferring file: null to http://repo1.maven.org/maven2/io/fabric8/jube/images/fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/quickstart-war-camel-servlet-2.2-SNAPSHOT.pom
Caused by: org.sonatype.aether.transfer.ArtifactTransferException: Could not transfer artifact io.fabric8.jube.images.fabric8:quickstart-war-camel-servlet:pom:2.2-SNAPSHOT from/to central (http://repo1.maven.org/maven2): Error transferring file: null to http://repo1.maven.org/maven2/io/fabric8/jube/images/fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/quickstart-war-camel-servlet-2.2-SNAPSHOT.pom
Caused by: org.apache.maven.wagon.TransferFailedException: Error transferring file: null to http://repo1.maven.org/maven2/io/fabric8/jube/images/fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/quickstart-war-camel-servlet-2.2-SNAPSHOT.pom
at org.sonatype.maven.wagon.AhcWagon.fillInputData(AhcWagon.java:301)
Caused by: java.io.IOException: null to http://repo1.maven.org/maven2/io/fabric8/jube/images/fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/quickstart-war-camel-servlet-2.2-SNAPSHOT.pom
Caused by: java.net.ConnectException: null to http://repo1.maven.org/maven2/io/fabric8/jube/images/fabric8/quickstart-war-camel-servlet/2.2-SNAPSHOT/quickstart-war-camel-servlet-2.2-SNAPSHOT.pom
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
Caused by: java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:127)
Any help to resolve this is greatly appreciated.
Thanks in advance
We untar spark-0.9.0-incubating.tgz and trying to build it for use with Yarn.
SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 SPARK_YARN=true sbt/sbt assembly
...
[info] Resolving io.netty#netty-all;4.0.13.Final ...
[error] Server access Error: Connection timed out url=https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
[error] Server access Error: Connection timed out url=https://oss.sonatype.org/service/local/staging/deploy/maven2/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
...
If I just cut-paste the url into a browser, I get:
404 - ItemNotFoundException
Retrieval of /io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom from M2Repository(id=snapshots) is forbidden by repository policy SNAPSHOT.
org.sonatype.nexus.proxy.ItemNotFoundException: Retrieval of /io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom from M2Repository(id=snapshots) is forbidden by repository policy SNAPSHOT.
at org.sonatype.nexus.proxy.maven.AbstractMavenRepository.doRetrieveItem(AbstractMavenRepository.java:380)
at org.sonatype.nexus.proxy.maven.maven2.M2Repository.doRetrieveItem(M2Repository.java:396)
at org.sonatype.nexus.proxy.repository.AbstractRepository.retrieveItem(AbstractRepository.java:765)
at org.sonatype.nexus.proxy.repository.AbstractRepository.retrieveItem(AbstractRepository.java:608)
at org.sonatype.nexus.proxy.router.DefaultRepositoryRouter.retrieveItem(DefaultRepositoryRouter.java:155)
at org.sonatype.nexus.web.content.NexusContentServlet.doGet(NexusContentServlet.java:359)
at org.sonatype.nexus.web.content.NexusContentServlet.service(NexusContentServlet.java:331)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
I have seen this reported in a number of places but no solution. Is this error because we are behind a corporate firewall, or is this due to something else? Please advise.
I had proxy set as environment variables, but it appears they are not being picked up. Adding them in sbt directly worked for me.
Edit $SPARK_HOME/sbt/sbt
For example,
EXTRA_ARGS="-Dhttp.proxySet=true -Dhttp.proxyHost=myproxy.mycompany.com -Dhttp.proxyPort=80 -Dhttps.proxySet=true -Dhttps.proxyHost=myproxy.mycompany.com -Dhttps.proxyPort=80 -Dftp.proxySet=true -Dftp.proxyHost=myproxy.mycompany.com -Dftp.proxyPort=80 -Dhttp.nonProxyHosts=mydomain -Dhttps.nonProxyHosts=mydomain -Dftp.nonProxyHosts=mydomain"
I am working on a J2ME project in Netbeans. I am having a problem in buiding the project. Whenever I ties to build it, it's giving me preverification error. below are the logs.
Executable: C:\WTK2.5.2\bin\preverify
Arguments : -classpath "C:\J2ME Client USE\S60CnB\lib\jsr257.jar;C:\J2ME Client USE\S60CnB\lib\JSR257Ext.jar;C:\WTK2.5.2\lib\mmapi.jar;C:\WTK2.5.2\lib\jsr75.jar;C:\WTK2.5.2\lib\cldcapi11.jar;C:\WTK2.5.2\lib\jsr239.jar;C:\WTK2.5.2\lib\midpapi20.jar;C:\WTK2.5.2\lib\jsr179.jar" -d "C:\J2ME Client USE\S60CnB\ec09b2f6.tmp" -target CLDC1.1 "C:\J2ME Client USE\S60CnB\build\NKTej2.jar"
JAR file creation failed with error 1
The preverified classes if any are in tmp26379. See jar log of errors in C:\J2ME Client USE\S60CnB\ec09b2f6.tmp\jarlog.txt
Error preferifying, attempting to print C:\J2ME Client USE\S60CnB\ec09b2f6.tmp\jarlog.txt
====C:\J2ME Client USE\S60CnB\ec09b2f6.tmp\jarlog.txt====
java.io.FileNotFoundException: tmp26379\META-INF\MANIFEST.MF (The system cannot find the path specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at java.io.FileInputStream.<init>(FileInputStream.java:79)
at sun.tools.jar.Main.run(Main.java:148)
at sun.tools.jar.Main.main(Main.java:1147)
Error: No error
jar -cfm "C:\J2ME Client USE\S60CnB\ec09b2f6.tmp\NKTej2.jar" tmp26379\\META-INF\MANIFEST.MF -C tmp26379 .
C:\J2ME Client USE\S60CnB\build.xml:146: Preverification failed (result=1)
Please help me out.
please details of your program. Which UI are you using? Have you gone through your program to see their is no bug? Have you successfully embedded the oracle sdk plugins for netbeans?
These are the possible problem. If it persists, reply...
J2me tutorial at http://www.tutorialmasterng.blogspot.com