I ran the below command to run the dependency check under a proxy server.
./dependency-check.sh \
--scan test/sample.jar \
--project test_owasp \
--out test/output \
It gives me the following output:
[DependencyCheck] [ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
[DependencyCheck] org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
[DependencyCheck] at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:347)
[DependencyCheck] at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded(NvdCveUpdater.java:385)
[DependencyCheck] at org.owasp.dependencycheck.data.update.NvdCveUpdater.update(NvdCveUpdater.java:122)
[DependencyCheck] at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:936)
[DependencyCheck] at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:737)
[DependencyCheck] at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:667)
[DependencyCheck] at org.owasp.dependencycheck.App.runScan(App.java:254)
[DependencyCheck] at org.owasp.dependencycheck.App.run(App.java:186)
[DependencyCheck] at org.owasp.dependencycheck.App.main(App.java:81)
[DependencyCheck] Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta'
[DependencyCheck] at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:131)
[DependencyCheck] at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:340)
[DependencyCheck] ... 8 common frames omitted
[DependencyCheck] Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; unable to connect.
[DependencyCheck] at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:238)
[DependencyCheck] at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:138)
[DependencyCheck] at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:126)
[DependencyCheck] ... 9 common frames omitted
[DependencyCheck] Caused by: java.net.SocketTimeoutException: connect timed out
[DependencyCheck] at java.net.PlainSocketImpl.socketConnect(Native Method)
[DependencyCheck] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[DependencyCheck] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[DependencyCheck] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[DependencyCheck] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[DependencyCheck] at java.net.Socket.connect(Socket.java:607)
[DependencyCheck] at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
[DependencyCheck] at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
[DependencyCheck] at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
[DependencyCheck] at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
[DependencyCheck] at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
[DependencyCheck] at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
[DependencyCheck] at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
[DependencyCheck] at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
[DependencyCheck] at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
[DependencyCheck] at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
[DependencyCheck] at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:178)
[DependencyCheck] ... 11 common frames omitted
[DependencyCheck] [INFO] Skipping RetireJS update since last update was within 24 hours.
[DependencyCheck] [WARN] Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
[DependencyCheck] [ERROR] Unable to continue dependency-check analysis.
[DependencyCheck] [ERROR] One or more fatal errors occurred
[DependencyCheck] [ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
[DependencyCheck] [ERROR] No documents exist
It was not able to download the Metadata thru' proxy, eventhough I passed the below parameters with command.
--proxyserver sample.proxy.com \
--proxyport 1234
Could anyone help me on resolving this issue, please!!
By creating a Maven Project and adding owasp dependency check dependency code in pom.xml, I was able to run owasp dependency check along with the smooth download of resources (nvd-cve's). Before running add the jars to scan, in src/main/resources location.
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>owasp</groupId>
<artifactId>dependency-check</artifactId>
<version>99.0.0.0.0</version>
<name>OWASP_Dependency_Check</name>
<dependencies>
<dependency>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>5.0.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Run the following command:
mvn clean install org.owasp:dependency-check-maven:check -DscanSet.fileSet=['src/main/resources/']
Regarding Proxy - I had to update proxy in two settings.xml.
1) .m2/settings.xml
2) apache-maven-3.6.3/conf/settings.xml
Related
How do I start the Jenkins process in offline mode on one of my servers?
I am trying to start Jenkins by running command java -jar jenkins.war, but it is failing at the below warning because of no connectivity.
Is there any way to skip this plugin upgrade through the internet and bring the Jenkins up and running?
2021-04-27 16:53:43.490+0000 [id=64] WARNING hudson.model.UpdateCenter#updateDefaultSite: Upgrading Jenkins. Failed to update the default Update Site 'default'. Plugin upgrades may fail.
2021-04-27 16:53:43.490+0000 [id=64] WARNING hudson.model.UpdateCenter#updateDefaultSite: Upgrading Jenkins. Failed to update the default Update Site 'default'. Plugin upgrades may fail.
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
at hudson.model.DownloadService.loadJSON(DownloadService.java:116)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:218)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:213)
at hudson.model.UpdateCenter.updateDefaultSite(UpdateCenter.java:2611)
at jenkins.install.SetupWizard.init(SetupWizard.java:212)
at jenkins.install.InstallState$InitialSecuritySetup.initializeState(InstallState.java:168)
at jenkins.model.Jenkins.setInstallState(Jenkins.java:1104)
at jenkins.install.InstallUtil.proceedToNextStateFrom(InstallUtil.java:98)
at jenkins.install.InstallState$Unknown.initializeState(InstallState.java:86)
at jenkins.model.Jenkins$16.run(Jenkins.java:3356)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1131)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-04-27 16:53:43.491+0000 [id=53] INFO jenkins.InitReactorRunner$1#onAttained: Completed initialization
2021-04-27 16:53:43.494+0000 [id=92] INFO hudson.util.Retrier#start: The attempt #1 to do the action check updates server failed with an allowed exception:
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
at hudson.model.DownloadService.loadJSON(DownloadService.java:116)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:218)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:213)
at hudson.PluginManager.checkUpdatesServer(PluginManager.java:1882)
at hudson.util.Retrier.start(Retrier.java:63)
at hudson.PluginManager.doCheckUpdatesServer(PluginManager.java:1853)
at jenkins.DailyCheck.execute(DailyCheck.java:93)
at hudson.model.AsyncPeriodicWork.lambda$doRun$0(AsyncPeriodicWork.java:100)
at java.lang.Thread.run(Thread.java:748)
2021-04-27 16:53:43.494+0000 [id=92] INFO hudson.util.Retrier#start: Calling the listener of the allowed exception 'connect timed out' at the attempt #1 to do the action check updates server
2021-04-27 16:53:43.497+0000 [id=92] INFO hudson.util.Retrier#start: Attempted the action check updates server for 1 time(s) with no success
2021-04-27 16:53:43.499+0000 [id=92] SEVERE hudson.PluginManager#doCheckUpdatesServer: Error checking update sites for 1 attempt(s). Last exception was: SocketTimeoutException: connect timed out
2021-04-27 16:53:43.504+0000 [id=36] INFO hudson.WebAppMain$3#run: Jenkins is fully up and running
2021-04-27 16:53:43.504+0000 [id=92] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 20,249 ms
2021-04-27 16:54:13.279+0000 [id=106] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Started Periodic background build discarder
2021-04-27 16:54:13.280+0000 [id=106] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Periodic background build discarder. 1 ms```
Make sure to pass all of these arguments before the -jar argument, otherwise they will be ignored. Example:
java -Dhudson.footerURL=http://example.org -jar jenkins.war
I am getting the below error while trying to read the Excel. Using ApachePOI to open the excel workbook. Using Serenity with Cucumber framework and executing the command "mvn clean verify" to run the tests. Is it not possible to use ApachePOI without Main method? It works fine when I include this code in Main method.
null
net.thucydides.core.reports.ReportGenerationFailedError: Failed to generate configuration report
at net.thucydides.core.reports.ReportService.waitForReportGenerationToFinish(ReportService.java:204)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:181)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:127)
at io.cucumber.core.plugin.SerenityReporter.generateReports(SerenityReporter.java:937)
at io.cucumber.core.plugin.SerenityReporter.handleTestRunFinished(SerenityReporter.java:323)
at io.cucumber.core.eventbus.AbstractEventPublisher.send(AbstractEventPublisher.java:45)
at io.cucumber.core.eventbus.AbstractEventBus.send(AbstractEventBus.java:9)
at io.cucumber.junit.CucumberSerenityRunner$RunCucumber.evaluate(CucumberSerenityRunner.java:255)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:405)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.apache.commons.io.FileUtils.listFilesAndDirs(Ljava/io/File;Lorg/apache/commons/io/filefilter/IOFileFilter;Lorg/apache/commons/io/filefilter/IOFileFilter;)Ljava/util/Collection;
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at net.thucydides.core.reports.ReportService.waitForReportGenerationToFinish(ReportService.java:201)
... 17 more
Caused by: java.lang.NoSuchMethodError: org.apache.commons.io.FileUtils.listFilesAndDirs(Ljava/io/File;Lorg/apache/commons/io/filefilter/IOFileFilter;Lorg/apache/commons/io/filefilter/IOFileFilter;)Ljava/util/Collection;
at net.thucydides.core.files.TheDirectoryStructure.maxDepth(TheDirectoryStructure.java:58)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.lambda$maxDirectoryDepthIn$2(FileSystemRequirementsTagProvider.java:206)
at java.base/java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:212)
at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1608)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:496)
at java.base/java.util.stream.IntPipeline.max(IntPipeline.java:459)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.maxDirectoryDepthIn(FileSystemRequirementsTagProvider.java:207)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.<init>(FileSystemRequirementsTagProvider.java:129)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.<init>(FileSystemRequirementsTagProvider.java:78)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.<init>(FileSystemRequirementsTagProvider.java:73)
at net.serenitybdd.cucumber.service.CucumberTagProviderStrategy.getTagProviders(CucumberTagProviderStrategy.java:34)
at net.thucydides.core.statistics.service.ClasspathTagProviderService.tagProvidersThatCanProcess(ClasspathTagProviderService.java:80)
at net.thucydides.core.statistics.service.ClasspathTagProviderService.loadTagProvidersFromPath(ClasspathTagProviderService.java:56)
at net.thucydides.core.statistics.service.ClasspathTagProviderService.tagProvidersFor(ClasspathTagProviderService.java:36)
at net.thucydides.core.statistics.service.ClasspathTagProviderService.getTagProviders(ClasspathTagProviderService.java:28)
at net.thucydides.core.model.TestOutcome.getTags(TestOutcome.java:1854)
at net.thucydides.core.model.TestOutcome.addVersionsDefinedInTagsTo(TestOutcome.java:1734)
at net.thucydides.core.model.TestOutcome.getVersions(TestOutcome.java:1729)
at net.thucydides.core.model.TestOutcome.calculateDynamicFieldValues(TestOutcome.java:349)
at net.thucydides.core.reports.json.gson.GsonJSONConverter.toJson(GsonJSONConverter.java:81)
at net.thucydides.core.reports.json.JSONTestOutcomeReporter.generateReportFor(JSONTestOutcomeReporter.java:60)
at net.thucydides.core.reports.ReportService.generateReportFor(ReportService.java:239)
at net.thucydides.core.reports.ReportService.lambda$generateReportsFor$0(ReportService.java:177)
... 5 more
It worked after I changed the following Maven dependency:
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
While creating the configuration of nutch in deploy mode using the config/create endpoint it is throwing NoSuchMethodError.
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectReader.forType(Lcom/fasterxml/jackson/databind/JavaType;)Lcom/fasterxml/jackson/databind/ObjectReader;
at com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:803)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBodyReader(JAXRSUtils.java:1379)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBody(JAXRSUtils.java:1330)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameter(JAXRSUtils.java:856)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameters(JAXRSUtils.java:815)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:215)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:78)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:234)
at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:76)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1065)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
19/03/22 01:46:25 WARN thread.QueuedThreadPool:
java.lang.NoSuchMethodError: javax.servlet.http.HttpServletResponse.getStatus()
at org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine$1.handle(JettyHTTPServerEngine.java:368)
at org.eclipse.jetty.server.Response.sendError(Response.java:597)
at org.eclipse.jetty.server.HttpChannel.handleException(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpConnection$HttpChannelOverHttp.handleException(HttpConnection.java:621)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:387)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
This method belongs to jackson-databind-2.9.5.jar when I checked into apache-nutch-1.15.job the same version jar exists in it.
I've use Apache Zeppelin 0.8.1.
I want user Groovy interpreter. And use #Grape dep menagement:
%groovy
#Grapes([
#Grab('com.github.twosigma.beakerx:beaker-kernel-groovy:0.9.0')
])
import com.twosigma.beakerx.widget.IntSlider
But got error:
java.lang.RuntimeException: Failed to parse groovy script: java.lang.NoClassDefFoundError: org/apache/ivy/Ivy
at org.apache.zeppelin.groovy.GroovyInterpreter.getGroovyScript(GroovyInterpreter.java:140)
Reasonable. By official documentation of Dependency Management for Interpreter I've provide artifact dep: org.apache.ivy:ivy:2.5.0-rc1:
In repositories I have enabled Central (http://repo1.maven.org/maven2/) which have that artifact:
But on interpreter restarting and notebook reload I just got error: Error setting properties for interpreter 'groovy.groovy': Cannot fetch dependencies for org.apache.ivy:ivy:2.5.0-rc1:
I have tried (as experiment, not long-term solution) enter into zeppelin docker container and download http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.5.0-rc1/ivy-2.5.0-rc1.jar into /zeppelin/interpreter/groovy, restart interpreter, reload nootebook and it looks like groovy try resolve deps - i see progress-bar for second, but then again got error:
java.lang.RuntimeException: Failed to parse groovy script: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: General error during conversion: Error grabbing Grapes -- [unresolved dependency: com.github.twosigma.beakerx#beaker-kernel-groovy;0.9.0: not found] java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: com.github.twosigma.beakerx#beaker-kernel-groovy;0.9.0: not found] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83) at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:77) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247) at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:426) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:169) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:573) at groovy.grape.GrapeIvy$resolve$1.callCurrent(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:540) at groovy.grape.GrapeIvy$resolve$0.callCurrent(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:182) at groovy.grape.GrapeIvy.grab(GrapeIvy.groovy:258) at groovy.grape.Grape.grab(Grape.java:167) at groovy.grape.GrabAnnotationTransformation.visit(GrabAnnotationTransformation.java:378) at org.codehaus.groovy.transform.ASTTransformationVisitor$3.call(ASTTransformationVisitor.java:321) at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931) at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593) at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569) at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546) at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) at groovy.lang.GroovyShell.parse(GroovyShell.java:700) at groovy.lang.GroovyShell.parse(GroovyShell.java:736) at org.apache.zeppelin.groovy.GroovyInterpreter.getGroovyScript(GroovyInterpreter.java:133) at org.apache.zeppelin.groovy.GroovyInterpreter.interpret(GroovyInterpreter.java:155) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:632) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 1 error at org.apache.zeppelin.groovy.GroovyInterpreter.getGroovyScript(GroovyInterpreter.java:140)
It looks like groovy does not see repository configurations and try resolve dependencies only locally.
What I should setup to change that?
groovy in zeppelin not designed to support #Grab
your artifact is present in a specific repository
http://maven.imagej.net/content/repositories/public/
check this artifact search engine:
https://mvnrepository.com/artifact/com.github.twosigma.beakerx/beaker-kernel-groovy/0.9.0
so, just add a new repo to zeppelin
UPD:
for org.apache.ivy:ivy:2.5.0-rc1 there are a lot of dependencies downloaded
and one of them fails. Here is what I see in console:
Error while downloading repos for interpreter group : groovy,
go to interpreter setting page click on edit and save it again
to make this interpreter work properly:
Cannot fetch dependencies for org.apache.ivy:ivy:2.5.0-rc1
...
Caused by: org.sonatype.aether.resolution.DependencyResolutionException:
Could not find artifact com.jcraft:jsch.agentproxy:jar:0.0.9 in central
(http://repo1.maven.org/maven2/)
...
(IHMO it's a bug of zeppelin)
but to make it work set
artifact= org.apache.ivy:ivy:2.4.0
exclude= com.jcraft:jsch.agentproxy
I'm very newbie with Linux/Java/Scriptella and I'm trying a jdbc connection with scriptella on a Firebird local database, but I'm receiving the following errors:
2-dic-2013 1.03.34 <INFO> Execution Progress.Initializing properties: 1%
2-dic-2013 1.03.34 <GRAVE> Script /home/maurizio/Scrivania/JATROPHA/applicazioni/prova_per_scriptella.etl execution failed.
javax/resource/ResourceException
2-dic-2013 1.03.34 <GRAVE> Scriptella bug report. Submit to issue tracker.
Scriptella version: 1.1
Exception:
scriptella.execution.EtlExecutorException: javax/resource/ResourceException
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:190)
at scriptella.tools.launcher.EtlLauncher.execute(EtlLauncher.java:276)
at scriptella.tools.launcher.EtlLauncher.launch(EtlLauncher.java:193)
at scriptella.tools.launcher.EtlLauncher.main(EtlLauncher.java:321)
Caused by: java.lang.NoClassDefFoundError: javax/resource/ResourceException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at scriptella.core.DriverFactory.getDriver(DriverFactory.java:53)
at scriptella.core.ConnectionManager.<init>(ConnectionManager.java:70)
at scriptella.core.Session.<init>(Session.java:51)
at scriptella.execution.EtlExecutor.prepare(EtlExecutor.java:248)
at scriptella.execution.EtlExecutor.execute(EtlExecutor.java:178)
... 3 more
Caused by: java.lang.ClassNotFoundException: javax.resource.ResourceException
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
... 10 more
I'm using Ubuntu 10.04 Lucid Lynx.
I start scriptella via console in directory /home/maurizio/Scrivania/JATROPHA/applicazioni/ with the command scriptella/scriptella-1.1/bin/scriptella.sh -debug "prova_per_scriptella.etl"
My ETL file prova_per_scriptella.etl contains the following rows:
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>Prova connessione Firebird</description>
<connection
id="fb_destination"
driver="org.firebirdsql.jdbc.FBDriver"
url="jdbc:firebirdsql:localhost/3050:/home/maurizio/Scrivania/JATROPHA/db/jatrofa.fdb"
user="user"
password="password"
classpath="/home/maurizio/Scrivania/JATROPHA/applicazioni/jaybird/Jaybird-2.2.3JDK_1.6/jaybird-2.2.3.jar"
/>
</etl>
The env var $_SCRIPTELLA_CP of batch command scriptella/scriptella-1.1/bin/scriptella.sh results in
:/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/commons-jexl.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/commons-logging.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/jaybird-2.2.3.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/jsqlparser-0.8.0.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/scriptella-core.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/scriptella-drivers.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/scriptella-tools.jar:
/home/maurizio/Scrivania/JATROPHA/applicazioni/scriptella/scriptella-1.1/lib/sqlsheet-6.5.jar
Any help would be very appreciated.
Thanks in advance.
You are missing the required dependency connector-api-1.5.jar, or you need to use jaybird-full-2.2.3.jar (which includes both the normal Jaybird and connector-api). See the releasenotes of Jaybird 2.2.3
The error means that you probably miss additional J2EE classes on classpath. Try downloading mini-j2ee.jar from http://www.firebirdsql.org/en/jdbc-driver/ and adding it to classpath attribute in etl.xml:
classpath="/path/to/mini-j2ee.jar:/home/maurizio/Scrivania/JATROPHA/applicazioni/jaybird/Jaybird-2.2.3JDK_1.6/jaybird-2.2.3.jar"