Environment - Cucumber V. 4.0.0 | Selenium V.3.8.1 | JUnit V.4.12 | cucumber-runner & cucumber-report-generator 1.3.3
First time i am trying to implement #ExtendedCucumberOptions so that i can run failed test case again. Getting initialization error -
More Info about Error -
java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
cucumber/runtime/ExtendedRuntime.runStep(Ljava/lang/String;Lgherkin/formatter/model/Step;Lgherkin/formatter/Reporter;Lgherkin/I18n;)V #94: invokeinterface
Reason:
Type 'cucumber/runtime/StepDefinitionMatch' (current frame, stack[1]) is not assignable to 'gherkin/formatter/model/Match'
Current Frame:
bci: #94
flags: { }
locals: { 'cucumber/runtime/ExtendedRuntime', 'java/lang/String', 'gherkin/formatter/model/Step', 'gherkin/formatter/Reporter', 'gherkin/I18n', 'cucumber/runtime/StepDefinitionMatch' }
stack: { 'gherkin/formatter/Reporter', 'cucumber/runtime/StepDefinitionMatch' }
Bytecode:
0x0000000: 2ab6 0070 2b2c 1904 b900 6504 003a 05a7
0x0000010: 0047 3a06 2d19 06b6 0072 03b9 0073 0200
0x0000020: c000 52b9 0074 0200 bb00 5059 1260 09b8
0x0000030: 0059 1906 b200 5ab7 005b 3a07 2d19 07b9
0x0000040: 0075 0200 2a19 07b7 0076 2a19 06b6 0061
0x0000050: 2a04 b500 0cb1 1905 c600 0e2d 1905 b900
0x0000060: 7402 00a7 0022 2db2 0077 b900 7402 002d
0x0000070: b200 78b9 0075 0200 2ab2 0078 b700 762a
0x0000080: 04b5 000c b12a b400 14b6 004c 9900 082a
0x0000090: 04b5 000c 2ab4 000c 9900 162a b200 79b7
0x00000a0: 0076 2db2 0079 b900 7502 00a7 0114 1251
0x00000b0: 3a06 013a 072a b400 15b9 0056 0100 1905
0x00000c0: 1904 b600 6f2a b400 15b9 0058 0100 3708
0x00000d0: bb00 5059 1906 1608 b800 5919 07b2 005a
0x00000e0: b700 5b3a 0a2a 190a b700 762d 190a b900
0x00000f0: 7502 00a7 00cc 3a08 1908 3a07 1260 3a06
0x0000100: 2a19 08b6 0061 2ab4 0015 b900 5801 0037
0x0000110: 08bb 0050 5919 0616 08b8 0059 1907 b200
0x0000120: 5ab7 005b 3a0a 2a19 0ab7 0076 2d19 0ab9
0x0000130: 0075 0200 a700 8b3a 0819 083a 0719 08b8
0x0000140: 0045 9900 0a12 5f3a 06a7 0007 1260 3a06
0x0000150: 2a19 08b6 0061 2a04 b500 0c2a b400 15b9
0x0000160: 0058 0100 3708 bb00 5059 1906 1608 b800
0x0000170: 5919 07b2 005a b700 5b3a 0a2a 190a b700
0x0000180: 762d 190a b900 7502 00a7 0036 3a0b 2ab4
0x0000190: 0015 b900 5801 0037 0cbb 0050 5919 0616
0x00001a0: 0cb8 0059 1907 b200 5ab7 005b 3a0e 2a19
0x00001b0: 0eb7 0076 2d19 0eb9 0075 0200 190b bfb1
0x00001c0:
Exception Handler Table:
bci [0, 15] => handler: 18
bci [190, 197] => handler: 246
bci [190, 197] => handler: 311
bci [190, 197] => handler: 396
bci [246, 262] => handler: 396
bci [311, 347] => handler: 396
bci [396, 398] => handler: 396
Stackmap Table:
same_locals_1_stack_item_frame(#18,Object[#311])
append_frame(#86,Object[#303])
same_frame(#102)
same_frame(#133)
same_frame(#148)
same_frame(#174)
full_frame(#246,{Object[#194],Object[#277],Object[#302],Object[#275],Object[#312],Object[#303],Object[#277],Object[#278]},{Object[#313]})
same_locals_1_stack_item_extended(#311,Object[#278])
append_frame(#332,Object[#278])
same_frame(#336)
full_frame(#396,{Object[#194],Object[#277],Object[#302],Object[#275],Object[#312],Object[#303],Object[#277],Object[#278]},{Object[#278]})
chop_frame(#447,2)
at com.github.mkolisnyk.cucumber.runner.ExtendedCucumber.createRuntime(ExtendedCucumber.java:93)
at com.github.mkolisnyk.cucumber.runner.ExtendedCucumber.<init>(ExtendedCucumber.java:52)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createUnfilteredTest(JUnit4TestLoader.java:87)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:73)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:46)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:522)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
First let me share my understanding on implementation, we have to add below 2 dependencies -
<dependency>
<groupId>com.github.mkolisnyk</groupId>
<artifactId>cucumber-runner</artifactId>
<version>1.3.3</version>
</dependency>
<dependency>
<groupId>com.github.mkolisnyk</groupId>
<artifactId>cucumber-report-generator</artifactId>
<version>1.3.3</version>
<scope>test</scope>
</dependency>
I also added below 2 as i was under impression, these shall be added as well by doing some research on net.
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>gherkin</artifactId>
<version>4.1.3</version>
</dependency>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.28</version>
</dependency>
Below is RunCukeTest.java Class -
#RunWith(ExtendedCucumber.class)
#ExtendedCucumberOptions(
jsonReport = "target/cucumber-json/cucumber.json",
retryCount = 3,
detailedReport = true,
detailedAggregatedReport = true,
overviewReport = true,
coverageReport = true,
jsonUsageReport = "target/cucumber-reports/cucumber-usage.json",
usageReport = false,
toPDF = true,
// excludeCoverageTags = {"#sanity" },
includeCoverageTags = {"#regression" },
outputFolder = "target/cucumber-reports/extended-report")
#CucumberOptions(features = "classpath:features/functional/",
glue = {"com.jacksparrow.automation.steps_definitions.functional" },
plugin = { "pretty","json:target/cucumber-json/cucumber.json",
"junit:target/cucumber-reports/Cucumber.xml", "html:target/cucumber-reports",
"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"},
tags = { "#regression" },
junit ={ "--step-notifications"},
strict = false,
dryRun = false,
monochrome = true)
public class RunCukeTest {
}
POM.XML
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.jacksparrow.automation</groupId>
<artifactId>TheDayAfterTomorrow</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scm>
<connection>scm:git:https://github.com/extent-framework/extentreports-cucumber4-adapter.git</connection>
<developerConnection>scm:git:https://github.com/extent-framework/extentreports-cucumber4-adapter.git</developerConnection>
<url>https://github.com/extent-framework/extentreports-cucumber4-adapter</url>
</scm>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<cucumberversion>4.0.0</cucumberversion>
<selenium.version>3.8.1</selenium.version>
<junit.version>4.12</junit.version>
<assertj-core.version>3.12.0</assertj-core.version>
<cucumberdatatableversion>1.1.12</cucumberdatatableversion>
<webdrivermanager.version>3.0.0</webdrivermanager.version>
<extent.report.version>3.1.1</extent.report.version>
<extent-cucumber-adapter.version>1.0.5</extent-cucumber-adapter.version>
<maven-surefire.plugin.version>3.0.0-M3</maven-surefire.plugin.version>
<maven.compiler.plugin.version>3.8.0</maven.compiler.plugin.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.failsafe.plugin.version>2.22.1</maven.failsafe.plugin.version>
<maven.build.helper.plugin.version>3.0.0</maven.build.helper.plugin.version>
<joda-time>2.5</joda-time>
<mysql>5.1.6</mysql>
<jackson-core>2.6.3</jackson-core>
<common-db-utils>1.6</common-db-utils>
<commons-io>2.6</commons-io>
<commons-lang>3.4</commons-lang>
<lombok>1.16.4</lombok>
<slf4jVersion>1.7.7</slf4jVersion>
<apache.poi.version>3.16</apache.poi.version>
<generated.runner.directory>${project.build.directory}/parallel/runners</generated.runner.directory>
<generated.feature.directory>${project.build.directory}/parallel/features</generated.feature.directory>
</properties>
<dependencies>
<!-- ********************************()() IO.CUCUMBER & SELENIUM DEPENDENCIES ()()******************************** -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>${cucumberversion}</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>${cucumberversion}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>${cucumberversion}</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-jvm</artifactId>
<version>2.0.0</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-core</artifactId>
<version>${cucumberversion}</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>datatable</artifactId>
<version>${cucumberdatatableversion}</version>
</dependency>
<dependency>
<groupId>com.github.mkolisnyk</groupId>
<artifactId>cucumber-runner</artifactId>
<version>1.3.3</version>
</dependency>
<dependency>
<groupId>com.github.mkolisnyk</groupId>
<artifactId>cucumber-report-generator</artifactId>
<version>1.3.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>gherkin</artifactId>
<version>4.1.3</version>
</dependency>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.28</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>${selenium.version}</version>
</dependency>
<dependency>
<groupId>io.github.bonigarcia</groupId>
<artifactId>webdrivermanager</artifactId>
<version>${webdrivermanager.version}</version>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>${assertj-core.version}</version>
<scope>test</scope>
</dependency>
<!-- ************************************()() EXTENT REPORT DEPENDENCY ()()******************************* -->
<!-- <dependency> -->
<!-- <groupId>com.aventstack</groupId> -->
<!-- <artifactId>extentreports</artifactId> -->
<!-- <version>4.0.9</version> -->
<!-- </dependency> -->
<!-- <dependency> -->
<!-- <groupId>com.vimalselvam</groupId> -->
<!-- <artifactId>cucumber-extentsreport</artifactId> -->
<!-- <version>${extent.report.version}</version> -->
<!-- </dependency> -->
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports-cucumber4-adapter</artifactId>
<version>${extent-cucumber-adapter.version}</version>
</dependency>
<!-- *******************************()() CUCUMBER MAVEN REPORTING DEPENDENCY ()()***************************** -->
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>1.0.0</version>
</dependency>
<!-- **************************************()() SLF4J DEPENDENCY ()()************************************ -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4jVersion}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4jVersion}</version>
</dependency>
<!-- ****************************************()() JSON DEPENDENCY ()()************************************** -->
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-cucumber4-jvm</artifactId>
<version>2.9.0</version>
</dependency>
<!-- **********************************()() JAVA MAIL API DEPENDENCY ()()************************************ -->
<dependency>
<groupId>javax.mail</groupId>
<artifactId>javax.mail-api</artifactId>
<version>1.5.5</version>
</dependency>
<!-- **************************************()() LOG4J DEPENDENCY ()()*************************************** -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- ************************************()() POI & XML DEPENDENCY ()()************************************** -->
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>3.11-beta3</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commons-io}</version>
</dependency>
<!-- ***************************************()() REPORTNG DEPENDENCY ()()********************************* -->
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>3.0</version>
</dependency>
<dependency>
<groupId>velocity</groupId>
<artifactId>velocity-dep</artifactId>
<version>1.4</version>
</dependency>
<!-- ***************************************()() APPIUM DEPENDENCY ()()********************************* -->
<dependency>
<groupId>io.appium</groupId>
<artifactId>java-client</artifactId>
<version>3.4.0</version>
</dependency>
<!-- *****************************************()() PDF DEPENDENCY ()()********************************** -->
<dependency>
<groupId>com.itextpdf</groupId>
<artifactId>itextpdf</artifactId>
<version>5.4.0</version>
</dependency>
<dependency>
<groupId>com.lowagie</groupId>
<artifactId>itext</artifactId>
<version>1.3.1</version>
</dependency>
<!-- ************************************()() ATUTestRecorder DEPENDENCY ()()**************************** -->
<dependency>
<groupId>com.CA.ATU21</groupId>
<artifactId>ATUTestRecorder</artifactId>
<version>2.1</version>
<scope>system</scope>
<systemPath>${basedir}/lib/ATUTestRecorder-2.1.jar</systemPath>
</dependency>
<dependency>
<groupId>com.codeborne</groupId>
<artifactId>phantomjsdriver</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>java-hamcrest</artifactId>
<version>2.0.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>${apache.poi.version}</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>${apache.poi.version}</version>
</dependency>
</dependencies>
<!-- ************************************()() MAVEN COMPILER PLUGIN ()()*************************************** -->
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerVersion>1.8</compilerVersion>
</configuration>
</plugin>
<!-- ************************************()() MAVEN SUREFIRE PLUGIN ()()*************************************** -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire.plugin.version}</version>
<configuration>
<parallel>methods</parallel>
<threadCount>1</threadCount>
<reuserForks>false</reuserForks>
<testFailureIgnore>true</testFailureIgnore>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<includes>
<include>**/*RunCukeTest.java</include>
</includes>
</configuration>
</plugin>
<!-- ***********************************()() CLUECUMBER REPORT PLUGIN ()()*********************************** -->
<plugin>
<groupId>com.trivago.rta</groupId>
<artifactId>cluecumber-report-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<id>report</id>
<phase>post-integration-test</phase>
<goals>
<goal>reporting</goal>
</goals>
</execution>
</executions>
<configuration>
<sourceJsonReportDirectory>${project.build.directory}\cucumber-json</sourceJsonReportDirectory>
<generatedHtmlReportDirectory>${project.build.directory}\cluecumber-report</generatedHtmlReportDirectory>
</configuration>
</plugin>
<!-- ******************************()() MAVEN CUCUMBER HTML REPORT PLUGIN ()()******************************** -->
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>4.2.3</version>
<executions>
<execution>
<id>execution</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>TheDayAfterTomorrow</projectName>
<!-- output directory for the generated report -->
<outputDirectory>${project.build.directory}/cucumber-maven-report</outputDirectory>
<inputDirectory>${project.build.directory}/cucumber-json</inputDirectory>
<jsonFiles>
<!-- supports wildcard or name pattern -->
<param>**/*.json</param>
</jsonFiles>
<skippedFails>true</skippedFails>
<enableFlashCharts>true</enableFlashCharts>
<buildNumber>10.2.1</buildNumber>
<parallelTesting>false</parallelTesting>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-maven</artifactId>
<version>2.9</version>
<configuration>
<reportVersion>2.7.0</reportVersion>
<allureDownloadUrl>https://github.com/allure-framework/allure2/releases/download/2.7.0/allure-2.7.0.zip</allureDownloadUrl>
</configuration>
</plugin>
</plugins>
</build>
</project>
What could be wrong here.
Now its clear #ExtendedCucumberOptions were supported till Cucumber V 1.2.5 and Someone working on Cucumber 4.0.0 or later of V 1.2.5 shall not use this option in case someone like to re-run their failed test cases.
I am trying to analyze facebook networks using spark. When I am creating an object of Spark Conf it's giving me the following error.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkConf.
I am using scala version 2.11 with which it should work fine. In my intelliJ IDEA also scala sdk 2.12 is loaded.
Code for the same is:-
import org.apache.spark.SparkConf
import org.apache.spark.graphx.GraphLoader
import org.apache.spark.sql.SparkSession
object Sample {
def main(args: Array[String]): Unit = {
print("Hello World")
val conf = new SparkConf().setMaster("local[2]")
val spark = SparkSession
.builder
.appName("SampleApp").config(conf)
.getOrCreate()
val sc = spark.sparkContext
val graph = GraphLoader.edgeListFile(sc, "facebook_combined.txt")
val ranks = graph.pageRank(1).vertices
print(ranks.collect().mkString("\n"))
spark.stop()
}
}
My pom.xml file is stated before :-
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Sample</groupId>
<artifactId>CMPE256</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<spark.version>2.1.1</spark.version>
<scala.dep.version>2.11</scala.dep.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.dep.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.dep.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-graphx_${scala.dep.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.6</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- mixed scala/java compile -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<id>compile</id>
<goals>
<goal>compile</goal>
</goals>
<phase>compile</phase>
</execution>
<execution>
<id>test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
<phase>test-compile</phase>
</execution>
<execution>
<phase>process-resources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Comment out the <scope>provided</scope> in your pom.xml and that should fix the problem.
More about why is here: MavenDoc
instead of this you can write like this in scala.
package Wordcount
import org.apache.spark.streaming._
import org.apache.spark.SparkContext
import org.apache.spark._
class FirstApp {
val conf = new SparkConf().setMaster("local[2]").setAppName("wordcount");
...
}
I am facing issue with JUnit while running unit test cases.I am using spring 4.3.2.RELEASE and JUnit 4.12 and Maven 3.3.9. I'm not sure why i'm getting this exception while running the test code.I tried all the possible ways to execute the code but i got failed to run test cases.
Exception is:
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:308)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:109)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:75)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:333)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:220)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:301)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:303)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:240)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:180)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from class path resource [spring-beans/applicationContext-b2b-transport1.xml]; nested exception is java.io.FileNotFoundException: class path resource [spring-beans/applicationContext-b2b-transport1.xml] cannot be opened because it does not exist
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:341)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:302)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:143)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:178)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:149)
at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:212)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:81)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:1)
at org.springframework.test.context.TestContext.loadApplicationContext(TestContext.java:280)
at org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:304)
... 24 more
Caused by: java.io.FileNotFoundException: class path resource [spring-beans/applicationContext-b2b-transport1.xml] cannot be opened because it does not exist
at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:158)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:328)
... 33 more
POM.Xml
<?xml version="1.0" encoding="utf-8"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.bt.b2b.gateway</groupId>
<artifactId>b2babc</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<name>b2babc_ws</name>
<url>http://maven.apache.org</url>
<!-- Shared version number properties -->
<properties>
<!--env>dev</env -->
<!-- <org.springframework.version>4.1.8.RELEASE</org.springframework.version> commented for Spring Library Updates-->
<org.springframework.version>4.3.2.RELEASE</org.springframework.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<timestamp>${maven.build.timestamp}</timestamp>
<maven.build.timestamp.format>yyyy-MM-dd-HH-mm</maven.build.timestamp.format>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<!-- <version>1.3</version> commented for Spring Library Updates -->
<version>1.8</version>
</plugin>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<!-- <version>2.4.1</version>commented for Spring Library Updates -->
<version>3.0.0</version>
<executions>
<execution>
<id>auto-clean</id>
<phase>initialize</phase>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<!--filters>
<filter>${env.M2_HOME}/conf/${env}.properties</filter>
</filters-->
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
<testResources>
<testResource>
<directory>src/test/resources</directory>
<filtering>true</filtering>
</testResource>
</testResources>
<!-- <finalName>OpenreachSSGatewayEAR-${maven.build.timestamp}</finalName> -->
</build>
<dependencies>
<!-- Open Source Dependencies -->
<dependency>
<groupId>emw-logging-framework</groupId>
<artifactId>logging</artifactId>
<version>3_3_12_2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-oxm</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>${org.springframework.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>${org.springframework.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.xmlbeans</groupId>
<artifactId>xmlbeans-xpath</artifactId>
<version>2.4.0</version>
<!--<scope>provided</scope>-->
</dependency>
<dependency>
<groupId>net.sf.saxon</groupId>
<artifactId>saxon</artifactId>
<version>9</version>
</dependency>
<dependency>
<groupId>net.sf.saxon</groupId>
<artifactId>saxon9-dom</artifactId>
<version>9.1.0.8</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<!-- B2B Specific Dependencies -->
<dependency>
<groupId>actionplan</groupId>
<artifactId>actionplan</artifactId>
<!-- <version>1.6</version> added storeBusiness Receipt in Action Plan 1.7 -->
<version>1.7</version>
</dependency>
<dependency>
<groupId>e537DeliveryReceipt</groupId>
<artifactId>e537DeliveryReceipt</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.bt</groupId>
<artifactId>aliasnamesplan</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>supplieroutboundactionplan</groupId>
<artifactId>supplieroutboundactionplan</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.bt</groupId>
<artifactId>AliasNamesPlanSingleStageCompr</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>noNamespace</groupId>
<artifactId>b2bpropertymappings</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>noNamespace</groupId>
<artifactId>b2bbesconfig</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>noNamespace</groupId>
<artifactId>b2bmisservices</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>noNamespace</groupId>
<artifactId>btacknowledgements</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>7.0</version>
<scope>provided</scope>
</dependency>
<!-- Added these dependencies for EassyMock Testing-->
<dependency>
<groupId>org.easymock.EasyMock</groupId>
<artifactId>EasyMock</artifactId>
<version>3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.objectweb.asm</groupId>
<artifactId>ASM</artifactId>
<version>3.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.sf.cglib</groupId>
<artifactId>cglib</artifactId>
<version>2.2.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>1.2</version>
<scope>test</scope>
</dependency>
<!-- EassyMock End -->
</dependencies>
<modules>
<module>components/p</module>
<module>components/b2ba</module>
<module>components/b2bb</module>
<module>components/b2bc</module>
<module>components/b2bd</module>
<module>components/b2be</module>
<module>components/f</module>
<module>components/g</module>
<module>components/h</module>
<module>components/i</module>
<module>components/j</module>
<module>components/k</module>
<module>components/l</module>
<module>components/m</module>
<module>ears/o</module>
<module>components/b2bn</module>
</modules>
<!-- profiles and their matching property filenames -->
<profiles>
<profile>
<id>BTR</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<build.profile.id>BTR</build.profile.id>
<b2blogpath></b2blogpath>
<earName>somexyzEAR</earName>
</properties>
<build>
<filters>
<filter>${env.M2_HOME}/conf/${build.profile.id}.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
</profile>
<profile>
<id>BTORSS</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<build.profile.id>BTORSS</build.profile.id>
<b2blogpath></b2blogpath>
<earName>somexyzEAR</earName>
<!-- <timestamp>${maven.build.timestamp}</timestamp> -->
<!-- <maven.build.timestamp.format>yyyy-MM-dd HH:mm</maven.build.timestamp.format> -->
</properties>
<build>
<filters>
<filter>${env.M2_HOME}/conf/${build.profile.id}.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
</profile>
<profile>
<id>BTWS</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<build.profile.id>BTWS</build.profile.id>
<b2blogpath></b2blogpath>
<earName>somexyzEAR</earName>
</properties>
<build>
<filters>
<filter>${env.M2_HOME}/conf/${build.profile.id}.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
</profile>
<profile>
<id>BTGS</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<build.profile.id>BTGS</build.profile.id>
<b2blogpath></b2blogpath>
<earName>somexyzEAR</earName>
</properties>
<build>
<finalName>${project.artifactId}-${project.version}-${timestamp}</finalName>
<filters>
<filter>${env.M2_HOME}/conf/${build.profile.id}.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
</profile>
<profile>
<id>BTOR</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<build.profile.id>BTOR</build.profile.id>
<b2blogpath></b2blogpath>
<earName>OpenreachGatewayEAR</earName>
</properties>
<build>
<filters>
<filter>${env.M2_HOME}/conf/${build.profile.id}.properties</filter>
</filters>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
</profile>
</profiles>
</project>
Below is the code for Junit Testcase
package com.java.autowire;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
#ContextConfiguration(locations = {"classpath:spring-beans/applicationContext-b2b-transport1.xml"})
#RunWith(SpringJUnit4ClassRunner.class)
public class TestMainApp {
/*#Autowired
private Student student;*/
#Test
public void getname(){
//Do something here.....
}
}
Below is the code for Studenet.java
package com.java.autowire;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Required;
public class Student {
private Integer age;
private String name;
public String getName() {
return name;
}
//#Autowired
public void setName(String name) {
this.name = name;
}
public Integer getAge() {
return age;
}
//#Autowired(required=false)
public void setAge(Integer age) {
this.age = age;
}
}
Below is the Configuration file named as Bean.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns = "http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context = "http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd">
<context:annotation-config/>
<!-- Definition for student bean -->
<bean id = "student" class = "com.java.autowire.Student">
</bean>
</beans>
Hint: This configuration file is under src/main/resource/spring-beans/applicationContext-b2b-transport1.xml
Tried all the possible ways. Please help me out from this issue.
The error seems pretty obvious...
class path resource [spring-beans/applicationContext-b2b-transport1.xml] cannot be opened because it does not exist
...where are you referencing that file?
You are also loading...
#ContextConfiguration(locations = {"classpath:sree/Bean.xml"})
not /spring-beans/Beans.xml.
While creating a hive context I am finding below error. I have attached my code, pom file and the exception with thins post.
POM File:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>ScalaTest</groupId>
<artifactId>Test</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.0</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.10</artifactId>
<version>2.2.6</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.5.0</version>
</dependency>
<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.5</version>
<scope>compile</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>2.10.5</version>
<scope>compile</scope>
<optional>true</optional>
</dependency>
<!-- END Scala -->
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2.1</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<sourceDir>src/main/scala</sourceDir>
<jvmArgs>
<jvmArg>-Xms64m</jvmArg>
<jvmArg>-Xmx1024m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
</plugins>
</build>
</project>
Code:
import org.apache.spark.SharedSparkContext
import org.scalatest.FunSuite
import org.apache.spark.{SparkContext, SparkConf}
import org.apache.spark.sql.hive.HiveContext
/**
* Created by PSwain on 4/5/2017.
*/
class Test extends FunSuite with SharedSparkContext {
test("test initializing spark context") {
//val list = List(1, 2, 3, 4)
//val rdd = sc.parallelize(list)
val hc = new HiveContext(sc)
//assert(list.length === rdd.count())
}
}
Exception:
A needed class was not found. This could be due to an error in your runpath. Missing class: org/apache/spark/sql/SQLContext$SQLSession
java.lang.NoClassDefFoundError: org/apache/spark/sql/SQLContext$SQLSession
at Test$$anonfun$1.apply$mcV$sp(Test.scala:15)
at Test$$anonfun$1.apply(Test.scala:12)
at Test$$anonfun$1.apply(Test.scala:12)
at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
at org.scalatest.FunSuite.runTest(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
at org.scalatest.Suite$class.run(Suite.scala:1424)
at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
at Test.org$scalatest$BeforeAndAfterAll$$super$run(Test.scala:9)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
at Test.run(Test.scala:9)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:55)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2563)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2557)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:2557)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1044)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1043)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:2722)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1043)
at org.scalatest.tools.Runner$.run(Runner.scala:883)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:138)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.SQLContext$SQLSession
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 55 more
Your version for spark-hive import doesn't match other spark dependencies. It should be:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.6.0</version>
</dependency>
I am trying to integrate Apache Ignite into an existing Spark Streaming project written in Java that counts the words in a text file. However, I'm getting a Class Not Found Error when I add in the dependency for ignite-spark:
java.lang.ClassNotFoundException: org.spark_project.protobuf.GeneratedMessage
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:67)
at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:66)
at scala.util.Try$.apply(Try.scala:161)
at akka.actor.ReflectiveDynamicAccess.getClassFor(DynamicAccess.scala:66)
at akka.serialization.Serialization$$anonfun$6.apply(Serialization.scala:181)
at akka.serialization.Serialization$$anonfun$6.apply(Serialization.scala:181)
at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:722)
at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224)
at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403)
at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:721)
at akka.serialization.Serialization.<init>(Serialization.scala:181)
at akka.serialization.SerializationExtension$.createExtension(SerializationExtension.scala:15)
at akka.serialization.SerializationExtension$.createExtension(SerializationExtension.scala:12)
at akka.actor.ActorSystemImpl.registerExtension(ActorSystem.scala:713)
at akka.actor.ExtensionId$class.apply(Extension.scala:79)
at akka.serialization.SerializationExtension$.apply(SerializationExtension.scala:12)
at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:175)
at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:620)
at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:617)
at akka.actor.ActorSystemImpl._start(ActorSystem.scala:617)
at akka.actor.ActorSystemImpl.start(ActorSystem.scala:634)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)
at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1920)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1911)
at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)
at org.apache.spark.rpc.akka.AkkaRpcEnvFactory.create(AkkaRpcEnv.scala:253)
at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:53)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:254)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:194)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:277)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:450)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:162)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:180)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:67)
at wordCount_test.TestSparkWordCount.setUp(TestSparkWordCount.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)`
Here is my pom.xml file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>wordCount_Spark</groupId>
<artifactId>wordCount_Spark</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>wordCount_Spark</name>
<description>Count words using spark</description>
<dependencies>
<!-- Spark Streaming dependencies -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.5.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.5.2</version>
<exclusions>
<exclusion>
<artifactId>protobuf-java</artifactId>
<groupId>com.google.protobuf</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.spark-project.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.5.0-spark</version>
</dependency>
<!-- Apache Ignite dependencies -->
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-spring</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-indexing</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-log4j</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-spark</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<!-- <scope>test</scope> -->
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
<useIncrementalCompilation>false</useIncrementalCompilation>
</configuration>
</plugin>
</plugins>
</build>
I have tried switching the order of Spark and Ignite in the POM, but it still throws an error when I try to run my JUnit test. Here is the test I am running:
package wordCount_test;
import static org.junit.Assert.*;
import java.io.File;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
public class TestSparkWordCount {
JavaSparkContext jsc;
File txtFile;
#Before
public void setUp() throws Exception {
jsc = new JavaSparkContext("local[2]", "testSparkWordCount");
txtFile = new File("AIW_WordCount");
if(txtFile.exists()){
txtFile.delete();
}
}
#After
public void tearDown() throws Exception {
if(jsc != null){
jsc.stop();
jsc = null;
}
}
#Test
public void testSparkInit() {
assertNotNull(jsc.sc());
}
}
Is it possible to have both Ignite and Spark Streaming in the same project? Am I missing a dependency? Did I do something wrong or miss a step?
Any help or guidance you can provide would be greatly appreciated!
The test works for me if I remove the exclusion for com.google.protobuf artifact.
My current dependencies list is the following:
<dependencies>
<!-- Spark Streaming dependencies -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.5.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.5.2</version>
</dependency>
<!-- Apache Ignite dependencies -->
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-spring</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-indexing</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-log4j</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-spark</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.0</version>
<!-- <scope>test</scope> -->
</dependency>
</dependencies>