I get stuck with setup Nutch 2.3 with hbase 0.94:
fx#fx:~$ $NUTCH_HOME/runtime/local/bin/nutch inject file:///home/fx/Abivin/apache-nutch-2.3/seed/urls.txt
InjectorJob: starting at 2015-06-17 14:46:35
InjectorJob: Injecting urlDir: file:/home/fx/Abivin/apache-nutch-2.3/seed/urls.txt
InjectorJob: Using class org.apache.gora.memory.store.MemStore as the Gora storage class.
InjectorJob: java.lang.RuntimeException: job failed: name=inject file:/home/fx/Abivin/apache-nutch-2.3/seed/urls.txt, jobid=job_local1999341506_0001
at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:231)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:284)
when seed/urls.txt stores urls. I've searched many similar errors but still get stuck with this. Please give me some ideas to resolve. Thanks
It seems that Nutch cannot inject URL to 'webpage' table. First, please check the configuration in gora-hbase. In the case the configuration is correct, you should delete the hbase data directory and start again.
Hope this helps
Related
We have been trying to start the claimcenter server from tomcat. The server is getting started even though it throws exception with the message "ClaimCenter unable to start" in the logs. Please find the log details below.
gw.pl.exception.GWLifecycleException: An exception was thrown while starting a component. Setting runlevel to NODAEMONS
Caused by: gw.pl.exception.GWLifecycleException:Messaging plugin does not implement MessageTransport interface error
Possible causes -
The xyzMessageTransport class failed to instantiate due to invalid parameters.
A class which is not implementation of MessageTransport inteface configured under
MessageTransport plugin. check messaging-config.xml.
Also - specify plugin name or attach whole log if possible.
I am trying to use databricks connect to run the spark job on databricks cluster from intellj .I followed the below link documentation.
https://docs.databricks.com/dev-tools/databricks-connect.html
However I could not make it work with intellj and it throws below exception
21/10/01 18:32:07 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
21/10/01 18:32:07 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
Exception in thread "main" java.lang.NoSuchFieldError: JAVA_9
at org.apache.spark.storage.StorageUtils$.<init>(StorageUtils.scala:207)
at org.apache.spark.storage.StorageUtils$.<clinit>(StorageUtils.scala)
at org.apache.spark.storage.BlockManagerMasterEndpoint.<init>(BlockManagerMasterEndpoint.scala:95)
at org.apache.spark.SparkEnv$.$anonfun$create$9(SparkEnv.scala:443)
at org.apache.spark.SparkEnv$.registerOrLookupEndpoint$1(SparkEnv.scala:384)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:432)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:262)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:291)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:495)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2834)
I could not find a workaround this as the documentation does not say anything clearly I cross checked from intellj its pointed to correct jar directory returned by (databricks-connect get-jar-dir).Any clue on this will be helpful?
Note:databricks-connect test is returning success
I am trying to insert data into sql server using spark using the below Jdbc methods.
Option 1:
prop.put("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
dataf.write.mode(org.apache.spark.sql.SaveMode.Append).jdbc(url,table_name, prop)
Table is already created. Appending new data.Job Error-ed with the below exception
Exception in thread "main"
com.microsoft.sqlserver.jdbc.SQLServerException: CREATE TABLE
permission denied in database
Question is : Why create table permission is required for appending the data?
Option2:
prop.put("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable(dataf, url, table_name, prop)
Above command working from spark-shell. when the same is used in scala code and packaged with dependencies giving below exception
Exception in thread "main" java.sql.SQLException: No suitable driver
at java.sql.DriverManager.getDriver(DriverManager.java:315)
I tried setting driver class-path and executor class-path and also --jars still no luck. Included sqljdbc4.jar in driver-classpath and --jars.
Copied sqljdbc4.jar to all worker nodes as well still no luck.
Any Ideas on this?
After Lot of searching and Testing, I found the answer. It might be useful for someone.
Option 1: This is because of bug in spark 1.5.X. the same was resolved
in 1.6.x and later. Because of the bug, It always try to create a new
table.
Option2: This causes because , driver name on classpath given
priority than properties we are passing as argument. Workaround for
this is to create connection and then invoke savetable.
workaround if you are using spark 1.5.x or lower.
JdbcUtils.createConnection(url, prop)
JdbcUtils.saveTable()
Im getting this error running my project.
type Exception report
messageInternal Server Error
description The server encountered an internal error that prevented it from fulfilling this request.
exception
javax.servlet.ServletException: javax.servlet.jsp.JspException: Unable to get connection, DataSource invalid: "java.sql.SQLException: No suitable driver found for jdbc/IFPWAFCAD"
root cause
javax.servlet.jsp.JspException: Unable to get connection, DataSource invalid: "java.sql.SQLException: No suitable driver found for jdbc/IFPWAFCAD"
Anybody had a clue why this is happening?
Thanks for the help
This error could happened for many reasons (and since you don't upload more code its pretty difficult to help you).
First Possible Reason
You are getting this error "no suitable driver" because the resource referens isn't beed added to the web.xml deployment descriptor, so add the following code.
<resource-ref>
<description>Database for IFPWAFCAD application</description>
<res-ref-name>jdbc/IFPWAFCAD</res-ref-name>
<res-type>javax.sql.data source</res-type>
<res-auth>Container</res-auth>
<res-sharing-scope>Shareable</res-sharing-scope>
</resource-ref>
Second possible Reason
This may be tricky but check that there is not 'blankspace' on your xml files
Third possible reason
If you are deploying the current app lets call them "testApp" to a Webserver, have in mind that you should copy the mysql connector to the lib directory of that web server
Hope this Help You
Hello Everyone,
I am working on jhipster.I have created a database on dev profile and it is created fine.but when i will try to load data into table using csv file then getting a exception.Please help me.
Error creating bean with name 'org.springframework.scheduling.annotation.SchedulingConfiguration': Initialization of bean failed; nested exception is java.lang.NoSuchMethodError: org.springframework.scheduling.annotation.SchedulingConfiguration.setBeanFactory(Lorg/springframework/beans/factory/BeanFactory;)V
Thanks in advance.
email;firstName;middleName;lastName;sex;aboutYou;dateOfBirth;password;picture;isActive;createdDate
test#gmail.com;charn;jeet;singh;b;aboutme;NULL;b8f57d6d6ec0a60dfe2e20182d4615b12e321cad9e2979e0b9f81e0d6eda78ad9b6dcfe53e4e22d1;NULL;1;NULL
The problem is that you hit a specific bug in Spring Boot, which masks your original exception:
https://github.com/spring-projects/spring-boot/issues/253
In order to see your "real" exception, either:
Migrate to the latest "SNAPSHOT" version of Spring Boot, as it looks to be solved now
Remove the configuration from your AsyncConfiguration class, which is responsible for this bug