Anyone configured apache solr FieldReaderDataSource? - search

Question

I have a datbase column containing xml and I want to index using apache solr content in that column i have following data-config.xml (configuration). The database name is "solrdb" and columns name is "xmlfield", There seems to be some problem in it, the error is specified at the bottom.
<dataConfig>
<!--Data source to connect to database-->
<dataSource
name="XmlDocDS"
type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://127.0.0.1/solrdb"
user="root"
password="root" />
<!-- Data Source for getting xml columne data-->
<dataSource
name="solrFieldReaderDS"
type="FieldReaderDataSource"/>
<document>
<entity
name="xmltable"
rootEntity="false"
datasource="XmlDocDS"
query="select xmlfield from xmltable">
<field column="xmldata" blob="true" />
<entity
name="page"
dataSource="solrFieldReaderDS"
dataField="xmltable.xmldata"
processor="XPathEntityProcessor"
forEach="/page">
<field column="id" xpath="/mediawiki/page/id"/>
<field column="Title" xpath="/mediawiki/page/title"/>
</entity>
</entity>
</document>
</dataConfig>
The error is following:
SEVERE: Exception while processing: xmltable document : null:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: select xmlfield from xmltable Processing Document # 1

Answer 1

The error is thrown in this part of JDBC importer code:
try {
Connection c = getConnection();
stmt = c.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(batchSize);
stmt.setMaxRows(maxRows);
LOG.debug("Executing SQL: " + query);
long start = System.currentTimeMillis();
if (stmt.execute(query)) {
resultSet = stmt.getResultSet();
}
LOG.trace("Time taken for sql :"
+ (System.currentTimeMillis() - start));
colNames = readFieldNames(resultSet.getMetaData());
} catch (Exception e) {
wrapAndThrow(SEVERE, e, "Unable to execute query: " + query);
}
So there can be error in connection or query (smth wrong with DB?). Also grep logs for "Executing SQL" and "Time taken for sql :"

Answer 2

there was an error in connection, for some reason it was nto able to connect to my local machine, i changed the database host and it connected!, the problem is that I have the configuration in place and FieldReaderDataSource seems to work fine, but now when it completes everything it says documents indexed/updated = 0
here is my xml configuration
<dataSource
name="jdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="xxxx"
user="yyyy"
password="zzzz" readOnly="true"/>
<dataSource
name="solrFieldReaderDS"
type="FieldReaderDataSource"/>
<document>
<entity
name="tabledata"
dataSource="jdbcDataSource"
query="select codeID,codeText from ArticlePoolState where codeID=3">
<entity
name="xmldata"
dataSource="solrFieldReaderDS"
forEach="/med"
dataField="tabledata.codeText"
processor="XPathEntityProcessor">
<field column="title" xpath="/title"/>
</entity>
</entity>
</document>
The query is fine.