Getting "produced no reply for request Message" - spring-integration

I am using Spring Integration framework in my project. The database is db2 running on AS400. I have to call two stored procedures one after another. My first stored procedure is executed without any issues and passing the message to the second stored-proc-outbound-gateway. Here, weird thing is happening. It appears that the second stored procedure is executed successfully, but strangely the the second stored-proc-outbound-gateway is not producing any reply message.
The following are the log statements after I connect to SQL server:
[org.springframework.jdbc.core.JdbcTemplate.doInCallableStatement] CallableStatement.execute() returned 'true'
[org.springframework.jdbc.core.JdbcTemplate.doInCallableStatement] CallableStatement.getUpdateCount() returned -1
[org.springframework.jdbc.core.JdbcTemplate.extractReturnedResults] CallableStatement.getUpdateCount() returned 1
[org.springframework.jdbc.core.JdbcTemplate.extractReturnedResults] CallableStatement.getUpdateCount() returned -1
The same statements are different when I connect to db2:
DEBUG [org.springframework.jdbc.core.JdbcTemplate.doInCallableStatement] CallableStatement.execute() returned 'false'
DEBUG [org.springframework.jdbc.core.JdbcTemplate.doInCallableStatement] CallableStatement.getUpdateCount() returned 0
DEBUG [org.springframework.jdbc.core.JdbcTemplate.extractReturnedResults] CallableStatement.getUpdateCount() returned -1
Can any one please say what's wrong here?
Please feel free to ask me if you need any additional info.
Here is the complete spring-integration file for your reference:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:int-xml="http://www.springframework.org/schema/integration/xml"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-jdbc="http://www.springframework.org/schema/integration/jdbc"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:int-http="http://www.springframework.org/schema/integration/http"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/xml
http://www.springframework.org/schema/integration/xml/spring-integration-xml.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd
http://www.springframework.org/schema/integration/jdbc
http://www.springframework.org/schema/integration/jdbc/spring-integration-jdbc.xsd
http://www.springframework.org/schema/integration/http
http://www.springframework.org/schema/integration/http/spring-integration-http.xsd ">
<int:channel id="PQPutUserBAInformation-InputChannel" />
<int:chain input-channel="PQPutUserBAInformation-InputChannel" output-channel="PQPutUserBAInformation-SetDateTag" >
<!-- Transformer to transform the resultXml to the user understandable form using XSLT -->
<int-xml:xslt-transformer xsl-resource="${stylesheet.PQPutUserBAInformationSetDateTag}" />
</int:chain>
<int:chain input-channel="PQPutUserBAInformation-SetDateTag" output-channel="PQPutUserBAInformation-SPCall" >
<!-- Split the Search Request Params from Xml -->
<int-xml:xpath-splitter>
<int-xml:xpath-expression expression="//AWDPQContactID" namespace-map="xmlMessageNamespace" />
</int-xml:xpath-splitter>
<!-- Store the original payload in header for future purpose -->
<int:header-enricher default-overwrite="true" should-skip-nulls="true" >
<int:header name="${headerNames.originalPayload}" expression="payload" />
</int:header-enricher>
</int:chain>
<!-- PQPutUserBAInformation Channel -->
<int:channel id="PQPutUserBAInformation-SPCall" />
<int:chain input-channel="PQPutUserBAInformation-SPCall" output-channel="PQPutUserBAInformation-SPCallStage1-Response" >
<int:service-activator ref="msgHandler" method="buildRequestBasedDataSource" />
<int-jdbc:stored-proc-outbound-gateway
id="PQPutUserBAInformation-AWD-StoredProcedure"
auto-startup="true"
data-source="routingDataSource"
stored-procedure-name="ZSPPQDELETEUSERIDBA"
skip-undeclared-results="true"
ignore-column-meta-data="true"
use-payload-as-parameter-source = "false"
expect-single-result="true" >
<int-jdbc:sql-parameter-definition name="P_USERID" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_BUSINESSAREA" direction="IN" type="VARCHAR" />
<int-jdbc:parameter name="P_USERID" expression="#xpath(payload, '//CurrentUserID')" />
<int-jdbc:parameter name="P_BUSINESSAREA" expression="#xpath(payload, '//SelectedBusinessArea/Code')" />
<int-jdbc:returning-resultset name="rowMapper" row-mapper="com.dsths.cs.awd.utils.ResultSetRowMapper"/>
</int-jdbc:stored-proc-outbound-gateway>
</int:chain>
<!-- Service Activator to build the Message from the Stored Procedure ResultSet -->
<int:channel id="PQPutUserBAInformation-SPCallStage1-Response" />
<int:service-activator input-channel="PQPutUserBAInformation-SPCallStage1-Response"
output-channel="PQPutUserBAInformation-SPCall2-Translate"
ref="msgHandler"
method="buildMessageFromExtSysResponse" />
<!-- Service Activator to build the Message from the Stored Procedure ResultSet -->
<int:channel id="PQPutUserBAInformation-SPCall2-Translate" />
<int:service-activator input-channel="PQPutUserBAInformation-SPCall2-Translate"
output-channel="PQPutUserBAInformation-SPCall2"
ref="cacheRequestHandler"
method="translatePassword" />
<!-- PQPutUserBAInformation Channel -->
<int:channel id="PQPutUserBAInformation-SPCall2" />
<int:chain input-channel="PQPutUserBAInformation-SPCall2" output-channel="PQPutUserBAInformation-SPCallStage2-Response" >
<int:service-activator ref="msgHandler" method="buildRequestBasedDataSource" />
<int-jdbc:stored-proc-outbound-gateway
id="PQPutUserBAInformation-AWD-StoredProcedure2"
auto-startup="true"
data-source="routingDataSource"
stored-procedure-name="ZSPPQINSERTUSERIDBA"
skip-undeclared-results="true"
ignore-column-meta-data="true"
use-payload-as-parameter-source = "false"
expect-single-result="true" >
<int-jdbc:sql-parameter-definition name="P_USERID" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_BUSINESSAREA" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_SELECTEDIND" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_DEFAULTIND" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_LEGACYSYSTEM" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_LEGACYLOGIN" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_LEGACYPASSWORD" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_OTHERLOGIN" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_OTHERPASSWORD" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_ADDSECURLOGIN" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_ADDSECURPASSWORD" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_LASTUPDATEUSERID" direction="IN" type="VARCHAR" />
<int-jdbc:sql-parameter-definition name="P_LASTUPDATE" direction="IN" type="VARCHAR" />
<int-jdbc:parameter name="P_USERID" expression="#xpath(payload, '//CurrentUserID')" />
<int-jdbc:parameter name="P_BUSINESSAREA" expression="#xpath(payload, '//SelectedBusinessArea/Code')" />
<int-jdbc:parameter name="P_SELECTEDIND" expression="#xpath(payload, '//SelectedBusinessArea/Selected')" />
<int-jdbc:parameter name="P_DEFAULTIND" expression="#xpath(payload, '//SelectedBusinessArea/Default')" />
<int-jdbc:parameter name="P_LEGACYSYSTEM" expression="#xpath(payload, '//SelectedBusinessArea/LegacySystem')" />
<int-jdbc:parameter name="P_LEGACYLOGIN" expression="#xpath(payload, '//SelectedBusinessArea/LegacyLogin')" />
<int-jdbc:parameter name="P_LEGACYPASSWORD" expression="headers.LegacyPassword" />
<int-jdbc:parameter name="P_OTHERLOGIN" expression="#xpath(payload, '//SelectedBusinessArea/OtherLogin')" />
<int-jdbc:parameter name="P_OTHERPASSWORD" expression="headers.otherPassword" />
<int-jdbc:parameter name="P_ADDSECURLOGIN" expression="#xpath(payload, '//SelectedBusinessArea/AddSecurLogin')" />
<int-jdbc:parameter name="P_ADDSECURPASSWORD" expression="headers.addSecurPassword" />
<int-jdbc:parameter name="P_LASTUPDATEUSERID" expression="#xpath(payload, '//userID')" />
<int-jdbc:parameter name="P_LASTUPDATE" expression="#xpath(payload, '//dateTimeStamp')" />
<int-jdbc:returning-resultset name="rowMapper" row-mapper="com.dsths.cs.awd.utils.ResultSetRowMapper"/>
</int-jdbc:stored-proc-outbound-gateway>
</int:chain>
<!-- Service Activator to build the Message from the Stored Procedure ResultSet -->
<int:channel id="PQPutUserBAInformation-SPCallStage2-Response" />
<int:service-activator input-channel="PQPutUserBAInformation-SPCallStage2-Response"
output-channel="PQPutUserBAInformation-Enrich-SPCallStage2"
ref="msgHandler"
method="buildMessageFromExtSysResponse" />
<!-- Service Activator to build the jobResponse xml making use of jobRequestXml and xlst transformed xml-->
<int:channel id="PQPutUserBAInformation-Enrich-SPCallStage2" />
<int:service-activator input-channel="PQPutUserBAInformation-Enrich-SPCallStage2"
output-channel="PQPutUserBAInformation-CleanUp"
ref="msgHandler"
method="enrichPayloadXml" />
<int:chain input-channel="PQPutUserBAInformation-CleanUp" >
<!-- Transformer to transform the resultXml to the user understandable form using XSLT -->
<int-xml:xslt-transformer xsl-resource="${stylesheet.PQPutUserBAInformationCleanUp}" />
</int:chain>
</beans>

I got the same problem, and I fixed it, the exception means the transform step returns null(shouldn't be null) because the message I created cant be properly transformed. Try to debug into the transform step, and see what is the outcome. the exception happened in my second line:
IntegrationFlows.from(testChannel() )
.transform( new ObjectTransformer() )
.<Object, Class<?>>route(Object::getClass,

It means the stored proc produced no result.

Related

Spring Integration Header Enricher Null check

I have the below header enricher in my configuration.
<int:header-enricher input-channel="" output-channel="" id="" >
<int:header name="OrderNo" expression="#gateway.exchange(#root).payload"/>
</int:header-enricher>
When the payload is null, I am getting the exception
java.lang.IllegalArgumentException: payload must not be null
at org.springframework.util.Assert.notNull(Assert.java:112)
at org.springframework.integration.support.MessageBuilder.<init>(MessageBuilder.java:56) ~[spring-integration-core-4.0.5.RELEASE.jar:na]
Is there a way to check for null payload from gateway and set some dummy value in header, if payload is null?
<int:gateway id="gateway" default-request-channel="defReqChannel" />
<int-jdbc:stored-proc-outbound-gateway request-channel="defReqChannel" data-source="dataSourceDD" expect-single-result="true" is-function="false" stored-procedure-name="SP_GET_CODE">
<int-jdbc:sql-parameter-definition name="param1" direction="IN" />
<int-jdbc:sql-parameter-definition name="param2" direction="IN" />
<int-jdbc:sql-parameter-definition name="param3" direction="INOUT" />
<int-jdbc:sql-parameter-definition name="param4" direction="IN" />
<int-jdbc:parameter name="param1" expression="exp1" />
<int-jdbc:parameter name="param2" expression="exp2" />
<int-jdbc:parameter name="param3" value="0"/>
<int-jdbc:parameter name="param4" value="0"/>
</int-jdbc:stored-proc-outbound-gateway>
The stored procedure that I am invoking can return a null value. (param3). The failure happens in those scenarios
Please, share more StackTrace. payload can't be null in the Message. The code you show is fully controlled by the Framework.
I think there is a code when you do something like MessageBuilder.withPayload(null).
But that fully isn't a case for config you demonstrate.
UPDATE
This is really a bug: https://jira.spring.io/browse/INT-4202.
The workaround is like do not use expect-single-result="true" and extract target value from the result Map downstream manually.

How to copy the message payload of direct channel to recovery channel in gateway?

I have a 'datachannel' which gets the result set from DB using inbound-channel adapter.
Here i am getting a field called 'process_id' from DB.After calling an external system through int:http-outbound gateway i am defining a recovery-channel. I want to do an update query only for that process_id.But i am unable to get the process id in the recovery channel.Getting an exception invalid property "payload[process_id]"..Is there anyway to pass the process_id to the recovery channel,so that i can perform my update query in a like this
int-jdbc:outbound-channel-adapter query="update TBL_RECEIPT set receipt_status=1
where process_id in (:payload[process_id])" data-source="dataSource" channel="errors"/>
For clarity,below is spring-integration xml configuration
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:int-jdbc="http://www.springframework.org/schema/integration/jdbc"
xmlns:context="http://www.springframework.org/schema/context" xmlns:p="http://www.springframework.org/schema/p"
xmlns:task="http://www.springframework.org/schema/task" xmlns:int-http="http://www.springframework.org/schema/integration/http"
xmlns:stream="http://www.springframework.org/schema/integration/stream"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-4.1.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration-4.1.xsd
http://www.springframework.org/schema/integration/stream
http://www.springframework.org/schema/integration/stream/spring-integration-stream-4.1.xsd
http://www.springframework.org/schema/integration/http
http://www.springframework.org/schema/integration/http/spring-integration-http-4.1.xsd
http://www.springframework.org/schema/integration/jdbc
http://www.springframework.org/schema/integration/jdbc/spring-integration-jdbc-4.1.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.xsd">
<int:channel id="requestchannel"></int:channel>
<int:channel id="xtifyrequestchannel"></int:channel>
<int:channel id="xtifyresponsechannel"></int:channel>
<int:channel id="tpgrequestchannel"></int:channel>
<int:channel id="tpgresponsechannel"></int:channel>
<int:channel id="xtifyerrorchannel">
</int:channel>
<int:channel id="tpgerrorchannel">
</int:channel>
<int:channel id="executerchannel">
<int:dispatcher task-executor="taskExecutor" />
</int:channel>
<task:executor id="taskExecutor" pool-size="2" />
<bean id="pollerdatamapper" class="main.java.com.as.poller.PollerDataMapper" />
<bean id="pollerservice" class="main.java.com.as.poller.PollerService" />
<bean id="requestFactory"
class="org.springframework.http.client.SimpleClientHttpRequestFactory">
<property name="connectTimeout" value="10000" />
<property name="readTimeout" value="10000" />
</bean>
<int:logging-channel-adapter id="logger"
level="INFO" />
<int-jdbc:inbound-channel-adapter id="datachannel"
query="select loyalty_id,process_id,mobile_uid,mobile_os from TBL_RECEIPT where r_cre_time=(select min(r_cre_time) from TBL_RECEIPT where receipt_status=0)"
data-source="dataSource" max-rows-per-poll="1" row-mapper="pollerdatamapper">
<int:poller fixed-rate="5000">
</int:poller>
</int-jdbc:inbound-channel-adapter>
<int:gateway id="requestGateway" service-interface="main.java.com.as.poller.RequestGateway"
default-request-channel="requestchannel" default-reply-timeout="20000">
<int:method name="pushNotification" />
<int:method name="sendTPGRequest" request-channel="tpgrequestchannel">
<int:header name="Content-Type" value="multipart/form-data" />
</int:method>
</int:gateway>
<int:object-to-json-transformer
input-channel="requestchannel" output-channel="xtifyrequestchannel"></int:object-to-json-transformer>
<int-http:outbound-gateway id="xtifygateway"
request-channel="xtifyrequestchannel" reply-channel="xtifyresponsechannel" request-factory="requestFactory"
url="${xtifyUrl}" http-method="POST">
<int-http:request-handler-advice-chain>
<int:retry-advice max-attempts="3" recovery-channel="xtifyerrorchannel">
</int:retry-advice>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<int-http:outbound-gateway id="tpggateway"
request-channel="tpgrequestchannel" reply-channel="tpgresponsechannel"
request-factory="requestFactory" expected-response-type="java.lang.String"
url="${tpg_url}" http-method="POST">
<int-http:request-handler-advice-chain>
<int:retry-advice max-attempts="3" recovery-channel="tpgerrorchannel">
</int:retry-advice>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<int:json-to-object-transformer
input-channel="tpgresponsechannel" type="main.java.com.as.rest.response.TPGResponse" />
<int:service-activator input-channel="datachannel"
output-channel="executerchannel" ref="pollerservice" method="getRecordFromPoller">
</int:service-activator>
<int:service-activator input-channel="executerchannel"
ref="pollerservice" method="getDataFromExecuterChannel">
</int:service-activator>
<int-jdbc:outbound-channel-adapter
id="tpgsystemfailure"
query="update TBL_RECEIPT set receipt_status=1
where process_id in (:payload.failedMessage.payload[process_id])"
data-source="dataSource" channel="tpgerrorchannel" />
<int-jdbc:outbound-channel-adapter
id="xtifysystemfailure"
query="update TBL_RECEIPT set receipt_status=4 where process_id in (:payload.failedMessage.payload[process_id])"
data-source="dataSource" channel="xtifyerrorchannel" />
<int-jdbc:outbound-channel-adapter
id="xtifysystemsuccess"
query="update TBL_RECEIPT set receipt_status=5 where process_id in (:payload.process_id)"
data-source="dataSource" channel="xtifyresponsechannel" />
</beans>
The recovery-channel gets an ErrorMessage. The payload is a MessagingException with two properties failedMessage and cause.
Use payload.failedMessage.payload[process_id].

Mobicents diameter no connection to peer error message

I am new to mobicents diameter and followed the example downloaded here.
Executing both ExampleClient and ExampleServer on the same machine with different ports the "Result-code" is 3002
"Error-message" is "No connection to peer"
Where am I wrong?
Protocol error:
DIAMETER_UNABLE_TO_DELIVER 3002
This error is given when Diameter can not deliver the message to
the destination, either because no host within the realm
supporting the required application was available to process the
request, or because Destination-Host AVP was given without the
associated Destination-Realm AVP.
client-jdiameter-config.xml
<?xml version="1.0"?>
<Configuration xmlns="http://www.jdiameter.org/jdiameter-server">
<LocalPeer>
<URI value="aaa://127.0.0.1:1812" />
<IPAddresses>
<IPAddress value="127.0.0.1" />
</IPAddresses>
<Realm value="mobicents.org" />
<VendorID value="0" />
<ProductName value="jDiameter" />
<FirmwareRevision value="1" />
<OverloadMonitor>
<Entry index="1" lowThreshold="0.5" highThreshold="0.6">
<ApplicationID>
<VendorId value="0" />
<AuthApplId value="333333" />
<AcctApplId value="0" />
</ApplicationID>
</Entry>
</OverloadMonitor>
</LocalPeer>
<Parameters>
<AcceptUndefinedPeer value="false" />
<DuplicateProtection value="true" />
<DuplicateTimer value="240000" />
<UseUriAsFqdn value="false" />
<QueueSize value="10000" />
<MessageTimeOut value="60000" />
<StopTimeOut value="10000" />
<CeaTimeOut value="10000" />
<IacTimeOut value="30000" />
<DwaTimeOut value="10000" />
<DpaTimeOut value="5000" />
<RecTimeOut value="10000" />
<Concurrent>
<Entity name="ThreadGroup" size="64" />
<Entity name="ProcessingMessageTimer" size="1" />
<Entity name="DuplicationMessageTimer" size="1" />
<Entity name="RedirectMessageTimer" size="1" />
<Entity name="PeerOverloadTimer" size="1" />
<Entity name="ConnectionTimer" size="1" />
<Entity name="StatisticTimer" size="1" />
</Concurrent>
</Parameters>
<Network>
<Peers>
<Peer name="aaa://127.0.0.1:3868" attempt_connect="true" rating="1" />
</Peers>
<Realms>
<Realm name="mobicents.org" peers="127.0.0.1"
local_action="LOCAL" dynamic="false" exp_time="1">
<ApplicationID>
<VendorId value="0" />
<AuthApplId value="333333" />
<AcctApplId value="0" />
</ApplicationID>
</Realm>
</Realms>
</Network>
<Extensions />
</Configuration>
server-jdiameter-config.xml
<?xml version="1.0"?>
<Configuration xmlns="http://www.jdiameter.org/jdiameter-server">
<LocalPeer>
<URI value="aaa://127.0.0.1:3868" />
<IPAddresses>
<IPAddress value="127.0.0.1" />
</IPAddresses>
<Realm value="mobicents.org" />
<VendorID value="0" />
<ProductName value="jDiameter" />
<FirmwareRevision value="1" />
<OverloadMonitor>
<Entry index="1" lowThreshold="0.5" highThreshold="0.6">
<ApplicationID>
<VendorId value="0" />
<AuthApplId value="333333" />
<AcctApplId value="0" />
</ApplicationID>
</Entry>
</OverloadMonitor>
</LocalPeer>
<Parameters>
<!-- set to true, we can safely remove client def in this case -->
<AcceptUndefinedPeer value="true" />
<DuplicateProtection value="true" />
<DuplicateTimer value="240000" />
<UseUriAsFqdn value="false" />
<QueueSize value="10000" />
<MessageTimeOut value="60000" />
<StopTimeOut value="10000" />
<CeaTimeOut value="10000" />
<IacTimeOut value="30000" />
<DwaTimeOut value="10000" />
<DpaTimeOut value="5000" />
<RecTimeOut value="10000" />
<Concurrent>
<Entity name="ThreadGroup" size="64" />
<Entity name="ProcessingMessageTimer" size="1" />
<Entity name="DuplicationMessageTimer" size="1" />
<Entity name="RedirectMessageTimer" size="1" />
<Entity name="PeerOverloadTimer" size="1" />
<Entity name="ConnectionTimer" size="1" />
<Entity name="StatisticTimer" size="1" />
</Concurrent>
</Parameters>
<Network>
<Peers>
<!-- our client, lets define it -->
<Peer name="aaa://127.0.0.1:1812" attempt_connect="false"
rating="1" />
</Peers>
<Realms>
<Realm name="mobicents.org" peers="127.0.0.1" local_action="LOCAL" dynamic="false" exp_time="1">
<ApplicationID>
<VendorId value="0" />
<AuthApplId value="333333" />
<AcctApplId value="0" />
</ApplicationID>
</Realm>
</Realms>
</Network>
<Extensions />
</Configuration>

Custom BCS indexing connector with changelog inremental crawl is not working properly

I am writing a custom indexing connector using changelog incremental crawl approach.
I'm using sample from http://msdn.microsoft.com/en-us/library/ff625800%28v=office.14%29.aspx and trying to change it for me.
My model has next stereotypes: IdEnumerator, ChangedIdEnumerator, DeletedIdEnumerator, SpecificFinder, Finder, StreamAccessor
If I'm starting full crawl, IdEnumerator, ChangedIdEnumerator, DeletedIdEnumerator will be called.
First problem: the SpecificFinder is not called.
If I'm starting incremental crawl, ChangedIdEnumerator and DeletedIdEnumerator will be called.
DeletedIdEnumerator is working: items with deleted ids are deleted from the index.
Second problem: ChangedIdEnumerator is not working. Nothing happens after I returned the changed ids.
There are now errors in the crowl log.
My model is here:
<Model xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Name="MyFileModel" xmlns="http://schemas.microsoft.com/windows/2007/BusinessDataCatalog">
<LobSystems>
<LobSystem Name="MyFileSystem" Type="Custom">
<Properties>
<Property Name="SystemUtilityTypeName" Type="System.String">MyFileConnector.MyFileConnector, MyFileConnector, Version=1.0.0.0, Culture=neutral, PublicKeyToken=15865f58b9878bf8</Property>
<Property Name="SystemUtilityInstallDate" Type="System.DateTime">2013-01-01 00:00:00Z</Property>
<Property Name="InputUriProcessor" Type="System.String">MyFileConnector.MyFileLobUri, MyFileConnector, Version=1.0.0.0, Culture=neutral, PublicKeyToken=15865f58b9878bf8</Property>
<Property Name="OutputUriProcessor" Type="System.String">MyFileConnector.MyFileNamingContainer, MyFileConnector, Version=1.0.0.0, Culture=neutral, PublicKeyToken=15865f58b9878bf8</Property>
</Properties>
<LobSystemInstances>
<LobSystemInstance Name="MyFileConnector_instance">
<Properties>
<Property Name="AuthenticationType" Type="System.String">Credentials</Property>
</Properties>
</LobSystemInstance>
</LobSystemInstances>
<Entities>
<Entity Name="MyFolder" Namespace="MyFileConnector" Version="1.0.0.1">
<Properties>
<Property Name="Title" Type="System.String">Name</Property>
</Properties>
<Identifiers>
<Identifier Name="ID" TypeName="System.String" />
</Identifiers>
<Methods>
<!-- IdEnumerator -->
<Method Name="ReadAllIds" DefaultDisplayName="ReadAllIds" IsStatic="false">
<Parameters>
<Parameter Name="returnIds" Direction="Return">
<TypeDescriptor Name="Nodes" TypeName="Microsoft.BusinessData.Runtime.DynamicType[]" IsCollection="true">
<TypeDescriptors>
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType" Name="Node">
<TypeDescriptors>
<TypeDescriptor Name="ID" TypeName="System.String" IdentifierName="ID" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Type="IdEnumerator" Name="ReadAllIds" DefaultDisplayName="ReadAllIds" ReturnParameterName="returnIds" Default="true">
<Properties>
<Property Name="RootFinder" Type="System.String">true</Property>
</Properties>
<AccessControlList>
<AccessControlEntry Principal="NT AUTHORITY\Authenticated Users">
<Right BdcRight="Execute" />
</AccessControlEntry>
<AccessControlEntry Principal="NT AUTHORITY\System">
<Right BdcRight="SetPermissions"/>
</AccessControlEntry>
</AccessControlList>
</MethodInstance>
</MethodInstances>
</Method>
<!-- ChangedIdEnumerator -->
<Method Name="ReadIncrementalList" IsStatic="false">
<FilterDescriptors>
<FilterDescriptor Name="LastCrawl" Type="InputOutput">
<Properties>
<Property Name="SynchronizationCookie" Type="System.String">x</Property>
</Properties>
</FilterDescriptor>
<FilterDescriptor Name="Timestamp" Type="Timestamp" />
</FilterDescriptors>
<Parameters>
<Parameter Name="lastCrawlDate" Direction="InOut">
<TypeDescriptor Name="LastCrawlDate" TypeName="System.DateTime" IsCollection="false" AssociatedFilter="LastCrawl">
<Interpretation>
<NormalizeDateTime LobDateTimeMode="Local" />
</Interpretation>
</TypeDescriptor>
</Parameter>
<Parameter Name="returnIds" Direction="Return">
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType[]" Name="Nodes" IsCollection="true" >
<TypeDescriptors>
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType" Name="Node">
<TypeDescriptors>
<TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name="ReadIncrementalListInstance" Type="ChangedIdEnumerator" ReturnParameterName="returnIds" Default="true">
<AccessControlList>
<AccessControlEntry Principal="NT AUTHORITY\Authenticated Users">
<Right BdcRight="Execute" />
<Right BdcRight="SetPermissions" />
</AccessControlEntry>
</AccessControlList>
</MethodInstance>
</MethodInstances>
</Method>
<!-- DeletedIdEnumerator -->
<Method Name="ReadDeletedIncrementalList" IsStatic="false" DefaultDisplayName="ReadDeletedIncrementalList">
<FilterDescriptors>
<FilterDescriptor Name="LastCrawl" Type="InputOutput">
<Properties>
<Property Name="SynchronizationCookie" Type="System.String">x</Property>
</Properties>
</FilterDescriptor>
<FilterDescriptor Name="Timestamp" Type="Timestamp" />
</FilterDescriptors>
<Parameters>
<Parameter Name="LastCrawlDate" Direction="InOut">
<TypeDescriptor Name="LastCrawlDate" TypeName="System.DateTime" IsCollection="false" AssociatedFilter="LastCrawl">
<Interpretation>
<NormalizeDateTime LobDateTimeMode="Local" />
</Interpretation>
</TypeDescriptor>
</Parameter>
<Parameter Name="deletedIds" Direction="Return">
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType[]" Name="Nodes" IsCollection="true">
<TypeDescriptors>
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType" Name="Node">
<TypeDescriptors>
<TypeDescriptor Name="ID" TypeName="System.String" IdentifierName="ID" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name="ReadDeletedIncrementalListInstance" Type="DeletedIdEnumerator" ReturnParameterName="deletedIds">
<AccessControlList>
<AccessControlEntry Principal="NT AUTHORITY\Authenticated Users">
<Right BdcRight="Execute" />
<Right BdcRight="SetPermissions" />
</AccessControlEntry>
</AccessControlList>
</MethodInstance>
</MethodInstances>
</Method>
<!-- Finder -->
<Method Name="ReadAllItems" DefaultDisplayName="ReadAllItems" IsStatic="false">
<Parameters>
<Parameter Name="returnAllItems" Direction="Return">
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType[]" Name="Nodes" IsCollection="true" >
<TypeDescriptors>
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType" Name="Node">
<TypeDescriptors>
<TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" />
<TypeDescriptor TypeName="System.String" Name="Name" />
<TypeDescriptor TypeName="System.String" Name="Title" />
<TypeDescriptor TypeName="System.String" Name="Path" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Type="Finder" Name="ReadAllItems" DefaultDisplayName="ReadAllItems" ReturnParameterName="returnAllItems" Default="true" ReturnTypeDescriptorName="Nodes" ReturnTypeDescriptorLevel="0">
<AccessControlList>
<AccessControlEntry Principal="NT AUTHORITY\Authenticated Users">
<Right BdcRight="Execute" />
</AccessControlEntry>
<AccessControlEntry Principal="NT AUTHORITY\System">
<Right BdcRight="SetPermissions"/>
</AccessControlEntry>
</AccessControlList>
</MethodInstance>
</MethodInstances>
</Method>
<!-- SpecificFinder -->
<Method Name="ReadItem" DefaultDisplayName="ReadItem" IsStatic="false">
<Parameters>
<Parameter Direction="In" Name="ID">
<TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" />
</Parameter>
<Parameter Direction="Return" Name="returnParameter">
<TypeDescriptor TypeName="Microsoft.BusinessData.Runtime.DynamicType" Name="Node">
<TypeDescriptors>
<TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" ReadOnly="true" />
<TypeDescriptor TypeName="System.String" Name="Title" />
<TypeDescriptor TypeName="System.String" Name="Author" />
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Type="SpecificFinder" ReturnParameterName="returnParameter" ReturnTypeDescriptorName="Node" Default="true" Name="ReadItem" DefaultDisplayName="ReadItem" ReturnTypeDescriptorLevel="0">
<AccessControlList>
<AccessControlEntry Principal="NT AUTHORITY\Authenticated Users">
<Right BdcRight="Execute" />
</AccessControlEntry>
<AccessControlEntry Principal="NT AUTHORITY\System">
<Right BdcRight="SetPermissions"/>
</AccessControlEntry>
</AccessControlList>
</MethodInstance>
</MethodInstances>
</Method>
</Methods>
</Entity>
</Entities>
</LobSystem>
What I'm doing wrong? I would really appreciate any input.
You have the same name for your SynchronizationCookie ("x"), give a different cookie name to each of your methods
I encountered similar issue recently (SpecificFinder not called) in my custom BCS connector and managed to sort it out. In my scenario I have two entities (parent and child), and SpecificFinder was called only for parent entities but was not called for the child ones.
It turned out that the issue was related to the way I constructed "access URIs". Initially the URIs were like this:
<protocol>://<entity_name>/<entity_id>
And my start URL (specified in the content source definition) was an URL of a "fake" parent entity (without any ID):
<protocol>://<parent_entity_name>
But it seems that SharePoint crawler treats access URIs in the same way as web URLs, and applies a filter by the URL path specified in the content source definition. In other words, in my case, it would only crawl URIs corresponding to the following pattern:
<protocol>://<parent_entity_name>/*
After I changed my access URI format to
<protocol>://root/<entity_name>
and set my start URL in the content source definition to
<protocol>://root
everything started working properly.

Error deploying SharePoint 2010 solution - Cannot create more than '500' IEntity objects per ILobSystem object

I am generating a sharepoint 2010 bdc model xml file along with the entities and associated service classes from a model 1st LLBLGen framework/ C# .net 3.5 project. All was going well when suddenly I started receiving this error while deploying the BDC solution;
"Error 178 Error occurred in deployment step 'Add Solution': Cannot create more than '500' IEntity objects per ILobSystem object. "
It had been deploying fine (after modifying the registry to extend the timeout settings) with the current number of entities. I can't find any reference to IEntity object limitations in MSDN nor via google and have tried changing the generated xml file in various ways to test. If I remove an entity, the error shifts to the beginning of the next entity. Visual Studio builds the solution just fine with only warnings about the datetime datatype (known problem apparently).
I only have 59 entities defined. Some of which are in inheritance hierarchies and there are numerous FK relationships expressed in the model. It doesn't make any sense to me that I have too many entities in my model. I have plenty more I would like to add. I am including EstimatedInstanceCount="10000" on each entity but that doesn't seem to effect anything. In fact I believe it is the default. The model file is quite large but I will include the following single entity snippet for reference as to what code is being generated.
<Entity Name="Load" Namespace="SharePoint.DataConnector.VoyagerModel" EstimatedInstanceCount="10000" Version="1.0.0.26">
<Properties>
<Property Name="Class" Type="System.String">SharePoint.DataConnector.VoyagerModel.LoadService, VoyagerModel</Property>
</Properties>
<Identifiers>
<Identifier Name="Id" TypeName="System.Int32" />
<!-- TODO: Change the name of the ID and if needed the TypeName of your identifier. -->
</Identifiers>
<Methods>
<!-- start finder method -->
<Method Name="ReadList">
<!-- TODO: Change the name of the method if needed. -->
<Parameters>
<Parameter Direction="Return" Name="returnParameter">
<TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[[SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel]]" IsCollection="true" Name="LoadList">
<TypeDescriptors>
<TypeDescriptor Name="Load" TypeName="SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierName="Id" IsCollection="false" ReadOnly="false" />
<!-- TODO: Add TypeDescriptors when you add properties to Load. -->
<TypeDescriptor Name="OrderId" IsCollection="false" ReadOnly="false" TypeName="System.Int32" />
<TypeDescriptor Name="Status" IsCollection="false" ReadOnly="false" TypeName="System.String" />
<TypeDescriptor Name="DriverId" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" IsCollection="false" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Type="Finder" ReturnParameterName="returnParameter" Default="true" Name="ReadList" DefaultDisplayName="Load List">
<Properties>
<Property Name="RootFinder" Type="System.String">x</Property>
</Properties>
</MethodInstance>
</MethodInstances>
</Method>
<!-- end finder method -->
<!-- start specific finder method -->
<Method Name="ReadItem">
<Parameters>
<Parameter Direction="In" Name="id">
<TypeDescriptor TypeName="System.Int32" IdentifierName="Id" Name="Id" IsCollection="false" />
</Parameter>
<Parameter Direction="Return" Name="returnParameter">
<TypeDescriptor TypeName="SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel" Name="Load">
<TypeDescriptors>
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierName="Id" IsCollection="false" ReadOnly="false" />
<!-- TODO: Add TypeDescriptors when you add properties to Load. -->
<TypeDescriptor Name="OrderId" TypeName="System.Int32" IsCollection="false" />
<TypeDescriptor Name="Status" TypeName="System.String" IsCollection="false" />
<TypeDescriptor Name="DriverId" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" IsCollection="false" />
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Type="SpecificFinder" ReturnParameterName="returnParameter" Default="true" Name="ReadItem" DefaultDisplayName="Read Load" />
</MethodInstances>
</Method>
<Method Name="Create">
<Parameters>
<Parameter Name="returnLoad" Direction="Return">
<TypeDescriptor Name="ReturnLoad" TypeName="SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierName="Id" IsCollection="false" ReadOnly="false" />
<TypeDescriptor Name="OrderId" TypeName="System.Int32" IsCollection="false" />
<TypeDescriptor Name="Status" TypeName="System.String" IsCollection="false" />
<TypeDescriptor Name="DriverId" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" IsCollection="false" />
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
<Parameter Name="newLoad" Direction="In">
<TypeDescriptor Name="NewLoad" TypeName="SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" IdentifierName="Id" IsCollection="false" ReadOnly="false" TypeName="System.Int32" CreatorField="false" />
<TypeDescriptor Name="OrderId" TypeName="System.Int32" IsCollection="false" CreatorField="true" />
<TypeDescriptor Name="Status" TypeName="System.String" IsCollection="false" CreatorField="true" />
<TypeDescriptor Name="DriverId" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" IsCollection="false" />
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name="Create" Type="Creator" ReturnParameterName="returnLoad" ReturnTypeDescriptorPath="ReturnLoad" />
</MethodInstances>
</Method>
<Method Name="Delete">
<Parameters>
<Parameter Name="id" Direction="In">
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierEntityName="Load" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name="Delete" Type="Deleter" />
</MethodInstances>
</Method>
<Method Name="Update">
<Parameters>
<Parameter Name="Load" Direction="In">
<TypeDescriptor Name="Load" TypeName="SharePoint.DataConnector.VoyagerModel.Load, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierName="Id" IsCollection="false" ReadOnly="false" />
<TypeDescriptor Name="OrderId" TypeName="System.Int32" IsCollection="false" UpdaterField="true" />
<TypeDescriptor Name="Status" TypeName="System.String" IsCollection="false" UpdaterField="true" />
<TypeDescriptor Name="DriverId" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" IsCollection="false" />
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name="Update" Type="Updater" />
</MethodInstances>
</Method>
<!-- start related entity methods -->
<Method Name="LoadToOrders">
<Parameters>
<Parameter Name="id" Direction="In">
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierEntityName="Load" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" ForeignIdentifierAssociationEntityName="Load" ForeignIdentifierAssociationEntityNamespace="SharePoint.DataConnector.VoyagerModel" ForeignIdentifierAssociationName="LoadToOrdersAssociationNavigator" />
</Parameter>
<Parameter Name="orderList" Direction="Return">
<TypeDescriptor Name="OrderList" TypeName="System.Collections.Generic.IEnumerable`1[[SharePoint.DataConnector.VoyagerModel.Order, VoyagerModel]]" IsCollection="true">
<TypeDescriptors>
<TypeDescriptor Name="Order" TypeName="SharePoint.DataConnector.VoyagerModel.Order, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" IsCollection="false" ReadOnly="true" TypeName="System.Int32" IdentifierEntityName="Order" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<Association Name="LoadToOrdersAssociationNavigator" Type="AssociationNavigator" ReturnParameterName="orderList" ReturnTypeDescriptorPath="OrderList">
<SourceEntity Name="Load" Namespace="SharePoint.DataConnector.VoyagerModel" />
<DestinationEntity Name="Order" Namespace="SharePoint.DataConnector.VoyagerModel" />
</Association>
</MethodInstances>
</Method>
<Method Name="LoadToDriver">
<Parameters>
<Parameter Name="id" Direction="In">
<TypeDescriptor Name="Id" TypeName="System.Int32" IdentifierEntityName="Load" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" ForeignIdentifierAssociationEntityName="Load" ForeignIdentifierAssociationEntityNamespace="SharePoint.DataConnector.VoyagerModel" ForeignIdentifierAssociationName="LoadToDriverAssociationNavigator" />
</Parameter>
<Parameter Name="driverList" Direction="Return">
<TypeDescriptor Name="DriverList" TypeName="System.Collections.Generic.IEnumerable`1[[SharePoint.DataConnector.VoyagerModel.Driver, VoyagerModel]]" IsCollection="true">
<TypeDescriptors>
<TypeDescriptor Name="Driver" TypeName="SharePoint.DataConnector.VoyagerModel.Driver, VoyagerModel">
<TypeDescriptors>
<TypeDescriptor Name="Id" IsCollection="false" ReadOnly="true" TypeName="System.Int32" IdentifierEntityName="Driver" IdentifierEntityNamespace="SharePoint.DataConnector.VoyagerModel" IdentifierName="Id" />
</TypeDescriptors>
</TypeDescriptor>
</TypeDescriptors>
</TypeDescriptor>
</Parameter>
</Parameters>
<MethodInstances>
<Association Name="LoadToDriverAssociationNavigator" Type="AssociationNavigator" ReturnParameterName="driverList" ReturnTypeDescriptorPath="DriverList">
<SourceEntity Name="Load" Namespace="SharePoint.DataConnector.VoyagerModel" />
<DestinationEntity Name="Driver" Namespace="SharePoint.DataConnector.VoyagerModel" />
</Association>
</MethodInstances>
</Method>
<!-- end related entity methods -->
</Methods>
<AssociationGroups>
<AssociationGroup Name="DriverToLoadAssociation">
<AssociationReference AssociationName="LoadToDriverAssociationNavigator" Reverse="true" />
</AssociationGroup>
</AssociationGroups>
</Entity>
Is this a misleading error message?
What changes could I try to the model?
Could there be any site settings effecting this?
Thanks for any help!
I have actually ran in to the same problem as you described and couldn't find much information about it back then either. The MSDN article you referred to however is for SharePoint 2007 and the BDC model, not for the Business Connectivity Services.
If you only have 59 entities, it is likely the LobSystem isn't completely removed after retracting your solution. The model and entities do get removed, but the LobSystem itself will still exist. You can check this from Central Administration.Thus every new deployment you do will add the entities to the system again and raising the ´entity count´, reaching the 500 limitation much faster then expected.
Try removing the LobSystem manually (through Central Administration) so a new LobSystem will get created during deployment as well.
OK, so apparently not too many people are working with large enterprise level BDC models? I'm scared...
I've found what I guess is my answer in MSDN. If you go to MSDN You'll find the LobSystem limitations detailed of which;
LobSystemInstances - Max instances per system: 300
Entities - Max entities per system: 200
Associations - Max associations per system: 1000
Individually, these don't sound like a problem. I am using a single Lob instance and I only have 59 entities. And on average 8 or so associations on each. But how the Max associations are calculated I have no idea. Does each side of the association count as an instance? Does each reference to the association add a reference?
I would love to hear some deeper insight into the LobSystem architecture.
In the mean time, I am going to split my model into separate schema groups and modify my templates to create associations in the bdc model schema file which fully qualify across different LobSystem models.
If anyone knows why this won't work, please stop me NOW!

Resources