spring-integration is losing headers when sent to a subscriber - spring-integration

I am using spring-integration with hornetQ. The problem is that I have put a custom header in the message (Method), but when it hits the subscriber the header is no longer available. I there some sort of configuration property I need to setup to preserve headers?
An application receives the message (I can see the Method header in the console log so I know it is actually getting the correct message). It basically just routes the message onto the outbound queue so that client can subscribe to it (if there is a cleaner way to do this please let me know)
<int:channel id="partsChannel" />
<int-jms:message-driven-channel-adapter
id="jmsPartsInbound"
acknowledge="transacted"
destination-name="parts.in"
channel="partsChannel"
connection-factory="jmsConnectionFactory"
/> <!-- error-channel="partsInboundFailedChannel" -->
<int-jms:outbound-channel-adapter
id="jmsPartsOutbound"
destination-name="parts.out"
channel="partsChannel"
connection-factory="jmsConnectionFactory"
pub-sub-domain="true"
>
<int-jms:request-handler-advice-chain>
<int:retry-advice max-attempts="3">
<int:exponential-back-off initial="2000" multiplier="2" />
</int:retry-advice>
</int-jms:request-handler-advice-chain>
</int-jms:outbound-channel-adapter>
Applications subscribe like so:
<int:channel id="partsInboundChannel" />
<int-jms:message-driven-channel-adapter
id="jmsPartsInbound"
acknowledge="transacted"
destination-name="parts.out"
channel="partsInboundChannel"
pub-sub-domain="true"
connection-factory="jmsConnectionFactory"/>
And this is the part that gets the message in the subscriber.
#ServiceActivator(inputChannel = "partsInboundChannel")
public void processPart(final Message message) {
...message.getHeaders does not contain the "Method" header
}

Isn't your issue here in the DefaultJmsHeaderMapper.fromHeaders:
if (value != null && SUPPORTED_PROPERTY_TYPES.contains(value.getClass())) {
try {
String propertyName = this.fromHeaderName(headerName);
jmsMessage.setObjectProperty(propertyName, value);
}
where SUPPORTED_PROPERTY_TYPESare:
private static List<Class<?>> SUPPORTED_PROPERTY_TYPES = Arrays.asList(new Class<?>[] {
Boolean.class, Byte.class, Double.class, Float.class, Integer.class, Long.class, Short.class, String.class });
So, if your method is really of Method type, it will be skipped.
Consider to use its name instead.

Related

Message Template receive gives Dispatcher has no subscribers for channel

This is my spring-integration inbound and out bound which gets a list from a end point.
<http:inbound-gateway id="webListGateway"
request-channel="fromWeb_List"
reply-channel="toWeb_List"
path="/api/profile/V1/get"
supported-methods="GET">
<http:header name="container" expression="#pathVariables.container"/>
<http:header name="groupName" expression="#pathVariables.groupName"/>
<http:header name="userId" expression="#pathVariables.userId"/>
</http:inbound-gateway>
<int:header-enricher input-channel="fromWeb_List" output-channel="toCloud_List">
<int:header name="apikey" value=“1234”/>
</int:header-enricher>
<http:outbound-gateway id="profileListGateway"
request-channel="toCloud_List"
reply-channel="sync_preferences"
url=“localhost:8081/containers/{container}/groups/{groupName}/values/hierarchy/{userId}"
http-method="GET"
expected-response-type="java.lang.String"
charset="UTF-8"
extract-request-payload="false"
header-mapper="headerMapper"
encode-uri="true" >
<http:uri-variable name="container" expression="headers.container"/>
<http:uri-variable name="groupName" expression="headers.groupName"/>
<http:uri-variable name="userId" expression="headers.userId"/>
</http:outbound-gateway>
This is my recipient-list-router which send backs the list to requestor and also saves the list in another end point.
<int:recipient-list-router id="syncRouter" input-channel="sync_preferences">
<int:recipient channel="toWeb_List"/>
<int:recipient channel="toCloud_Save"/>
</int:recipient-list-router>
I am also trying to call the outbound gateway from java code and trying to get the response from toWeb_List channel by using receive method on MessageTemplate, which is giving me error
MessagingTemplate template = new MessagingTemplate();
Message<String> message1 = MessageBuilder.withPayload("")
.setHeader("container", “fwd”)
.setHeader("groupName", “foo”)
.setHeader("userId", “user”)
.build();
template.send((MessageChannel) CONTEXT.getBean("fromWeb_List"),message1);
PreList pre = (PreList) template.receive((MessageChannel)CONTEXT.getBean("toWeb_List"));
error
Dispatcher has no subscribers for channel 'application:springboot.toWeb_List'
Any Idea what I am doing wrong here.
You can't use DirectChannel for the MessagingGateway.receive():
protected final Message<?> doReceive(MessageChannel channel, long timeout) {
Assert.notNull(channel, "MessageChannel is required");
Assert.state(channel instanceof PollableChannel, "A PollableChannel is required to receive messages");
Another issue that reply-channel="toWeb_List" has to be registered as correlator in the Inbound Gateway to be able to receive messages, as Gary pointed. And that is done on demand, on the first request. That's why you get
that Dispatcher has no subscribers.
And really, please, try to explain what you would like to do.
UPDATE
If you are going to reuse that <int:recipient-list-router> from the HTTP Inbound and from some other similar request-reply place, you should consider to drop off usage of the reply-channel and just rely on the replyChannel in headers.
I mean there can be that toWeb_List channel bean definition, but should not use it from the reply-channel. In this case your config should be like this:
<int:recipient-list-router id="syncRouter" input-channel="sync_preferences">
<int:recipient channel="toWeb_List"/>
<int:recipient channel="toCloud_Save"/>
</int:recipient-list-router>
<int:bridge input-channel="toWeb_List"/>
The bridge is such a components to shift message from input channel to the output one, if present. Otherwise it consults MessageHeaders for the replyChannel value. And this is is populated exactly via those Inbound request-reply components such as <http:inbound-gateway> or plain <int:gateway> when you call directly from Java.
See more information in the Reference Manual.

How to make aggregator wait for response from all methods in publish-subsribe channel?

I have used the below spring configuration to combine results from publish subscribe channel using aggregator. But the aggregator populates response only from the first service activator in the publish subscribe channel and it does not wait for the response from the other service activators. How should i modify my configuration to make the aggregator wait for response from all 4 service activators?
<int:bridge id="ValidationsBridge" input-channel="RequestChannel" output-channel="bridgeOutputChannel"></int:bridge>
<int:publish-subscribe-channel id="bridgeOutputChannel" apply-sequence="true" />
<int:service-activator input-channel="bridgeOutputChannel" output-channel="aggregatorInput"
method="populateResponse1" ref="WebServiceImpl" >
</int:service-activator>
<int:service-activator input-channel="bridgeOutputChannel" method="populateResponse2" ref="WebServiceImpl" output-channel="aggregatorInput"
>
</int:service-activator>
<int:service-activator input-channel="bridgeOutputChannel" method="populateResponse3" ref="WebServiceImpl" output-channel="aggregatorInput"
>
</int:service-activator>
<int:service-activator input-channel="bridgeOutputChannel" method="populateResponse4" ref="WebServiceImpl" output-channel="aggregatorInput"
>
</int:service-activator>
<task:executor id="executor" pool-size="4" keep-alive="20"/>
<int:aggregator input-channel="aggregatorInput" output-channel="aggregatorOutput" ref="vehicleAggregator" method="populateResponse"
></int:aggregator>
<int:service-activator id="processorServiceActivator" input-channel="aggregatorOutput" ref="Processor" method="mapResponse" output-channel="ResponseChannel"/>
<int:channel id="bridgeOutputChannel" />
<int:channel id="aggregatorInput" />
<int:channel id="aggregatorOutput" />
</beans>
Below is a snippet from my aggregator
public Message<?> populateResponse(Collection<Message<?>> message){
MessageBuilder<?> MsgBuilder =null;
MsgBuilder=MessageBuilder.withPayload(message.iterator().next().getPayload());
for (Message<?> message2 : message) {
if(null!=message2.getHeaders().get(Constants.RESPONSE1)){
MsgBuilder.setHeader(Constants.RESPONSE1, message2.getHeaders().get(Constants.RESPONSE1));
}
if(null!=message2.getHeaders().get(Constants.RESPONSE2)){
MsgBuilder.setHeader(Constants.RESPONSE2, message2.getHeaders().get(Constants.RESPONSE2));
}
}
return (Message<?>) MsgBuilder.build();
}
You should use apply-sequence="true" on that publish-subscribe-channel (which should be there by default) and don't use any correlation options on the aggregator - just rely on the default correlationKey header populated by that apply-sequence.
Having the correlation strategy based on the message id (your code), makes the aggregator to build new groups for each message, just because message id is always unique.
By the way, you don't need release strategy as well. The aggregator easily do that by the populate sequenceNumber header.
And I'm not sure that you need group-timeout too.
In other words what you need for your use-case is just rely on the out-of-the-box sequence details feature:
http://docs.spring.io/spring-integration/docs/4.3.9.RELEASE/reference/html/messaging-channels-section.html#channel-configuration-pubsubchannel
http://docs.spring.io/spring-integration/docs/4.3.9.RELEASE/reference/html/messaging-routing-chapter.html#aggregator

Aggregator is failing after upgrading to Spring-Integration 4

We are using Spring-Integration in our project. We are trying to migrate from spring-integration-core:jar:3.0.1.RELEASE to spring-integration-core:jar:4.3.2.RELEASE, java 8, spring 4. We are running into issues with aggregator. Weirdly, the aggregator's method is not called during the execution. The configuration is shown below:
<!-- Store the original payload in header for future purpose -->
<int:header-enricher default-overwrite="true" should-skip-nulls="true" >
<int:header name="${headerNames.originalPayload}" expression="payload" />
</int:header-enricher>
<!-- split the issues-->
<int-xml:xpath-splitter >
<int-xml:xpath-expression expression="//transaction"/>
</int-xml:xpath-splitter>
<int:service-activator ref="httpOutboundGatewayHandler" method="buildHttpOutboundGatewayRequest" />
<int:header-filter header-names="accept-encoding"/>
<int-http:outbound-gateway url-expression="headers.restResourceUrl"
http-method-expression="headers.httpMethod"
extract-request-payload="true"
expected-response-type="java.lang.String">
</int-http:outbound-gateway>
<int:service-activator ref="msgHandler" method="buildMessageFromExtSysResponse" />
<int-xml:xslt-transformer xsl-resource="${stylesheet.PQGetWorklist-Response-MoveSources}" />
</int:chain>
<int:aggregator input-channel="PQGetWorklist-Aggregate-sources" output-channel="PQGetWorklist-MoveSourcesUnderIssues"
ref="xmlAggregator" method="aggregateSources">
</int:aggregator>
In the above code, <int-http:outbound-gateway is getting executed, but the XmlAggregator.aggregateSources is not called for unknown reasons. I could see that the message is sent to on the channel PQGetWorklist-Aggregate-sources. But from there the aggregator's method aggregateSources is not called. As a result, we are getting No reply received within timeout. Remember the same configuration is working fine with spring-integration-core:jar:3.0.1.RELEASE. The problem is seen only when we upgrade it to spring-integration-core:jar:4.3.2.RELEASE .
Here is my XmlAggregator.java
public class XmlAggregator {
private static final Logger logger = Logger.getLogger(XmlAggregator.class);
public Message aggregateSources(List < Message > messages) throws DocumentException {
Document mainDom = XmlParserUtil.convertString2Document("<Results> </Results>");
Document splitMessageDom = XmlParserUtil.convertString2Document(messages.get(0).getPayload().toString());
Document IssuesDom = XmlParserUtil.convertString2Document("<Issues> </Issues>");
Document sourcesDom = XmlParserUtil.convertString2Document("<RetrievedSources> </RetrievedSources>");
if(messages.get(0).getHeaders().get("jobDesignerJobName").equals("PQIssueInquiry")){
//extract callerType node
Element callerType = XmlParserUtil.getXmlElements(XmlParserUtil.convertString2Document(messages.get(0).getPayload().toString()), "//callerType").get(0);
//add callerType to root node
mainDom.getRootElement().content().add(callerType);
}
//extract sort node
Element sort = XmlParserUtil.getXmlElements(XmlParserUtil.convertString2Document(messages.get(0).getPayload().toString()), "//sort").get(0);
//add sort to root node
mainDom.getRootElement().content().add(sort);
//get all the issues
List < Element > transactionElements = XmlParserUtil.getXmlElements(splitMessageDom, "//transaction");
for (Element issue: transactionElements) {
// add all the issues to the IssuesDom
IssuesDom.getRootElement().content().add(issue);
}
//add all the issues to the root node
XmlParserUtil.appendChild(mainDom, IssuesDom, null);
for (Message source: messages) {
Document sourcesTempDom = XmlParserUtil.convertString2Document(source.getPayload().toString());
Reader xml = new StringReader((String) source.getPayload());
SAXReader reader = new SAXReader();
Document document = reader.read(xml);
//get all the sources
List < Element > sourceElements = XmlParserUtil.getXmlElements(sourcesTempDom, "//sources");
for (Element sources: sourceElements) {
//add all the sources to sourcesDom
sourcesDom.getRootElement().content().add(sources);
}
}
// add all the sources to the root node
XmlParserUtil.appendChild(mainDom, sourcesDom, null);
MessageBuilder < ? > msgBuilder = MessageBuilder.withPayload(mainDom.asXML());
Message message = msgBuilder.build();
logger.debug("aggregateSources Results after aggregation " + mainDom.asXML());
return message;
}
}
Any thoughts?
Starting with version 4.2 the XPathMessageSplitter is based on the iterator functionality by default:
<xsd:attribute name="iterator" default="true">
<xsd:annotation>
<xsd:documentation>
The iterator mode: 'true' (default) to return an 'java.util.Iterator'
for splitting 'payload', 'false to return a 'java.util.List'.
Note: the 'list' contains transformed nodes whereas with the
'iterator' each node is transformed while iterating.
</xsd:documentation>
</xsd:annotation>
<xsd:simpleType>
<xsd:union memberTypes="xsd:boolean xsd:string" />
</xsd:simpleType>
</xsd:attribute>
Do not overhead the memory with the size interest and pull the data from the target source on demand.
The functionality can be turned off via iterator="false" option.

Unable to get Aggregator to work

I am trying to understand the Aggregator basics. Below is the use case I am trying to implement:
1) Read message (order details) from queue.
<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="http://www.example.org/orders">
<orderItem>
<isbn>12333454443</isbn>
<quantity>4</quantity>
</orderItem>
<orderItem>
<isbn>545656777</isbn>
<quantity>50</quantity>
</orderItem>
..
..
</order>
One order message will contain multiple orderItem. And we can expect hundreds of order messages in the queue.
2) End Result ::
a) Each orderitem should be written to a file.
b) 4 such files should be written to a unique folder.
To give an example, lets say we got two order messages - each containing three orderitem.
So we need to create 2 folders :
In "folder 1", there should be 4 files(1 orderitem in each file)
In "folder 2", there should be 2 files(1 orderitem in each file). Here for simplicity we assume no more order messages came and we can write after 5 mins.
Implementation:
I am able to read the message from the queue (websphere MQ) and unmarshall the message successfully.
Used splitter to split the message based on orderitem count.
Used Aggregator to group the message in size of 4.
I unable to get the aggregator to work as per my understanding.
I push one order when 4 orderitem, the message is getting aggregated correctly.
I push one order with 5 orderitem, the first 4 is getting aggregated but the last one is sent to discard channel. This is expected as the MessageGroup is released so the last message is discarded.
I push two orders each containing 2 orderitem. The last 2 orderitem are sent to discard channel.
The correlation strategy is hardcoded (OrderAggregator.java) but the above case should have worked.
Need pointers on how to implement this use case where I can group them in 4 and write to unique folders.
Please note that the orderitem are all independent book orders and have no relation amongst them.
Below is the configuration.
spring-bean.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans">
<int:channel id="mqInbound"/>
<int:channel id="item"/>
<int:channel id="itemList"/>
<int:channel id="aggregatorDiscardChannel"/>
<int-jms:message-driven-channel-adapter id="jmsIn"
channel="mqInbound"
destination="requestQueue"
message- converter="orderMessageConverter"/>
<int:splitter input-channel="mqInbound" output-channel="item" expression="payload.orderItem"/>
<int:chain id="aggregateList" input-channel="item" output-channel="itemList" >
<int:header-enricher>
<int:header name="sequenceSize" expression="4" overwrite="true"/>
</int:header-enricher>
<int:aggregator correlation-strategy="orderAggregator" correlation-strategy-method="groupOrders" discard-channel="aggregatorDiscardChannel" />
</int:chain>
<int:service-activator input-channel="itemList" ref="displayAggregatedList" method="display"/>
<int:service-activator input-channel="aggregatorDiscardChannel" ref="displayAggregatedList" method="displayDiscarded"/>
<bean id="orderAggregator" class="com.samples.Aggregator.OrderAggregator"/>
<bean id="displayAggregatedList" class="com.samples.Aggregator.DisplayAggregatedList"/>
...
....
</beans>
OrderAggregator.java
public class OrderAggregator {
#Aggregator
public List<OrderItemType> sendList(List<OrderItemType> orderItemTypeList) {
return orderItemTypeList;
}
#CorrelationStrategy
public String groupOrders( OrderItemType orderItemType) {
return "items";
}
}
DisplayAggregatedList.java
public class DisplayAggregatedList {
public void display(List <OrderItemType> orderItemTypeList) {
System.out.println("######## Display Aggregated ##############");
for(OrderItemType oit : orderItemTypeList) {
System.out.println("### Isbn :" + oit.getIsbn() + ":: Quantity :" + oit.getQuantity());
}
}
public void displayDiscarded(Message<?> message) {
System.out.println("######## Display Discarded ##############" + message);
}
}
What you need is called expire-groups-upon-completion:
When set to true (default false), completed groups are removed from the message store, allowing subsequent messages with the same correlation to form a new group. The default behavior is to send messages with the same correlation as a completed group to the discard-channel.
If you need to release uncompleted groups anyway (2 orders left, for example), consider to use group-timeout: http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#agg-and-group-to
Please, use expire-groups-upon-completion="true" and consider to use MessageCountReleaseStrategy` for release-strategy – Artem Bilan

unable to set correlation id from a pojo

I am able to set correlation id using <int:correlation-id /> tag but I want to set it from a POJO. So I tried
<int:header-enricher id="he" input-channel="in" output-channel="test">
<int:correlation-id ref="heSimple" method="resolve" />
</int:header-enricher>
this doesnt works. CorrelationId is not modified. However if i change it to
<int:header-enricher id="he" input-channel="in" output-channel="test">
<int:header name="foo" ref="heSimple" method="resolve" />
</int:header-enricher>
this works fine. So basically I can modify any other header key other than correlation id from this pojo. I also tried
<int:header-enricher id="he" input-channel="in" output-channel="test">
<int:header name="correlationId" ref="heSimple" method="resolve" />
</int:header-enricher>
Interestingly that doesnt works either.
EDIT
private MessageChannel in;
in.send(MessageBuilder.withPayload("a1").build());
in.send(MessageBuilder.withPayload("b1").setCorrelationId("1").build());
somehow the problem seems to be related to setting the correlation id explicitly while sending the message. So in this case a1 message does ends up using the header enricher to set the new corelation id but the message b2 continue to use 1 as the correlation id.
Looking at debug logs it seems like a1 message is sent to transformer and invokes the header enricher but for message b2 the header enricher is not invoked
[11:44:23:052] [main] DEBUG o.s.i.t.MessageTransformingHandler - org.springframework.integration.transformer.MessageTransformingHandler#0 received message: [Payload=a1][Headers={timestamp=1399045463050, id=d20d68e1-ea7a-df20-4faf-d1864855851c}]
in hesimple for [Payload=a1][Headers={timestamp=1399045463050, id=d20d68e1-ea7a-df20-4faf-d1864855851c}]
[11:44:23:055] [main] DEBUG o.s.i.t.MessageTransformingHandler - handler 'org.springframework.integration.transformer.MessageTransformingHandler#0' sending reply Message: [Payload=a1][Headers={timestamp=1399045463055, id=3b72891c-aaa5-3570-7715-e020529b5f83, correlationId=test}]
for message b1 header enricher is skipped
[11:44:23:056] [main] DEBUG o.s.i.t.MessageTransformingHandler - org.springframework.integration.transformer.MessageTransformingHandler#0 received message: [Payload=b1][Headers={timestamp=1399045463055, id=9aa2bb7e-9c6c-66e5-0d64-3afea4a57ee4, correlationId=1}]
[11:44:23:056] [main] DEBUG o.s.i.t.MessageTransformingHandler - handler 'org.springframework.integration.transformer.MessageTransformingHandler#0' sending reply Message: [Payload=b1][Headers={timestamp=1399045463056, id=c8d54223-e506-727d-6bff-703755017403, correlationId=1}]
there is not much in the header enricher
public static class HESimple {
public String resolve(Message<String> message){
System.out.println("in hesimple for " + message);
return "test";
}
}
I just tested the first case and it works just fine for me.
What version of Spring Integration are you using?
What is the signature of the method?
There is nothing "special" about that header that would cause the different behavior you describe. I suggest you turn on DEBUG logging to figure out what's happening.
EDIT:
In order to overwrite existing values, you need to set overwrite="true".

Resources