We have a requirement where I need the same message(payload) to be processed in two different channels. We are under the impression that using PubliSHSubscribe Channel will help us deal with that by making a copy of message to both the channels. However we figured that each channel was getting executed one after the other and if e make any changes in the payload in one channel, its effecting the payload of other channel as well.
#Bean
public IntegrationFlow bean1() {
return IntegrationFlows
.from("Channel1")
.handle(MyMessage.class, (payload, header) -> obj1.method1(payload))
.channel(MessageChannels.publishSubscribe("subscribableChannel").get())
.get();
}
#Bean
public IntegrationFlow bean21() {
return IntegrationFlows
.from("subscribableChannel")
.handle(MyMessage.class, (payload, header) -> obj2.method2(payload,header))
.channel("nullChannel")
.get();
}
#Bean
public IntegrationFlow bean22() {
return IntegrationFlows
.from("subscribableChannel")
.handle(MyMessage.class, (payload, header) -> obj3.method3(payload))
.channel("nullChannel")
.get();
}
IN the above example, if i make changes to payload in bean21, its effecting the input payload passed to bean 22.
My requirement is to pass the same payload to bean21 and bean22 and to execute them parallely? Can you please advise how to accomplish that?
That's correct. Spring Integration is just Java and there is no any magic in copying payload between different messages. It is really just the same object in the memory. Now imagine you have a pure Java and would like to call two different method with the same Foo object. And then you modify that object in one method. What happens in another one? Right, it will see a modifications on the object.
To achieve your goal with definitely copying object to a new instance, you have to ensure it yourself. For example implement Cloneable interface on your class or provide copy constructor or any other possible solution to create a new object.
In the beginning of the flow of one of the subscriber you should perform that clone operation and you will have a new object without impacting another subscriber.
See more info in this JIRA: https://jira.spring.io/browse/INT-2979
Related
I am trying to learn about how to build IntegrationFlows as units, and join them up.
I set up a very simple processing integration flow:
IntegrationFlow processingFlow = f -> f
.<String>handle((p, h) -> process(p))
.log();
flowContext.registration(processingFlow)
.id("testProcessing")
.autoStartup(false)
.register();
Processing is very simple:
public String process(String process) {
return process + " has been processed";
}
Then I compose a flow from a source, using .gateway() to join the source to the processing:
MessageChannel beginningChannel = MessageChannels.direct("beginning").get();
StandardIntegrationFlow composedFlow = IntegrationFlows
.from(beginningChannel)
.gateway(processingFlow)
.log()
.get();
flowContext.registration(composedFlow)
.id("testComposed")
.autoStartup(false)
.addBean(processingFlow)
.register();
Then I start the flow and send a couple of messages:
composedFlow.start();
beginningChannel.send(MessageBuilder.withPayload(new String("first string")).build());
beginningChannel.send(MessageBuilder.withPayload(new String("second string")).build());
The logging handler confirms the handle method has been called for the first message, but the main thread then sits idle, and the second message is never processed.
Is this not the correct way to compose integration flow from building blocks? Doing so with channels requires registering the channel as a bean, and I'm trying to do all this dynamically.
It must be logAndReply() in the processingFlow. See their JavaDocs for difference. The log() in the end of flow makes it one-way. That’s why you are blocked since gateway waits for reply, but there is no one according your current flow definition. Unfortunately we can’t determine that from the framework level: there might be cases when you indeed don’t return according some routing or filtering logic. The gateway can be configured with a reply timeout. By default it is an infinite.
Error message: Caused by: java.lang.IllegalStateException: Null correlation not allowed. Maybe the CorrelationStrategy is failing?
My implementation,
#Bean
public IntegrationFlow start() {
return IntegrationFlows
.from("getOrders")
.split()
.publishSubscribeChannel(c -> c.subscribe(s -> s.channel(q -> q.queue(1))
.<Order, Message<?>>transform(p -> MessageBuilder.withPayload(new Item(p.getItems())).setHeader(ORDERID, p.getOrderId()).build())
.split(Item.class, Item::getItems)
.transform() // let's assume, an object created for each item, let's say ItemProperty to the object.
// Transform returns message; MessageBuilder.withPayload(createItemProperty(getItemName, getItemId)).build();
.aggregate() // so, here aggregate method needs to aggregate ItemProperties.
.handle() // handler gets List<ItemProperty> as an input.
))
.get();
}
Both splitters works fine. I've also tested the transformer after the second splitter, works fine. But, when it comes to aggregate it is failing. What I am missing here?
You are missing the fact that transformer is that type of endpoint which deal with the whole message as is. And if you create a message by yourself, it doesn't modify it.
So, with your MessageBuilder.withPayload(createItemProperty(getItemName, getItemId)).build(); you just miss important sequence details headers after splitter. Therefore an aggregator after that doesn't know what to do with your message since you configure it for default correlation strategies but you don't provide respective headers in the message.
Technically I don't see a reason to create a message over there manually: the simple return createItemProperty(getItemName, getItemId); should be enough for you. And the framework will take about message creation on your behalf with respective request message headers copying.
If you really still think that you need to create a message yourself in that transform, then you need to consider to copyHeaders() on that MessageBuilder from the request message to carry required sequence details headers.
I have been working on a "paved road" for setting up asynchronous messaging between two micro services using AMQP. We want to promote the use of separate domain objects for each service, which means that each service must define their own copy of any objects passed across the queue.
We are using Jackson2JsonMessageConverter on both the producer and the consumer side and we are using the Java DSL to wire the flows to/from the queues.
I am sure there is a way to do this, but it is escaping me: I want the consumer side to ignore the __TypeID__ header that is passed from the producer, as the consumer may have a different representation of that event (and it will likely be in in a different java package).
It appears there was work done such that if using the annotation #RabbitListener, an inferredArgumentTypeargument is derived and will override the header information. This is exactly what I would like to do, but I would like to use the Java DSL to do it. I have not yet found a clean way in which to do this and maybe I am just missing something obvious. It seems it would be fairly straight forward to derive the type when using the following DSL:
return IntegrationFlows
.from(
Amqp.inboundAdapter(factory, queueRemoteTaskStatus())
.concurrentConsumers(10)
.errorHandler(errorHandler)
.messageConverter(messageConverter)
)
.channel(channelRemoteTaskStatusIn())
.handle(listener, "handleRemoteTaskStatus")
.get();
However, this results in a ClassNotFound exception. The only way I have found to get around this, so far, is to set a custom message converter, which requires explicit definition of the type.
public class ForcedTypeJsonMessageConverter extends Jackson2JsonMessageConverter {
ForcedTypeJsonMessageConverter(final Class<?> forcedType) {
setClassMapper(new ClassMapper() {
#Override
public void fromClass(Class<?> clazz, MessageProperties properties) {
//this class is only used for inbound marshalling.
}
#Override
public Class<?> toClass(MessageProperties properties) {
return forcedType;
}
});
}
}
I would really like this to be derived, so the developer does not have to really deal with this.
Is there an easier way to do this?
The simplest way is to configure the Jackson converter's DefaultJackson2JavaTypeMapper with TypeIdMapping (setIdClassMapping()).
On the sending system, map foo:com.one.Foo and on the receiving system map foo:com.two.Foo.
Then, the __TypeId__ header gets foo and the receiving system will map it to its representation of a Foo.
EDIT
Another option would be to add an afterReceiveMessagePostProcessor to the inbound channel adapter's listener container - it could change the __TypeId__ header.
I am trying to transfer remote files from an FTP repo to a local repo. At the moment it works in terms of the initial transfer and if the local file is deleted but I would like it to pick up on remote file changes from the last modified timestamp. I have read around trying to create a custom filter but can't find much information on doing this via Java DSL.
#Bean
public IntegrationFlow ftpInboundFlow(){
return IntegrationFlows
.from(s -> s
.ftp(this.ftpSessionFactory())
.preserveTimestamp(true)
.remoteDirectory(ftpData.getRemoteDirectory())
.localDirectory(new File(ftpData.getLocalDirectory())),
e -> e.id("ftpInboundAdapter").autoStartup(true))
.channel(MessageChannels.publishSubscribe())
.get();
}
It has been fixed only recently: https://jira.spring.io/browse/INT-4232.
Meanwhile you don't have choice unless delete local files after processing.
You have to use FtpPersistentAcceptOnceFileListFilter any way, because of: https://jira.spring.io/browse/INT-4115.
There is nothing from Java DSL perspective.
UPDATE
can you point me towards how to delete local files via Java DSL
The FtpInboundFileSynchronizingMessageSource produces a message already for local file as a payload. In addition there are some headers like:
.setHeader(FileHeaders.RELATIVE_PATH, file.getAbsolutePath()
.replaceFirst(Matcher.quoteReplacement(this.directory.getAbsolutePath() + File.separator),
""))
.setHeader(FileHeaders.FILENAME, file.getName())
.setHeader(FileHeaders.ORIGINAL_FILE, file)
When you're done with the file downstream already you can delete it via regular File.delete() operation. That can be done for example using ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public Advice deleteFileAdvice() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnSuccessExpressionString("headers[file_originalFile].delete()");
return advice;
}
...
.<String>handle((p, h) -> ..., e -> e.advice(deleteFileAdvice()))
Given a MessageChannel or Message object, how is it possible to get from one of them the name of the underlying JMS Queue which the message was received on ?
Here is the scenario:
Several jms:message-driven-channel-adapter instances are defined in the xml. The destination-name of each adapter uses SEL to receive from different queues. This SEL is dynamic, and is not possible to know these queue names ahead of time. All channel adapters output to the same internal Spring Integration channel.
I want to add the actual underlying queue name which the message was received on to the header of the message.
The idea is to setup a ChannelInterceptor for either the channel-adapters or the internal channel. The postReceive() method has both the Message and MessageChannel as arguments. Using either of these, is it possible to get the name of the underlying Queue name which the message came in on?
Thanks
Looks like you need to extend a bit DefaultJmsHeaderMapper:
class DestinationJmsHeaderMapper extends DefaultJmsHeaderMapper {
public Map<String, Object> toHeaders(javax.jms.Message jmsMessage) {
Map<String, Object> headers = super.toHeaders(jmsMessage);
headers.put("JMS_DESTINATION", ((Queue) jmsMessage.getJMSDestination()).getQueueName());
}
}
And inject it to your <jms:message-driven-channel-adapter>s
This is how we did it:
<int:header-enricher>
<int:header name="JMS_DESTINATION" expression="payload.JMSDestination.queueName"/>
</int:header-enricher>
It requires extract-payload="false" in your <jms:message-driven-channel-adapter>.
P.S. The answer of Artem is missing the return statement (I do not have enough reputations to comment).