I'm having issues with the following Message Driver Channel Adapter
#Bean
public IntegrationFlow jmsInboundFlow() {
return IntegrationFlows.from(Jms.messageDriverChannelAdapter(this.jmsConnectionFactory)
.outputChannel(MessageChannels.queue("inbound").get())
.destination("test"))
.get();
}
#Bean
public IntegrationFlow channelFlow() {
return IntegrationFlows.from("inbound")
.transform("hello "::concat)
.handle(System.out::println)
.get();
}
I'm getting an error about "Dispatcher has no subscribers for channel". What's the preferred configuration for sending the message payload to another integration flow?
With that Java DSL channel auto-creation you should be careful. For example that .outputChannel(MessageChannels.queue("inbound").get()) doesn't populate a MessageChannel bean to the bean factory. But from other side IntegrationFlows.from("inbound") does that.
To fix your issue I suggest to extract #Bean for your inbound channel, or just rely on the DSL:
return IntegrationFlows.from(Jms.messageDriverChannelAdapter(this.jmsConnectionFactory)
.destination("test"))
.channel(MessageChannels.queue("inbound").get())
.get();
Feel free to raise a GH-issue to fix JavaDocs on that .outputChannel() or remove it alltogether, since it is confused.
Related
How can I setup a reactive flow using DSL for the following steps:
Receive an SQS Message using SqsMessageDrivenChannelAdapter
Validate the Json message [JsonSchemaValidator class with validate method]
Transform the json to objects
Pass the objects to a service activator (BusinessService : business logic, state machine)
Persist the Objects R2DBC outbound adapter
I was looking at this : https://github.com/spring-projects/spring-integration/blob/master/spring-integration-core/src/test/java/org/springframework/integration/dsl/reactivestreams/ReactiveStreamsTests.java
In the above example, there are dedicated flows created that return a Publisher and in the tests the Publishers are subscribed. However, my flow will be triggered when SqsMessageDrivenChannelAdapter brings in a message into a channel.
How to achieve a reactive flow configuration, for the scenario above steps 1 to 5?
Update : Sample code added
#Bean
public IntegrationFlow importFlow() {
IntegrationFlows.from(sqsInboundChannel())
.handle((payload, messageHeaders) -> jsonSchemaValidator.validate(payload.toString()))
.transform(Transformers.fromJson(Entity.class))
.handle((payload, messageHeaders) ->businessService.process((Entity) payload))
.handle(
Jpa.outboundAdapter(this.entityManagerFactory)
.entityClass(Entity)
.persistMode(PersistMode.PERSIST),
ConsumerEndpointSpec::transactional)
.get();
}
#Bean
public MessageProducer sqsMessageDrivenChannelAdapter() {
SqsMessageDrivenChannelAdapter sqsMessageDrivenChannelAdapter =
new SqsMessageDrivenChannelAdapter(asyncSqsClient, queueName);
sqsMessageDrivenChannelAdapter.setAutoStartup(true);
sqsMessageDrivenChannelAdapter.setOutputChannel(sqsInboundChannel());
return sqsMessageDrivenChannelAdapter;
}
#Bean
public MessageChannel sqsInboundChannel() {
return MessageChannels.flux().get();
}
Update 2 : Moved JPA to a diff thread using executor channel
#Bean
public IntegrationFlow importFlow() {
IntegrationFlows.from(sqsInboundChannel())
.handle((payload, messageHeaders) -> jsonSchemaValidator.validate(payload.toString()))
.transform(Transformers.fromJson(Entity.class))
.handle((payload, messageHeaders) ->businessService.process((Entity) payload))
.channel(persistChannel())
.handle(
Jpa.outboundAdapter(this.entityManagerFactory)
.entityClass(Entity)
.persistMode(PersistMode.PERSIST),
ConsumerEndpointSpec::transactional)
.get();
}
#Bean
public MessageProducer sqsMessageDrivenChannelAdapter() {
SqsMessageDrivenChannelAdapter sqsMessageDrivenChannelAdapter =
new SqsMessageDrivenChannelAdapter(asyncSqsClient, queueName);
sqsMessageDrivenChannelAdapter.setAutoStartup(true);
sqsMessageDrivenChannelAdapter.setOutputChannel(sqsInboundChannel());
return sqsMessageDrivenChannelAdapter;
}
#Bean
public MessageChannel sqsInboundChannel() {
return MessageChannels.flux().get();
}
#Bean
public MessageChannel persistChannel() {
return MessageChannels.executor(Executors.newCachedThreadPool()).get();
}
You probably need to make yourself more familiar with what we have so far for Reactive Streams in Spring Integration: https://docs.spring.io/spring-integration/docs/current/reference/html/reactive-streams.html#reactive-streams
The sample you show with that test class is fully not relevant to your use case. In that test we try to cover some API we expose in Spring Integration, kinda unit tests. It has nothing to do with the whole flow.
Your use-case is really just a full black box flow starting with SQS listener and ending in the R2DBC. Therefore there is no point in your flow to try to convert part of it into the Publisher and then bring it back to another part of the flow: you are not going to track some how and subscribe to that Publisher yourself.
You may consider to place a FluxMessageChannel in between endpoints in your flow, but it still does not make sense for your use-case. It won't be fully reactive as you expect just because a org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer is not blocking on the consumer thread to be ready for a back-pressure from downstream.
The only really reactive part of your flow is that R2DBC outbound channel adapter, but probably it does not bring you too much value because the source of data is not reactive.
As I said: you can try to place a channel(channels -> channels.flux()) just after an SqsMessageDrivenChannelAdapter definition to start a reactive flow from that point. At the same time you should try to set a maxNumberOfMessages to 1 to try to make it waiting for a free space in before pulling the next mesasge from SQS.
In this link, the author use java domain specific language to build connection from spring integration channel to activeMQ. If i don't want to use java dsl and I just want to use general java method. how should I do?
#Bean
public IntegrationFlow outboundFlow(ActiveMQConnectionFactory connectionFactory) {
return IntegrationFlows
.from(requests())
.handle(Jms.outboundAdapter(connectionFactory).destination("requests"))
.get();
}
https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/java/org/springframework/batch/sample/remotepartitioning/aggregating/WorkerConfiguration.java
The Jms.outboundAdapter(connectionFactory) is a syntax sugar wrapper around JmsSendingMessageHandler.
The .handle() is an equivalent of the #ServiceActivator.
So, what you need with plain Java is like this:
#Bean
#ServiceActivator(inputChannel="requests")
public MessageHandler jmsMessageHandler((ActiveMQConnectionFactory connectionFactory) {
JmsSendingMessageHandler handler = new JmsSendingMessageHandler(new JmsTemplate(connectionFactory));
handler.setDestinationName("requests");
return handler;
}
See some explanation in the Docs: https://docs.spring.io/spring-integration/docs/current/reference/html/overview.html#programming-tips
And more about JmsSendingMessageHandler: https://docs.spring.io/spring-integration/docs/current/reference/html/jms.html#jms-outbound-channel-adapter
I have following spring integration flow:
#Bean
public IntegrationFlow innerInFlow(#Value("${jms.concurrency:10}") String concurrency) {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(
Jms.container(connectionFactory, innerQueue)
.concurrency(concurrency)
.taskExecutor(taskExecutor()).get())
.extractPayload(true))
.transform(Transformers.deserializer())
.route(eventRouter())
.get();
}
And after routing
#Bean
public IntegrationFlow findPersonClienFlow(FindClientHandler findClientHandler) {
return IntegrationFlows.from(findPersonClienChannel())
.transform(findClientHandler, "queryToFindClientRequest")
.handle(Jms.outboundGateway(connectionFactory).requestDestination(cifRequestQueue)
.replyDestination(cifResponseQueue).get())
.get();
}
}
In the Jms.outboundGateway I have org.springframework.integration.MessageTimeoutException and I cant understand how I can handle this error?
Thank you.
I believe the MessageTimeoutException is there because the other side doesn't send you a reply into the cifResponseQueue.
You can configure there a receiveTimeout(), but it is 5 secs by default anyway.
Also you can configure a RequestHandlerRetryAdvice or ExpressionEvaluatingRequestHandlerAdvice on this Jms.outboundGateway() to really handle this exception some specific way, using an advice(...) of the ConsumerEndpointSpec.
See Docs on the matter: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints-chapter.html#message-handler-advice-chain
UPDATE
Also as Gary pointed you can catch an exception from the Jms.messageDrivenChannelAdapter() level using its errorChannel() and some flow subscribed to this channel.
I am using java dsl to configure sfp outbound flow.
Gateway:
#MessagingGateway
public interface SftpGateway {
#Gateway(requestChannel = "sftp-channel")
void sendFiles(List<Message> messages);
}
Config:
#Bean
public IntegrationFlow sftpFlow(DefaultSftpSessionFactory sftpSessionFactory) {
return IntegrationFlows
.from("sftp-channel")
.split()
.handle(Sftp.outboundAdapter(sftpSessionFactory, FileExistsMode.REPLACE)
.useTemporaryFileName(false)
.remoteDirectory(REMOTE_DIR_TO_CREATE).autoCreateDirectory(true)).get();
}
#Bean
public DefaultSftpSessionFactory sftpSessionFactory() {
...
}
How can i configure flow to make my gateway reply with Messages that were failed?
In other words i want my gateway to be able to return list of messages which were failed, not void.
I marked gateway with
#MessagingGateway(errorChannel = "errorChannel")
and wrote error channel
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("errorChannel").handle(new GenericHandler<MessagingException>() {
public Message handle(MessagingException payload, Map headers) {
System.out.println(payload.getFailedMessage().getHeaders());
return payload.getFailedMessage();
}
})
.get();
}
#Bean
public MessageChannel errorChannel() {
return MessageChannels.direct().get();
}
and in case of some errors(i.e. no connection to SFTP) i get only one error (payload of first message in list).
Where should i put Advice to aggregate all messages?
This is not the question to Spring Integration Java DSL.
This is mostly a design and architecture task.
Currently you don't have any choice because you use Sftp.outboundAdapter() which is one-way, therefore without any reply. And your SftpGateway is ready for that behavior with the void return type.
If you have a downstream errorr, you can only throw them or catch and send to some error-channel.
According to your request of:
i want my gateway to be able to return list of messages which were failed, not void.
I'd say it depends. Actually it is just return from your gateway. So, if you return an empty list into gateway that may mean that there is no errors.
Since Java doesn't provide multi-return capabilities we don't have choice unless do something in our stream which builds that single message to return. As we decided list of failed messages.
Since you have there .split(), you should look into .aggregate() to build a single reply.
Aggregator correlates with the Splitter enough easy, via default applySequence = true.
To send to aggregator I'd suggest to take a look into ExpressionEvaluatingRequestHandlerAdvice on the Sftp.outboundAdapter() endpoint (second param of the .handle()). With that you should send both good and bad messages to the same .aggregate() flow. Than you can iterate a result list to clean up it from the good result. The result after that can be send to the SftpGateway using replyChannel header.
I understand that it sounds a bit complicated, but what you want doesn't exist out-of-the-box. Need to think and play yourself to figure out what can be reached.
I want to consume messages from a Queue channel using java-dsl, but Integrations.from doesn't have a signature allowing me to specify a Poller.
How can I achieve this?
Ex.:
#Bean
IntegrationFlow flow() {
return IntegrationFlows.from(this.channel())
.handle(...)
.get();
}
#Bean
MessageChannel channel() {
return MessageChannels.queue().get();
}
Well, actually it is an endpoint responsibility to provide poller properties.
If you are familiar with an XML configuration you should remember that to poll from <queue> we should configure <poller> sub-element for <service-activator> etc.
The same approach is applied in Java DSL as well. The next endpoint definition should be with desired poller:
IntegrationFlows.from(this.channel())
.handle(..., e -> e.poller(Pollers...))
.get();
I've had trouble for some reason setting a poller on the endpoint definition as Artem described - for some reason it gets ignored. You can always set a default poller. This has worked for me:
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedRate(500).get();
}