I am using
spring 4.2.4.RELEASE and
spring-integration-java-dsl:1.1.2.RELEASE
I have Jms.inboundGateway and some transformer. When transformer fails to transform a message, inbound gateway retries again and again. But I want to stop the entire flow in case of exception. Is it possible to do?
This is my flow configuration:
#Bean
public IntegrationFlow nonStop {
return IntegrationFlows
.from(Jms.inboundGateway(emsConnectionFactory)
.destination(myDestination)
.configureListenerContainer(spec -> spec
.sessionTransacted(true)
.subscriptionDurable(true)
.durableSubscriptionName(durableSubscriptionName)
.errorHandler((ErrorHandler) t -> {
t.printStackTrace();
throw new RuntimeException(t);
}))
.errorChannel(errorChannel)
.autoStartup(true)
.id(myNonStoppableFlow))
.filter(...)
.transform(...)
.handle(Jms.outboundAdapter(emsConnectionFactory)
.destination(myOnotherDestination))
.get();
}
One interesting notice. When errorHandler swallows an exception inbound gateway retires without any delay. When it throws Runtime exception there is delay about 5 seconds (which is not configured anywhere).
That retry in JMS is called like redelivery: http://www.javaworld.com/article/2074123/java-web-development/transaction-and-redelivery-in-jms.html.
So, any downstream exception makes for your message rollback and redelivery, which can be configured on Broker for the destination.
Actually if your .errorChannel(errorChannel) flow doesn't re-throw the exception, it will be treated as a successful handling and as commit.
From other side you should reconsider if you really need there a Jms.inboundGateway(). Because this one requires a reply, which doesn't look possible with your Jms.outboundAdapter() in the end.
Related
Expect the below configuration to rollback the JMS message to the backout queue, but it does not. The failed message is reaching the application's custom ErrorChannel. Please suggest why is the rollback not happening. Thanks
#Bean
public DefaultMessageListenerContainer mqAdapterListenerContainer(final ConnectionFactory mqCachingConnectionFactory) {
var defaultMessageListenerContainer = new DefaultMessageListenerContainer();
defaultMessageListenerContainer.setConnectionFactory(mqCachingConnectionFactory);
defaultMessageListenerContainer.setSessionTransacted(true);
return defaultMessageListenerContainer;
}
return IntegrationFlows.from(Jms.messageDrivenChannelAdapter(mqAdapterListenerContainer)
.errorChannel(appErrorChannel))
.enrichHeaders(headerEnricherSpec -> headerEnricherSpec.errorChannel(appErrorChannel, true))
That JMS transaction can rollback only if listener throws exception, but since you do error handling via that appErrorChannel, there is no exception thrown to that JMS container.
So, you may still have a .errorChannel(appErrorChannel) and that error handling, but you need to re-throw some exception to let JMS container catch it rollback transaction respectively.
I have to listen a queue using spring integration flow and intgeration sqs. Once message is received from queue it should trigger a integration flow. Below is the things which I am trying but everythings fine in but afater receiving test it is not triggering any Integration flow. Please let me know where I am doing wrong:
UPDATED as per comment from Artem
Adapter for SQS.
#Bean
public MessageProducerSupport sqsMessageDrivenChannelAdapter() {
SqsMessageDrivenChannelAdapter adapter = new SqsMessageDrivenChannelAdapter(amazonSQSAsync, "Main");
adapter.setOutputChannel(inputChannel());
adapter.setAutoStartup(true);
adapter.setMessageDeletionPolicy(SqsMessageDeletionPolicy.NEVER);
adapter.setMaxNumberOfMessages(10);
adapter.setVisibilityTimeout(5);
return adapter;
}
Queue configured:
#Bean
public MessageChannel inputChannel() {
return new DirectChannel();
}
Now the main integration flow trigger point:
#Bean
public IntegrationFlow inbound() {
return IntegrationFlows.from("inputChannel").transform(i -> "TEST_FLOW").get();
}
}
Appreciate any type of help.
The sqsMessageDrivenChannelAdapter() must be declared as a #Bean
The inbound() must be declared as a #Bean
This one fully does not make sense IntegrationFlows.from(MessageChannels.queue()). What is the point to start the flow from anonymous channel? Who and how is going to produce messages to that channel?
Make yourself familiar with different channels: https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#channel-implementations
Pay attention that QueueChannel must be consumed via polling endpoint.
Right, there is a default poller auto-configured by Spring Boot, but it is based on a single thread in the TaskScheduler and has a polling period as 10 millis.
I wouldn't recommend to hand off SQS messages to the QueueChannel: when consumer fails, you lose the data. It is better to process those messages in the consumer thread.
Otherwise your intention is not clear in the provided code.
Can you, please, share with us what error you get or anything else?
You also can turn on DEBUG logging level for org.springframework.integration to see how your messages are processed.
I have a process where I need to upload file to Sftp server asynchronously. So after exploring more about Async in gateways I found that I need to have error channel defined in #MessagingGateway parameter then handler to handle the Exception propagated to error channel but I felt handling this way is complicated for me, as I will have to update Pojo field and persist into DB depending on the file upload, success or failure.
So I thought of having a custom method annotated with #Async and call the gateway method. Also surround gateway method with try block and catch any exception occurred in the downstream
Code Sample:
#Async
void upload(Resource file, FileStatus fileStatus){
try{
uploadGateway.upload(file,fileStatus.getFilePath(),fileStatus.getFileName());
}catch(RuntimeException e){
fileStatus.setUploadStatus("Failed");
//save into db
}
}
Upload Gateway without error channel so that error can be sent back to caller
#MessagingGateway
public interface UploadGateway {
#Gateway(requestChannel = "input.channel")
void upload(#Payload Resource file, #Header("path") String path, #Header("name") String fileName);
}
Handler:
#Bean
public IntegrationFlow uploadDocument() {
return IntegrationFlows.from("input.channel")
.log(LoggingHandler.Level.WARN)
.handle(Sftp.outboundAdapter(sftpSessionFactory(), FileExistsMode.FAIL)
.autoCreateDirectory(true)
.remoteDirectoryExpression("headers['path']")
.fileNameExpression("headers['name']"))
.get();
}
Question:
What will be the consequences if I'm handling error this way? Is this the right way to handle any error occurred in downstream flow?
Since #MessagingGateway is like an RPC in messaging, it is fully OK to catch an exception on its method call like that. Since you make your flow fully sync, it works like typical Java exceptions sub-system.
Your concern about async error handling with errorChannel really makes sense since it is similar in complexity with standard Java async method handling and its errors processing.
On the other hand it is really commended to handle errors downstream via errorChannel if that is going to be some complex logic in some other flow. Plus you are going to return back some compensation message.
However in the end of day the choice is yours: there is no drawbacks to handle errors yourself.
See Error Handling chapter for more food to think on.
How to route the flow from the channel myChannel to the error channel myErrorChannel if there occurs an error in the Http.outboundGateway call?
#Bean
private IntegrationFlow myChannel() {
return f -> f
.handle(Http.outboundGateway("http://localhost:8080/greeting")
...
.expectedResponseType(String.class));
}
#Bean
private IntegrationFlow myErrorChannel() {
return f -> f
...
}
In the error handler I will wrap the error message inside my custom JSON and I will send that as a part of the normal flow back to the source system.
Is this a good way to handle errors in the Spring Integration Java DSL?
You can use an ExpressionEvaluatingRequestHandlerAdvice with its returnFailureExpressionResult = true and use it in the second argument of the .handle(..., e -> e.advice(...)).
You configure that advice for the onFailureExpression to be able to return something meaningful. If you still think that you need to send to the channel and get reply back, then you need to have a #MessagingGateway and use it in that onFailureExpression to send and receive. The normal failureChannel configuration in the ExpressionEvaluatingRequestHandlerAdvice doesn't expect reply.
Another approach can be done using the same #MessagingGateway, but in front of that myChannel IntegrationFlow. Then you can configure that gateway for the errorChannel and here a reply from the error flow is expected.
This question is more of a design question than a real problem. Given following basic flow:
#Bean
public DirectChannel getFileToSftpChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow sftpOutboundFlow() {
return IntegrationFlows.from(getFileToSftpChannel())
.handle(Sftp.outboundAdapter(this.sftpSessionFactory)
.useTemporaryFileName(false)
.remoteDirectory("test")).get();
}
#Bean
public IntegrationFlow filePollingInboundFlow() {
return from(s -> s.file(new File("path")).patternFilter("*.ext"),
e -> e.poller(fixedDelay(60, SECONDS).channel(getFileToSftpChannel()).get();
}
There is an inbound file polling flow which publishes messages via a DirectChannel to an outbound SFTP flow uploading the file.
After the entire flow finishes, I want to execute a "success" action: move the original file (locally) to an archive folder.
Using the DirectChannel, I understand that the upload will happen in the same thread as the file polling.
In other words, the file poller blocks untill the upload completes (or an error message is returned which is then pushed to the error channel).
Knowing this, I want to place the 'success' action (= moving the original file) on the inbound flow. Things I already know about and don't want to use:
Another 'handle' on the sftpOutbound. Reason: moving the file is tied to the inboud flow not the outbound flow. For ex. if I would introduce another, 2nd, producer later on (eg. a JMS inbound flow) publishing to the same channel, there would be no 'file' to be moved.
Adding an interceptor on the DirectChannel and use the 'afterSendCompletion'. Reason: same as above, I want to logic to be tied to the inbound flow
Add transaction semantics on the inbound flow and react on 'commit'. Reason: as all of this is non transactional (file system/SFTP based) I want to avoid using this.
Another thing I tried was adding an 'handle' on the inbound flow. However, I learned as the inbound flow has no real 'reply', the handle is executed before the message is sent, so this doesn't work as the move has to be executed after successful processing of the message.
Question in short: what is the standard way of executing an action supplied by the producer (=inbound flow) after the message was successfully processed by a consumer (=outbound flow) via the DirectChannel?
Well, the standard way to do something similar is transaction and that's why we some time ago introduced the PseudoTransactionManager and the XML sample for similar task looks like:
<int-file:inbound-channel-adapter id="realTx" channel="txInput" auto-startup="false"
directory="${java.io.tmpdir}/si-test2">
<int:poller fixed-rate="500">
<int:transactional synchronization-factory="syncFactory"/>
</int:poller>
</int-file:inbound-channel-adapter>
<bean id="transactionManager" class="org.springframework.integration.transaction.PseudoTransactionManager"/>
<int:transaction-synchronization-factory id="syncFactory">
<int:after-commit expression="payload.delete()"/>
</int:transaction-synchronization-factory>
As you see we remove the file in the end of transaction which is caused really after your move to SFTP.
I'd say it is the best way to be tied with only the producer.
Another way is to introduce one more channel before getFileToSftpChannel() and apply the ChannelInterceptor.afterSendCompletion which will be invoked in the end too, by the same single-thread reason. With this approach you should just bridge all your producers with their specific DirectChannels to that single getFileToSftpChannel() for the SFTP adapter.
So, it's up to you what to choose. You have good argument from the architectural perspective to divide the logic by the responsibility levels, but as you see there is no so much choice...
Any other ideas are welcome!
You can try something like the following
#Bean
public DirectChannel getFileToSftpChannel() {
DirectChannel directChannel = new DirectChannel();
directChannel.addInterceptor(new ChannelInterceptorAdapter() {
#Override
public void afterSendCompletion(final Message<?> message,
final MessageChannel channel, final boolean sent, final Exception ex) {
if (ex == null) {
new Archiver().archive((File) message.getPayload());
}
}
});
return directChannel;
}