log channel errors multiple stacktrace - spring-integration

Question

Trying to log channel error and using a retry policy, i log two exception stacktrace instead of only one.
Channel configuration :
<bean id="retryInterceptor" class="org.springframework.amqp.rabbit.config.StatelessRetryOperationsInterceptorFactoryBean">
<property name="messageRecoverer" ref="rejectAndDontRequeueRecoverer"/>
<property name="retryOperations" ref="retryTemplate" />
</bean>
<bean id="rejectAndDontRequeueRecoverer" class="org.springframework.amqp.rabbit.retry.RejectAndDontRequeueRecoverer"/>
<!-- Configuration du retry -->
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<!-- Intervalle de temps en millisecondes entre deux tentatives -->
<property name="initialInterval" value="2000" />
</bean>
</property>
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<!-- Nombre de retry max -->
<property name="maxAttempts" value="3" />
</bean>
</property>
</bean>
<bean id="log" class="com.logger.Logger"/>
<!-- Logger -->
<int:channel id="one" ></int:channel>
<int:channel id="error1" ></int:channel>
<int:publish-subscribe-channel id="processChannel1" />
<int:logging-channel-adapter channel="processChannel1" logger-name="log1" level="ERROR" expression="payload.message"/>
<int:service-activator input-channel="processChannel1" ref="log" output-channel="error1" />
<!-- Source -->
<rabbit:connection-factory id="connectionFactory" username="guest" password="guest" addresses="XX.XX.XX.XX:5672" cache-mode="CONNECTION" virtual-host="/" />
<int-amqp:inbound-channel-adapter channel="one" id="inboundChannelAdapter1" queue-names="myqueue" connection-factory="connectionFactory" error-channel="processChannel1" channel-transacted="true" advice-chain="retryInterceptor" />
<!-- Destination -->
<rabbit:connection-factory id="connectionFactoryRmqDest" username="guest" password="guest111" addresses="YY.YY.YY.YY:5672" cache-mode="CONNECTION" connection-cache-size="50" virtual-host="/" />
<rabbit:template id="rabbitTemplateRmqDest" connection-factory="connectionFactoryRmqDest" />
<int-amqp:outbound-channel-adapter channel="one" id="outboundChannelAdapter1" routing-key="keyMyQueue" exchange-name="direct.exchange" amqp-template="rabbitTemplateRmqDest" default-delivery-mode="PERSISTENT"/>
Logger.java
package com.logger;
public class Logger {
public Message<?> log(Message<?> message) {
System.out.println("*************************************");
throw new AmqpException("Error on channel");
}
}
Log file :
2018-03-12 16:34:43.008 [(inner bean)#160e9f2e-1] ERROR log1 - error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint#0]; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
2018-03-12 16:34:43.021 [(inner bean)#160e9f2e-1] ERROR log1 - Message conversion failed
2018-03-12 16:34:45.181 [(inner bean)#160e9f2e-1] ERROR log1 - error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint#0]; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
2018-03-12 16:34:45.182 [(inner bean)#160e9f2e-1] ERROR log1 - Message conversion failed
2018-03-12 16:34:49.345 [(inner bean)#160e9f2e-1] ERROR log1 - error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint#0]; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
2018-03-12 16:34:49.346 [(inner bean)#160e9f2e-1] ERROR log1 - Message conversion failed
Removing the expression attributes, stack traces are (just one attempt on the 3 retries):
2018-03-12 17:24:03.303 [(inner bean)#76e1e924-1] ERROR log1 - org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint#0]; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile., failedMessage=GenericMessage [payload=byte[20], headers={timestamp=1520871843104, id=7c9f713f-90a1-2720-60e4-a0e33ef98ecb, amqp_receivedRoutingKey=myqueue, amqp_consumerQueue=myqueue, amqp_consumerTag=amq.ctag-ENdQ-j6xD1LRXDxGr_iJuQ, amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_redelivered=false, amqp_deliveryTag=1}]
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:139)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:425)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter.access$1100(AmqpInboundChannelAdapter.java:56)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.processMessage(AmqpInboundChannelAdapter.java:246)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.onMessage(AmqpInboundChannelAdapter.java:203)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:848)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:771)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:102)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:198)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:74)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:276)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:172)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy9.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1311)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:752)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1254)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1224)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:102)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1470)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:65)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:368)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:603)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1430)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1411)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:712)
at org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint.send(AmqpOutboundEndpoint.java:134)
at org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint.handleRequestMessage(AmqpOutboundEndpoint.java:122)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
... 38 more
Caused by: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:342)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:909)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:859)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:799)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:352)
... 46 more
2018-03-12 17:24:03.314 [(inner bean)#76e1e924-1] ERROR log1 - org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Message conversion failed
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.onMessage(AmqpInboundChannelAdapter.java:223)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:848)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:771)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$001(SimpleMessageListenerContainer.java:102)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1.invokeListener(SimpleMessageListenerContainer.java:198)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:74)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:276)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:172)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy9.invokeListener(Unknown Source)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.invokeListener(SimpleMessageListenerContainer.java:1311)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:752)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1254)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1224)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:102)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1470)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.MessageHandlingException: nested exception is org.springframework.amqp.AmqpException: Erreur sur le channel, failedMessage=EnhancedErrorMessage [payload=org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.amqp.outbound.AmqpOutboundEndpoint#0]; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile., failedMessage=GenericMessage [payload=byte[20], headers={timestamp=1520871843104, id=7c9f713f-90a1-2720-60e4-a0e33ef98ecb, amqp_receivedRoutingKey=myqueue, amqp_consumerQueue=myqueue, amqp_consumerTag=amq.ctag-ENdQ-j6xD1LRXDxGr_iJuQ, amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_redelivered=false, amqp_deliveryTag=1}], headers={timestamp=1520871843291, id=dbde6a9f-3c26-086f-53a1-7c98482efcce, amqp_raw_message=(Body:'[B#56360d58(byte[20])' MessageProperties [headers={}, timestamp=null, messageId=null, userId=null, receivedUserId=null, appId=null, clusterId=null, type=null, correlationId=null, correlationIdString=null, replyTo=null, contentType=null, contentEncoding=null, contentLength=0, deliveryMode=null, receivedDeliveryMode=NON_PERSISTENT, expiration=null, priority=null, redelivered=false, receivedExchange=, receivedRoutingKey=myqueue, receivedDelay=null, deliveryTag=1, messageCount=0, consumerTag=amq.ctag-ENdQ-j6xD1LRXDxGr_iJuQ, consumerQueue=myqueue])}] for original GenericMessage [payload=byte[20], headers={timestamp=1520871843104, id=7c9f713f-90a1-2720-60e4-a0e33ef98ecb, amqp_receivedRoutingKey=myqueue, amqp_consumerQueue=myqueue, amqp_consumerTag=amq.ctag-ENdQ-j6xD1LRXDxGr_iJuQ, amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_redelivered=false, amqp_deliveryTag=1}]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:96)
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:89)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.BroadcastingDispatcher.invokeHandler(BroadcastingDispatcher.java:236)
at org.springframework.integration.dispatcher.BroadcastingDispatcher.dispatch(BroadcastingDispatcher.java:185)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:425)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.endpoint.MessageProducerSupport.sendErrorMessageIfNecessary(MessageProducerSupport.java:207)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:191)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter.access$1100(AmqpInboundChannelAdapter.java:56)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.processMessage(AmqpInboundChannelAdapter.java:246)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.onMessage(AmqpInboundChannelAdapter.java:203)
... 25 more
Caused by: org.springframework.amqp.AmqpException: Erreur sur le channel
at com.logger.Logger.log(Logger.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:117)
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:129)
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:49)
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:347)
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:88)
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:132)
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:360)
at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:169)
at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:319)
at org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:155)
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:93)
... 41 more
In the log file i have 3 attemps for transfer the message. => Retry is ok
I would like to have only one stacktrace? What the mean of the "Message conversion failed"? How can i prevent it?
Is there a way to test if the cause of an exception is empty or null? I would like to use "payload.cause.message" in the expression?
Thanks for your help,
Regards
Eric

Answer 1

The first log message comes from the <int:logging-channel-adapter> just because you use an error-channel="processChannel1" on the <int-amqp:outbound-channel-adapter>.
The second one comes from the AbstractMessageListenerContainer because your Logger doesn't log message, but does exactly this:
throw new AmqpException("Error on channel");
So, when we do in the AmqpInboundChannelAdapter.Listener this:
getMessagingTemplate().send(getErrorChannel(), buildErrorMessage(null,
new ListenerExecutionFailedException("Message conversion failed", e, message)));
Send error message to the error channel, you just re-throw an exception from the to the top-level AMQP listener and that one logs it one more time.
You should consider do not re-throw an exception from your Logger component, or don't use use one more <int:logging-channel-adapter> subscriber, or just don't use error-channel approach at all!
For the payload.cause.message expression you should use a SpEL ternary operator : payload.cause != null ? payload.cause.message : payload.message
UPDATE
OK. I see what's going on.
We send an exception to the errorChannel during MessageProducerSupport.sendMessage() process:
try {
this.messagingTemplate.send(getOutputChannel(), message);
}
catch (RuntimeException e) {
if (!sendErrorMessageIfNecessary(message, e)) {
throw e;
}
}
And since you have one more subscriber to throw a fresh exception (or just re-throw), that sendErrorMessageIfNecessary() finishes with a bubbled exception.
This one come to the catch (RuntimeException e) { of the AmqpInboundChannelAdapter.Listener.onMessage() like mentioned before. And a new ErrorMessage is sent to the errorChannel. That's how you see that one more logged message. But since the second subscriber re-throws an exception, that's how it comes back to the listener container and initiate retries. And that's how you see a stack trace only once, after all the retry attempts. However I can guess that it is already a wrong stack trace, because you would like to see the real reason of the sending to the AMQP problem.
What I suggest here for you as a solution is like this:
No error-channel on the <int-amqp:inbound-channel-adapter>
No second subscriber to the processChannel1 - the existing <int:logging-channel-adapter> is enough.
You configure ExpressionEvaluatingRequestHandlerAdvice with the failureChannel to the processChannel1 and onFailureExpression to the #this
Use this ExpressionEvaluatingRequestHandlerAdvice as a <request-handler-advice-chain> reference for the <int-amqp:outbound-channel-adapter>
This way you'll get logs on each retry and according the trapException = false, a real exception will be re-thrown to the listener cotnainer for retry.