There appear to be multiple prior questions about this, mostly with cold links to the old Spring forums.
Here is the subflow I am trying to capture exceptions/errors from:
return flowDef
.filter(getFilterExpression(rule)).channel(new DirectChannel())
.handle(inboundAdapter)
.split(insertDeleteSplitter)
.publishSubscribeChannel(c ->
c.subscribe(s -> s
.filter ("....")
.transform(genericTransformer)
.handle(insertUpdateMessageHandler(rule))) // a JDBCMessageHandler
.subscribe(s -> s
.filter("....")
.transform(genericTransformer)
.handle(deleteMessageHandler(rule))) // a JDBCMessageHandler
.subscribe(sub -> sub
.handle(cleanupMessageHandler(rule))) // a JDBCMessageHandler
// .errorHandler(new CustomErrorHandler()); // Obviously not working
);
The symptom I am seeing is that the errors are propagated back to the poller, which does not have the detail necessary to resolve the issues.
My intent is that any exceptions or errors will be caught and rerouted from this subflow level, where the message is complete, to an error flow that consists of code to record the failure and core data then completes cleanup.
Starting condition: Message ABC has been properly routed to this subflow
Message ABC has been split and routed to (example) the insertUpdateMessageHandler (JDBCMessageHandler)
Handling fails - Database server throws duplicate row (for example)
Actual outcome:
Exception is caught by the poller, which does not have the necessary information to record/handle the error
Desired outcome:
Exception is handled at this level, and the flow ends.
My suspicisons:
Ideally, I've made a simple typo and will feel like a fool when it's pointed out
I suspect that this is a design consideration, with the intent that the Poller be the level at which errors or handled, probably for transactionality.
I may need to build my exception handling into custom message handlers that wrap the JDBC message handler then handle the exceptions
(Edit)
See also: How do I configure this JdbcMessageHandler to pull parameters from the message instead of static beans?
It's not clear what you mean by "the poller does not have the necessary information...".
You can add an errorChannel to the poller and the downstream flow from that channel will get an ErrorMessage with a payload of type MessagingException with failedMessage and cause properties. The ErrorMessage also has the original message property.
Alternatively, you can add an ExpressionEvaluatingRequestHandlerAdvice to the adapter endpoint's advice chain.
See https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain
and
https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#expression-advice
Related
When I try to shutdown my spring-integration process, the flow using an inbound Jms.messageDrivenChannelAdapter throws the following error message:
"org.springframework.jms.listener.DefaultMessageListenerContainer - Rejecting received message because of the listener container been stopeed in the meantime"
my inbound adapter is defined as follows:
Jms.messageDrivenChannelAdapter(
Jms.container(jmsConnectionFactory, destinationName)
.concurrency(highConcurrency)
.get()
)
I believe that my problem is that the default "receiveTimeout" on my jms container is too small and that I need to increase that value to cater for my "high-concurrency" (right ?), as "receiveTimeout" seems to be the only value the container "doShutdown" method cares about.
Now, the sourceCode for the receiveTimeout property says "this value needs to be smaller than the transaction timeout". Also the spring-integration doco regarding inbound jms adapters says "if you want the entire flow to be transactional [...] consider using a jms-message-driven-channel-adapter with acknowledge set to transacted (the default)", which seems to imply that the jms adapter is transactional by default.
Hence, my main question is: even though I'm not using any explicit transaction manager, do I need to not only explicitely set "receiveTimeout" on my container but also "transactionTimeout" with transactionTimeout > receiveTimeout ?
Thanks a lot in advance for your expertise and your time.
Best Regards
That is not "throws". That is just warn:
protected void doExecuteListener(Session session, Message message) throws JMSException {
if (!isAcceptMessagesWhileStopping() && !isRunning()) {
if (logger.isWarnEnabled()) {
logger.warn("Rejecting received message because of the listener container " +
"having been stopped in the meantime: " + message);
}
rollbackIfNecessary(session);
throw new MessageRejectedWhileStoppingException();
}
And pay attention to that rollbackIfNecessary(session);. So, even if the received message slips somehow into this listener function, the whole environment makes it sure that the state is not broken and the data is not lost - the session is rolled back.
The transactionTimeout does not make sense if you don't use a transactionManager. Spring Integration makes it transacted exactly for the use-case we see around that warn log.
I am trying to learn Spring Integration. As a test project to explore all the features. My project goes like this
AMQP Queue -> Header Router -> Transformer -> OutBound SOAP WS Call -> Log Results -> End
1.) Start the flow by Listening to a RabbitMq Queue. The RabbitMq Message that are sent will have a String Value (Example: 32) and a Header like {"operation" "CelsiusToFahrenheit"}.
2.) The RabbitMq message is then transformed to the corresponding Java Object based on the RabbitMQ Header Value ("operation").
3.) Then I am using a MarshallingWebServiceOutboundGateway to invoke the SOAP WebService # (http://www.w3schools.com/xml/tempconvert.asmx). When I do that I am getting an Error.
ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: error occurred in message handler [wsOutboundGateway]; nested exception is org.springframework.ws.soap.client.SoapFaultClientException: Server did not recognize the value of HTTP Header SOAPAction: ., headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#601affec, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#601affec, id=7f6624cc-e7a4-37f6-0a1f-578dc78c4afa, timestamp=1482356790717}]
As you can see the SOAPAction was Blank and the server did NOT like it.
If I hard code the SOAPAction like this
public MessageHandler wsOutboundGateway()
{
MarshallingWebServiceOutboundGateway temperatureConversionWebServiceGateway = new MarshallingWebServiceOutboundGateway(WEBSERVICE_URL, jaxb2Marshaller(), jaxb2Marshaller());
temperatureConversionWebServiceGateway.setRequestCallback(new SoapActionCallback("http://www.w3schools.com/xml/CelsiusToFahrenheit"));
temperatureConversionWebServiceGateway.setOutputChannelName("webServiceResponseChannel");
return temperatureConversionWebServiceGateway;
}
it works!
However, the endpoint exposes 2 Services (2 Possible Actions). See below.
FahrenheitToCelsius soapAction="http://www.w3schools.com/xml/FahrenheitToCelsius"
CelsiusToFahrenheit soapAction="http://www.w3schools.com/xml/CelsiusToFahrenheit"
I want to be able to set the SoapAction at run time based on the RabbitMQ Header Value ("operation"). How do I do it?
On a Side note, I have done a few different experiments to solve this problem but none seems to have yielded the results I am looking for. One of the solutions I tried was to set the SoapAction as part of the Transformation process (Converting the Input from Rabbit MQ message to the corresponding Java Type based on the RabbitMQ Header Value) and send that request to the Outbound Gateway (See Code below).
#Bean
public IntegrationFlow generateCelsiusToFahrenheitRequestFlow() {
Map<String, Object> headers = new HashMap<>();
headers.put("SOAPAction", "http://www.w3schools.com/xml/CelsiusToFahrenheit");
return IntegrationFlows.from(celsiusToFahrenheitChannel)
.transform(celsiusToFahrenheitTransformer)
.enrichHeaders(headers)
.channel(webServiceRequestChannel)
.get();
}
After I do that and tap the 'webServiceRequestChannel', I see the SoapAction Header that I added previously (See Below).
GenericMessage [payload=CelsiusToFahrenheit [celsius=32], headers={SOAPAction=http://www.w3schools.com/xml/CelsiusToFahrenheit, amqp_receivedDeliveryMode=NON_PERSISTENT,........
However, when the MarshallingWebServiceOutboundGateway picked up the request and makes the outbound call to the SOAP WebService, the "SOAPAction: " is still Blank. I am not sure why the Soap Action got blanked out. What am I missing?
Sorry if I have not followed conventions when Writing / Formatting my question. This is my first time here on Stack Overflow. Any inputs or suggestions to address this issues will be greatly appriciated.
Yeah, the mistake here that the header name must be exactly WebServiceHeaders.SOAP_ACTION, where its name is ws_soapAction. Only now a DefaultSoapHeaderMapper is able to map it into request WebserviceMessage:
public DefaultSoapHeaderMapper() {
super(WebServiceHeaders.PREFIX, STANDARD_HEADER_NAMES, Collections.<String>emptyList());
}
I wrote a simple flow for AMQP inbound messages with Json payloads, something like
IntegrationFlows
.from(Amqp.inboundGateway(connectionFactory, new Queue("qin"))
.errorChannel(Amqp.channel("dlx", connectionFactory))
)
.handle(new MessageTransformingHandler(m -> {
Object result = null;
try {
result = (...)
} catch (Exception e) {
throw new MessageTransformationException(m, e.getMessage());
}
(...)
}))
.transform(Transformers.toJson(...))
.handle(Amqp.outboundAdapter(new RabbitTemplate(connectionFactory))
.routingKey("qout"))
.get();
}
This works perfectly OK, except when there's errors! As it is now I do get the error in DLX but in content_type: application/x-java-serialized-object and it is required to be application/json.
I could do this by having the error channel specify 2 converters
.amqpMessageConverter(...)
.messageConverter(...)
but the problem is that I have to implement then myself which is not easy because I have to deal with converting messages to ampqmessages, plus the business objects, plues the error object and text, and so on...
So I was thinking if I couldn't have a adapter in front of the error channel that at least took care of message->amqpmessage conversion (hopefully the payloads as well).
I also tried having a errorHandler instead of a errorChannel but the problems are the same.
Any sugestion?
Thanks in advance.
EDITED
Many thanks for your reply. However I'm struggling with it. After many tries and errors, I finally think I understand the solution (to use a "intermediary" channel so I can handle the message before send it to Amqp?) but I still can't get it to work. I have now
.errorChannel(MessageChannels.direct("amqpErrorChannel").get())
and the a flow listening to that channel
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("amqpErrorChannel")
.handle(new MessageTransformingHandler(m ->(...)
but I still have a error
MessageDeliveryException: Dispatcher has no subscribers for channel
'amqpErrorChannel'.
Any pointers to what I'm doing wrong?
Cheers.
Yes, you can have .transform() or any other adapter in front of (Amqp.channel("dlx", connectionFactory). Actually .errorChannel() is just a hook to send error to the error handling flow. So, you can use there any simple Spring Integration channel (not an AMQP one) and build any complex error handling logic.
Correct, in the end of that flow you can send a result message (after a bunch of transformation, enrichment etc.) to the AMQP dlx, but for this purpose the simple one-way Amqp.outboundAdapter() would be enough.
To be honest Amqp.channel() is two-way and that really would be better that you have a subscriber for it. But your case is one-way, so you should use Amqp.outboundAdapter() there instead.
The Azure Service Bus supports a built-in retry mechanism which makes an abandoned message immediately visible for another read attempt. I'm trying to use this mechanism to handle some transient errors, but the message is made available immediately after being abandoned.
What I would like to do is make the message invisible for a period of time after it is abandoned, preferably based on an exponentially incrementing policy.
I've tried to set the ScheduledEnqueueTimeUtc property when abandoning the message, but it doesn't seem to have an effect:
var messagingFactory = MessagingFactory.CreateFromConnectionString(...);
var receiver = messagingFactory.CreateMessageReceiver("test-queue");
receiver.OnMessageAsync(async brokeredMessage =>
{
await brokeredMessage.AbandonAsync(
new Dictionary<string, object>
{
{ "ScheduledEnqueueTimeUtc", DateTime.UtcNow.AddSeconds(30) }
});
}
});
I've considered not abandoning the message at all and just letting the lock expire, but this would require having some way to influence how the MessageReceiver specifies the lock duration on a message, and I can't find anything in the API to let me change this value. In addition, it wouldn't be possible to read the delivery count of the message (and therefore make a decision for how long to wait for the next retry) until after the lock is already required.
Can the retry policy in the Message Bus be influenced in some way, or can a delay be artificially introduced in some other way?
Careful here because I think you are confusing the retry feature with the automatic Complete/Abandon mechanism for the OnMessage event-driven message handling. The built in retry mechanism comes into play when a call to the Service Bus fails. For example, if you call to set a message as complete and that fails, then the retry mechanism would kick in. If you are processing a message an exception occurs in your own code that will NOT trigger a retry through the retry feature. Your question doesn't get explicit on if the error is from your code or when attempting to contact the service bus.
If you are indeed after modifying the retry policy that occurs when an error occurs attempting to communicate with the service bus you can modify the RetryPolicy that is set on the MessageReciver itself. There is an RetryExponitial which is used by default, as well as an abstract RetryPolicy you can create your own from.
What I think you are after is more control over what happens when you get an exception doing your processing, and you want to push off working on that message. There are a few options:
When you create your message handler you can set up OnMessageOptions. One of the properties is "AutoComplete". By default this is set to true, which means as soon as processing for the message is completed the Complete method is called automatically. If an exception occurs then abandon is automatically called, which is what you are seeing. By setting the AutoComplete to false you required to call Complete on your own from within the message handler. Failing to do so will cause the message lock to eventually run out, which is one of the behaviors you are looking for.
So, you could write your handler so that if an exception occurs during your processing you simply do not call Complete. The message would then remain on the queue until it's lock runs out and then would become available again. The standard dead lettering mechanism applies and after x number of tries it will be put into the deadletter queue automatically.
A caution of handling this way is that any type of exception will be treated this way. You really need to think about what types of exceptions are doing this and if you really want to push off processing or not. For example, if you are calling a third party system during your processing and it gives you an exception you know is transient, great. If, however, it gives you an error that you know will be a big problem then you may decide to do something else in the system besides just bailing on the message.
You could also look at the "Defer" method. This method actually will then not allow that message to be processed off the queue unless it is specifically pulled by its sequence number. You're code would have to remember the sequence number value and pull it. This isn't quite what you described though.
Another option is you can move away from the OnMessage, Event-driven style of processing messages. While this is very helpful you don't get a lot of control over things. Instead hook up your own processing loop and handle the abandon/complete on your own. You'll also need to deal some of the threading/concurrent call management that the OnMessage pattern gives you. This can be more work but you have the ultimate in flexibility.
Finally, I believe the reason the call you made to AbandonAsync passing the properties you wanted to modify didn't work is that those properties are referring to Metadata properties on the method, not standard properties on BrokeredMessage.
I actually asked this same question last year (implementation aside) with the three approaches I could think of looking at the API. #ClemensVasters, who works on the SB team, responded that using Defer with some kind of re-receive is really the only way to control this precisely.
You can read my comment to his answer for a specific approach to doing it where I suggest using a secondary queue to store messages that indicate which primary messages have been deferred and need to be re-received from the main queue. Then you can control how long you wait by setting the ScheduledEnqueueTimeUtc on those secondary messages to control exactly how long you wait before you retry.
I ran into a similar issue where our order picking system is legacy and goes into maintenance mode each night.
Using the ideas in this article(https://markheath.net/post/defer-processing-azure-service-bus-message) I created a custom property to track how many times a message has been resubmitted and manually dead lettering the message after 10 tries. If the message is under 10 retries it clones the message increments the custom property and sets the en queue of the new message.
using Microsoft.Azure.ServiceBus;
public PickQueue()
{
queueClient = new QueueClient(QUEUE_CONN_STRING, QUEUE_NAME);
}
public async Task QueueMessageAsync(int OrderId)
{
string body = JsonConvert.SerializeObject(OrderId);
var message = new Message(Encoding.UTF8.GetBytes(body));
await queueClient.SendAsync(message);
}
public async Task ReQueueMessageAsync(Message message, DateTime utcEnqueueTime)
{
int resubmitCount = (int)(message.UserProperties["ResubmitCount"] ?? 0) + 1;
if (resubmitCount > 10)
{
await queueClient.DeadLetterAsync(message.SystemProperties.LockToken);
}
else
{
Message clone = message.Clone();
clone.UserProperties["ResubmitCount"] = ++resubmitCount;
await queueClient.ScheduleMessageAsync(message, utcEnqueueTime);
}
}
This question asks how to implement exponential backoff in Azure Functions. If you do not want to use the built-in RetryPolicy (only available when autoComplete = false), here's the solution I've been using:
public static async Task ExceptionHandler(IMessageSession MessageSession, string LockToken, int DeliveryCount)
{
if (DeliveryCount < Globals.MaxDeliveryCount)
{
var DelaySeconds = Math.Pow(Globals.ExponentialBackoff, DeliveryCount);
await Task.Delay(TimeSpan.FromSeconds(DelaySeconds));
await MessageSession.AbandonAsync(LockToken);
}
else
{
await MessageSession.DeadLetterAsync(LockToken);
}
}
I am building a system with Spring Integration that processes all lines in a file as records. Because some of the String records are malformed I have multiple paths through the application via a Splitter and Aggregator combination (I'm building the Aggregator as we speak).
Further, some of the records are so malformed that they are effectively errors. However I have a requirement that all records must be processed therefore I must identify and log gross malformation errors separately and finish processing the file. In other words, I can not fail to process the file but instead must only log errors.
Aggregator
I intend to do achieve the goal of processing grossly malformed records by modifying the headers on the incoming message and passing the message on-ward to the Aggregator which can search for the existence of such a parameter. I'll effectively be hand coding in some error handling situations to my processors and aggregator.
My Release Strategy for the Aggregator will be when all messages are processed.
Code Extract
This code comes from a blog entry by Matt Vickery. He constructs an entirely new message (using MessageBuilder and transferring headers) whereas I will just add something to the Message headers. He includes this code in a gateway which subsequently transfers the Message onto the Aggregator.
public Message<AvsResponse> service(Message<AvsRequest> message) {
Assert.notNull(message, MISSING_MANDATORY_ARG);
Assert.notNull(message.getPayload(), MISSING_MANDATORY_ARG);
MessageHeaders requestMessageHeaders = message.getHeaders();
Message<AvsResponse> responseMessage = null;
try {
logger.debug("Entering AVS Gateway");
responseMessage = avsGateway.send(message);
if (responseMessage == null)
responseMessage = buildNewResponse(requestMessageHeaders,
AvsResponseType.NULL_RESULT);
logger.debug("Exited AVS Gateway");
return responseMessage;
}
catch (Exception e) {
return buildNewResponse(responseMessage, requestMessageHeaders,
AvsResponseType.EXCEPTION_RESULT, e);
}
}
Confusion (...at least, that which I know about)
My questions are as follows:
When I have such a release strategy (all messages processed), is that the best way to ensure all messages get through to the Aggregator?
When using an Aggregator it seems like in practical cases, it would be very common to need access to the Message in some previous step, as opposed to just passing and processing simple POJOs. Would that be true or is there something I should be doing to simplify my design so I can avoid Message
I came across a blog entry by Matt Vickery showing how he achieves what seems to be similar with an Aggregator. I'm using his work as a guide.
P.S. Per Artem Bilan's advice, I'm avoiding creating my own messages and letting SI turn them into Messages
There is no difference for Aggregator if payload is valid or not. Its general purpose is to build a List (by default) of payloads to one Message. And it does it via some sequenceDetails from MessageHeaders. It is first.
If you use Splitter, it is responsible to enrich each produced Message with default sequenceDetails. So, if you have this configuration:
<splitter/>
<aggregator/>
And if your inbound payload is List, you end up with List after aggregator as well.
I assume, that your Splitter just produces String payloads from File lines.
Then you pass each Message to some service/transformer.
The result of that you may pass to the Aggregator.
But as you say some of payloads are not valid and your processor fails with an Exception.
So, how about just try...catch within that POJO method and return some payload with error indicator, e.g. simple String "Oops!".
As I described before: the result of POJO method will be pushed to payload of the Message by Framework. And what is magic, that sequenceDetails will be there in the MessageHeaders too.
I don't see reason to write some custom ReleaseStrategy for this task, or even any other Aggregator's strategies...
Let me know, what you don't understand.
UPDATE
To add some error-indicator to message headers and don't throw Exception, it really will be simpler to build a new Message from code, not via some error-channel flow:
try {
return [GOOD_RESULT];
}
catch(Exception e) {
return MessageBuilder.withPayload(payload).setHeader("ERROR", e.getMessage()).build();
}
But in this case you should use <service-activator> instead of <transformer>, because the last one doesn't copy headers from inbound Message. And you really need them - setHeader for aggregator.