How-to make an arbitrary part of a message flow (pseudo) transactional? - spring-integration

I would like to make a flow initiated by syslog inbound-channel-adapter messages transactional. Unfortunately the adapter does not take a poller which could be made transactional (the typical approach used in many examples with other inbound adapters).
Is there any workaround?
EDIT
After some thinking I realized that my intent is a little different than initially described (hence change of the title). Basically all I want to do is some simple and straight-forward way of making an arbitrary part (starting from some arbitrary channel in the flow) of my message flow pseudo-transactional. Meaning - I want to execute some custom code if the flow completes without any exceptions (but note that I don't want my custom pseudo commit code to be a part (step) of a flow itself). And I want to execute some custom code if any exception occured.
Semantics of using TransactionSynchronizationFactory would suite me very well.
<int:transaction-synchronization-factory id="syncFactory">
<int:after-commit expression="payload.renameTo('/success/' + payload.name)" channel="committedChannel" />
<int:after-rollback expression="payload.renameTo('/failed/' + payload.name)" channel="rolledBackChannel" />
</int:transaction-synchronization-factory>
The only problem is how-to wire it together with the rest of the flow. What I tried is to define intermediate dummy service-activator endpoint that receives messages from the channel where I want the transaction to begin. And then add transactional poller to that service-activator. But this approach has problems of its own because in order to use poller you have to define the incoming channel as a queue channel which seem to make execution of the flow in a separate thread (or at least I observed some async behaveour).

Any flow from a message-driven adapter can be run in the scope of a transaction by making the adapter's channel start a transaction:
<tx:advice id="txAdvice" transaction-manager="transactionManager">
<tx:attributes>
<tx:method name="send"/>
</tx:attributes>
</tx:advice>
<aop:config>
<aop:advisor advice-ref="txAdvice" pointcut="bean(fromSyslog)"/>
</aop:config>
<int:channel id="fromSyslog" />
The channel (and all downstream channels) must be a DirectChannel (no queue channels, or task executors). After an async handoff to another thread, the transaction will commit.
I am sure you understand, but for the benefit of other readers, this does not make the syslog adapter itself transactional, just the downstream flow.
EDIT
Another technique is to use a mid-flow transactional gateway...
#Transactional
public interface TxGate {
void oneWay(Message<?> message);
}
<int:service-activator input-channel="fromSyslog" ref="txGate" />
<int:gateway id="txGate" service-interface="foo.TxGate"
default-request-channel="txSyslog" error-channel="foo" />
That way, you can handle an exception within the scope of the transaction and decide whether or not to commit (throwing an exception from the error flow will rollback).
The void return is important; since the downstream flow doesn't return a reply.
EDIT2
In response to your edited question.
So, it seems your issue with the solutions I provided (specifically in the mid-flow transactional gateway) only allows you to take some action if there is an error when you also want to take some (different) action after success.
There are two ways to do that.
Make the last channel in the subflow a publish-subscribe-channel; add a second consumer (use order to explicitly define the order in which they are called) and take your 'success' action on the second subscriber - he won't be called after an exception (by default) so you'd continue to handle the exception case on the gateway's error channel.
Use a ChannelInterceptor. Start the transaction in preSend(). The afterSendCompletion() method will be invoked after the subflow completes. The presence (or not) of the Exception argument is populated if the subflow throws an exception.
We could consider adding such an interceptor to the framework, if you want to consider contributing it.

Related

Spring Integration TransactionSynchronizationFactory deleting file before Flow ends

Our TransactionSynchronizationFactory is deleting the source file even before the flow ends and this is causing a failure in the flow. After reading the file, we split(), make a WebClient Gateway call, resequence() and then aggregate(). Just after aggregation the TransactionSynchronizationFactory is performing a commit. Please suggest why is the behavior?
syncProcessor.setAfterCommitExpression(parser.parseExpression("payload.delete()"));
syncProcessor.setAfterRollbackExpression(parser.parseExpression("payload.delete()"));
return Pollers.fixedDelay(Duration.ofMinutes(pollInterval))
.maxMessagesPerPoll(maxMessagesPerPoll)
.transactionSynchronizationFactory(transactionSynchronizationFactory)
.transactional(pseudoTransactionManager)
.advice(loggingAdvice);
The transaction synchronization is tied to a thread which has started transaction. Whenever you leave that thread (kinda unblock), the end of transaction is triggered. Be sure that you don't shift a message after that aggregate() to some other thread, e.g. via an ExecutorChannel or QueueChannel.
In addition I would look into some other solution where you are not tied to transaction and threading model. Just have the file stored in the headers and whenever you done call its delete()! No reason to deal with transaction with simple files.

Poller with backoff policy for temporarily unavailable pollable message source

I'm trying to implement a Poller with a DynamicPeriodicTrigger which would backoff (increase duration between polls) if the pollable message source (e.g. FTP server) becomes unavailable, a bit like what is already done through SimpleActiveIdleMessageSourceAdvice but the advice would need to be able to catch the exception thrown during the poll. Unfortunately the invoke method of AbstractMessageSourceAdvice is final, so I can't overwrite it.
I also tried a different approach which is to catch the poll exception by having the poller forward it to an error-channel, where I can increase the duration of the trigger (that part works ok). The problem in that case is how to reset the trigger the next time the poll succeed (i.e. the message source is available again). I can't just reset the trigger in a downstream handler method because the message source may have recovered, but there could still be no message available (in which case my downstream handler method is never called to reset the duration of the trigger).
Thank you very much advance for your expertise and your time.  
  
Best Regards
You don't have to override AbstractMessageSourceAdvice; as you can see its invoke method is pretty trivial; just copy it and add functionality as needed (just be sure to implement MessageSourceMutator so it's detected as a receive-only advice).
Maybe it's as simple as moving the invocation.proceed() to a protected non-final method.
If you come up with something that you think will be generally useful to the community, consider contributing it back to the framework.

Delayer transaction management

I want to make sure the Delayer tied to a PersistentMessageStore will rollback to the DB if there was an exception proceeding from the Delayer after the delay time.
Will the transactional attribute take care of this or I need to have a txAdvice?
<int:delayer id="abcDelayer"
default-delay="1000"
message-store="JDBCMessageStore">
<int:transactional/>
</int:delayer>
Quoting Reference Manual:
The <delayer> can be enriched with mutually exclusive sub-elements <transactional> or <advice-chain>. The List of these AOP Advices is applied to the proxied internal DelayHandler.ReleaseMessageHandler, which has the responsibility to release the Message, after the delay, on a Thread of the scheduled task. It might be used, for example, when the downstream message flow throws an Exception and the ReleaseMessageHandler's transaction will be rolled back. In this case the delayed Message will remain in the persistent MessageStore.

Spring-Integration: Execute poller trigger by last message from previous poll

I have poller configuration like this:
<int:poller id="myPoller" default="true" max-messages-per-poll="2" task-executor="executor"
error-channel="errorChannel" receive-timeout="20000" trigger="dynamicTrigger">
</int:poller>
But i need to trigger the poller with last time message from previous poll. For example of the flow:
If i have 4(A,B,C,D) Post rest the app will process 2(A,B) message first and then wait for the last response (either from A or B), afterward trigger the poll for next message (C,D).
How to achieve that condition ? Thx.
Well, I suggest you to take a look into poller's <advice-chain>. With some custom MethodInterceptor you perform your decision logic and call your polling source stop(). When you have a desired response somewhere downstream you just perform start() of that endpoint again to let proceed with a new bunch of messages.
With enough big polling-interval you even don't need that <advice-chain> and just perform stop() in the downstream as well, via some state checking.
Also consider to use Control Bus for stop/start operations.

How does group timeout work?

Aggregator is a passive component and the release logic is only triggered when a new message arrives. How then does the group timeout work?
Is it a scheduled task similar to reaper that constantly monitors the state of the aggregator. Does that also mean it repeatedly evaluates the group-timeout-expression to determine the value of group-timeout, or is it evaluated once at the start? I am assuming, since there are some examples based on size of payload, that means it must evaluate the group-timeout-expression repeatedly but if that's the case how often does that happen? Can that frequency of evaluation be controlled/modified. Along the same lines if aggregator is a POJO, has this group-timeout functionality already flexible enough to be able to specify the timout from a POJO method.
Another interesting thing I noticed is that for my group-timeout-expression I was trying a spell expression and was passing payload or headers but those apparently aren't available in the context. Seems like the context within this group-timeout-expression points to SimpleMessageGroup which doesn't have payload or headers properties. So, how can I access payload or headers within the spel expression of group-timeout-expression?
In fact in my case I want the actual message (the wrapper around the payload) because my method signature expects an actual SI message passed to it not the payload.
Prior to Spring Integration 4.0, the aggregator was a passive component and you had to configure a MessageGroupStoreReaper to expire groups.
The group-timeout* attributes were added in 4.0; when a new message arrives, a task is scheduled to time out the group. If the next message arrives before the timeout, the task is cancelled, the new message is added to the group and, if the release doesn't happen, a new task is scheduled. The expression is re-evaluated each time (the example in the documentation looks at the group size).
Yes, the root object for expression evaluation is the message group.
Which "payload and headers" to you need? There are likely multiple messages in the group. You can easily access the first message in the group using one.payload or one.headers['foo'] (these expressions use group.getOne() which peeks at the first message in the group).
If you need to access a different message, you would need to write some code to iterate over the messages in the group. We don't currently, but it would be possible to make the newly arrived message available as a variable #newMessage or similar; feel free to open an 'Improvement' JIRA issue for that.

Resources