Our TransactionSynchronizationFactory is deleting the source file even before the flow ends and this is causing a failure in the flow. After reading the file, we split(), make a WebClient Gateway call, resequence() and then aggregate(). Just after aggregation the TransactionSynchronizationFactory is performing a commit. Please suggest why is the behavior?
syncProcessor.setAfterCommitExpression(parser.parseExpression("payload.delete()"));
syncProcessor.setAfterRollbackExpression(parser.parseExpression("payload.delete()"));
return Pollers.fixedDelay(Duration.ofMinutes(pollInterval))
.maxMessagesPerPoll(maxMessagesPerPoll)
.transactionSynchronizationFactory(transactionSynchronizationFactory)
.transactional(pseudoTransactionManager)
.advice(loggingAdvice);
The transaction synchronization is tied to a thread which has started transaction. Whenever you leave that thread (kinda unblock), the end of transaction is triggered. Be sure that you don't shift a message after that aggregate() to some other thread, e.g. via an ExecutorChannel or QueueChannel.
In addition I would look into some other solution where you are not tied to transaction and threading model. Just have the file stored in the headers and whenever you done call its delete()! No reason to deal with transaction with simple files.
Related
Is it possible to cancel a scheduled operation in azure function durable entity ? Below is an example code.I want to cancel the call to operation "DeviceTimeout"after it is scheduled.
Entity.Current.SignalEntity(Entity.Current.EntityId, DateTime.UtcNow.Add(TimeSpan.FromSeconds(30)), nameof(DeviceTimeout));
Unfortunately not. There is an open issue requesting this ability but afaik it has not been implemented yet:
I can' think of a very straightforward API to provide this. One approach would be to include some sort of unique identifier for the signal so that a cancellation request can be precisely matched to the signal being cancelled. But there are some open questions on what exactly the implementation would have to guarantee, and whether that can lead to new issues. For example, would we need to store a cancellation request that we can't match to a signal until such a signal arrives? what if it never arrives?
I suppose the next best thing is to just exit immediately once the operation is executed.
I'm trying to implement a Poller with a DynamicPeriodicTrigger which would backoff (increase duration between polls) if the pollable message source (e.g. FTP server) becomes unavailable, a bit like what is already done through SimpleActiveIdleMessageSourceAdvice but the advice would need to be able to catch the exception thrown during the poll. Unfortunately the invoke method of AbstractMessageSourceAdvice is final, so I can't overwrite it.
I also tried a different approach which is to catch the poll exception by having the poller forward it to an error-channel, where I can increase the duration of the trigger (that part works ok). The problem in that case is how to reset the trigger the next time the poll succeed (i.e. the message source is available again). I can't just reset the trigger in a downstream handler method because the message source may have recovered, but there could still be no message available (in which case my downstream handler method is never called to reset the duration of the trigger).
Thank you very much advance for your expertise and your time.
Best Regards
You don't have to override AbstractMessageSourceAdvice; as you can see its invoke method is pretty trivial; just copy it and add functionality as needed (just be sure to implement MessageSourceMutator so it's detected as a receive-only advice).
Maybe it's as simple as moving the invocation.proceed() to a protected non-final method.
If you come up with something that you think will be generally useful to the community, consider contributing it back to the framework.
I want to make sure the Delayer tied to a PersistentMessageStore will rollback to the DB if there was an exception proceeding from the Delayer after the delay time.
Will the transactional attribute take care of this or I need to have a txAdvice?
<int:delayer id="abcDelayer"
default-delay="1000"
message-store="JDBCMessageStore">
<int:transactional/>
</int:delayer>
Quoting Reference Manual:
The <delayer> can be enriched with mutually exclusive sub-elements <transactional> or <advice-chain>. The List of these AOP Advices is applied to the proxied internal DelayHandler.ReleaseMessageHandler, which has the responsibility to release the Message, after the delay, on a Thread of the scheduled task. It might be used, for example, when the downstream message flow throws an Exception and the ReleaseMessageHandler's transaction will be rolled back. In this case the delayed Message will remain in the persistent MessageStore.
I would like to make a flow initiated by syslog inbound-channel-adapter messages transactional. Unfortunately the adapter does not take a poller which could be made transactional (the typical approach used in many examples with other inbound adapters).
Is there any workaround?
EDIT
After some thinking I realized that my intent is a little different than initially described (hence change of the title). Basically all I want to do is some simple and straight-forward way of making an arbitrary part (starting from some arbitrary channel in the flow) of my message flow pseudo-transactional. Meaning - I want to execute some custom code if the flow completes without any exceptions (but note that I don't want my custom pseudo commit code to be a part (step) of a flow itself). And I want to execute some custom code if any exception occured.
Semantics of using TransactionSynchronizationFactory would suite me very well.
<int:transaction-synchronization-factory id="syncFactory">
<int:after-commit expression="payload.renameTo('/success/' + payload.name)" channel="committedChannel" />
<int:after-rollback expression="payload.renameTo('/failed/' + payload.name)" channel="rolledBackChannel" />
</int:transaction-synchronization-factory>
The only problem is how-to wire it together with the rest of the flow. What I tried is to define intermediate dummy service-activator endpoint that receives messages from the channel where I want the transaction to begin. And then add transactional poller to that service-activator. But this approach has problems of its own because in order to use poller you have to define the incoming channel as a queue channel which seem to make execution of the flow in a separate thread (or at least I observed some async behaveour).
Any flow from a message-driven adapter can be run in the scope of a transaction by making the adapter's channel start a transaction:
<tx:advice id="txAdvice" transaction-manager="transactionManager">
<tx:attributes>
<tx:method name="send"/>
</tx:attributes>
</tx:advice>
<aop:config>
<aop:advisor advice-ref="txAdvice" pointcut="bean(fromSyslog)"/>
</aop:config>
<int:channel id="fromSyslog" />
The channel (and all downstream channels) must be a DirectChannel (no queue channels, or task executors). After an async handoff to another thread, the transaction will commit.
I am sure you understand, but for the benefit of other readers, this does not make the syslog adapter itself transactional, just the downstream flow.
EDIT
Another technique is to use a mid-flow transactional gateway...
#Transactional
public interface TxGate {
void oneWay(Message<?> message);
}
<int:service-activator input-channel="fromSyslog" ref="txGate" />
<int:gateway id="txGate" service-interface="foo.TxGate"
default-request-channel="txSyslog" error-channel="foo" />
That way, you can handle an exception within the scope of the transaction and decide whether or not to commit (throwing an exception from the error flow will rollback).
The void return is important; since the downstream flow doesn't return a reply.
EDIT2
In response to your edited question.
So, it seems your issue with the solutions I provided (specifically in the mid-flow transactional gateway) only allows you to take some action if there is an error when you also want to take some (different) action after success.
There are two ways to do that.
Make the last channel in the subflow a publish-subscribe-channel; add a second consumer (use order to explicitly define the order in which they are called) and take your 'success' action on the second subscriber - he won't be called after an exception (by default) so you'd continue to handle the exception case on the gateway's error channel.
Use a ChannelInterceptor. Start the transaction in preSend(). The afterSendCompletion() method will be invoked after the subflow completes. The presence (or not) of the Exception argument is populated if the subflow throws an exception.
We could consider adding such an interceptor to the framework, if you want to consider contributing it.
Example: Business rules states that the customer should get a confirmation message (email or similar) when an order has been placed.
Lets say that a NewOrderRegisteredEvent is dispatched from the domain and is picked up by an event listener that sends of the confirmation message. When that is done some other event handler throws an exception or something else goes wrong and the unit of work is rolled back. We've now sent the user a confirmation message for something that was rolled back.
What is the "cqrs" way of solving problems like this where you want to do something after a unit of work has been committed? Another complicating factor is replaying of events. I don't want old confirmation messages to be re-sent whenever I replay recorded events in order to build a new view / projection.
My best theory so far: I've just started to look into the fascinating world of cqrs and was wondering whether this is something that would be implemented as a saga? If a saga is like a state machine where each transition only can take place a single time then I guess that would solve this problem? I just have a hard time visualizing how this will fit together with the command bus and domain events..
An Event should only occur after the transaction has been completed. If anything goes wrong and there's a rollback, then the event didn't occur from an external point of view. Therefore it shouldn't be published at all. Though an OrderRegistrationFailed event could be published if necessary.
You wouldn't want the mail to be sent unless the command has sucessfully been executed.
First a few reasons why the command handler -- as proposed in another answer -- would be the wrong place: Under some circumstances the command handler wouldn't be able to tell if the command will eventually succeed or not. Having the command handler invoke the mail sending would also put process knowledge inside the command handler, which would break the SRM and too tightly couple business rules with the application layer.
The mail should be sent after the fact, i.e. from an event handler.
To prevent this handler from firing during replay, you can just not register it. This works similar to how you test your application. You only register the handlers that you actually need.
Production system -> register all event handlers
Tests -> register only the tested event handlers
Replay -> register only the projection/denormalization handlers
Another - even more loosely coupled, though a bit more complex - possibility would be to have a Saga handle the NewOrderRegisteredEvent and issue a SendMail command to the appropriate bounded context (thanks, Yves Reynhout, for pointing this out in the question's comments).
There are two likely solutions
1) The publishing of the event and the handling of the event (i.e. the email) are part of a single transaction. In this case, your transaction framework takes care of it for you. If the email fails, then the event is rolled back. You'll likely retry the command. This is conceptually clean and easy to think about. No event is finished publishing until everyone that has something to say about it has had their say. However practically speaking, this can be painful, as it typically involves distributed transactions. These are hard to come by. Can your email client enroll in the same transaction as the database which is holding your events?
2) The publishing of the event is transactional, but the event handlers each deal with transactions in their own way. The event handler which sends emails could keep track of which events it had seen. If it crashed, it would request old events and process them. You could make a business decision as to how big a deal it would be if people had missing or duplicate emails. (For money-related transactions, the answer is probably you shouldn't allow it.)
Solution (2) is typically what you see promoted in DDD/CQRS circles as it's the more loosely coupled solution. Solution (1) is quite practical in a small system where the event store and the projections are in a single database and the projections don't change often. Solution (2) allows a diversity of event handlers to work in their own way. Solution (1) can cause lots of non-overlapping concerns to become entagled. In this case your order business rules don't complete until the many bizarre things that happen in emailing are taken care of. For one thing, it may slow you down quite a bit.
If the sending of the email were more interesting than "saw the event, sent the email", then you're right, you might have a saga or workflow on your hands. Email in large operations is often a complex system in its own right which you're unlikely to have to implement much of. You just need to be sure you put your email into a request queue of some sort (using approach (2)), and the email system is likely to do retries/batching/spam avoidance/working overnight/etc.