I am using spring-rabbit-1.7.3.RELEASE.jar
I have defined a SimpleMessageListenerContainer in my xml with shutdownTimeout parameter.
bean id="aContainer"
class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="rabbitConnectionFactory" />
<property name="queueNames" value="aQueue" />
<property name="adviceChain" ref="retryAdvice" />
<property name="acknowledgeMode" value="AUTO" />
<property name="shutdownTimeout" value="900000" />
</bean>
When my service shuts down and there are still messages in "aQueue", I expect that the shutdownTimeout would allow the messages to get processed. But this doesn't seem to happen.
On further investigation I found out that the await() method defined in SimpleMessageListenerContainer is always returning true.
this.cancellationLock.await(Long.valueOf(this.shutdownTimeout), TimeUnit.MILLISECONDS);
I would like to understand how the logic for await works, how it acquires lock and what additional configuration is required at my end to make the code work.
It waits for the consumers on-the-fly, those who are busy to process already fetched but not yet acknowledged messages. Nobody is going to poll fresh messages from the queue during shutdown.
The ActiveObjectCounter awaits on the all internal CountDownLatchs to be released. And that happens when we:
public void handleShutdownSignal(String consumerTag, ShutdownSignalException sig) {
So, that really might be a fact that all your consumers (private volatile int concurrentConsumers = 1; by default) are cancelled and released during that shutdownTimeout.
But again: no body is going to poll new messages from the Broker when the state is shutdown.
Related
I asked a question a bit ago about a specific use case when multi-threading in spring integration, basically I have to have one thread of execution that stays on the initial and then a second that spawns a thread. I have implemented this like so
<int:channel id="newThread" >
<int:dispatcher task-executor="workerThreadPoolAdapter"/>
</int:channel>
<!--This bean is used to split incoming messages -->
<bean id="splitterBean" class="orchestration.MessageSplitter">
<property name="channels" ref="splitterChannelsList" />
</bean>
<util:list id="splitterChannelsList" value-type="java.lang.String">
<value>newThread</value>
<value>mainThread</value>
</util:list>
<!-- This bean is used to aggregate incoming messages -->
<bean id="aggregator" class="orchestration.MessageAggregator">
<property name="wrapperNode" value="container" />
</bean>
<!-- Channel for aggregator output and that will be input for response transformer -->
<int:publish-subscribe-channel id="gatherChannel" apply-sequence="true"/>
<!-- This splitter splits request and send -->
<!-- will add a header called channelHeader which is the channel the message should be routed to using the recipient-list-router -->
<int:splitter input-channel="splitter"
output-channel="recipientListRouter"
apply-sequence="true"
ref="splitterBean" method="split" />
<!-- Aggregator that aggregates responses received from the calls -->
<int:aggregator input-channel="gatherChannel"
output-channel="transformResponse"
ref="aggregator"
method="aggregateMessages"
send-partial-result-on-expiry="false"
expire-groups-upon-completion="true"
message-store="removeMessageFromStore"
release-strategy-expression="size() == 2"/>
<int:recipient-list-router input-channel="recipientListRouter">
<int:recipient channel="mainThread"
selector-expression="headers.get('channelHeader') == 'mainThread'"/>
<int:recipient channel="newThread"
selector-expression="headers.get('channelHeader') == 'newThread'"/>
</int:recipient-list-router>
<!-- only route to call if boolean is populated -->
<int:header-value-router header-name="shouldMakeExtraOutboundCall"
input-channel="newThread"
default-output-channel="gatherChannel" >
<int:mapping value="true" channel="outboundCall" />
</int:header-value-router>
<int:chain input-channel="outboundCall" output-channel="gatherChannel">
<!-- make an outbound call -->
</int:chain>
<int:chain input-channel="mainThread" output-channel="gatherChannel">
<!-- make a bunch of outbound calls -->
</int:chain>
<int:chain input-channel="transformResponse" output-channel="backToClient">
<!-- do some stuff and respond back to client -->
</int:chain>
I've had the output channel of the aggregator as both a publish-subscribe and a direct channel and had the issue for both.
When I look at the logs I can see that one of the threads has a preSend to the 'gatherChannel' then AggregatingMessageHandler saying it received the message, then another log from AggregatingMessageHandler saying Handling message with correlationKey [f2b16b6a-3605-778f-a628-870ed8ce3f5e] then a postSend (sent=True) on channel 'gatherChannel'.
I thought that it would not send to the transformResponse channel until both messages that got split out from the splitter got to it. I even added the size() == 2 as the release strategy expression as an extra layer but that doesn't seem to be causing the aggregator to wait either.
I'm a little perplexed why this is happening, it's happening when both the main thread or the spawned thread gets to the aggregator, I'm trying to figure out how to get that aggregator to wait to send to the output channel until BOTH messages that were split from the splitter are received.
According to your current configuration it really does not happen.
Do you really have some evidences that first message in the group is really sent to that transformResponse unconditionally?
You probably is a bit confused with those preSend and postSend since an aggregator is really non-blocking until release. It does accept the message and store it into the MessageStore for grouping if release condition is false. Just after this it returns immediately to the caller, which is that gatherChannel and therefore your see postSend on a first message.
Currently am working with spring integration for new application and started poc to know how to handle the failure cases.
In my application spring integration will receive message from IBM mq and validate the header information and route to different queue depends on the message type. incoming message could be bulk message, so i've used splitter and aggregator from spring integration and am having good progress and control over the technical workflow.
Currently am facing few issues, we have IBM mq and also webservice as our gateway. Both gateway receive the message and send to splitter channel where splitter splits the message and send to outbound channel( executor channel). so message will be send to destination in parallel and status update service activator will receive the message with same channel with order=2 and send to aggregator. so for its good with implementation.
Problem:
if jms outbound gateway throws the execption i've added advise as exception handler which wil send to another service activator to update failure status to DTO object and will have same aggregator channel as output but am not receiving the message in aggregator channel in this case and aggregator receive only in happy flow.
I want to aggregate the outbound successful message and failure message(other service activator update the status) and then the complete status needs to posted to response queue as another outbound or as response in webservice.
i tried to have ordered succesful service activator and failure error handler service activator to have same channel which is input channel for aggregator and its not working.
Appreciated for your guidance to proceed with this workflow
using Spring Integration 2.2.2
<channel id="inbound"/>
<channel id="splitterInChannel"/>
<channel id="splitterOutChannel">
<dispatcher task-executor="splitterExecutor"/>
</channel>
<beans:bean id="splitterExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<beans:property name="corePoolSize" value="5" />
<beans:property name="maxPoolSize" value="10" />
<beans:property name="queueCapacity" value="25" />
</beans:bean>
<channel id="ValidatorChannel"/>
<channel id="outBoundjmsChannel"/>
<channel id="outBoundErrorChannel"/>
<channel id="finalOutputChannel"></channel>
<channel id="aggregatorChannel"/>
<jms:inbound-channel-adapter connection-factory="AMQConnectionFactory"
destination="AMQueue" channel="inbound" auto-startup="true"
extract-payload="false" acknowledge="transacted"></jms:inbound-channel-adapter>
<service-activator ref="InBoundProcessor" input-channel="inbound" output-channel="splitterInChannel"></service-activator>
<!-- splitter -->
<splitter ref="Splitter" method="splitInput" input-channel="splitterInChannel" output-channel="splitterOutChannel"/>
<!-- validator -->
<service-activator ref="Validator" method="validate" input-channel="splitterOutChannel" output-channel="ValidatorChannel"/>
<!-- need to add enricher -->
<service-activator ref="Enricher" method="enrich" input-channel="ValidatorChannel" output-channel="outBoundjmsChannel"/>
<!-- outbound gateway -->
<jms:outbound-channel-adapter channel="outBoundjmsChannel" connection-factory="AMQConnectionFactory" destination-name="outputQueue"
message-converter="customMessageConvertor" order="1" >
<jms:request-handler-advice-chain>
<beans:bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<beans:property name="retryTemplate" >
<beans:bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<beans:property name="retryPolicy">
<beans:bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<beans:property name="maxAttempts" value="2" />
</beans:bean>
</beans:property>
<beans:property name="backOffPolicy">
<beans:bean class="org.springframework.retry.backoff.FixedBackOffPolicy">
<beans:property name="backOffPeriod" value="1000" />
</beans:bean>
</beans:property>
</beans:bean>
</beans:property>
<beans:property name="recoveryCallback">
<beans:bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<beans:constructor-arg ref="outBoundErrorChannel" />
</beans:bean>
</beans:property>
</beans:bean>
</jms:request-handler-advice-chain>
</jms:outbound-channel-adapter>
<!-- outBound error processor -->
<service-activator ref="ErrorProcessor" method="errorHandling" input-channel="outBoundErrorChannel" output-channel="aggregatorChannel" />
<!-- Post send processor -->
<service-activator ref="PostProcessor" method="Postprocessing" input-channel="outBoundjmsChannel" output-channel="aggregatorChannel" order="2"/>
<!-- aggregator -->
<aggregator ref="Aggregator" correlation-strategy-method="aggregateStrategy" input-channel="aggregatorChannel" output-channel="finalOutputChannel"
release-strategy-method="isRelease" method="aggregate" expire-groups-upon-completion="true"/>
<!-- final processor or responder -->
<service-activator ref="FinalProcessor" method="endProcessing" input-channel="finalOutputChannel"/>
</beans:beans>
In the above configuration as of now i've given the release strategy as false and correlation method as empty string if this works, i will generate UUID for the batch and will attach the UUID in splitter to corrlate.
when debugging the above configuration i noticed the outbound error channel receive whenever it attempts to send to the outbound adapter(in my case its send twice ). I don't want to make an reattempt in one of the application and in another application it needs to attempt for reposting the message.
In both case i want to send the message to outbound error channel after the final attempt to aggregate, if fails i will update the status in ErrorProcessor as failed to send.
Two issues.
1. am receiving duplicate message to the channel and difficult to identify the last failure or success.
2.Couldn't make logic for release strategy and difficult to identify which is the duplicate and whether its successful or not.
In the above case i couldn't find a generic way to compare objects because equals method doesn't have proper attributes to compare and it will not be
correct way to compare with boolean field.
please help me out to resolve this issue to proceed my workflow design and completion.
Much appreciated for guiding me to proceed.
Thanks,
Krish S
currently
public Object errorHandling(Object object){
OutBoundMessage outBoundMessage = null;
if(object instanceof MessagingException){
outBoundMessage =((MessagingException) object).getFailedMessage();
}else{
//TODO: log the message
}
return outBoundMessage;
}
public String aggregateStrategy(OutBoundMessage outBoundMessage){
//TODO: get the UUID from outbound message and return
return "";
}
public List<OutBoundMessage> splitter(InBoundMessage inBoundMessage){
String[] message = inBoundMessage.getRawMessage().split(",");
long uuid = java.util.UUID.randomUUID().getLeastSignificantBits();
List<OutBoundMessage> outBoundMessagelist = new ArrayList<OutBoundMessage>();
for (String string : message) {
OutBoundMessage outBoundMessage = new outBoundMessage();
outBoundMessage.setCorrelationID(uuid);
outBoundMessagelist.add(outBoundMessage);
}
}
Added as default false in the following method to validate
public boolean isRelease(List<OutBoundMessage> outBoundMessage){
//TODO: need to define condition for closing the list aggregation
return false;
}
Please, share your ErrorProcessor source code. And correlation-strategy-method="aggregateStrategy" as well.
I would like to know how you deal with ErrorMessage there and how you restore correlationKey from the message after your ErrorProcessor.
Not sure how you build your own correlationKey, but the <splitter> provide applySequence = true by default. So, the Correlation Details are available in each splitted message to be able to aggregate afterwards.
For your ErrorMessage from the ErrorMessageSendingRecoverer I can recommend to pay attention to the Exception payload there. It looks like (from the ErrorMessageSendingRecoverer source code):
else if (!(lastThrowable instanceof MessagingException)) {
lastThrowable = new MessagingException((Message<?>) context.getAttribute("message"),
lastThrowable.getMessage(), lastThrowable);
}
....
messagingTemplate.send(new ErrorMessage(lastThrowable));
So, that MessagingException, has a "guilty" message for the Exception and exactly that message has an appropriate Correlation Details headers for aggregator. Therefore you should rely on them if you'd like to aggregate errors to the same message group.
Finally I understood how it works,
I've a boolean set to true in message convertor and in Errorhandle I set it to false and return null so the recovery is that message is received as failed message to aggregator and understood what happens when I return the object
Thanks #ArtemBilan, your code block gave me an insight of what's happening and what should I do
I'm using Spring Batch 3.0.1.
I have a two-steps job.
First step is a read-process-write chunk. It generates PDF files with iText,and writes in two databases with Atomikos and JTA.
Second one is only a dummy step to log in the file "process ended"
I have configured it, launched it with CommandLineJobRunner way, and the work I expected has been done, both PDFs and databases.
The final message "process ended" is written; and, in batch_job_execution and batch_step_execution database tables, the rows are written with COMPLETED status and exit code, and end time is fulfilled.
So, my problem is that prompt is not coming back, and the process seems to be executing.
When I debug the process with Eclipse, the behaviour is the same: work is done, but the process does not end. Threads related to Atomikos, database pool and step remain as running, and I don't understand why.
The test case has 9 PDFs (items) to process.
The first step configuration is this:
<bean id="printingTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10" />
<property name="maxPoolSize" value="20" />
<property name="queueCapacity" value="50" />
<property name="waitForTasksToCompleteOnShutdown" value="false" />
</bean>
<step id="pdfStep" next="endLoggerStep">
<tasklet transaction-manager="jtaTransactionManager" task-executor="printingTaskExecutor" throttle-limit="10" >
<batch:chunk reader="pdfItemReader" processor="pdfItemProcessor" writer="pdfItemWriter" commit-interval="20">
</batch:chunk>
</tasklet>
</step>
pdfItemReader is a JdbcPagingItemReader with pageSize parameter with 20 value too.
How can I actually finalize the process in command line or Eclipse? Any wrong configuration?
Any help is very appreciated. Thanks
[SOLVED]
I was going mad with this. Finally, when logging with trace level, the last line was:
Invoking destroy method 'close' on bean with name 'atomikosTransactionManager'
I changed the flag forceShutdown to true, and finally the program ends gratefully.
<bean id="atomikosTransactionManager" class="com.atomikos.icatch.jta.UserTransactionManager"
init-method="init" destroy-method="close">
<property name="forceShutdown" value="true" /><!-- Change this to true -->
</bean>
Hope it helps.
You can use System.exit(0); at the end of your main class which will terminate JVM eventually terminating the batch execution.
I have been following previous answers that talk about inserting gateway / service-activator to start a new transaction mid way in SI processing. I have tried below code but for some reason it does not work, if you could point to error in this configuration please.
All the threads seems to be waiting for something to happen and the sp outbound gateway is not invoked, but i cant figure out what.
The idea here is to process each task produced by splitter in a thread pool under a new transaction.
<int:splitter...output-channel="taskChannel"/>
<int:channel id="taskChannel">
<int:dispatcher task-executor="taskExecutor"/>
</int:channel>
<int:gateway id="txGw" service-interface="com.some.test.StartTransactionalGateway"
default-request-channel="taskChannel" default-reply-channel="individualTask">
</int:gateway>
<int:service-activator ref="txGw"
input-channel="taskChannel"
output-channel="individualTask"
method="startTx"
auto-startup="true">
</int:service-activator>
<int-jdbc:stored-proc-outbound-gateway ...request-channel="individualTask" .....
interface StartTransactionalGateway {
#Transactional
Message<?> startTx(Message<?> m);
}
default-request-channel="taskChannel" the gateway is sending messages to itself.
You can't mix channels in the subflow with the main channels. You need something like...
<int:splitter...output-channel="taskChannel"/>
<int:channel id="taskChannel">
<int:dispatcher task-executor="taskExecutor"/>
</int:channel>
<int:service-activator ref="txGw"
input-channel="taskChannel"
output-channel="whereWeWantTheResultsToGo"
method="startTx"
reply-timeout="0"
auto-startup="true" />
<int:gateway id="txGw" service-interface="com.some.test.StartTransactionalGateway"
default-request-channel="toStoredProc" />
<int-jdbc:stored-proc-outbound-gateway ...request-channel="toStoredProd" .....
You don't need a default-reply-channel; simply omit the reply-channel on the stored proc gateway and the reply will automatically go back.
Well, Don't forget that you can mark something transactional not only using #Transactional annotation. There is <tx:advice> for the XML declaration.
If you need to wrap to the TX just only that <int-jdbc:stored-proc-outbound-gateway>, there is <request-handler-advice> sub-element to wrap the handleRequestMessage of the underlying component to any Advice:
<int-jdbc:request-handler-advice-chain>
<tx:advice>
<tx:attributes>
<tx:method name="*"/>
</tx:attributes>
</tx:advice>
</int-jdbc:request-handler-advice-chain>
From other side your code can not work, because <int-jdbc:stored-proc-outbound-gateway> may not return result. The procedure is one-way. But the <gateway> interface is request-reply.
UPDATE
See my answer here Keep transaction within Spring Integration flow. It is another trick how to make down-stream flow transactional.
With <gateway> you should be sure that your transactional subflow returns to the replyChannel in the end. Or... Make your gateway's method as void to achieve one-way behaviour.
I have an application relying on Spring Integration (4.0.4.RELEASE) and RabbitMQ. My flow is as follow:
Messages are put in queue via a process (they do not expect any answer):
Gateway -> Channel -> RabbitMQ
And then drained by another process:
RabbitMQ --1--> inbound-channel-adapter A --2--> chain B --3--> aggregator C --4--> service-activator D --5--> final service-activator E
Explanations & context
The specific thing is that nowhere in my application I am using a splitter: aggregator C just waits for enough messages to come, or for a timeout to expire, and then forwards the batch to service D. Messages can get stuck in aggregator C for quite a long time, and should NOT be considered as consumed there. They should only be consumed once service D successfully completes. Therefore, I am using MANUAL acknowledgement on inbound-channel-adapter A and service E is in charge of acknowledging the batch.
Custom aggregator
I solved the acknowledgement issue I had when set to AUTO by redefining the aggregator. Indeed, messages are acknowledged immediately if any asynchronous process occurs in the flow (see question here). Therefore, I switched to MANUAL acknowledgement and implemented the aggregator like this:
<bean class="org.springframework.integration.config.ConsumerEndpointFactoryBean">
<property name="inputChannel" ref="channel3"/>
<property name="handler">
<bean class="org.springframework.integration.aggregator.AggregatingMessageHandler">
<constructor-arg name="processor">
<bean class="com.test.AMQPAggregator"/>
</constructor-arg>
<property name="correlationStrategy">
<bean class="com.test.AggregatorDefaultCorrelationStrategy" />
</property>
<property name="releaseStrategy">
<bean class="com.test.AggregatorMongoReleaseStrategy" />
</property>
<property name="messageStore" ref="messageStoreBean"/>
<property name="expireGroupsUponCompletion" value="true"/>
<property name="sendPartialResultOnExpiry" value="true"/>
<property name="outputChannel" ref="channel4"/>
</bean>
</property>
</bean>
<bean id="messageStoreBean" class="org.springframework.integration.store.SimpleMessageStore"/>
<bean id="messageStoreReaperBean" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="messageStore" />
<property name="timeout" value="${myapp.timeout}" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="messageStoreReaperBean" method="run" fixed-rate="2000" />
</task:scheduled-tasks>
I wanted indeed to aggregate the headers in a different way, and keep the highest value of all the amqp_deliveryTag for later multi-acknoledgement in service E (see this thread). This works great so far, apart from the fact that it is far more verbose than the typical aggregator namespace (see this old Jira ticket).
Services
I am just using basic configurations:
chain-B
<int:chain input-channel="channel2" output-channel="channel3">
<int:header-enricher>
<int:error-channel ref="errorChannel" /> // Probably useless
</int:header-enricher>
<int:json-to-object-transformer/>
<int:transformer ref="serviceABean"
method="doThis" />
<int:transformer ref="serviceBBean"
method="doThat" />
</int:chain>
service-D
<int:service-activator ref="serviceDBean"
method="doSomething"
input-channel="channel4"
output-channel="channel5" />
Error management
As I rely on MANUAL acknowledgement, I need to manually reject messages as well in case an exception occurs. I have the following definition for inbound-channel-adapter A:
<int-amqp:inbound-channel-adapter channel="channel2"
queue-names="si.queue1"
error-channel="errorChannel"
mapped-request-headers="*"
acknowledge-mode="MANUAL"
prefetch-count="${properties.prefetch_count}"
connection-factory="rabbitConnectionFactory"/>
I use the following definition for errorChannel:
<int:chain input-channel="errorChannel">
<int:transformer ref="errorUnwrapperBean" method="unwrap" />
<int:service-activator ref="amqpAcknowledgerBean" method="rejectMessage" />
</int:chain>
ErrorUnwrapper is based on this code and the whole exception detection and message rejection works well until messages reach aggregator C.
Problem
If an exception is raised while processing the messages in service-activator D, then I see this exception but errorChannel does not seem to receive any message, and my ErrorUnwrapper unwrap() method is not called. The tailored stack traces I see when an Exception("ahahah") is thrown are as follow:
2014-09-23 16:41:18,725 ERROR o.s.i.s.SimpleMessageStore:174: Exception in expiry callback
org.springframework.messaging.MessageHandlingException: java.lang.Exception: ahahaha
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:78)
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:71)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:170)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
(...)
Caused by: java.lang.Exception: ahahaha
at com.myapp.ServiceD.doSomething(ServiceD.java:153)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
(...)
2014-09-23 16:41:18,733 ERROR o.s.s.s.TaskUtils$LoggingErrorHandler:95: Unexpected error occurred in scheduled task.
org.springframework.messaging.MessageHandlingException: java.lang.Exception: ahahaha
(...)
Question
How can one tell the services that process messages coming from such an aggregator to publish errors to errorChannel? I tried to specify in the header via a header-enricher the error-channel with no luck. I am using the default errorChannel definition, but I tried as well to change its name and redefine it. I am clueless here, and even though I found this and that, I have not managed to get it to work. Thanks in advance for your help!
As you see by StackTrace your process is started from the MessageGroupStoreReaper Thread, which is initiated from the default ThreadPoolTaskScheduler.
So, you must provide a custom bean for that:
<bean id="scheduler" class="org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler">
<property name="errorHandler">
<bean class="org.springframework.integration.channel.MessagePublishingErrorHandler">
<property name="defaultErrorChannel" ref="errorChannel"/>
</bean>
</property>
</bean>
<task:scheduled-tasks scheduler="scheduler">
<task:scheduled ref="messageStoreReaperBean" method="run" fixed-rate="2000" />
</task:scheduled-tasks>
However I see the benefits from having the error-channel on the <aggregator>, where we really have several points from different detached Threads, with wich we can't get deal normally.