I am trying to create a generic Error Handling process for All message Handlers that will be used in my SI flow. This will,
1. Retry on Connection Exception.
2. Stop the SI flow using Circuit Breaker.
3. Rollback the failed message to the channel.
I have achieved the Retry and Circuit Break Functionality. But, I am unable to rollback message to the channel.
I tried using transaction advice. But it does not work.
Here is the code.
<bean id="retryAdvice"
class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="retryTemplate">
<bean class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="2000" />
<property name="multiplier" value="2" />
</bean>
</property>
</bean>
</property>
<property name="recoveryCallback">
<bean
class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref="recoveryChannel" />
</bean>
</property>
<property name="retryStateGenerator">
<bean
class="org.springframework.integration.handler.advice.SpelExpressionRetryStateGenerator">
<constructor-arg value="headers['uniqueId']" />
</bean>
</property>
</bean>
<int:channel id="recoveryChannel" />
<int:transformer id="defaultTransformer" input-channel="recoveryChannel"
output-channel="loggerChannel" ref="defaultTransformer" method="transform">
</int:transformer>
<int:logging-channel-adapter id="loggerChannel"
level="INFO" log-full-message="true" auto-startup="true">
</int:logging-channel-adapter>
<bean id="defaultTransformer"
class="com.bestbuy.ingestion.foundation.core.util.DefaultTransformer" />
<bean id="circuitBreakerAdvice"
class="org.springframework.integration.handler.advice.RequestHandlerCircuitBreakerAdvice">
<property name="threshold" value="2" /> <!-- close after 2 failures -->
<property name="halfOpenAfter" value="60000" /> <!-- half open after 15 seconds -->
</bean>
<tx:advice id="txansactionAdvice" transaction-manager="transactionManager">
</tx:advice>
What type of Transaction Manager I need to use.
I may be using different message Handlers, on different data source.
Here is how I add these advises to the Message Handlers.
public Object postProcessBeforeInitialization(Object bean, String beanName)
throws BeansException {
logger.error("called for bean id :: "+beanName+" with bean class "+bean.getClass().getName());
if(bean instanceof AbstractSimpleMessageHandlerFactoryBean){
logger.error("************ Bean "+beanName+" is instance of AbstractSimpleMessageHandlerFactoryBean **********");
}
if(bean instanceof ConsumerEndpointFactoryBean){
logger.error("Bean is of type ConsumerEndpointFactoryBean");
return populateRequestHandlerAdviceChain((ConsumerEndpointFactoryBean)bean);
}
if(bean instanceof AbstractSimpleMessageHandlerFactoryBean){
logger.error("Bean is of type AbstractSimpleMessageHandlerFactoryBean");
return populateRequestHandlerAdviceChain((AbstractSimpleMessageHandlerFactoryBean<?>)bean);
}
return bean;
}
private Object populateRequestHandlerAdviceChain(ConsumerEndpointFactoryBean bean){
ArrayList<Advice> list = new ArrayList<Advice>();
logger.error("Adding Retry Advice");
list.add((Advice)factory.getBean("retryAdvice"));
logger.error("Adding Cricuit Breaker Advice");
list.add((Advice)factory.getBean("circuitBreakerAdvice"));
logger.error("Adding Transactional Advice");
list.add((Advice)factory.getBean("txansactionAdvice"));
bean.setAdviceChain(list);
return bean;
}
If the bean of type ConsumerEndpointFactoryBean I add these advices. I need transaction management in all these Handlers.
First of all: since your txansactionAdvice is nested to the retryAdvice you rollback here the each retry.
From other side it isn't clear why you apply the circuitBreakerAdvice for each retry. I'd say this patter would be better to use 'around' retryAdvice.
And the txansactionAdvice should on top. So, it might look like this:
txansactionAdvice
circuitBreakerAdvice
retryAdvice
And one more point: your transaction won't be rolled back, bacause your use recoveryCallback, which just sends ErrorMessage and strangles the Exception.
HTH and you'll change your mind on the matter.
Related
I implemented a sftp-inbound-channel-adapter, and when an exception is handled , i should display a customized message.
I tried :
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
auto-startup="true" channel="receiveChannel" session-factory="sftpSessionFactory"
local-directory="file:${directory.files.local}" remote-directory="${directory.files.remote}"
auto-create-local-directory="true" delete-remote-files="true"
filename-pattern="*.txt" >
<int:poller fixed-delay="${sftp.interval.request}"
max-messages-per-poll="-1" />
<int-sftp:request-handler-advice-chain>
<bean: class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<property name="onSuccessExpression" value="payload" />
<property name="successChannel" ref="afterSuccessDeleteChannel" />
<property name="onFailureExpression" value="payload.renameTo(new java.io.File(payload.absolutePath + '.failed.to.send'))" />
<property name="failureChannel" ref="afterFailRenameChannel" />
</bean>
</int-sftp:request-handler-advice-chain>
But an element
<int-sftp:request-handler-advice-chain>
is not accepted. Can you explain another solution?
The request handler advice goes on some downstream component, not an inbound channel adapter.
You can add an error-channel to the <poller/> element. The message sent to the error channel will be an ErrorMessage with the exception as a payload. If it's an exception on the downstream flow, the payload will be a MessagingException with failedMessage and cause properties.
Add some component to consume the error messages.
I'm using JPA and Hibernate for persistence, with some autoconfiguration help from Spring Boot. I'm running a JUnit test which saves some record in the JPA Repository. Then it instantiates a new Spring-managed-thread, and it's run by ThreadPoolTaskExecutor. That thread will try to get that previously added record with no success.
Here is the relevant code of the test and the runnable thread:
public class RtmpSpyingTests extends AbstractTransactionalJUnit4SpringContextTests {
#Autowired
ThreadPoolTaskExecutor rtmpSpyingTaskExecutor;
#Autowired
ApplicationContext ctx;
#Autowired
RtmpSourceRepository rtmpRep;
#Test
public void test() {
RtmpSource rtmpSourceSample = new RtmpSource("test");
rtmpRep.save(rtmpSourceSample);
rtmpRep.flush();
List<RtmpSource> rtmpSourceList = rtmpRep.findAll(); // Here I get a list containing rtmpSourceSample
RtmpSpyingTask rtmpSpyingTask = ctx.getBean(RtmpSpyingTask.class,
"arg1","arg2");
rtmpSpyingTaskExecutor.execute(rtmpSpyingTask);
}
}
public class RtmpSpyingTask implements Runnable {
#Autowired
RtmpSourceRepository rtmpRep;
String nameIdCh;
String rtmpUrl;
public RtmpSpyingTask(String nameIdCh, String rtmpUrl) {
this.nameIdCh = nameIdCh;
this.rtmpUrl = rtmpUrl;
}
public void run() {
// Here I should get a list containing rtmpSourceSample, but instead of that
// I get an empty list
List<RtmpSource> rtmpSource = rtmpRep.findAll();
}
}
So, after I have inserted the rtmpSourceSample object, I can check it's been inserted from the test method, it is indeed on the rtmpSourceList list. However, when I do the same from the thread, what I get is an empty list.
Here is the JPA/Hibernate configuration in my spring-context.xml configuration file:
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;MVCC=true" />
<property name="username" value="user" />
<property name="password" value="user" />
</bean>
<!-- Define the JPA transaction manager -->
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<constructor-arg ref="entityManagerFactory" />
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter" ref="vendorAdaptor" />
<property name="packagesToScan" value="guiatv.persistence.domain" />
</bean>
<bean id="abstractVendorAdaptor" abstract="true">
<property name="generateDdl" value="true" />
<property name="database" value="H2" />
<property name="showSql" value="false" />
</bean>
<bean id="entityManager"
class="org.springframework.orm.jpa.support.SharedEntityManagerBean">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="vendorAdaptor"
class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"
parent="abstractVendorAdaptor">
</bean>
<context:annotation-config />
<tx:annotation-driven />
<context:component-scan base-package="guiatv.persistence" />
Note that the persistence-unit.xml is not needed since I'm using Spring Boot.
And finally this is the xml configuration of the taskexecutor bean and the runnable thread:
<bean id="rtmpSpyingTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="5" />
<property name="queueCapacity" value="5" />
</bean>
<bean id="rtmpSpyingTask" class="guiatv.realtime.rtmpspying.RtmpSpyingTask"
scope="prototype">
<constructor-arg ref="rtmpSpyingTaskExecutor" />
</bean>
I've been looking for topics about this problematic Spring's persistence and threading combination. One of the solutions I've found so far is to create some #Service annotated class with a #Transactional method, which I am supposed to call from my run() method. It does not work for me.
Some other solutions involve the use of the EntityManager or some other Hibernate dependent bean, instead of querying the JPA Repository directly. Doesn't work too.
So, any solution that could fit my needs? Thank you!
SOLUTION (from cproinger):
Create a #Service annotated class:
#Service
public class AsyncTransactionService {
#Autowired
RtmpSourceRepository rtmpRep;
#Transactional(readOnly = true)
public List<RtmpSource> getRtmpSources() {
return rtmpRep.findAll();
}
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void insertRtmpSource(RtmpSource rtmpSource) {
rtmpRep.save(rtmpSource);
}
}
Then, I autowire that AsyncTransactionService from both the JUnit test, and the Runnable class. In order to insert the record from JUnit test I invoke insertRtmpSource(), and then I get the records from my thread by calling getRtmpSources().
I tried to do it without the #Service. This is, by putting an annotated insertRtmpSource() method on my JUnit class and a getRtmpSources() method on my Runnable class, but it did not work.
Thank you for you quick response cproinger!
the thread does not see the record because the test runs inside a transaction that is not committed yet. since the transaction is bound to the executing thread the task that is forked does not use the same transaction. in order for this to work the insert must run in a method that is annotated with #Transactional(propagation = REQUIRES_NEW). please note that transactional rollback will not work then though
I have implemented the DynamicFtpChannelResolver found in the spring integration samples in order to allow dynamic FTP locations in my integration app.
As part of the outbound adapter I have added an advice chain. The FtpCompleteAdvice requires access to an existing service bean but at runtime this is not injected presumably because the context is dynamically created.
Is there a way for autowiring to work or another way to get access to this service bean?
Here is an extract of the xml:
<int:channel id="toFtpChannel" />
<bean id="ftpClientFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="${host}" />
<property name="port" value="${port}" />
<property name="user" value="${user}" />
<property name="password" value="${password}" />
</bean>
<int-sftp:outbound-channel-adapter id="ftpOutbound" session-factory="ftpClientFactory" channel="toFtpChannel" remote-directory="${remote.directory}" remote-file-separator="/" remote-filename-generator-expression="headers.filename">
<int-sftp:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="retryTemplate" ref="retryTemplate" />
</bean>
<bean class="com.bstonetech.ptms.integration.util.FtpCompleteAdvice">
<property name="interfaceType" value="OUTBOUND" />
<property name="interfaceName" value="TEST" />
</bean>
</int-sftp:request-handler-advice-chain>
</int-sftp:outbound-channel-adapter>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="3" />
</bean>
</property>
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="maxInterval" value="600000" />
<property name="initialInterval" value="3000" />
<property name="multiplier" value="2" />
</bean>
</property>
</bean>
public class FtpCompleteAdvice extends AbstractRequestHandlerAdvice {
#Autowired
private IEmailUtilities emailUtilities;
#Autowired
private IFileService fileService;
private String interfaceType;
private String interfaceName;
public void setInterfaceType(String interfaceType) {
this.interfaceType = interfaceType;
}
public void setInterfaceName(String interfaceName) {
this.interfaceName = interfaceName;
}
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) throws Exception {
Object result = callback.execute();
String filename = (String)message.getHeaders().get("filename");
//insert ftp row into file_ctl
fileService.insertFtpFile(filename, interfaceName, interfaceType, new Date());
//send email to confirm ftp
emailUtilities.afterFtp(filename, interfaceName);
return result;
}
}
You need to make the new context a child of the main context that has those beans. This is discussed in the forum posts referenced in the sample's README, when using a similar technique for inbound endpoints.
Then, any beans in the parent context are available for wiring.
The jdbc inbound channel adapter and message stores in our application consistenly stops querying for data and I've been able to track it down to a blocked Oracle session. In Oracle, we are getting enq: TX - row lock contention. The DBA tracked it down to the following:
SELECT COMPLETE, LAST_RELEASED_SEQUENCE, CREATED_DATE, UPDATED_DATE
from INT_MESSAGE_GROUP
where GROUP_KEY = :1 and REGION=:2
Any suggestions on how to resolve this would be greatly appreciated.
SI configuration (extract):
<int-jdbc:message-store id="jdbc-messageStore" data-source="dataSource" />
<int-jdbc:inbound-channel-adapter id="JDBCInboundChannel" query="${cache.integration.jdbc.selectQuery}"
channel="inboundMessagesChannel" data-source="dataSource" update="update CACHE_REPOSITORY set STATUS='P' WHERE GUID IN (:guid)" row-mapper="rowMapper"
max-rows-per-poll="${cache.integration.jdbc.maxRowsPerPoll}" auto-startup="false" >
<int:poller id="jdbcPoller" fixed-delay="${cache.integration.jdbc.fixedDelay}" >
<int:transactional />
</int:poller>
</int-jdbc:inbound-channel-adapter>
<int:chain...>
<int:aggregator id="reportTypeAggregator" ref="aggregatorBean" method="collect"
message-store="jdbc-messageStore" release-strategy="releaseStrategry"
release-strategy-method="canRelease" correlation-strategy="reportTypeCorrelationStrategry"
correlation-strategy-method="correlate" send-partial-result-on-expiry="true"
expire-groups-upon-completion="true" empty-group-min-timeout="30000" />
</int:chain>
<bean id="queryProvider" class="org.springframework.integration.jdbc.store.channel.OracleChannelMessageStoreQueryProvider"/>
<bean id="jdbc-channel-messageStore" class="org.springframework.integration.jdbc.store.JdbcChannelMessageStore">
<property name="dataSource" ref="dataSource"/>
<property name="channelMessageStoreQueryProvider" ref="queryProvider"/>
<property name="region" value="${cache.integration.channelMessageStore.region}"/>
</bean>
<int:channel id="archiveCreationChannel" >
<int:queue message-store="jdbc-channel-messageStore" />
<int:interceptors>
<int:wire-tap channel="logger" timeout="-1"/>
</int:interceptors>
</int:channel>
<bean id="documentMessageStoreReaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="jdbc-messageStore" />
<property name="timeout" value="${cache.integration.reaper.timeout}" />
<property name="autoStartup" value="false" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="documentMessageStoreReaper" method="run" fixed-rate="10000" />
</task:scheduled-tasks>
I suppose the main issue is around fixed-rate for documentMessageStoreReaper, which initiate the DELETE from INT_MESSAGE_GROUP. You should change it to the fixed-dealy to avoid concurrent tasks.
Since you say that the DELETE from INT_MESSAGE_GROUP is very long, hence 10 sec between tasks might be not enough.
In the below xml configuartion, i have a sql query which needs to be injected to empDAO.
<bean id="propertyPlaceholderConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>/WEB-INF/conf/db.properties</value>
<value>/WEB-INF/conf/query.properties</value>
</list>
</property>
</bean>
<bean id="empDAO" class="com.dao.EmployeeDAO">
<!-- How to do Annotation-based autowire for the string-->
<property name="selectTradeQ" value="${select.emp}" />
</bean>
My question is How to use Annotation-autowire for the String? Some thing like below
//This is not possible ?? Then how to do this
<bean id="selectTradeQ" value="${select.emp}>
#Component
public class EmployeeDAO {
#Value("${select.emp}")
private String selectTradeQ;
}