Translate JMS Queue XML config to Java config - spring-integration

In order to learn spring integration I've been attempting to create a simple, resilient log processor. I'm also wanting to stick with a java configuration approach.
I've been having a difficult time translating existing XML configuration, mostly due to being so new to spring in general.
In a question on the spring forums Gary Russell presented a similar solution to this using a publish-subscribe + JMS model with a simple XML config.
I've been attempting to translate his suggestion into a Java config, but am stuck. Namely I'm not sure of the proper entities to use for the outbound-channel-adapter, service-activators or how to set the order of messages properly.
Here is Gary's XML config:
<int-file:inbound-channel-adapter id="dispatcher"
directory="spool"
channel="fileChannel">
<int:poller fixed-delay="2000">
<int:transactional/>
</int:poller>
</int-file:inbound-channel-adapter>
<int:channel id="fileChannel" />
<int-file:file-to-string-transformer input-channel="fileChannel" output-channel="dispatchChannel" />
<int:publish-subscribe-channel id="dispatchChannel" />
<int-jms:outbound-channel-adapter id="dispatcherJms" channel="dispatchChannel" order="1"
connection-factory="connectionFactory"
destination="dispatcher.queue" />
<!-- If JMS Send was successful, remove the file (within the transaction)-->
<int:service-activator input-channel="dispatchChannel" order="2"
output-channel="nullChannel"
expression="headers.file_originalFile.delete()">
<bean id="transactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
UPDATE
Based on the comments below I've updated the java config.
However I'm still receiving errors and most likely am not understanding the flow and connections between the entities, but the original question has been answered.
#Bean
#Transactional
#InboundChannelAdapter(channel = "dispatchChannel", poller = #Poller(fixedDelay = "2000"))
public MessageSource<?> dispatcher() {
CompositeFileListFilter<File> filters = new CompositeFileListFilter<>();
filters.addFilter(new SimplePatternFileListFilter(sourceFilenamePattern));
//filters.addFilter(persistentFilter());
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File(sourceDirectory));
source.setFilter(filters);
return source;
}
#Bean
public MessageChannel fileChannel() {
return new DirectChannel();
}
#Bean
public PublishSubscribeChannel dispatchChannel() {
return new PublishSubscribeChannel();
}
#Autowired
JmsTemplate jmsTemplate;
#Autowired
ConnectionFactory connectionFactory;
#Bean
#Order(1)
#ServiceActivator(inputChannel = "dispatchChannel")
public MessageHandler dispatcherJmsOutboundChannelAdapter(Message<File> message) {
JmsSendingMessageHandler handler = new JmsSendingMessageHandler(jmsTemplate);
handler.setDestinationName("dispatcher.queue");
return handler;
}
#Bean
#Order(2)
#ServiceActivator(inputChannel = "dispatchChannel")
public void removeFile(Message<?> message) {
//message.getHeaders().get(FileHeaders.ORIGINAL_FILE, File.class).delete();
log.info("delete");
}
#Bean
public JmsTransactionManager transactionManager(ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
I'm using spring boot and several starter components, such as activemq. I've added the #Bean for JmsListenerContainerFactory and a #JmsListener, though I'm not sure those are truly necessary.
I couldn't get anything to run until adding #EnableJms to my configuration file as well as #Autowiring the jmstemplate and connectionfactory.
When running, the error I'm receiving now is:
org.springframework.beans.factory.NoSuchBeanDefinitionException:
No qualifying bean of type [org.springframework.messaging.Message] found for dependency
[org.springframework.messaging.Message<?>]:
expected at least 1 bean which qualifies as autowire candidate for this dependency.
Dependency annotations: {}

This one
<int:service-activator input-channel="dispatchChannel" order="2"
output-channel="nullChannel"
expression="headers.file_originalFile.delete()">
is pretty simple in Java:
#ServiceActivator(inputChannel = "dispatchChannel")
public void removeFile(Message<?> message) {
message.getHeaders().get(FileHeaders.ORIGINAL_FILE, File.class).delete();
}
and
<int-jms:outbound-channel-adapter>
is translated to this:
#Bean
#ServiceActivator(inputChannel = "dispatchChannel")
public MessageHandler dispatcherJmsOutboundChannelAdapter() {
JmsSendingMessageHandler handler =
new JmsSendingMessageHandler(new JmsTemplate(this.connectionFactory));
handler.setDestinationName("dispatcher.queue");
return handler;
}
Pay attention to this paragraph in the Reference Manual.

The last piece of jigsaw puzzle is FileWritingMessageHandler
#Bean
public FileWritingMessageHandler fileWritingMessageHandler() {
SpelExpressionParser parser = new SpelExpressionParser();
Expression expression = parser.parseExpression("headers.file_originalFile.delete()");
FileWritingMessageHandler fileWritingMessageHandler = new FileWritingMessageHandler(expression);
fileWritingMessageHandler.setOutputChannel(new NullChannel());
fileWritingMessageHandler.setDeleteSourceFiles(true);
return fileWritingMessageHandler;
}

Related

Exception Behaviour without errorChannel on inbound adapters

I have int-aws:sqs-message-driven-channel-adapter on which if I set the errorChannel, the downstream exceptions go there.
However, when I don't set an errorChannel, the exception does not get logged. It does not go to the errorChannel which is expected. Is there a way, that such exceptions at least get logged? Is there a default errorlogger which can simply log such errors?
UPDATE
Posting XML and DSL config as per the comments. The error is simulated in the persistence layer by setting null for a #NotBlank field on the ServiceObject.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-aws="http://www.springframework.org/schema/integration/aws"
xmlns:int-jpa="http://www.springframework.org/schema/integration/jpa"
xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/jpa https://www.springframework.org/schema/integration/jpa/spring-integration-jpa.xsd
http://www.springframework.org/schema/integration/aws https://www.springframework.org/schema/integration/aws/spring-integration-aws.xsd">
<int:channel id="serviceLogChannel">
<int:interceptors>
<int:wire-tap channel="loggingChannel"/>
</int:interceptors>
</int:channel>
<int-aws:sqs-message-driven-channel-adapter sqs="amazonSQS"
auto-startup="true"
channel="serviceLogChannel"
id="sqsMessageDrivenChannelAdapter"
queues="${app.queue-name}"
max-number-of-messages="10"
visibility-timeout="5"
wait-time-out="20"
error-channel="errorChannel"/>
<int:chain input-channel="serviceLogChannel">
<int:json-to-object-transformer type="ServiceObject"/>
<int-jpa:outbound-channel-adapter entity-class="ServiceObject"
persist-mode="PERSIST"
entity-manager-factory="entityManagerFactory">
<int-jpa:transactional/>
</int-jpa:outbound-channel-adapter>
</int:chain>
<int:logging-channel-adapter log-full-message="true"
logger-name="tapInbound"
id="loggingChannel"/>
<int:service-activator input-channel="errorChannel" expression="#reThrow.rethrow(payload)" order="100"/>
</beans>
The ReThrow service-activator:
#Component
public class ReThrow {
public void rethrow(Exception exception) throws Exception {
throw exception;
}
}
The DSL config for the same is :
#Configuration
public class IntegrationConfiguration {
#Bean
public MessageProducer createSqsMessageDrivenChannelAdapter(
AmazonSQSAsync amazonSQSAsync,
MessageChannel serviceChannel,
MessageChannel errorChannel,
#Value("${app.queue-name}") String queueName) {
SqsMessageDrivenChannelAdapter adapter =
new SqsMessageDrivenChannelAdapter(amazonSQSAsync, queueName);
adapter.setVisibilityTimeout(5);
adapter.setWaitTimeOut(20);
adapter.setAutoStartup(true);
adapter.setMaxNumberOfMessages(10);
adapter.setOutputChannel(serviceChannel);
adapter.setErrorChannel(errorChannel);
adapter.setMessageDeletionPolicy(SqsMessageDeletionPolicy.NO_REDRIVE);
return adapter;
}
#Bean
public IntegrationFlow messageProcessingFlow(
MessageChannel serviceChannel, EntityManagerFactory entityManagerFactory) {
return IntegrationFlows.from(serviceChannel)
.transform(Transformers.fromJson(ServiceObject.class))
.handle(
Jpa.outboundAdapter(entityManagerFactory)
.entityClass(ServiceObject.class)
.persistMode(PersistMode.PERSIST),
e -> e.transactional())
.get();
}
#Bean
public IntegrationFlow errorProcessingFlow(MessageChannel errorChannel) {
return IntegrationFlows.from(errorChannel)
.handle(
m -> {
throw (RuntimeException) m.getPayload();
})
.get();
}
#Bean
public MessageChannel serviceChannel() {
return MessageChannels.publishSubscribe().get();
}
}
The SqsMessageDrivenChannelAdapter is fully based on the SimpleMessageListenerContainerFactory from Spring Cloud AWS and that one just delegates to a listener we provide. Looking to the code there is just no any error handling. So, the best way to deal with it at the moment to explicitly set an error-channel="errorChannel" and it is going to be logged via default logger subscribed to that global errorChannel.
And yes: it is not expected to go to the errorChannel by default. I'm not sure that there is such an official claim in our docs. Probably better to think about it as "no error channel by default", so it is up to underlying protocol client to handle thrown errors. Since there is no one there, then we don't have choice unless set error channel explicitly.

How to have Spring ContextConfiguration load both from XML and from JavaConfig

From what I've read of the Spring #ContextConfiguration annotation, it's possible to load multiple XML context files, or multiple JavaConfig classes. What I need is to load from one XML context file and one class. All the examples I've seen either load all XML, or all classes, but not both.
I'm trying to do this because I want my test class, which is just there to verify expected Spring wiring, to load my default applicationContext.xml file (presently just a copy stored in "src/test/resources, and trying to figure out how to directly specify the default one) along with a JavaConfig class that specifies some JNDI resources that need to be available. For the purposes of my test, I only need to set those JNDI resources to dummy strings, but I'd really like to specify them in an inline static class instead of an external XML file, because my tests are going to have to verify that some settings are equal to those dummy strings, and it's more maintainable if both the values and the checks are in the same file.
What I have so far, and what I've tried, can be illustrated with this:
#RunWith(SpringRunner.class)
#ContextConfiguration(value = {"/testApplicationContext.xml", "/testResources.xml"})
//#ContextHierarchy({
// #ContextConfiguration("/testApplicationContext.xml"),
// #ContextConfiguration(classes = SpringWiringTest.Config.class)
//})
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
And this at the end of the class:
#Configuration
public static class Config {
#Bean public String uslDatasourcesList() { return "abc"; }
#Bean public String atgDatasourcesList() { return "abc"; }
#Bean public String uslTableNamePrefixsList() { return "abc"; }
#Bean public String atgTableNamePrefixsList() { return "abc"; }
#Bean public String doNotifications() { return "false"; }
#Bean public DataSource abc() { return new DriverManagerDataSource(); }
}
If I comment out the first #ContextConfiguration and comment back in the #ContextHierarchy block, I get an error like this:
Error creating bean with name 'uslDatasourcesList': Invocation of init
method failed; nested exception is
javax.naming.NoInitialContextException: Need to specify class name in
environment or system property, or as an applet parameter, or in an
application resource file: java.naming.factory.initial
Update:
Using the guideline of picking either JavaConfig or XML as the "entry point" to configuration, here are some modified excerpts that show what I'm trying:
#RunWith(SpringRunner.class)
#ContextConfiguration
//#ContextConfiguration(value = {"file:src/main/webapp/WEB-INF/applicationContext.xml", "/testResources.xml"})
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
...
#BeforeClass
public static void setup() throws Exception {
SimpleNamingContextBuilder builder = SimpleNamingContextBuilder.emptyActivatedContextBuilder();
DataSource ds = new DriverManagerDataSource();
builder.bind("java:comp/env/abc", ds);
}
...
#Configuration
#ImportResource("file:src/main/webapp/WEB-INF/applicationContext.xml")
public static class Config {
#Bean public String uslDatasourcesList() { return "abc"; }
#Bean public String atgDatasourcesList() { return "abc"; }
#Bean public String uslTableNamePrefixsList() { return "abc"; }
#Bean public String atgTableNamePrefixsList() { return "abc"; }
#Bean public String doNotifications() { return "false"; }
#Bean public DataSource abc() { return new DriverManagerDataSource(); }
}
}
When I run my test, the bottom "Caused by" in the exception says this:
Caused by: javax.naming.NameNotFoundException: Name
[uslDatasourcesList] not bound; 1 bindings: [java:comp/env/abc]
In the alternate version, using the commented-out "#ContextConfiguration" (and commenting out the Config class and its annotations), this error does not occur.
Note that this the meat of my "testResources.xml" file:
<bean id="uslDatasourcesList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="atgDatasourcesList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="uslTableNamePrefixList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="atgTableNamePrefixList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="doNotifications" class="java.lang.String"> <constructor-arg value="false"/> </bean>
<bean id="abc" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
</bean>
Note that the bean mentioned in the error message, "uslDatasourcesList" is defined in both versions, but it's not working in the version with JavaConfig and XML mixed.
It almost appears that the beans in the "#ImportResource" annotation are evaluated on their own, before the beans declared in the JavaConfig class are merged into it.
This is clearly documented in the Spring Reference Manual in the section named Mixing XML, Groovy scripts, and annotated classes.
In summary, ...
... you will have to pick one as the entry point, and that one will have to include or import the other.
Thus, the following should hopefully solve your problem.
#RunWith(SpringRunner.class)
#ContextConfiguration
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
// ...
#Configuration
#ImportResource({"/testApplicationContext.xml", "/testResources.xml"})
static class Config {
// ...
}
}

creating pollable message source

I am trying to create a pollable message source and I have tried to do that by extending MessageProducerSupport, however I was able to see the message from receive method only once and was not successful in making it pollable. (The receive method is not getting called based on my polling schedule.)
My code snippet is as below:
#Component
public class MyAdapter extends MessageProducerSupport {
#Override
protected void doStart() {
receive();
}
public void receive() {
System.out.println("polled at : "+ new Date());
sendMessage(MessageBuilder.withPayload("Hello WOrld! "+ new Date()).build());
}
}
And my applicationContext is as below:
<context:component-scan base-package="com.mypackage" />
<context:annotation-config />
<bean id="pollerTaskExecutor" class="org.springframework.core.task.SyncTaskExecutor"/>
<int:inbound-channel-adapter ref="myAdapter" channel="output">
<int:poller task-executor="pollerTaskExecutor">
<int:interval-trigger interval="3000" fixed-rate="true" time-unit="MILLISECONDS"/>
</int:poller>
</int:inbound-channel-adapter>
I would like to know what am I missing to make this message source pollable.
You are right: the pollable message source is based . erm... on org.springframework.integration.core.MessageSource.
So, to make it working you just should move your MessageProducerSupport code to the AbstractMessageSource implementation.
See more info in the Reference Manual.

How to convert Spring Integration XML to Java DSL for errorChannel

I have the below xml configuration in my application and I would like to convert it to the Java DSL.
So in this reference I'm explicitly defining the name for the error channel. Mostly for example reason. With this reference what I'm expecting to happen is when a downstream process throws and exception that it should route the payload back through the error channel. What would the Java code look like?
<int-jms:message-driven-channel-adapter
id="notification"
connection-factory="connectionFactory"
destination="otificationQueue"
channel="notificationChannel"
error-channel="error"
/>
<int:chain id="chainError" input-channel="error">
<int:transformer id="transformerError" ref="errorTransformer" />
<int-jms:outbound-channel-adapter
id="error"
connection-factory="connectionFactory"
destination="errorQueue" />
</int:chain>
#Bean
public IntegrationFlow jmsMessageDrivenFlow() {
return IntegrationFlows
.from(Jms.messageDriverChannelAdapter(this.jmsConnectionFactory)
.id("notification")
.destination(this.notificationQueue)
.errorChannel(this.errorChannel))
...
.get();
}
#Bean
public IntegrationFlow ErrorFlow() {
return IntegrationFlows
.from(this.errorChannel)
.transform(errorTransformer())
.handle(Jms.outboundAdapter(this.jmsConnectionFactory)
.destination(this.errorQueue), e -> e.id("error"))
.get();
}
EDIT:
#Bean
public MessageChannel errorChannel() {
return new DirectChannel();
}
and autowire it, or reference it as
.errorChannel(errorChannel())

Spring equivalent of CompletionService?

In my app I have to process multiple jobs asynchronously from the main application thread and collect the result of each job. I have a plain Java solution that does this using a ExecutorService and a ExecutorCompletionService that collects the job results.
Now I would like to convert my code to a Spring solution. The docs show me how the use the ExecutorService and the #Async annotation, but I am not sure how and if I can collect the results of multiple jobs.
In other words: I am looking for the Spring equivalent of the CompletionService. Is there such a thing?
My current code:
class MyService {
private static ExecutorService executorService;
private static CompletionService<String> taskCompletionService;
// static init block
static {
executorService = Executors.newFixedThreadPool(4);
taskCompletionService = new ExecutorCompletionService<String>(executorService);
// Create thread that keeps looking for results
new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
Future<String> future = taskCompletionService.take();
String s = future.get();
LOG.debug(s);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}
}).start();
}
// This method can and will be called multiple times,
// so multiple jobs are submitted to the completion service
public void solve(List<Long> ids) throws IOException, SolverException {
String data = createSolverData(ids);
taskCompletionService.submit(new SolverRunner(data, properties));
}
}
You need to consider what's your main goal, because your current code will work fine alongside other Spring-associated classes. Spring provides support for native Java ExecutorService as well as other popular 3rd party library such as Quartz
Probably what you're after is setting up the executor service on the spring container (eg: using following config on your spring beans xml)
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="queueCapacity" value="25" />
</bean>
And decorate your MyService class with #Service annotation and inject the reference to the executor service
I ended up defining my beans in the Spring application context and injection the completionservice into MyService. Works as a charm.
<task:executor id="solverExecutorService" pool-size="5" queue-capacity="100" />
<spring:bean id="solverCompletionService" class="nl.marktmonitor.solver.service.SolverCompletionService" scope="singleton">
<constructor-arg name="executor" ref="solverExecutorService"/>
</spring:bean>

Resources