In my app I have to process multiple jobs asynchronously from the main application thread and collect the result of each job. I have a plain Java solution that does this using a ExecutorService and a ExecutorCompletionService that collects the job results.
Now I would like to convert my code to a Spring solution. The docs show me how the use the ExecutorService and the #Async annotation, but I am not sure how and if I can collect the results of multiple jobs.
In other words: I am looking for the Spring equivalent of the CompletionService. Is there such a thing?
My current code:
class MyService {
private static ExecutorService executorService;
private static CompletionService<String> taskCompletionService;
// static init block
static {
executorService = Executors.newFixedThreadPool(4);
taskCompletionService = new ExecutorCompletionService<String>(executorService);
// Create thread that keeps looking for results
new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
Future<String> future = taskCompletionService.take();
String s = future.get();
LOG.debug(s);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}
}).start();
}
// This method can and will be called multiple times,
// so multiple jobs are submitted to the completion service
public void solve(List<Long> ids) throws IOException, SolverException {
String data = createSolverData(ids);
taskCompletionService.submit(new SolverRunner(data, properties));
}
}
You need to consider what's your main goal, because your current code will work fine alongside other Spring-associated classes. Spring provides support for native Java ExecutorService as well as other popular 3rd party library such as Quartz
Probably what you're after is setting up the executor service on the spring container (eg: using following config on your spring beans xml)
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="queueCapacity" value="25" />
</bean>
And decorate your MyService class with #Service annotation and inject the reference to the executor service
I ended up defining my beans in the Spring application context and injection the completionservice into MyService. Works as a charm.
<task:executor id="solverExecutorService" pool-size="5" queue-capacity="100" />
<spring:bean id="solverCompletionService" class="nl.marktmonitor.solver.service.SolverCompletionService" scope="singleton">
<constructor-arg name="executor" ref="solverExecutorService"/>
</spring:bean>
Related
From what I've read of the Spring #ContextConfiguration annotation, it's possible to load multiple XML context files, or multiple JavaConfig classes. What I need is to load from one XML context file and one class. All the examples I've seen either load all XML, or all classes, but not both.
I'm trying to do this because I want my test class, which is just there to verify expected Spring wiring, to load my default applicationContext.xml file (presently just a copy stored in "src/test/resources, and trying to figure out how to directly specify the default one) along with a JavaConfig class that specifies some JNDI resources that need to be available. For the purposes of my test, I only need to set those JNDI resources to dummy strings, but I'd really like to specify them in an inline static class instead of an external XML file, because my tests are going to have to verify that some settings are equal to those dummy strings, and it's more maintainable if both the values and the checks are in the same file.
What I have so far, and what I've tried, can be illustrated with this:
#RunWith(SpringRunner.class)
#ContextConfiguration(value = {"/testApplicationContext.xml", "/testResources.xml"})
//#ContextHierarchy({
// #ContextConfiguration("/testApplicationContext.xml"),
// #ContextConfiguration(classes = SpringWiringTest.Config.class)
//})
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
And this at the end of the class:
#Configuration
public static class Config {
#Bean public String uslDatasourcesList() { return "abc"; }
#Bean public String atgDatasourcesList() { return "abc"; }
#Bean public String uslTableNamePrefixsList() { return "abc"; }
#Bean public String atgTableNamePrefixsList() { return "abc"; }
#Bean public String doNotifications() { return "false"; }
#Bean public DataSource abc() { return new DriverManagerDataSource(); }
}
If I comment out the first #ContextConfiguration and comment back in the #ContextHierarchy block, I get an error like this:
Error creating bean with name 'uslDatasourcesList': Invocation of init
method failed; nested exception is
javax.naming.NoInitialContextException: Need to specify class name in
environment or system property, or as an applet parameter, or in an
application resource file: java.naming.factory.initial
Update:
Using the guideline of picking either JavaConfig or XML as the "entry point" to configuration, here are some modified excerpts that show what I'm trying:
#RunWith(SpringRunner.class)
#ContextConfiguration
//#ContextConfiguration(value = {"file:src/main/webapp/WEB-INF/applicationContext.xml", "/testResources.xml"})
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
...
#BeforeClass
public static void setup() throws Exception {
SimpleNamingContextBuilder builder = SimpleNamingContextBuilder.emptyActivatedContextBuilder();
DataSource ds = new DriverManagerDataSource();
builder.bind("java:comp/env/abc", ds);
}
...
#Configuration
#ImportResource("file:src/main/webapp/WEB-INF/applicationContext.xml")
public static class Config {
#Bean public String uslDatasourcesList() { return "abc"; }
#Bean public String atgDatasourcesList() { return "abc"; }
#Bean public String uslTableNamePrefixsList() { return "abc"; }
#Bean public String atgTableNamePrefixsList() { return "abc"; }
#Bean public String doNotifications() { return "false"; }
#Bean public DataSource abc() { return new DriverManagerDataSource(); }
}
}
When I run my test, the bottom "Caused by" in the exception says this:
Caused by: javax.naming.NameNotFoundException: Name
[uslDatasourcesList] not bound; 1 bindings: [java:comp/env/abc]
In the alternate version, using the commented-out "#ContextConfiguration" (and commenting out the Config class and its annotations), this error does not occur.
Note that this the meat of my "testResources.xml" file:
<bean id="uslDatasourcesList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="atgDatasourcesList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="uslTableNamePrefixList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="atgTableNamePrefixList" class="java.lang.String"> <constructor-arg value="abc"/> </bean>
<bean id="doNotifications" class="java.lang.String"> <constructor-arg value="false"/> </bean>
<bean id="abc" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
</bean>
Note that the bean mentioned in the error message, "uslDatasourcesList" is defined in both versions, but it's not working in the version with JavaConfig and XML mixed.
It almost appears that the beans in the "#ImportResource" annotation are evaluated on their own, before the beans declared in the JavaConfig class are merged into it.
This is clearly documented in the Spring Reference Manual in the section named Mixing XML, Groovy scripts, and annotated classes.
In summary, ...
... you will have to pick one as the entry point, and that one will have to include or import the other.
Thus, the following should hopefully solve your problem.
#RunWith(SpringRunner.class)
#ContextConfiguration
#TestPropertySource(properties = { "env = tomcat", "doNotifications = false" })
public class SpringWiringTest {
// ...
#Configuration
#ImportResource({"/testApplicationContext.xml", "/testResources.xml"})
static class Config {
// ...
}
}
In order to learn spring integration I've been attempting to create a simple, resilient log processor. I'm also wanting to stick with a java configuration approach.
I've been having a difficult time translating existing XML configuration, mostly due to being so new to spring in general.
In a question on the spring forums Gary Russell presented a similar solution to this using a publish-subscribe + JMS model with a simple XML config.
I've been attempting to translate his suggestion into a Java config, but am stuck. Namely I'm not sure of the proper entities to use for the outbound-channel-adapter, service-activators or how to set the order of messages properly.
Here is Gary's XML config:
<int-file:inbound-channel-adapter id="dispatcher"
directory="spool"
channel="fileChannel">
<int:poller fixed-delay="2000">
<int:transactional/>
</int:poller>
</int-file:inbound-channel-adapter>
<int:channel id="fileChannel" />
<int-file:file-to-string-transformer input-channel="fileChannel" output-channel="dispatchChannel" />
<int:publish-subscribe-channel id="dispatchChannel" />
<int-jms:outbound-channel-adapter id="dispatcherJms" channel="dispatchChannel" order="1"
connection-factory="connectionFactory"
destination="dispatcher.queue" />
<!-- If JMS Send was successful, remove the file (within the transaction)-->
<int:service-activator input-channel="dispatchChannel" order="2"
output-channel="nullChannel"
expression="headers.file_originalFile.delete()">
<bean id="transactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
UPDATE
Based on the comments below I've updated the java config.
However I'm still receiving errors and most likely am not understanding the flow and connections between the entities, but the original question has been answered.
#Bean
#Transactional
#InboundChannelAdapter(channel = "dispatchChannel", poller = #Poller(fixedDelay = "2000"))
public MessageSource<?> dispatcher() {
CompositeFileListFilter<File> filters = new CompositeFileListFilter<>();
filters.addFilter(new SimplePatternFileListFilter(sourceFilenamePattern));
//filters.addFilter(persistentFilter());
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File(sourceDirectory));
source.setFilter(filters);
return source;
}
#Bean
public MessageChannel fileChannel() {
return new DirectChannel();
}
#Bean
public PublishSubscribeChannel dispatchChannel() {
return new PublishSubscribeChannel();
}
#Autowired
JmsTemplate jmsTemplate;
#Autowired
ConnectionFactory connectionFactory;
#Bean
#Order(1)
#ServiceActivator(inputChannel = "dispatchChannel")
public MessageHandler dispatcherJmsOutboundChannelAdapter(Message<File> message) {
JmsSendingMessageHandler handler = new JmsSendingMessageHandler(jmsTemplate);
handler.setDestinationName("dispatcher.queue");
return handler;
}
#Bean
#Order(2)
#ServiceActivator(inputChannel = "dispatchChannel")
public void removeFile(Message<?> message) {
//message.getHeaders().get(FileHeaders.ORIGINAL_FILE, File.class).delete();
log.info("delete");
}
#Bean
public JmsTransactionManager transactionManager(ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
I'm using spring boot and several starter components, such as activemq. I've added the #Bean for JmsListenerContainerFactory and a #JmsListener, though I'm not sure those are truly necessary.
I couldn't get anything to run until adding #EnableJms to my configuration file as well as #Autowiring the jmstemplate and connectionfactory.
When running, the error I'm receiving now is:
org.springframework.beans.factory.NoSuchBeanDefinitionException:
No qualifying bean of type [org.springframework.messaging.Message] found for dependency
[org.springframework.messaging.Message<?>]:
expected at least 1 bean which qualifies as autowire candidate for this dependency.
Dependency annotations: {}
This one
<int:service-activator input-channel="dispatchChannel" order="2"
output-channel="nullChannel"
expression="headers.file_originalFile.delete()">
is pretty simple in Java:
#ServiceActivator(inputChannel = "dispatchChannel")
public void removeFile(Message<?> message) {
message.getHeaders().get(FileHeaders.ORIGINAL_FILE, File.class).delete();
}
and
<int-jms:outbound-channel-adapter>
is translated to this:
#Bean
#ServiceActivator(inputChannel = "dispatchChannel")
public MessageHandler dispatcherJmsOutboundChannelAdapter() {
JmsSendingMessageHandler handler =
new JmsSendingMessageHandler(new JmsTemplate(this.connectionFactory));
handler.setDestinationName("dispatcher.queue");
return handler;
}
Pay attention to this paragraph in the Reference Manual.
The last piece of jigsaw puzzle is FileWritingMessageHandler
#Bean
public FileWritingMessageHandler fileWritingMessageHandler() {
SpelExpressionParser parser = new SpelExpressionParser();
Expression expression = parser.parseExpression("headers.file_originalFile.delete()");
FileWritingMessageHandler fileWritingMessageHandler = new FileWritingMessageHandler(expression);
fileWritingMessageHandler.setOutputChannel(new NullChannel());
fileWritingMessageHandler.setDeleteSourceFiles(true);
return fileWritingMessageHandler;
}
I am trying to create a pollable message source and I have tried to do that by extending MessageProducerSupport, however I was able to see the message from receive method only once and was not successful in making it pollable. (The receive method is not getting called based on my polling schedule.)
My code snippet is as below:
#Component
public class MyAdapter extends MessageProducerSupport {
#Override
protected void doStart() {
receive();
}
public void receive() {
System.out.println("polled at : "+ new Date());
sendMessage(MessageBuilder.withPayload("Hello WOrld! "+ new Date()).build());
}
}
And my applicationContext is as below:
<context:component-scan base-package="com.mypackage" />
<context:annotation-config />
<bean id="pollerTaskExecutor" class="org.springframework.core.task.SyncTaskExecutor"/>
<int:inbound-channel-adapter ref="myAdapter" channel="output">
<int:poller task-executor="pollerTaskExecutor">
<int:interval-trigger interval="3000" fixed-rate="true" time-unit="MILLISECONDS"/>
</int:poller>
</int:inbound-channel-adapter>
I would like to know what am I missing to make this message source pollable.
You are right: the pollable message source is based . erm... on org.springframework.integration.core.MessageSource.
So, to make it working you just should move your MessageProducerSupport code to the AbstractMessageSource implementation.
See more info in the Reference Manual.
How to notify with a listener when the Future method has finished using #Async annotation in spring based service layer and with a thread pool in application context called from a ManagedBean . I have tried with p:poll listener with a listener method future.isDone() but is not very effective because it makes many requests to the server.
Edit.
here is the example
#ManagedBean
#ViewScoped
public class Controller
{
Future<SomeModel> someFuture;
#ManagedProperty(value = "#{someService}")
private SomeService someService;
private String things;
private SomeModel someModel;
private boolean renderSomeModel;
public void callAsyncMethod()
{
someFuture = someService.thingsToBeInsered(things);
//here make the poll in jsf start
}
public void methodAsyncFinished{
if(someFuture!=null)
{
if(this.someFuture.isDone())
{
renderSomeModel = true;
//here make the poll to stop
}
}
}
#Service
public class SomeService
{
#Async
Future<SomeModel> thingsToBeInsered(things)
{
//Calling the dao here
return new AsyncResult<SomeModel>(things);
}
}
//spring context
<task:annotation-driven executor="taskExecutor" proxy-target-class="true"/>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="${thread_pool.corePoolSize}" />
<property name="maxPoolSize" value="${thread_pool.maxPoolSize}" />
<property name="WaitForTasksToCompleteOnShutdown" value="${thread_pool.waitForTasksToCompleteOnShutdown}" />
</bean>
jsf
<p:poll autoStart="false" widgetVar="asyncGenPoll" interval="6" listener="#{controller.methodAsyncFinished}" update="resultPanel"/>
We recently upgraded our spring version to 4.x from 2.x which started causing some issues with one of our 'threadpooltaskexecutor' implementation.
The implementation was as below:
public class A
{
..............................
..............................
public void create()
{
this.threadPoolTaskExecutor.execute(new Runnable()
{
public void run()
{
createABCAsync();
}
});
}
public void createABCAsync()
{
createABC();
}
public void createABC()
{
.....................................
.....................................
abcDAO.saveOrUpdate(abc);
xyzDAO.saveOrUpdate(xyz);
}
...................................................
...................................................
}
The bean definitions for class A has entries like below:
`<bean id="clsIntegrationManager" parent="txProxyTemplate"`>
.................................................
.................................................
<property name="transactionAttributes">
<props>
<prop key="create">PROPAGATION_REQUIRED</prop>
<prop key="createABCAsync">PROPAGATION_REQUIRES_NEW</prop>
</props>
</property>
After spring upgrade, the above code threw an exception as below:
Exception in thread "threadPoolTaskExecutor-1" org.springframework.dao.InvalidDataAccessApiUsageException: Write operations are not allowed in read-only mode (FlushMode.MANUAL): Turn your Session into FlushMode.COMMIT/AUTO or remove 'readOnly' marker from transaction definition.
To resolve the above I added an entry like follows in the DAO layer bean definition file:
<bean id ="hibernateTemplate" class="org.springframework.orm.hibernate4.HibernateTemplate">
<property name="sessionFactory" ref="sessionFactory"/>
<property name="checkWriteOperations" value="false"/>
</bean>
The above entry resolved the exception, but the asynchronous method execution which starts a separate transaction using REQUIRES_NEW is not doing any database commits. We have a couple of 'saveOrUpdate()' calls inside the createABC() method but nothing gets saved in database.
Can someone please help in understanding what is going wrong above ?
I tried a few different solution approaches, one of which is described here: [Spring TaskExecutor Transaction Lost (Proxy vs. Direct call) but this did not help.