How can I initiate a build process with Apache Ant that
is executed in parallel on multiple threads? That is,
I am searching for that corresponds to '-j' in GNU Make.
You can execute an Ant build target that has the parallel task. (see docs here)
Assuming you have a macro called 'dbpurge' that takes a 'file' argument. The following example would describe running 40 calls to dbpurge with a thread count (jobs count in GNU Make) of '4'.
<parallel threadCount="4">
<dbpurge file="db/one" />
<dbpurge file="db/two" />
<dbpurge file="db/three" />
<dbpurge file="db/four" />
<dbpurge file="db/five" />
<dbpurge file="db/six" />
<dbpurge file="db/seven" />
<dbpurge file="db/eight" />
<!-- repeated about 40 times -->
</parallel>
Related
Lets say that a task needs to be done and there are no data model changes needed(i.e items.xml does not need to be touched).
For example a new Interceptor is needed for an existing Item Type. In this case I just need a new spring bean and a new Java class.
After I do the changes, If I run an "ant build" it takes approximately 1:30(one minute and a half), sometimes even more than that.
From what I noticed Hybris tries to check every extension that is included in localExtension.xml with their required extensions as well, and that is taking a lot of time.
How can I perform a faster build ? It should not take that much time since the only thing that is needed in my Interceptor case is to compile the new Interceptor class, and that's it.
I understand that when data model is changed the models.jar needs to be deleted, the new sources need to be generated and compiled in a new models.jar and that requires time. But in the more simple scenario it should work a lot faster.
PS: I know about JRebel but this question addresses the cases in which the developer does not have JRebel.
In platform/build.xml add below ant target:
<target name="compileExtensions" description="Compiles only the provided extensions">
<compile_only_specified_extensions/>
</target>
In platform/resources/ant/compiling.xml add the macro definition:
<macrodef name="compile_only_specified_extensions">
<sequential>
<foreachextprovidedincli>
<do>
<if>
<not>
<isset property="ext.#{extname}.warextension" />
</not>
<then>
<extension_compile extname="#{extname}" />
</then>
<else>
<external_extension_build extname="#{extname}"/>
</else>
</if>
</do>
</foreachextprovidedincli>
</sequential>
</macrodef>
Define foreachextprovidedincli in platform/resources/ant/util.xml
<macrodef name="foreachextprovidedincli">
<element name="do" optional="false" />
<attribute name="param" default="extname" />
<sequential>
<for list="${extensions.to.compile}" param="#{param}" delimiter=";">
<sequential>
<do />
</sequential>
</for>
</sequential>
</macrodef>
Now what I simply have to do to compile my classes is run the following command:
ant compileExtensions -Dextensions.to.compile="extensionName1;extensionName2;extensionName3"
With above command the build was reduced to 4 seconds.
I am using a JmsOutboundGateway with a MessageChannelPartitionHandler for a partitioned batch job. I would like to be able to interrupt this code in the handle method
Message<Collection<StepExecution>> message = messagingGateway.receive(replyChannel);
From another thread I tried to call interrupt() on this thread. What is the best way to add this functionality?
CLARIFICATION:
The interruption does work. However, the thread for each partition that did receive a response has a stack trace(below). These threads remain blocked which pulls them from that threadpool and they are unavailable for subsequent partitioned jobs.
<b>Thread: springbatch.partitioned.jms.taskExecutor-1</b> :
priority:5, demon:false, threadId:823, threadState:TIMED_WAITING<br />
<blockquote>
- waiting on <0xf483c36> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)<br />sun.misc.Unsafe.park(Native
Method)<br />java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)<br />
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)<br />
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)<br />
org.springframework.integration.jms.JmsOutboundGateway.obtainReplyFromContainer(JmsOutboundGateway.java:865)<br />
org.springframework.integration.jms.JmsOutboundGateway.doSendAndReceiveAsync(JmsOutboundGateway.java:809)<br />
org.springframework.integration.jms.JmsOutboundGateway.sendAndReceiveWithContainer(JmsOutboundGateway.java:649)<br />
org.springframework.integration.jms.JmsOutboundGateway.handleRequestMessage(JmsOutboundGateway.java:580)<br />
org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:134)<br />
org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73)<br />
org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:115)<br />
org.springframework.integration.dispatcher.UnicastingDispatcher.access$000(UnicastingDispatcher.java:52)<br />
org.springframework.integration.dispatcher.UnicastingDispatcher$1.run(UnicastingDispatcher.java:97)<br />
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)<br />
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br />java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br />java.lang.Thread.run(Thread.java:744)<br />
<br />
How can I access these threads to interrupt also?
Interrupting the thread will work; however, the interrupt is caught (and the interrupt bit re-set in the QueueChannel).
After this, the received message is null so the partition handler will throw a MessageTimeoutException instead.
There is currently no way to determine whether the timeout is because of a real timeout or the interrupt.
I am newbie to camel. Let me explain my business requirement, i have some 10 files in Folder A. In particular time all these 10 files will be processed and do the business logic and move the files to Folder B. Each file place one done file respectively. I need to process files from Folder B for next business operation. But i need to start this process only when all the 10 done files are placed. until complete all the 10 files in my previous process, i should not start this process. How to do this in camel.
Note:
i dont want to use any cron for my second route.
10 file is assumption. it may be dynamic
Thanks in advance
You can an Aggregator to aggregate the processed files before continuing the second part of processing.
You will have something similar to the following:
<route>
<from uri="file://<Folder A>" />
<!-- your business logic -->
<aggregate strategyRef="aggregatorStrategy" ...>
...
</aggregate>
<!-- here goes the business logic used to poll files from Folder B -->
</route>
Maybe you can use a "alldone" file:
<route>
<from uri="file://<Folder A>?scheduler=<particular time>" />
<!-- your business logic part 1-->
<to uri="file://<Folder B>" />
<choice>
<when>
<simple>${exchangeProperty.CamelBatchComplete}</simple>
<setBody><constant>null</constant></setBody>
<setHeader headerName="CamelFileName">
<simple>alldone</simple>
</setHeader>
<to uri="file://<Folder B>" />
</when>
<choice>
</route>
<route>
<from uri="file://<Folder B>?doneFileName=alldone" />
<!-- your business logic part 2-->
</route>
I have a rest call to do inside splitter and then aggregate. I planned to implement parallelism for this rest call. So i introduced task executor as per this link. Now parallelism works but sometimes it is working and sometimes not. Guessing the aggregator is not waiting for all the thread to finish. Not sure exactly the problem. Can you help me here.
<int:enricher ipChann="som" opCahnnel="inputChannel" />
<int:splitter ipchan="inputChannel" opChannel="opchannel" ref="customersplitter" />
<int:channel id="opchannel" >
<int:dispatcher task-executor="exec" />
</int:channel>
<task:executor id="exec" pool-size="4" queue-capacity="10"/>
<int:enricher ipChannel="opchannel" opChannel="aggregatorChan"></int:enricher>
<int:aggregator ipChann="aggregatorChan" />
For simplicity i didnt expand enricher, but the flow are same.
Does the current version of Log4net have a way to create a RollingFileAppender with composite rolling style where the rolled files always preserves the given extension (.log in my case)?
Example of the format I would like:
MyLog.log
MyLog.2011-04-10.1.log
MyLog.2011-04-10.2.log
MyLog.2011-04-10.3.log
I found this post which says that there is a "PreserveLogFileNameExtension" property, but that it's not included in the official binaries. Is this still the case?
If so: Can anyone explain why this property is still not an offical part of Log4Net? I am a bit sceptical to use a custom build, but maybe I should not be?
I am also curious to know why the default functionality does not preserve the file extension. I do not see why it would gain the user that all the log files have different extensions.
Edit: Got it working by doing this:
1: Downloading and building the log4net source code
2: Applying these patches: https://issues.apache.org/jira/browse/LOG4NET-64
3: Setting PreserveLogFileNameExtension to "true" in the config.
Have you tried these parameters?
<file value="log-files\MyLog" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyy-MM-dd'.log'" />
<param name="StaticLogFileName" value="false" />
It will preserve the extension, but will give you a date in every filename like this.
MyLog2011-05-16.log
MyLog2011-05-17.log
MyLog2011-05-18.log
MyLog2011-05-19.log
Maybe it is possible to combine this with the size rolling?
The situation is unchanged. There is no newer release of log4net. It is quite unclear to me when (if) there will be a new release...
I think you do not need to worry to much about using a custom build. Test your software, if it works it is good enough.
EDIT: There is a new release that should include LOG4NET-64. Of course you can still stick to your custom build.
I'm using this configuration:
<file value="" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyyMMdd'.log'" />
<staticLogFileName value="false" />
To get filenames like:
20111101.log
20111102.log
20111103.log