Pollers and Memory Leaks - spring-integration

Question

I came across the following in the spring integration reference doc
The receiveTimeout property specifies the amount of time the poller should wait if no messages are available when it invokes the receive operation. For example, consider two options that seem similar on the surface but are actually quite different: the first has an interval trigger of 5 seconds and a receive timeout of 50 milliseconds while the second has an interval trigger of 50 milliseconds and a receive timeout of 5 seconds. The first one may receive a message up to 4950 milliseconds later than it arrived on the channel (if that message arrived immediately after one of its poll calls returned). On the other hand, the second configuration will never miss a message by more than 50 milliseconds. The difference is that the second option requires a thread to wait, but as a result it is able to respond much more quickly to arriving messages. This technique, known as long polling, can be used to emulate event-driven behavior on a polled source.
Based on my experience, the second option can cause a problem because an interval of 50 milliseconds will make the poller run every 50 millies, but if there's no messages to pick up each threads created will wait for 5 seconds for a message to appear. In that 5 seconds the poller will get executed another 100 times, potentially creating another 100 threads, etc.
This quickly runs away.
My question is did I misunderstand the way this all works? Because if I'm correct I think the reference documentation should be changed, or at least a warning added.
e<bean id="store" class="org.springframework.integration.jdbc.store.JdbcChannelMessageStore">
<property name="dataSource" ref="channelServerDataSource"/>
<property name="channelMessageStoreQueryProvider" ref="queryProvider"/>
<property name="region" value="${user.name}_${channel.queue.region:default}"/>
<property name="usingIdCache" value="false"/>
</bean>
<int:transaction-synchronization-factory id="syncFactory">
<int:after-commit expression="#store.removeFromIdCache(headers.id.toString())" />
<int:after-rollback expression="#store.removeFromIdCache(headers.id.toString())"/>
</int:transaction-synchronization-factory>
<int:channel id="transacitonAsyncServiceQueue">
<int:queue message-store="store"/>
<!-- <int:queue/> -->
</int:channel>
<bean id="rxPollingTrigger" class="org.springframework.scheduling.support.PeriodicTrigger">
<constructor-arg value="500"/>
<constructor-arg value="MILLISECONDS"/>
<property name = "initialDelay" value = "30000"/>
<!-- initialDelay important to ensure channel doesnt start processing before the datasources have been initialised becuase we
now persist transactions in the queue, at startup (restart) there maybe some ready to go which get processed before the
connection pools have been created which happens when the servlet is first hit -->
</bean>
<int:service-activator ref="asyncChannelReceiver" method="processMessage" input-channel="transacitonAsyncServiceQueue">
<int:poller trigger="rxPollingTrigger" max-messages-per-poll="20" task-executor="taskExecutor" receive-timeout="400">
<int:transactional transaction-manager="transactionManagerAsyncChannel" />
</int:poller>
<int:request-handler-advice-chain>
<ref bean="databaseSessionContext" />
</int:request-handler-advice-chain>
</int:service-activator>
<task:executor id="taskExecutor" pool-size="100-200" queue-capacity="200" keep-alive="1" rejection-policy="CALLER_RUNS" />

Answer 1

My question is did I misunderstand the way this all works?
Yes, you misunderstand.
The trigger (in this case a PeriodicTrigger with an interval of 50ms) is only consulted to calculate the next poll time when the current poll exits.
There is only one poller thread running concurrently. If there is no message, the poll thread is suspended for 5 seconds; the trigger is then consulted (t.nextExecutionTime()) and the next poll is scheduled for +50ms; so, with no data a single thread will run every 5.05 seconds.
When messages are present, and you wish to have a concurrency greater than one, you would use a task executor to allow the poller thread to hand off to another thread so that the trigger is immediately consulted for the next poll time.
Based on my experience
Please clarify "your experience" and show configuration, evidence etc.
If you have a suspected thread leak, the first step, generally, is to take a thread dump to figure out what they are all doing.
EDIT: (in response to your comments below).
There's not really a downside of CALLER_RUNS in this scenario because, although the current thread "jumps ahead" of the queued tasks, it's not like this poll has newer data than the queued tasks, it's just a poll after all. However, poller threads are a limited resource (although the limit can be changed) so long-running tasks on a poller thread are generally discouraged.
ABORT could cause some noise in the logs; an alternative is to configure a PollSkipAdvice where the advice can look at the task queue and silently ignore the current poll. In 4.2, we've added even more flexibility to the poller.
You will find many articles on the internet that say that using RDBMS as a queue is not the greatest idea; you might want to consider a JMS or rabbitmq-backed channel instead. If you are tied to JDBC, you should be sure to use the JdbcChannelMessageStore and not the JdbcMessageStore. The former is preferred for backing channels since it only uses 1 table; the latter has some performance issues when used to back a channel, because of contention on the message group table. See Backing Message Channels in the JDBC Support chapter for more information.