I have a requirement to fetch jms message from IBM MQ. But the messages should be fetched only when required, say when a method is invoked. The picked up message need to be put to another queue in turn. That is , this scenario is for copying message from one queue to another when required.
How could I achieve this in Spring Integration.
I believe i could create a flow using inbound-channel-adapter, set to poll message at regular interval, along with outbound-channel-adapter. But with this, I could not pick the message as and when required.
Please advise.
Question updated:
1) While searching on the web to implement the suggested solution, I stepped on the spring documentation for JmsDestinationPollingSource saying - "This source is only recommended for very low message volume".
2) Since we need for high volume environment and in need of local transaction, I looked into the suggested JmsMessageDrivenEndpoint solution which looked like using MessageDrivenChannelAdapter. Can you please adviseas how i could proceed.
3) Also, in that case how can i detect if all messages have been read to stop the listener . Please advise
Question updated:
I am able to invoke start and stop on DefaultMessageListener container used with MessageDrivenChannelAdapter.
Can you also please advise as how I could find when there are no messages on the queue or when the queue is empty so that I could invoke stop.
Question updated:
Thanks Gary. Sorry for the late update.
To find when the queue becomes empty, can we override noMessageReceived method of DefaultMessageListenerContainer (inherited from AbstractPollingMessageListenerContainer) and invoke stop on the DMLC. This kind of solution is given in
Spring integration inbound-gateway Fire an event when queue is empty.
I thought this might be easier to implement. Please advise if it will be correct to do in that way.
Set auto-startup="false" on the inbound channel adapter (so it won't poll).
Get a reference to its MessageSource (either by #Autowire or otherwise injecting or context.getBean()). The bean name is adapterid.source and is of type JmsDestinationPollingSource.
Then invoke the receive() method.
Or you can just wire up a JmsDestinationPollingSource bean yourself without using the namespace support.
Related
Following the example found in GitHub https://github.com/spring-cloud/spring-cloud-gcp/tree/master/spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-polling-binder-sample regarding polling messages from a PubSub subscription, I was wondering...
Is it possible to make a PollableMessageSource retrieve List<Message<?>> instead of a single message per poll?
I've seen the #Poller notation only being used in Source typed objects, never in Processor or Sink. Is it possible to use in such context when for example using #StreamListener or with a functional approach?
The PollableMessageSource binding and Source stream applications are fully based on the Poller and MessageSource abstraction from Spring Integration where its contract is to produce a single message to the channel configured. The point of the messaging is really to process a single message not affecting others. The failure for one message doesn't mean to fail others in the flow.
On the other hand you probably mean GCP Pub/Sub messages to be produced as a list in the Spring message payload. That is really possible, but via some custom code from Pub/Sub consumer and MessageSource impl. Although I would think twice to expect some batched from the source. Probably you may utilize an aggregator to build some small windows if your further logic is about processing as list. But again: it is going to be a single Spring message.
May be better to start thinking about a reactive function implementation where you indeed can expect a Flux<Message<?>> as an input and Spring Cloud Stream framework will take care for you how to emit the data from Pub/Sub into the reactive stream you expect.
See more info in docs: https://docs.spring.io/spring-cloud-stream/docs/3.1.0/reference/html/spring-cloud-stream.html#_reactive_functions_support
I would like to know if I can have persistence in my Spring Integration setup when I use a aggregator, which is not backed by a MessageStore, by leveraging the persistence of AMQP (RabbitMQ) queues before and after the aggregator.
I imagine that this would use ack's: The aggregator won't ack a message before it's collected all the parts and sent out the resulting message.
Additionally I would like to know if this is ever a good idea :)
I am new working with queue's, and am trying to get a good feel for patterns to use.
My business logic for this is as follows:
I receive a messages on one queue.
Each message must result in two unrelated webservice calls (preferably in parallel).
The results of these two calls must be combined with details from the original message.
The combination must then be sent out as a new message on a queue.
Messages are important, so they must not be lost.
I was/am hoping to use only one 'persistent' system, namely RabbitMQ, and not having to add a database as well.
I've tried to keep the question specific, but any other suggestions on how to approach this are greatly appreciated :)
What you would like to do recalls me Scatter-Gather EI Pattern.
So, you get a message from the AMQP send it into the ScatterGather endpoint and wait for the aggregated reply. That's enough for to stick with the default acknowledge.
Right, the scatterChannel can be PublishSubscribeChannel with an executor to call Web Services in parallel. Anyway the gatherer process will wait for replies according the release strategy and will block the original AMQP listener do not ack the message prematurely.
I currently am using Spring Integration to get messages off of a queue and send them to a service using a service activator. My issue is that the service I am calling requires a security context to be in place for the current thread. This can be setup by calling a no-argument method, handleAuthentication(), of another bean. I am wondering what the best way is to call this whenever a new message is received, prior to calling the service activator service? I was originally thinking I would chain together two service activators, with the first one calling handleAuthentication(), but this seems incorrect as handleAuthentication() does not require any information from the actual message.
Yes, your assumption about the security handling is correct. It is really just a side-effect aspect which should not be tied with the business logic.
Therefore we should use something which allows us to follow with the same behavior in the program. It is called as an Aspect in the programming as well.
For this purpose Spring Integration suggests a hook like MessageChannelInterceptor, where you can implement your handleAuthentication() exactly in the preReceive() callback, according to your explanation.
Another trick can be achieved with the <request-handler-advice-chain> and MethodInterceptor implementation which should populate the SecurityContext into the current thread just before target service invocation.
In my spring integration application i have several stored-proc-outbound-gateway, i would to log how much time each call is taking, any help would be appreciated.
I would ideally like to be able to enable/disable logging for the parameters used, time taken and total rows retrieved (returning-resultset) to monitor and performance tuning purpose.
Thanks
You can add a ChannelInterceptor (subclass of ChannelInterceptorAdapter) to the request channel which will give you raw timing (preSend/postSend), but the time will include any processing downstream of the gateway (on direct channels).
Since you want to examine the results too, you could start a timer (e.g. Spring StopWatch) in the interceptor (preSend) on the request channel and stop the timer in an interceptor on the reply channel). If you use the same interceptor bean you can store the timer in a ThreadLocal.
You can turn on/off collection using a boolean property on the interceptor.
Alternatively, you can add a custom advice to the gateway.
EDIT
The advice is probably the best approach because with a ThreadLocal you will need to add code to the first interceptor to handle failures and clean up. With an around advice, the timer would just be a local method variable.
Does the spring integration framework have any features I can use to guarantee message order?
We are needing to run our spring integration flow on two different nodes to keep up with the message volume. I have seen several solutions for solving this issue, but I wanted to see if the framework already has something.
This article might better explain what I'm trying to ask.
http://sleeplessinslc.blogspot.com/2010/02/message-ordering-in-jms-using-coherence.html
Yes, it is here. And its name Resequencer.
From Spring Integration Reference Manual: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/messaging-routing-chapter.html#resequencer.
Each of message for reordering has to have a correlationId header to compare with other messages and sequenceNumber to determine the real order of that message to emit.
The release-partial-sequences="true" does the stuff to release messages, when the correct order is achieved by default SequenceSizeReleaseStrategy on the next message.