I would like process the message programmatically using message-driven-channel-adapter. Here is scenario which I have to implement:
My application during the startup read the configuration from a service. The configuration provides information about the queues which will contain the messages. Hence I would like to create a message-driven-channel-adapter for each queue to listen to messages asynchronously.
Any example which initializes all the spring integration context programatically instead of using XML will be useful.
If you are going to do everything programmatically, I'd suggest you bypass Spring Integration magic and just use DefaultMessageListenerContainer directly.
Afterwards you can send messages to an existing MessageChannel directly from the MessageListener implementation or using Messaging Gateway.
Please, be careful with programmatic configuration with that do not miss important attributes like ApplicationContext or invocation for afterPropertiesSet().
Related
My flow starts with a polling inbound adapter which hands off the processing to a downstream components on pub/sub channels with async executors and service activator (orchestrator using 2 gateways)
Questions
1. I would like to log a correlation id for every execution of a poll so that I can track the logs for that flow execution independently from another execution of the same flow using this correlationId, something similar to a MDC with UUID. How do I pass this to all the threads involved in the processing and output the correlationId in the log statement?
2. I am also trying to figure out what is the best approach for logging. Currently I have a logging wireTap (consumed by a logging adapter) on all channels. For this one, I want to be able to enable the wireTap interception only when the logging mode is DEBUG
For example,
<int:logging-channel-adapter log-full-message="true"
logger-name="tapInbound"
level="DEBUG"
id="loggingChannel"/>
This will only log when application logger is DEBUG, but the wireTap interception will always happen. How to enable the wireTap interception only when log level is DEBUG? Probably by using selector-expression with value of something like isloggingLevelDebug(). How do I do this?
You can use an advice on the poller, or a custom interceptor on the first channel, to set the correlation.
Instead of declaring the wire taps declaratively in XML, you could programmatically add them to the channels in a SmartLifecycle bean in an early phase (Integer.MIN_VALUE).
AFAIK the Spring Cloud Stream project is based on Spring Integration. Hence I was wondering if there is a nice way to resequence a subset of inbound messages before the StreamListener handler is triggered? Or do I need to assemble the whole IntegrationFlow from scratch using XML or Java DSL config from Spring Integration?
My use case is as follows. Most of the time I process inbound messages on a Kafka topic as they come. However, a few events have to be resequenced based on CORRELATION_ID, SEQUENCE_NUMBER, and SEQUENCE_SIZE headers. In other words I'd like to keep using StreamListener as much as possible and simply plug in resequencing strategy for some events.
Yes, you would need to use Spring Integration for it. In fact Spring Cloud Stream is effectively a binding framework only. It binds message handlers to the message brokers via binders. The message handlers themselves are provided by the users.
The #StreamListener annotation is pretty much an equivalent of Spring Integration's #ServiceActivator with few extra features (e.g., conditional routing), but other then it is just a message handler.
Now, as you eluded to, you are aware that you can use Spring Integration (SI) to implement a message handler or an internal SI flow, and that is normal and recommended for complex cases.
That said, we do provide out of the box apps that implements certain EIP components and we do have, for example, and aggregator app which you can use as a starting point in implementing resequencer. Further more, given that we have an aggregator app and not resequencer, we would be glad to accept a contribution for it if you're interested.
I hope this answers you question.
We are using Spring Batch and partitioned job extensively with our project. Occasionally we see problems with partitioned jobs getting "hung" because of what apepars to be lost messages. The remote partitions all complete but the parent step stays in STARTED. Our configuration uses 1 connection factory for reading messages from the queues (inbound gateway) and a different clustered connection to send out the partition messages (outbound gateway). The reason for this is the JBoss messaging doesnt uniformly distribute messages around the cluster and the client connection factory provides that functionality.
Redhat came in and frankly threw mud at Spring and the configuration. The following are excerpts from their report
The Spring JMSTemplate code employs several anti-patterns, like creating a new connection, session, producer just to send a message, then closing the connection. Also, when receiving a message it can create a consumer each time,
receive the message, then close the consumer. This can results in poor performance under load. The use of anti-patterns not only results in poor performance, but can deplete operating system resources such as
threads and file handles, since some of the connection resources are released asynchronously. Moreover, with non-durable topic subscribers you can end up losing messages, since any messages received between the closing of
the last and opening of the next consumer will be lost. There is one place where it may be acceptable to use the Spring JMSTemplate is inside the application server using the JCA managed connection factory (normally at "java:/JmsXA") and that only works when you're sending messages.
The JCA managed connection factory caches connections so they will not actually be created each time. However using the JCA managed connection factory will not resolve the issue with consumers since they are not cached.
In summary, the Spring JMSTemplate is not safe to use apart from the very specific use case of using it inside the application server with the JCA managed connection factory (java:/JmsXA) and only in that case to send messages
(do not use it to consume messages).
Using it from a JMS client application outside the application server is never safe, and using it with a standard connection factory (e.g. "ConnectionFactory," "ClusteredConnectionFactory", "jms/RemoteConnectionFactory," etc.) is
never safe; also using it to receive messages is never safe. To safely receive messages using Spring, consider the use of MessageListenerContainers [7] with MessageDriven Pojos [8].
Finally, note that issues encountered are based on JMS anti-patterns and is thus not a problem specific to JBoss EAP. For example, see a similar discussion with regard to ActiveMQ [9].
Red Hat does not support using the Spring JMSTemplate with JBoss Messaging apart from the one acceptable use case for sending message via JCA managed connection factory.
RECOMMENDATIONS
● As to Spring JMS, as a rule, use JCA managed connection factories configured in JBoss EAP. Do not use the Spring configured connection factories. Use JNDI template to pull in the connection factories to Spring from JBoss. This will get rid of most of the Spring JMS problems.
● Use standard JMS instead of Spring JMS for the batch job. Spring is a non-standard (and probably sub-standard implementation of JMS). Standard JMS uses a pool of a few senders to send the message and close the session after the message is sent. On the listener side, standard JMS uses a pool of works listening to a distributed Queue or Topic. Each web server has JMS listener deployed as singleton and uses standard java observer to
notify any caller that is expecting a call back.
The JMS connection factories are configured in JBoss and loaded via JNDI.
Can you provide your feedback on their assessment?
To avoid the overhead of creating new connections/sessions per send, you need to wrap the provider's connection factory in a CachingConnectionFactory. It reuses the same connection for sends and caches sessions, producers, consumers.
I currently am using Spring Integration to get messages off of a queue and send them to a service using a service activator. My issue is that the service I am calling requires a security context to be in place for the current thread. This can be setup by calling a no-argument method, handleAuthentication(), of another bean. I am wondering what the best way is to call this whenever a new message is received, prior to calling the service activator service? I was originally thinking I would chain together two service activators, with the first one calling handleAuthentication(), but this seems incorrect as handleAuthentication() does not require any information from the actual message.
Yes, your assumption about the security handling is correct. It is really just a side-effect aspect which should not be tied with the business logic.
Therefore we should use something which allows us to follow with the same behavior in the program. It is called as an Aspect in the programming as well.
For this purpose Spring Integration suggests a hook like MessageChannelInterceptor, where you can implement your handleAuthentication() exactly in the preReceive() callback, according to your explanation.
Another trick can be achieved with the <request-handler-advice-chain> and MethodInterceptor implementation which should populate the SecurityContext into the current thread just before target service invocation.
We previously used to have a Spring Integration flow (XML configuration-based) where we would do an update in a database after sending a message to a JMS queue. To achieve this, the SI flow was configured with a publish-subscribe queue channel as an input to a JMS Outbound Channel Adapter (order 0) and a Service Activator (order 1). The idea here being that after a successful JMS send, the service activator would be called thus, updating the data in the database.
We are now in the process of updating our flows to work with spring-integration:4.0.x APIs and wanted to use this opportunity to see if the described flow pattern is still a good/recommended way of doing a database update after a successful JMS send or if there is now a simpler/better way of achieving this? As a side note, our flows are now being implemented using spring-integration-java-dsl:1.0.0.M3 APIs.
Thanks in advance for any input on this,
PM.
publish-subscribe queue channel
There's no such thing as a pub-sub queue channel; by definition, it's a subscribable channel; so I assume that's what you mean.
It is one of the ways to do what you need, and perfectly fine; you can also achieve the same result with a RecipientListRouter. The dsl syntax is quite nice, especially with Java 8; see the SpringOne demo app for an example.