This is an observation we have seen while using replyChannel("letFrameworkCreateChannel") auto create replyChannel via the framework as against replyChannel(constructDirectChannel()). In the former the consumer did NOT get the response and there were NO errors in the application. This happened to us in more than one application on Kubernetes & Linux environment, whereas we did NOT face this in Windows. We had to switch from replyChannel("letFrameworkCreateChannel") to replyChannel(constructDirectChannel()) for the request-reply pattern to work. Kindly suggest why is the behavior with replyChannel("letFrameworkCreateChannel")
Related
AFAIK the Spring Cloud Stream project is based on Spring Integration. Hence I was wondering if there is a nice way to resequence a subset of inbound messages before the StreamListener handler is triggered? Or do I need to assemble the whole IntegrationFlow from scratch using XML or Java DSL config from Spring Integration?
My use case is as follows. Most of the time I process inbound messages on a Kafka topic as they come. However, a few events have to be resequenced based on CORRELATION_ID, SEQUENCE_NUMBER, and SEQUENCE_SIZE headers. In other words I'd like to keep using StreamListener as much as possible and simply plug in resequencing strategy for some events.
Yes, you would need to use Spring Integration for it. In fact Spring Cloud Stream is effectively a binding framework only. It binds message handlers to the message brokers via binders. The message handlers themselves are provided by the users.
The #StreamListener annotation is pretty much an equivalent of Spring Integration's #ServiceActivator with few extra features (e.g., conditional routing), but other then it is just a message handler.
Now, as you eluded to, you are aware that you can use Spring Integration (SI) to implement a message handler or an internal SI flow, and that is normal and recommended for complex cases.
That said, we do provide out of the box apps that implements certain EIP components and we do have, for example, and aggregator app which you can use as a starting point in implementing resequencer. Further more, given that we have an aggregator app and not resequencer, we would be glad to accept a contribution for it if you're interested.
I hope this answers you question.
I am using Kafka producer to publish messages to some other kafka topic and its working pretty fine. sample template below:
<int-kafka:outbound-channel-adapter
kafka-template="template"
channel="inputToKafka"
topic="foo"/>
Does the above statement support errorchannel as supported in kafka message driven inbound channel adapter?
I needed that to audit the error count whenever my outgoing kafka server is down and i am unable to publish it.
Since any Outbound is passive component and it can only do its purpose by the external call, that is not a surprise that the error handling should be similar to the try...catch in Java when we call service method.
So, one way is to have error channel upstream - Messaging Gateway or Inbound channel Adapter.
Another way is to use ExpressionEvaluatingRequestHandlerAdvice in the request-handler-advice-chain of the <int-kafka:outbound-channel-adapter>.
Also, bear in mind that you should use async = false option to get all the errors from Kafka interaction in the same thread.
We are using Spring Batch and partitioned job extensively with our project. Occasionally we see problems with partitioned jobs getting "hung" because of what apepars to be lost messages. The remote partitions all complete but the parent step stays in STARTED. Our configuration uses 1 connection factory for reading messages from the queues (inbound gateway) and a different clustered connection to send out the partition messages (outbound gateway). The reason for this is the JBoss messaging doesnt uniformly distribute messages around the cluster and the client connection factory provides that functionality.
Redhat came in and frankly threw mud at Spring and the configuration. The following are excerpts from their report
The Spring JMSTemplate code employs several anti-patterns, like creating a new connection, session, producer just to send a message, then closing the connection. Also, when receiving a message it can create a consumer each time,
receive the message, then close the consumer. This can results in poor performance under load. The use of anti-patterns not only results in poor performance, but can deplete operating system resources such as
threads and file handles, since some of the connection resources are released asynchronously. Moreover, with non-durable topic subscribers you can end up losing messages, since any messages received between the closing of
the last and opening of the next consumer will be lost. There is one place where it may be acceptable to use the Spring JMSTemplate is inside the application server using the JCA managed connection factory (normally at "java:/JmsXA") and that only works when you're sending messages.
The JCA managed connection factory caches connections so they will not actually be created each time. However using the JCA managed connection factory will not resolve the issue with consumers since they are not cached.
In summary, the Spring JMSTemplate is not safe to use apart from the very specific use case of using it inside the application server with the JCA managed connection factory (java:/JmsXA) and only in that case to send messages
(do not use it to consume messages).
Using it from a JMS client application outside the application server is never safe, and using it with a standard connection factory (e.g. "ConnectionFactory," "ClusteredConnectionFactory", "jms/RemoteConnectionFactory," etc.) is
never safe; also using it to receive messages is never safe. To safely receive messages using Spring, consider the use of MessageListenerContainers [7] with MessageDriven Pojos [8].
Finally, note that issues encountered are based on JMS anti-patterns and is thus not a problem specific to JBoss EAP. For example, see a similar discussion with regard to ActiveMQ [9].
Red Hat does not support using the Spring JMSTemplate with JBoss Messaging apart from the one acceptable use case for sending message via JCA managed connection factory.
RECOMMENDATIONS
● As to Spring JMS, as a rule, use JCA managed connection factories configured in JBoss EAP. Do not use the Spring configured connection factories. Use JNDI template to pull in the connection factories to Spring from JBoss. This will get rid of most of the Spring JMS problems.
● Use standard JMS instead of Spring JMS for the batch job. Spring is a non-standard (and probably sub-standard implementation of JMS). Standard JMS uses a pool of a few senders to send the message and close the session after the message is sent. On the listener side, standard JMS uses a pool of works listening to a distributed Queue or Topic. Each web server has JMS listener deployed as singleton and uses standard java observer to
notify any caller that is expecting a call back.
The JMS connection factories are configured in JBoss and loaded via JNDI.
Can you provide your feedback on their assessment?
To avoid the overhead of creating new connections/sessions per send, you need to wrap the provider's connection factory in a CachingConnectionFactory. It reuses the same connection for sends and caches sessions, producers, consumers.
I have a request/reply implemented using spring integration and amqp. The requests may take long time to process (they may take even an hour for some cases) , for some reason the client throws exception -
Exception in thread "main" org.springframework.integration.handler.ReplyRequiredException: No reply produced by handler 'client', and its 'requiresReply' property is set to true.
My client config is below.
<int-amqp:outbound-gateway
id="client"
request-channel="in"
reply-channel="res"
exchange-name="reportingServer"
routing-key-expression="'report.req.'+headers.id"
amqp-template="amqpTemplate" requires-reply="true">
</int-amqp:outbound-gateway>
I believe reply-timeout default value is -1, means wait indefinitely, but not sure why its not working, any help would be appreciated.
Also are there any known issues with implementing such long waiting operations in amqp or it should be just fine?
Thank You
It will be fine as long as you don't have too many concurrent requests - it won't scale very well if you have gazillions of threads hanging waiting for a reply.
If you need to scale it, you could devise an asynchronous equivalent with a pair of outbound and inbound adapters, but it's a little more involved than using a gateway and the actual implementation will depend on the rest of your flow. Essentially you'll have to set up the replyTo header to cause the reply to come to the inbound adapter.
If you have a simple <gateway/> upstream of the AMQP gateway, you'll need to be sure that the replyChannel header is not lost. See Header Channel Registry in this section.
We previously used to have a Spring Integration flow (XML configuration-based) where we would do an update in a database after sending a message to a JMS queue. To achieve this, the SI flow was configured with a publish-subscribe queue channel as an input to a JMS Outbound Channel Adapter (order 0) and a Service Activator (order 1). The idea here being that after a successful JMS send, the service activator would be called thus, updating the data in the database.
We are now in the process of updating our flows to work with spring-integration:4.0.x APIs and wanted to use this opportunity to see if the described flow pattern is still a good/recommended way of doing a database update after a successful JMS send or if there is now a simpler/better way of achieving this? As a side note, our flows are now being implemented using spring-integration-java-dsl:1.0.0.M3 APIs.
Thanks in advance for any input on this,
PM.
publish-subscribe queue channel
There's no such thing as a pub-sub queue channel; by definition, it's a subscribable channel; so I assume that's what you mean.
It is one of the ways to do what you need, and perfectly fine; you can also achieve the same result with a RecipientListRouter. The dsl syntax is quite nice, especially with Java 8; see the SpringOne demo app for an example.