I have a request/reply implemented using spring integration and amqp. The requests may take long time to process (they may take even an hour for some cases) , for some reason the client throws exception -
Exception in thread "main" org.springframework.integration.handler.ReplyRequiredException: No reply produced by handler 'client', and its 'requiresReply' property is set to true.
My client config is below.
<int-amqp:outbound-gateway
id="client"
request-channel="in"
reply-channel="res"
exchange-name="reportingServer"
routing-key-expression="'report.req.'+headers.id"
amqp-template="amqpTemplate" requires-reply="true">
</int-amqp:outbound-gateway>
I believe reply-timeout default value is -1, means wait indefinitely, but not sure why its not working, any help would be appreciated.
Also are there any known issues with implementing such long waiting operations in amqp or it should be just fine?
Thank You
It will be fine as long as you don't have too many concurrent requests - it won't scale very well if you have gazillions of threads hanging waiting for a reply.
If you need to scale it, you could devise an asynchronous equivalent with a pair of outbound and inbound adapters, but it's a little more involved than using a gateway and the actual implementation will depend on the rest of your flow. Essentially you'll have to set up the replyTo header to cause the reply to come to the inbound adapter.
If you have a simple <gateway/> upstream of the AMQP gateway, you'll need to be sure that the replyChannel header is not lost. See Header Channel Registry in this section.
Related
To clarify the title: I am using ActiveMQ 5.15.15 (NOT the Artemis engine), and I am using AMQP 1.0 without official JMS libraries. And to be more specific, I am using the AmazonMQ version of this, which will soon upgrade to 5.16.2. I could force the upgrade, if needed.
I'm using an AMQP 1.0 compatible library (rhea) that has served us well so far, but I'm not finding any documentation for how to get ActiveMQ's redelivery plugin to work with my library. The library maintainers are unaware with how this is exposed via ActiveMQ, as well.
I've not been able to get the redelivery plugin to work, despite trying to add various headers, delivery annotations, message annotations, or application properties. I do have schedulerSupport="true" in my broker element for the server config.
These are the keys I've tried, and the values are numeric. E.g., 30000 for 30 seconds before allowing a consumer/subscriber see the message in the queue. I saw them in various docs, and it didn't hurt to try them.
AMQ_SCHEDULED_DELAY
x-opt-delivery-delay
_AMQ_SCHED_DELIVERY
I have also released the message from the client, meaning it failed to deliver (also passing a value that signals the failure to the broker and increases the attempted delivery count). While the number of delivery attempts increased, the delay and exponential backoff have not seemed to be working at the broker level.
I see that the STOMP protocol allows for headers when publishing, which allow setting options a bit more clearly. However, I don't want to switch everything over unless it makes sense to do so.
I also saw another ability to send a delayed message as a topic via the REST API, but I'm not sure if that was intended to be a production use case.
So right now, I'm either looking at:
hold the message in memory for a bit and attempt to republish or release it after a delay
Investigate STOMP, see if the redelivery plugin works with that
But I'm hoping someone knows where to implement this.
My redeliveryPolicy is basic:
<!--
The Redelivery plugin extends the capabilities of destination policies with respect to message redelivery.
For more information, see http://activemq.apache.org/message-redelivery-and-dlq-handling.html
-->
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<redeliveryPolicyEntries>
<!--<redeliveryPolicy maximumRedeliveries="4" queue="SpecialQueue" redeliveryDelay="10000"/>-->
</redeliveryPolicyEntries>
<defaultEntry>
<!-- 5s -> 15s -> 45s -> 135s -> 405s -->
<redeliveryPolicy backOffMultiplier="3" initialRedeliveryDelay="5000" maximumRedeliveries="5"/>
</defaultEntry>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
Update
I am using the auth plugin, and there's an entry that seems like it's for a built-in process. I think this came from a sample/default config. There doesn't appear to be a whole lot of documentation around this from a quick search. I can try opening access to other users, but each update/restart can take up to 15 minutes with the current setup.
<authorizationEntry admin="administrators" queue="SchedulingProcessor.>" write="scheduling-processor"/>
Comment Clarifications
My main objective is to delay redeliveries, so consumers don't see a failed message that was placed back into the queue for n seconds.
I started with no special headers/properties/annotations + the redelivery plugin, which also didn't work.
There is a distinction around message delivery delay and message redelivery delay that I think you are confusing or at least the question is mixing up.
A sender can request that a message be delivered after some delay from an AMQP client using the Message Annotations section of the sent message and adding in 'x-opt-delivery-delay' or 'x-opt-delivery-time' annotation assuming the broker has enabled scheduled deliveries. Some examples of this can be found in the ActiveMQ unit test. The delay annotation indicates a relative delay from time of receipt in milliseconds while the delivery time annotation indicates a time in UTC to deliver the message.
The ActiveMQ 5 redelivery policy affects messages that have been explicitly tagged as not deliverable on the client and therefore the AMQP Released outcome is not the right choice to trigger this behavior as it simply indicates that the client isn't going to process it and the remote should consider it undelivered and send it elsewhere. You would need to use one of Rejected or Modified(undeliverableHere=true) to "poison" the message and trigger the redelivery policy. This should if things go right trigger a redelivery after some delay although since ActiveMQ 5 has a relatively basic AMQP protocol head it will likely resend to the same consumer even if you've explicitly set the undeliverable here flag. I don't know how much that bit has been tested if any so your mileage may vary.
I would like to know if I can have persistence in my Spring Integration setup when I use a aggregator, which is not backed by a MessageStore, by leveraging the persistence of AMQP (RabbitMQ) queues before and after the aggregator.
I imagine that this would use ack's: The aggregator won't ack a message before it's collected all the parts and sent out the resulting message.
Additionally I would like to know if this is ever a good idea :)
I am new working with queue's, and am trying to get a good feel for patterns to use.
My business logic for this is as follows:
I receive a messages on one queue.
Each message must result in two unrelated webservice calls (preferably in parallel).
The results of these two calls must be combined with details from the original message.
The combination must then be sent out as a new message on a queue.
Messages are important, so they must not be lost.
I was/am hoping to use only one 'persistent' system, namely RabbitMQ, and not having to add a database as well.
I've tried to keep the question specific, but any other suggestions on how to approach this are greatly appreciated :)
What you would like to do recalls me Scatter-Gather EI Pattern.
So, you get a message from the AMQP send it into the ScatterGather endpoint and wait for the aggregated reply. That's enough for to stick with the default acknowledge.
Right, the scatterChannel can be PublishSubscribeChannel with an executor to call Web Services in parallel. Anyway the gatherer process will wait for replies according the release strategy and will block the original AMQP listener do not ack the message prematurely.
I'm trying to control the amount of maxThreadsActive and maxThreadsIdle for outgoing HTTP connections in Mule.
Setting the default-threading-profile doesn't affect the amount of threads that are allocated HTTP requesters.
For HTTP listeners it's possible to set the threading profile via the http:worker-threading-profile, like this:
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration">
<http:worker-threading-profile maxThreadsActive="2" maxThreadsIdle="1" threadTTL="60000"/>
</http:listener-config>
But i can't find a way to apply a threading profile on a http:request element.
Besides this i'm wondering how the http:worker-threading-profile in this case works for listeners, when i use a profiler (VisualVM) i don't see any changes in the amount of threads that are allocated for the HTTP listener.
Any ideas regarding threads for HTTP endpoints and how to control them and verify it?
Screenshot below is from a simple test app with the threading profile applied as mentioned above.
The same app has a simple http:request config, for an outbound HTTP connection (requester) i always get this number of threads:
Never tried it myself, but some info from research and training says this: If your flow is using a synchronous processing strategy, which Mule sets based on your message source and flow behavior, processing is done in the same thread. This might explain why you don't see any changes in the amount of threads that are allocated for the HTTP listener. The flow is set to synchronous if the message source is request-response--sender of the message is expecting a response or the flow is involved in a transaction.
Otherwise, Mule sets the flow to queued-asynchronous. In this case you set threads using the flow's properties view (in Studio, select the flow itself and look for Processing Strategy in the properties). Set properties for the flow as described in the docs. You do not set threads for the HTTP Requester afaik.
Our application integration flow is defined as splitter -> ws gateway -> aggregator. The splitter splits request into a list of account numbers; so that for each account number a web service call is initiated and the responses from multiple web service calls are aggregated in the aggregator.The channel between splitter and ws gateway is defined with dispatcher "commonj WorkManagerTaskExecutor" so that each webservice call is initiated parallel in different threads.
We have added timeout for each webservice call. But we would like to set a single timeout for the whole process. i.e. all the webservice calls should be completed in, say 50 secs, rather than setting 50 secs timeout for each individual call. commonj WorkManagerTaskExecutor, provides this feature by waitForAll(Collection workItems, long timeout_ms) method when implemented directly through code. Is there any way to use this or a similar feature to achieve our requirement.
Unfortunately, no, we can't use such a custom feature of that specific TaskExecutor.
From other side if you say "single timeout for the whole process" I can help you with the <gateway> pattern:
<chain>
<gateway request-channel="splitterChannel" reply-timeout="50000"/>
</chain>
Where reply-timeout is:
Specifies how long this gateway will wait for the reply message
before returning. By default it will wait indefinitely. 'null' is returned
if the gateway times out.
Does it make sense for you?
I'm trying to make sure I understand the meaning of the poolExhaustedAction values for a threading profile. I'm not seeing a lot of examples out there.
Assume I have a thread pool on an HTTP endpoint that has maxThreadsActive set to "16". I receive 20 inbound requests in a short period (faster than I can process any of them).
If poolExhaustedAction is set to "WAIT" then the last 4 requests will wait for threadWaitTimeout. Is this correct?
If poolExhaustedAction is set to "RUN" then the last 4 requests will ????...use the thread that carried the request to the endpoint to run the flow???? I'm a bit confused on this one. Specifically, if set to "RUN", will the service ever reject a request (assuming Mule has threads to deliver messages to it)?
Have you read http://www.mulesoft.org/documentation/display/current/Tuning+Performance? Especially this part?
Answers to your questions are:
Yes.
Indeed the thread that received the request will be used to process it in the flow. The service will start rejecting requests when inbound socket connections will time-out because the thread in charge of routing them in Mule is too busy to accept them.