In my spring integration application i have several stored-proc-outbound-gateway, i would to log how much time each call is taking, any help would be appreciated.
I would ideally like to be able to enable/disable logging for the parameters used, time taken and total rows retrieved (returning-resultset) to monitor and performance tuning purpose.
Thanks
You can add a ChannelInterceptor (subclass of ChannelInterceptorAdapter) to the request channel which will give you raw timing (preSend/postSend), but the time will include any processing downstream of the gateway (on direct channels).
Since you want to examine the results too, you could start a timer (e.g. Spring StopWatch) in the interceptor (preSend) on the request channel and stop the timer in an interceptor on the reply channel). If you use the same interceptor bean you can store the timer in a ThreadLocal.
You can turn on/off collection using a boolean property on the interceptor.
Alternatively, you can add a custom advice to the gateway.
EDIT
The advice is probably the best approach because with a ThreadLocal you will need to add code to the first interceptor to handle failures and clean up. With an around advice, the timer would just be a local method variable.
Related
I have a flow that starts with a poller and hands off the message to several async flows downstream using task-executors to execute in parallel for a given dataset. A downstream aggregator completes the flow and notifies the poller that the flow is complete.
I would like to track every execution of the poller by using MDC so that the logs can be mapped to a particular execution of the flow.
I started by adding MDC to the poller thread (using Advice), however with this approach there could be a couple of issues:
How do I stamp the MDC on the executor thread when the async hand off happens?
Since executor uses a a thread pool, do I need to clear the MDC before the thread returns to the pool? Will there be any side effects?
Another approach would be to add MDC to the Message header and set it manually on the new thread during the async handoff. How to do that? For example, if I turn on the debug logs, the MDC should be stamped right from the beginning of the new thread execution and not from the point where my logic starts in the service activator.
How to set this on the task-executor thread (and probably also remove before returning to the pool) using XML configuration? Something like an MdcAwareThreadPoolExecutor seen here.
Also, I would not want the MDC logic to be spread across all the async handoff endpoints, may be there is some generic way to configure it?
Is there a better way to achieve this? Any known solutions?
I would like to track every execution of the poller by using MDC so that the logs can be mapped to a particular execution of the flow.
It is fully sound as "you would like to track the message journey in your flow". As you noticed there is the way to set some message header. So, why just don't map your logs by this specific header?
You can take a look into Message History pattern how to gather the whole path for the message, so then in logs you can track it back looking into message headers.
See here: https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#message-history
If you really still insist on the MDC, then you definitely need to take a look into some MDCDelegatingExecutorDecorator. Some sample you can borrow from Spring Security and its DelegatingSecurityContextExecutor`: https://docs.spring.io/spring-security/site/docs/5.4.0/reference/html5/#concurrency
I'm trying to control the amount of maxThreadsActive and maxThreadsIdle for outgoing HTTP connections in Mule.
Setting the default-threading-profile doesn't affect the amount of threads that are allocated HTTP requesters.
For HTTP listeners it's possible to set the threading profile via the http:worker-threading-profile, like this:
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration">
<http:worker-threading-profile maxThreadsActive="2" maxThreadsIdle="1" threadTTL="60000"/>
</http:listener-config>
But i can't find a way to apply a threading profile on a http:request element.
Besides this i'm wondering how the http:worker-threading-profile in this case works for listeners, when i use a profiler (VisualVM) i don't see any changes in the amount of threads that are allocated for the HTTP listener.
Any ideas regarding threads for HTTP endpoints and how to control them and verify it?
Screenshot below is from a simple test app with the threading profile applied as mentioned above.
The same app has a simple http:request config, for an outbound HTTP connection (requester) i always get this number of threads:
Never tried it myself, but some info from research and training says this: If your flow is using a synchronous processing strategy, which Mule sets based on your message source and flow behavior, processing is done in the same thread. This might explain why you don't see any changes in the amount of threads that are allocated for the HTTP listener. The flow is set to synchronous if the message source is request-response--sender of the message is expecting a response or the flow is involved in a transaction.
Otherwise, Mule sets the flow to queued-asynchronous. In this case you set threads using the flow's properties view (in Studio, select the flow itself and look for Processing Strategy in the properties). Set properties for the flow as described in the docs. You do not set threads for the HTTP Requester afaik.
I have a requirement to fetch jms message from IBM MQ. But the messages should be fetched only when required, say when a method is invoked. The picked up message need to be put to another queue in turn. That is , this scenario is for copying message from one queue to another when required.
How could I achieve this in Spring Integration.
I believe i could create a flow using inbound-channel-adapter, set to poll message at regular interval, along with outbound-channel-adapter. But with this, I could not pick the message as and when required.
Please advise.
Question updated:
1) While searching on the web to implement the suggested solution, I stepped on the spring documentation for JmsDestinationPollingSource saying - "This source is only recommended for very low message volume".
2) Since we need for high volume environment and in need of local transaction, I looked into the suggested JmsMessageDrivenEndpoint solution which looked like using MessageDrivenChannelAdapter. Can you please adviseas how i could proceed.
3) Also, in that case how can i detect if all messages have been read to stop the listener . Please advise
Question updated:
I am able to invoke start and stop on DefaultMessageListener container used with MessageDrivenChannelAdapter.
Can you also please advise as how I could find when there are no messages on the queue or when the queue is empty so that I could invoke stop.
Question updated:
Thanks Gary. Sorry for the late update.
To find when the queue becomes empty, can we override noMessageReceived method of DefaultMessageListenerContainer (inherited from AbstractPollingMessageListenerContainer) and invoke stop on the DMLC. This kind of solution is given in
Spring integration inbound-gateway Fire an event when queue is empty.
I thought this might be easier to implement. Please advise if it will be correct to do in that way.
Set auto-startup="false" on the inbound channel adapter (so it won't poll).
Get a reference to its MessageSource (either by #Autowire or otherwise injecting or context.getBean()). The bean name is adapterid.source and is of type JmsDestinationPollingSource.
Then invoke the receive() method.
Or you can just wire up a JmsDestinationPollingSource bean yourself without using the namespace support.
I've a couple of questions regarding the (excellent) CouchDB .NET client MyCouch:
Is there some built-in retry policy in case of "transient" failure (like the server responding 503)?
Should instances of MyCouchClient or MyCouchStore be cached to be reused? Right now I'm creating one for each incoming request, but I'm wondering if that incurs a performance penalty.
I would like to customize the configuration of Json.NET as used by MyCouch, like adding a new StringEnumConverter { CamelCaseText = true } to the list of Converters. Is there a way to achieve that through the API?
Thanks
1) There's no magic in the MyCouchClient, it's just simple requests and responses. The MyCouchStore how-ever, I would gladly accept pull request to have options for retries or e.g. auto-batching queries.
2) Here are some links to information that would help you to decide on per request or per application.
Is async HttpClient from .Net 4.5 a bad choice for intensive load applications?
http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit.aspx
So doing one per application would probably need a reconf of the connection limit.
I have this centralized in my IoC-config, and by default I'm not doing per application. The first "connection" can take a bit longer, but the second has been measured down to milliseconds against Cloudant by other users so that should in general not be an issue.
3)
You can configure the serializer by providing a custom MyCouchClientBootstrapper and providing a custom implementation of: https://github.com/danielwertheim/mycouch/blob/master/source/projects/MyCouch.Net45/MyCouchClientBootstrapper.cs#L170
And you also have to extend this guy: https://github.com/danielwertheim/mycouch/blob/master/source/projects/MyCouch.Net45/Serialization/SerializationConfiguration.cs#L9
Feel free to suggest changes that makes this process simpler for you.
Is it possible to have serial processing in spring integration that is the response of one request is get to the next as its request. I have a requirement where only after getting response from service-1 I can initiate call to service-2. This was suggested so because only service-1 has a roll back service implemented.
Is it possible to control which request is processed first, I want request 1 to be processed first. Is this also possible
It really depends on what you are trying to do, but the general solution would be to use a <publish-subscribe-channel/> set the order on the first service to "1" and the second to "2".
By default, the second service will only be called if the first is successful.
If you need to aggregate the results, add an aggregator downstream of both services.