Thinking to use Spring Integration for dynamic Inbound / Outbound configuration. There can be 'n' number of customers using our system, each of them will have their own inbound FTP / webservice configuration to pull files for processing into our system. Likewise after processing each of those customers can have outbound FTP / webservice configuration where the reports (end result) need to be pushed. Is spring integration appropriate for such cases? if yes, can you please suggest or point to any of the examples. Webservice includes both REST and SOAP.
Also, instead of polling at fixed interval, can each customer have their own cron expression as to when to pull the files during Inbound?
Your use-case looks fully as an appropriate for the Spring Cloud Data Flow.
Each customer will configure a stream with desired options for each module.
Otherwise your question looks very broad and it has a big chance to be closed here.
See the dynamic ftp sample (XML configuration). This is outbound; see the readme for some links about doing it the same way for inbound.
See this answer - mail inbound adapters using Java Config
And the Dynamic TCP example using the Java DSL dynamic flow registration.
Dynamic WebService inbound is a bit more tricky because you have to register the new endpoints
Related
If one application have Azure serviceBus, EventHub in diff Azure Namesapces, web application and also other azure services (eg: cognitive services). can these be accessed with one URL by using Gateway or Load balancer or traffic manager or any other option ?
My problem is - if we have diff namesapces, we need to whitelist every time when there is new Namespaces and it could so too much of a work. so wondering if we can have one common DNS/URL that would make life easier.
Today, Service Bus and Event Hubs don't support any sort of network gateway. This is due to fact that namespace in the connection string used for authorization purpose at the service side.
To add a bit of context to Serkant's statement, support for this scenario is something that is on our roadmap, and hopefully in the near term. Unfortunately, I don't have a date to share currently. The work is being tracked [here] should you wish to keep an eye on it.
I am implementing a spring boot app that connects to different imap servers to receive emails.
There are lot of imap servers and their number will increase.
The app will be run in multiple instances.
I want to use spring integration mail. But I wouldn't like to create an integration flow for each imap server in each instance of app.
I am wondering if I can avoid creation of separate integration flow for each imap server?
It could be look like: we create one flow that will be run manually (e.g by an event) and this flow would dynamically create needed imap receiver, receive mails and finish.
Is it possible to implement something like that using spring integration mail?
If not and I have to run a separate flow for each imap server on each instance, each flow will probably have to use polling. Does spring integration provide any solution that controls that the same flows of different instances are not run at the same time or controls the polling time of flows of the same sources on different app instances? Maybe there is a load balancer that can control flows on different instances?
Thank you in advance!
I think the dynamic flows registration is an answer for you.
See docs: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl-runtime-flows
And see this sample: https://github.com/spring-projects/spring-integration-samples/tree/master/advanced/dynamic-tcp-client. It uses TCP/IP for dynamic registration, but you can borrow an idea what you should do for your imap servers and respective IntegrationFlow to provide mail channel adapters.
I am building the pollers based on the data from the database. Mainly file pollers.
Does Spring integration provide any way to monitor the flows via the spring admin /actuator?
If yes, is there any example?
Well, need to understand what you mean with the monitor.
There is an /integrationgraph since Spring Boot 2.1 endpoint to show all the integration components in the application context as a JSON graph. You have enable it explicitly though: https://docs.spring.io/spring-boot/docs/2.1.0.RELEASE/reference/htmlsingle/#production-ready-endpoints-exposing-endpoints.
There is also an endpoint for metrics to obtain: https://docs.spring.io/spring-boot/docs/2.1.0.RELEASE/reference/htmlsingle/#production-ready-metrics-meter
I have an application that subscribes to an Azure Service Bus topic and processes the incoming messages. I would like to do a performance/stress test by constantly sending a very large batch of messages from different locations to the Service Bus and measure the processing time of my application.
Is there a tool out there to flood Service Bus?
I would recommend going for Apache JMeter, it is free, open source, scalable, GUI-based multiprotocol load testing tool.
Given you have Azure Libraries For Java in JMeter Classpath you should be able to use the following Test Elements:
JMS Publisher
JMS Subscriber
JMS Point-to-Point
to mimic the load on ASB.
See Building a JMS Testing Plan - Apache JMeter for details.
Another free and open source tool which can be used is Gatling, see JMS API Example for reference.
Have a look at the sample ASB team has once provided. Saying that, I'd like to warn about tooling - your benchmarking will be off if you're in the standard tier.
Challenge with standard tier is the noisy neighbor effect, when because of your (or someone else) operation you can get throttled.
With premium tier you can get the consistent and promised throughput.
My question may be a repeating one.. but I was unable to find answer for this..
Our team is evaluating Spring Integration to replace the Message Broker flows.. Meanwhile I am writing the integration for consuming two different web service where in a single configuration I have defined separate channels for each web service outbound-gateway.
I figured out to use chaining to reduce the channel definition between the endpoints & it works well.
All i need to ask is, is it possible to share the same channel between various
web service outbound-gateway? Can I define only one channel where 2 web service outbound will be listening for messages?
If yes then how will SI identify that message-1 on channel-1 is for gateway-1 and message-2 on channel-1 is for gateway-2 and so on?
Is it necessary to define a unique channel for each web service outbound gateway?
-Thanks
MS
Yes, generally, each gateway needs its own channel.
However...
If you want to send the SAME message to both gateways, you can use a <publish-subscribe-channel/>.
If you want to "round robin" (load balance) across multiple instances of the same service then, yes, you simply subscribe each gateway to the same channel and they will alternate.
You could use a custom load balancing strategy to decide which gateway will get a message but it's probably better to simply use a router and have each gateway have its own channel.
It really depends on your use case, but there are lots of options.
However, don't fall into the trap of trying to eliminate channels everywhere - that's one of the compelling features of Spring Integration - the ability to change channel types/features without rewiring your entire application.