Vaadin and receiving email asynchronouosly - spring-integration

My Vaadin 14 application should receive emails in the background. If emails with a certain subject have been received, the user should be informed about this via PUSH message on the UI.
For the entire email handling I implemented the email / message handling from Spring integration and that works too. Two beans (IntegrationFlow and a ServiceActivator) are generated via #Configuration and #Bean annotation in the Spring Application Context like so:
#Configuration
public class EmailReceiver {
#Bean
public HeaderMapper<MimeMessage> mailHeaderMapper() {
return new DefaultMailHeaderMapper();
}
#Bean
public IntegrationFlow imapMailFlow() {
IntegrationFlow flow = IntegrationFlows
.from(Mail.imapInboundAdapter("imaps://user:pass#imap.ionos.de/INBOX")
.userFlag("testSIUserFlag")
.javaMailProperties(new Properties()),
e -> e.autoStartup(true)
.poller(p -> p.fixedDelay(5000)))
.transform(Mail.toStringTransformer())
.channel(MessageChannels.queue("imapChannel"))
.get();
return flow;
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
pollerMetadata.setTrigger(new PeriodicTrigger(1000));
return pollerMetadata;
}
#Bean
#ServiceActivator(inputChannel = "imapChannel")
public MessageHandler processNewEmail() {
MessageHandler messageHandler = new MessageHandler() {
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
System.out.println("new email received");
}
};
return messageHandler;
}
}
See also here: https://docs.spring.io/spring-integration/docs/current/reference/html/mail.html#mail-java-dsl-configuration
With such a #Configuration annotated class, the emails are received in the background of the Vaadin app. Check.
But how can I integrate a callback into a Vaadin view in the method EmailReceiver.processNewEmail?
#Bean
#ServiceActivator(inputChannel = "imapChannel")
public MessageHandler processNewEmail(UI ui) {
This always throws an error at application start: Scope vaadin-ui is not active for the current thread; consider defining a scoped proxy for this bean.
There is the example for asynchronous updates with Vaadin https://vaadin.com/docs/v14/flow/advanced/tutorial-push-access.
In contrast to this, I have to create a #Bean for #ServiceActivator handling. As soon as that is the case, there is always the error There is no UI available. The UI scope is not active.
If I move the method processNewEmail() into a separate class I still cannot reference a Vaadin UI:
#MessageEndpoint
class EmailMessageHandler {
private UI ui;
public EmailMessageHandler(UI ui) {
this.ui = ui;
}
#Bean
#ServiceActivator(inputChannel = "imapChannel")
public MessageHandler processNewEmail() {
MessageHandler messageHandler = new MessageHandler() {
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
System.out.println("new email received" + message);
}
};
return messageHandler;
}
}
How can I combine Vaadin asynchronous handling and Spring-Integration Email/ServiceActivator processing?

The point is that your mail receiving functionality is singleton per your application. On the other hand you are going to have as many UIs as many users do HTTP requests to your application. So, you need to think about some intermediary to dump email and get them from there when UI request happens.
You already have that imapChannel as a QueueChannel. So, you can take it from your UI scoped code and call its receive() API to pull the next message. Only the problem that it is a queue: as long as one call receive(), the other won't see the same message. Probably this is OK for your so far, but better to think about something what could be treated as topic in messaging terms. A good candidate easy to use is a Reactor's Sinks.Many: https://projectreactor.io/docs/core/release/reference/#sinks

Related

Spring Kafka outboundChannelAdapter's control does not return back in the integration flow

After a messsage is sent, it gets published to Kafka topic but the Message from KafkaSuccessTransformer does not return back to the REST controller. I am trying to return the message as-is if sent successfully but nothing after Kafka handler seems to be invoked.
#MessagingGateway
public interface MyGateway<String, Message<?>> {
#Gateway(requestChannel = "enrollChannel")
Message<?> sendMsg(#Payload String payload);
}
------------------------
#RestController
public class Controller {
MyGateway<String, Message<?>> myGateway;
#PostMapping
public Message<?> send(#RequestBody String request) throws Exception {
Message<?> resp = myGateway.sendMsg(request);
log.info("I am back"); // control doesn't come to this point
return resp;
}
}
--------------------------
#Component
public class MyIntegrationFlow {
KafkaSuccessTransformer stransformer;
#Bean
public MessageChannel enrollChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel kafkaSuccessChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow enrollIntegrationFlow() {
return IntegrationFlows.from("enrollChannel")
//another transformer which turns the string to Message<?>
.handle(Kafka.outboundChannelAdapter(kafkaTemplate) //kafkaTemplate has the necesssary config
.topic("topic1")
.messageKey(messageKeyFunction -> messageKeyFunction.getHeaders()
.get("key1")
.sendSuccessChannel("kafkaSuccessChannel"));
}
#Bean
public IntegrationFlow successfulKafkaSends() {
return f -> IntegrationFlows.from("kafkaSuccessChannel").transform(stransformer);
}
}
--------------
#Component
public class KafkaSuccessTransformer {
#Transformer
public Message<?> transform(Message<?> message) {
log.info("Message is sent to Kafka");
return message; //control comes here but does not return to REST controller
}
}
Channel adapters are for one-way traffic; there is no result.
Add a publishSubscribe channel with two subflows; the second one can be just a bridge to nowhere - .bridge() ends the flow. It will then return the outbound message to the gateway.
See https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl-subflows
Per Artem:
Something is off in the configuration or code. The logic is like this: processSendResult(message, producerRecord, sendFuture, getSendSuccessChannel());. Then: getMessageBuilderFactory().fromMessage(message). So, the replyChannel header is present in this "success" message. Therefore that transform(stransformer) should really produce its return to the replyChannel for a gateway in the beginning. Only the problem could be in the KafkaSuccessTransformer code where it does not copy request message headers for reply message. Please, share its whole code.

Spring Integration - Messaging Gateway with a returning value

I'm new with Spring Integration. I need to implement a Messaging Gateway with a returning value. In order to continue some processing asynchronously after executing some synchronous steps. So I made 2 activators
#Slf4j
#MessageEndpoint
public class Activator1 {
#ServiceActivator(inputChannel = "asyncChannel")
public void async(){
log.info("Just async message");
try {
Thread.sleep(500);
} catch (InterruptedException e) {
log.error("I don't want to sleep now");
}
}
}
and
#Slf4j
#MessageEndpoint
public class Activator2 {
#ServiceActivator(inputChannel = "syncChannel")
public ResponseEntity sync(){
try {
Thread.sleep(500);
return ResponseEntity.ok("Return Http Message");
} catch (InterruptedException e) {
log.error("I don't want to sleep");
}
return ResponseEntity.badRequest().build();
}
}
The pipeline
#Configuration
public class Pipeline {
#Bean
MessageChannel asyncChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel syncChannel() {
return MessageChannels.direct().get();
}
}
the gateway
#MessagingGateway
public interface ReturningGateway {
#Gateway(requestChannel = "asyncChannel", replyChannel = "syncChannel")
public ResponseEntity getSyncHttpResponse();
}
And Controller
#Slf4j
#RestController
#RequestMapping("/sync")
public class ResponseController {
#Autowired
ReturningGateway returningGateway;
#PostMapping("/http-response")
public ResponseEntity post() {
return returningGateway.getSyncHttpResponse();
}
}
So I'm not sure if thats the correct way to do what I want to do
Can you give me hand?
Let me try to explain some things first of all!
#Gateway(requestChannel = "asyncChannel", replyChannel = "syncChannel")
The requestChannel is where a gateway sends a message. But since you don't have any arguments in the gateway method and there is no a payloadExpression, the behavior is to "receive" from that channel. See docs for more info: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#gateway-calling-no-argument-methods.
The replyChannel is where to wait for a reply, not send. In most cases a gateway relies on the replyChannel header for correlation. The request-reply pattern in messaging. We need an explicit replyChannel if it is a PublishSubscribeChannel to track a reply somehow or when we deal with the flow which we can't modify to rely on the replyChannel header. See the same gateway chapter in the docs.
Your use-case is not clear for me: you say async continuation, but at the same time the return from your gateway contract looks like a result of that sync() method. From here, please make yourself familiar with the gateway contract and then come back to us with refreshed vision for your solution.

Thread safety in executor channel

I have a message producer which produces around 15 messages/second
The consumer is a spring integration project which consumes from the Message Queue and does a lot of processing. I have used the Executor channel to process messages in parallel and then the flow passes through some common handler class.
Please find below the snippet of code -
baseEventFlow() - We receive the message from the EMS queue and send it to a router
router() - Based on the id of the message" a particular ExecutorChannel instance is configured with a singled-threaded Executor. Every ExecutorChannel is going to be its dedicated executor with only single thread.
skwDefaultChannel(), gjsucaDefaultChannel(), rpaDefaultChannel() - All the ExecutorChannel beans are marked with the #BridgeTo for the same channel which starts that common flow.
uaEventFlow() - Here each message will get processed
#Bean
public IntegrationFlow baseEventFlow() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(Jms.container(this.emsConnectionFactory, this.emsQueue).get()))
.wireTap(FLTAWARE_WIRE_TAP_CHNL)
.route(router()).get();
}
public AbstractMessageRouter router() {
return new AbstractMessageRouter() {
#Override
protected Collection<MessageChannel> determineTargetChannels(Message<?> message) {
if (message.getPayload().toString().contains("\"id\":\"RPA")) {
return Collections.singletonList(skwDefaultChannel());
}else if (message.getPayload().toString().contains("\"id\":\"ASH")) {
return Collections.singletonList(rpaDefaultChannel());
} else if (message.getPayload().toString().contains("\"id\":\"GJS")
|| message.getPayload().toString().contains("\"id\":\"UCA")) {
return Collections.singletonList(gjsucaDefaultChannel());
} else {
return Collections.singletonList(new NullChannel());
}
}
};
}
#Bean
#BridgeTo("uaDefaultChannel")
public MessageChannel skwDefaultChannel() {
return MessageChannels.executor(SKW_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
}
#Bean
#BridgeTo("uaDefaultChannel")
public MessageChannel gjsucaDefaultChannel() {
return MessageChannels.executor(GJS_UCA_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
}
#Bean
#BridgeTo("uaDefaultChannel")
public MessageChannel rpaDefaultChannel() {
return MessageChannels.executor(RPA_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
}
#Bean
public IntegrationFlow uaEventFlow() {
return IntegrationFlows.from("uaDefaultChannel")
.wireTap(UA_WIRE_TAP_CHNL)
.transform(eventHandler, "parseEvent")
.handle(uaImpl, "process").get();
}
My concern is in the uaEVentFlow() the common transform and handler method are not thread safe and it may cause issue. How can we ensure that we inject a new transformer and handler at every message invocation?
Should I change the scope of the transformer and handler bean as prototype?
Instead of bridging to a common flow, you should move the .transform() and .handle() to each of the upstream flows and add
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
to their #Bean definitions so each gets its own instance.
But, it's generally better to make your code thread-safe.

Time-limited aggregation with publish-subscribe in Spring Integration

I am trying to implement the following using Spring Integration with DSL and lambda:
Given a message, send it to N consumers (via publish-subscribe). Wait for limited time and return all results that have arrived form consumers (<= N) during that interval.
Here is an example configuration I have so far:
#Configuration
#EnableIntegration
#IntegrationComponentScan
#ComponentScan
public class ExampleConfiguration {
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedRate(1000).maxMessagesPerPoll(1).get();
}
#Bean
public MessageChannel publishSubscribeChannel() {
return MessageChannels.publishSubscribe(splitterExecutorService()).applySequence(true).get();
}
#Bean
public ThreadPoolTaskExecutor splitterExecutorService() {
final ThreadPoolTaskExecutor executorService = new ThreadPoolTaskExecutor();
executorService.setCorePoolSize(3);
executorService.setMaxPoolSize(10);
return executorService;
}
#Bean
public DirectChannel errorChannel() {
return new DirectChannel();
}
#Bean
public DirectChannel requestChannel() {
return new DirectChannel();
}
#Bean
public DirectChannel channel1() {
return new DirectChannel();
}
#Bean
public DirectChannel channel2() {
return new DirectChannel();
}
#Bean
public DirectChannel collectorChannel() {
return new DirectChannel();
}
#Bean
public TransformerChannel1 transformerChannel1() {
return new TransformerChannel1();
}
#Bean
public TransformerChannel2 transformerChannel2() {
return new TransformerChannel2();
}
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from(errorChannel())
.handle(m -> System.err.println("[" + Thread.currentThread().getName() + "] " + m.getPayload()))
.get();
}
#Bean
public IntegrationFlow channel1Flow() {
return IntegrationFlows.from(publishSubscribeChannel())
.transform("1: "::concat)
.transform(transformerChannel1())
.channel(collectorChannel())
.get();
}
#Bean
public IntegrationFlow channel2Flow() {
return IntegrationFlows.from(publishSubscribeChannel())
.transform("2: "::concat)
.transform(transformerChannel2())
.channel(collectorChannel())
.get();
}
#Bean
public IntegrationFlow splitterFlow() {
return IntegrationFlows.from(requestChannel())
.channel(publishSubscribeChannel())
.get();
}
#Bean
public IntegrationFlow collectorFlow() {
return IntegrationFlows.from(collectorChannel())
.resequence(r -> r.releasePartialSequences(true),
null)
.aggregate(a ->
a.sendPartialResultOnExpiry(true)
.groupTimeout(500)
, null)
.get();
}
}
TransformerChannel1 and TransformerChannel2 are sample consumers and have been implemented with just a sleep to emulate delay.
The message flow is:
splitterFlow -> channel1Flow \
-> channel2Flow / -> collectorFlow
Everything seem to work as expected, but I see warnings like:
Reply message received but the receiving thread has already received a reply
which is to be expected, given that partial result was returned.
Questions:
Overall, is this a good approach?
What is the right way to gracefully service or discard those delayed messages?
How to deal with exceptions? Ideally I'd like to send them to errorChannel, but am not sure where to specify this.
Yes, the solution looks good. I guess it fits for the Scatter-Gather pattern. The implementation is provided since version 4.1.
From other side there is on more option for the aggregator since that version, too - expire-groups-upon-timeout, which is true for the aggregator by default. With this option as false you will be able to achieve your requirement to discard all those late messages. Unfortunately DSL doesn't support it yet. Hence it won't help even if you upgrade your project to use Spring Integration 4.1.
Another option for those "Reply message received but the receiving thread has already received a reply" is on the spring.integraton.messagingTemplate.throwExceptionOnLateReply = true option using spring.integration.properties file within the META-INF of one of jar.
Anyway I think that Scatter-Gather is the best solution for you use-case.
You can find here how to configure it from JavaConfig.
UPDATE
What about exceptions and error channel?
Since you get deal already with the throwExceptionOnLateReply I guess you send a message to the requestChannel via #MessagingGateway. The last one has errorChannel option. From other side the PublishSubscribeChannel has errorHandler option, for which you can use MessagePublishingErrorHandler with your errorChannel as a default one.
BTW, don't forget that Framework provides errorChannel bean and the endpoint on it for the LoggingHandler. So, think, please, if you really need to override that stuff. The default errorChannel is PublishSubscribeChannel, hence you can simply add your own subscribers to it.

Spring Integration 4 - configuring a LoadBalancingStrategy in Java DSL

I have a simple Spring Integration 4 Java DSL flow which uses a DirectChannel's LoadBalancingStrategy to round-robin Message requests to a number of possible REST Services (i.e. calls a REST service from one of two possible service endpoint URIs).
How my flow is currently configured:
#Bean(name = "test.load.balancing.ch")
public DirectChannel testLoadBalancingCh() {
LoadBalancingStrategy loadBalancingStrategy = new RoundRobinLoadBalancingStrategy();
DirectChannel directChannel = new DirectChannel(loadBalancingStrategy);
return directChannel;
}
#Bean
public IntegrationFlow testLoadBalancing0Flow() {
return IntegrationFlows.from("test.load.balancing.ch")
.handle(restHandler0())
.channel("test.result.ch")
.get();
}
#Bean
public IntegrationFlow testLoadBalancing1Flow() {
return IntegrationFlows.from("test.load.balancing.ch")
.handle(restHandler1())
.channel("test.result.ch")
.get();
}
#Bean
public HttpRequestExecutingMessageHandler restHandler0() {
return createRestHandler(endpointUri0, 0);
}
#Bean
public HttpRequestExecutingMessageHandler restHandler1() {
return createRestHandler(endpointUri1, 1);
}
private HttpRequestExecutingMessageHandler createRestHandler(String uri, int order) {
HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(uri);
// handler configuration goes here..
handler.setOrder(order);
return handler;
}
My configuration works, but I am wondering whether there is a simpler/better way of configuring the flow using Spring Integration's Java DSL?
Cheers,
PM
First of all the RoundRobinLoadBalancingStrategy is the default one for the DirectChannel.
So, can get rid of the testLoadBalancingCh() bean definition at all.
Further, to avoid duplication for the .channel("test.result.ch") you can configure it on the HttpRequestExecutingMessageHandler as setOutputChannel().
From other side your configuration is so simple that I don't see reason to use DSL. You can achieve the same just with annotation configuration:
#Bean(name = "test.load.balancing.ch")
public DirectChannel testLoadBalancingCh() {
return new DirectChannel();
}
#Bean(name = "test.result.ch")
public DirectChannel testResultCh() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "test.load.balancing.ch")
public HttpRequestExecutingMessageHandler restHandler0() {
return createRestHandler(endpointUri0, 0);
}
#Bean
#ServiceActivator(inputChannel = "test.load.balancing.ch")
public HttpRequestExecutingMessageHandler restHandler1() {
return createRestHandler(endpointUri1, 1);
}
private HttpRequestExecutingMessageHandler createRestHandler(String uri, int order) {
HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(uri);
// handler configuration goes here..
handler.setOrder(order);
handler.setOutputChannel(testResultCh());
return handler;
}
From other side there is MessageChannels builder factory to allow to simplify loadBalancer for your case:
#Bean(name = "test.load.balancing.ch")
public DirectChannel testLoadBalancingCh() {
return MessageChannels.direct()
.loadBalancer(new RoundRobinLoadBalancingStrategy())
.get();
}
However, I can guess that you want to avoid duplication within DSL flow definition to DRY, but it isn't possible now. That's because IntegrationFlow is linear to tie endoints bypassing the boilerplate code for standard objects creation.
As you see to achieve Round-Robin we have to duplicate, at least, inputChannel, to subscribe several MessageHandlers to the same channel. And we do that in the XML, via Annotations and, of course, from DSL.
I'm not sure that it will be so useful for real applications to provide a hook to configure several handlers using single .handle() for the same Round-Robin channel. Because the further downstream flow may not be so simple as your .channel("test.result.ch").
Cheers

Resources