I have this flow connected thru Spring Cloud DataFlow. One microservice gets the email from a queue and sends to this other microservice that should send the email.
I want to be able to catch failed emails and have the original message before .transform() and the exception message.
How to do it?
#Bean
public IntegrationFlow sendMailFlow() {
return IntegrationFlows.from(Sink.INPUT)
.transform(Transformers.converter(converter))
.handle(Mail.outboundAdapter(EmailSinkApplication.this.props.getServer())
.port(EmailSinkApplication.this.props.getPort())
.credentials(EmailSinkApplication.this.props.getUser(), EmailSinkApplication.this.props.getPass())
.protocol(EmailSinkApplication.this.props.getProto())
.javaMailProperties(p -> {
p.put("mail.smtp.starttls.enable", "true");
p.put("mail.smtp.auth", "true");
}),
e -> e.id("sendMailEndpoint"))
.get();
}
I suggest you to implement a ChannelInterceptor and add your logic into the afterSendCompletion(). Use a #GlobalChannelInterceptor on this interceptor bean to specify a patterns as a Sink.INPUT. So, you are going to have a try..catch around the whole flow on that Sink.INPUT channel.
Related
I am new to Spring Integration and i am trying to use the HttpRequestExecutingMessageHandler and the HttpRequestHandlingMessagingGateway. The Handler is sending a POST Request and expecting reply. The Gateway is consuming the request. It works fine, BUT the Handler is not getting a reply back. I dont know how to setup my flow, that i can direcly send a reply back and continue my flow.
#Bean
public HttpRequestExecutingMessageHandler httpOutbound() {
HttpRequestExecutingMessageHandler handler = Http.outboundGateway(restUrl)
.httpMethod(HttpMethod.POST)
.messageConverters(new MappingJackson2HttpMessageConverter())
.mappedRequestHeaders("Content-Type")
.get();
handler.setExpectReply(true);
handler.setOutputChannel(httpResponseChannel());
return handler;
}
#Bean
public HttpRequestHandlingMessagingGateway httpRequestGateway() {
HttpRequestHandlingMessagingGateway gateway = new HttpRequestHandlingMessagingGateway(true);
RequestMapping mapping = new RequestMapping();
mapping.setMethods(HttpMethod.POST);
mapping.setPathPatterns(httpRequestHandlingPathPattern);
gateway.setRequestMapping(mapping);
gateway.setErrorChannel(errorChannel());
gateway.setReplyChannel(replyChannel());
gateway.setRequestChannel(requestChannel());
gateway.setRequestPayloadTypeClass(DocumentConverterInput.class);
return gateway;
}
#Bean
public IntegrationFlow documentConverterFlow() {
return IntegrationFlows
.from(requestChannel())
.publishSubscribeChannel(publishSubscribeSpec ->
publishSubscribeSpec.subscribe(flow -> flow
.enrichHeaders(headerEnricherSpec -> headerEnricherSpec.header("http_statusCode", HttpStatus.OK))
.channel(replyChannel())))
.enrichHeaders(headerEnricherSpec ->
headerEnricherSpec.headerExpression(Constants.DOCUMENT_CONVERTER_INPUT, "payload"))
.handle(Jms.outboundAdapter(jmsTemplate(connectionFactory)))
.get();
}
My HttpRequestExecutingMessageHandler is successfully posting the request. The HttpRequestHandlingMessagingGateway is successfully consuming it. At first i had an error "No reply received within timeout", so that i added a publishSubscriberChannel. I don't know if it is the right way, but i cannot find working examples, which show how to reply correctly.
My above code is working, BUT not sending reply back to HttpRequestExecutingMessageHandler!
My goal is to receive the request message and directly send back 200 OK. After that i want to continue my integration flow, do some stuff and send the result to queue.
Any suggestions?
For just 200 OK response you need to consider to configure an HttpRequestHandlingMessagingGateway with a expectReply = false. This way it is going to work as an Inbound Channel Adapter, but having the fact that HTTP is always request-response, it will just do this setStatusCodeIfNeeded(response, httpEntity); and your HttpRequestExecutingMessageHandler on the client side will get an empty, but OK resposnse.
Not sure though why your channel(replyChannel()) doesn't work as expected. It might be the fact that you return a request palyoad into a reply message which eventually becomes as a HTTP response, but somehow it fails there, may be during conversion...
UPDATE
Here is a simple Spring Boot application demonstrating a reply-and-process scenario: https://github.com/artembilan/sandbox/tree/master/http-reply-and-process
I have got an Integrationflow which returns me a Message and now I would like to validate it in a second channel and after sending it back I would like to transform it.
But the transformation failed.
#Bean
public IntegrationFlow httpUserGetUserByUsername() {
return IntegrationFlows.from(httpGetGateUsersGetUserByUsername())
.channel("http.users.getUserByUsername").handle("userEndpoint", "getUserByUsername")
.channel("http.users.checkUserAuth").handle("userEndpoint","checkUserByUsername")
.transform(User u -> new UserResponseHelper(u))
.get();
}
public void checkUserByUsername(Message<User> msg) {
MessageChannel replayChannel = (MessageChannel) msg.getHeaders().getReplyChannel();
User u = msg.getPayload();
if (Authorization.isAllowByUsername(u.getUsername())) {
replayChannel.send(MessageBuilder.withPayload(u).build());
}else{
replayChannel.send(MessageBuilder.withPayload(new ResponseEntity(new ApiResponse(HttpStatus.FORBIDDEN,false,"You are not allow to get this ressource"), HttpStatus.FORBIDDEN)).build());
}
}
I get the user with all 8 properties and what i expect ist a user with 4 props
OK. I see what is your problem. You really have just missed the fact that your .transform(User u -> new UserResponseHelper(u)) is not called at all. So, that your "transformation failed" has confused me.
So, what is going on here:
Your void checkUserByUsername(Message<User> msg) returns void. Therefore there is nothing to wrap into an output Message. And from here nothing is going to trigger the next .transform() in your flow.
What you see for your REST service, that everything from that checkUserByUsername(Message<User> msg) is sent to the replyChannel header - the place where Inbound Gateway waits for reply.
In case of validation failure you send some custom ApiResponse. Why just don't throw an Exception, e.g. ResponseStatusException? This is going to be handled properly on the REST layer and returned as an error to the REST client.
I suggest you to really return that u from this checkUserByUsername() method and throw an exception otherwise. This way the User object is going to come to your .transform(User u -> new UserResponseHelper(u)) - and all good!
You have just confused yourself that void doesn't trigger transform and replyChannel is just for sending reply directly to the initiator gateway.
#Bean
public HttpMessageHandlerSpec documentumPolledInbound() {
return Http
.outboundGateway("/fintegration/getReadyForReimInvoices/Cz", restTemplate)
.expectedResponseType(String.class)
.errorHandler(new DefaultResponseErrorHandler())
;
}
How to poll the above, to get the payload for further processing
The HTTP Client endpoint is not pollable, but event-driven. So, as you usually call some REST Service from the curl, for example, the same way happens here. You have some .handle(), I guess, for this documentumPolledInbound() and there is some message channel to send message to trigger this endpoint to call your REST service.
Not clear how you are going to handle a response, but there is rally a way to trigger an event periodically to call this endpoint.
For that purpose we only can * mock* an inbound channel adapter which can configured with some trigger policy:
#Bean
public IntegrationFlow httpPollingFlow() {
return IntegrationFlows
.from(() -> "", e -> e.poller(p -> p.fixedDelay(10, TimeUnit.SECONDS)))
.handle(documentumPolledInbound())
.get();
}
This way we are going to send a message with an empty string as a payload to the channel for the handle(). And we do that every 10 seconds.
Since your HttpMessageHandlerSpec doesn't care about an inbound payload, it is really doesn't matter what we return from the MessageSource in the polling channel adapter.
I have a spring integration flow which produces messages that should be kept around waiting for an appropriate consumer to come along and consume them.
#Bean
public IntegrationFlow messagesPerCustomerFlow() {
return IntegrationFlows.
from(WebFlux.inboundChannelAdapter("/messages/{customer}")
.requestMapping(r -> r
.methods(HttpMethod.POST)
)
.requestPayloadType(JsonNode.class)
.headerExpression("customer", "#pathVariables.customer")
)
.channel(messagesPerCustomerQueue())
.get();
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerSpec poller() {
return Pollers.fixedRate(100);
}
#Bean
public QueueChannel messagesPerCustomerQueue() {
return MessageChannels.queue()
.get();
}
The messages in the queue should be delivered as server-sent events via http as shown below.
The PublisherSubscription is just a holder for the Publisher and the IntegrationFlowRegistration, the latter is used to destroy the dynamically created flow when it is no longer needed (note that the incoming message for the GET has no content, which is not handled properly ATM by the Webflux integration, hence a small workaround is necessary to get access to the path variable shoved to the customer header):
#Bean
public IntegrationFlow eventMessagesPerCustomer() {
return IntegrationFlows
.from(WebFlux.inboundGateway("/events/{customer}")
.requestMapping(m -> m.produces(TEXT_EVENT_STREAM_VALUE))
.headerExpression("customer", "#pathVariables.customer")
.payloadExpression("''") // neeeded to make handle((p,h) work
)
.log()
.handle((p, h) -> {
String customer = h.get("customer").toString();
PublisherSubscription<JsonNode> publisherSubscription =
subscribeToMessagesPerCustomer(customer);
return Flux.from(publisherSubscription.getPublisher())
.map(Message::getPayload)
.doFinally(signalType ->
publisherSubscription.unsubscribe());
})
.get();
}
The above request for server-sent events dynamically registers a flow which subscribes to the queue channel on demand with a selective consumer realized by a filter with throwExceptionOnRejection(true). Following the spec for Message Handler chain that should ensure that the message is offered to all consumers until one accepts it.
public PublisherSubscription<JsonNode> subscribeToMessagesPerCustomer(String customer) {
IntegrationFlowBuilder flow = IntegrationFlows.from(messagesPerCustomerQueue())
.filter("headers.customer=='" + customer + "'",
filterEndpointSpec -> filterEndpointSpec.throwExceptionOnRejection(true));
Publisher<Message<JsonNode>> messagePublisher = flow.toReactivePublisher();
IntegrationFlowRegistration registration = integrationFlowContext.registration(flow.get())
.register();
return new PublisherSubscription<>(messagePublisher, registration);
}
This construct works in principle, but with the following issues:
Messages sent to the queue while there are no subscribers at all lead to a MessageDeliveryException: Dispatcher has no subscribers for channel 'application.messagesPerCustomerQueue'
Messages sent to the queue while no matching subscriber is present yet lead to an AggregateMessageDeliveryException: All attempts to deliver Message to MessageHandlers failed.
What I want is that the message remains in the queue and is repeatedly offered to all subscribers until it is either consumed or expires (a proper selective consumer). How can I do that?
note that the incoming message for the GET has no content, which is not handled properly ATM by the Webflux integration
I don't understand this concern.
The WebFluxInboundEndpoint works with this algorithm:
if (isReadable(request)) {
...
else {
return (Mono<T>) Mono.just(exchange.getRequest().getQueryParams());
}
Where GET method really goes to the else branch. And the payload of the message to send is a MultiValueMap. And also we recently fixed with you the problem for the POST, which is released already as well in version 5.0.5: https://jira.spring.io/browse/INT-4462
Dispatcher has no subscribers
Can't happen on the QueueChannel in principle. There is no any dispatcher on there at all. It is just queue and sender offers message to be stored. You are missing something else to share with us. But let's call things with its own names: the messagesPerCustomerQueue is not a QueueChannel in your application.
UPDATE
Regarding:
What I want is that the message remains in the queue and is repeatedly offered to all subscribers until it is either consumed or expires (a proper selective consumer)
Only what we see is a PollableJmsChannel based on the embedded ActiveMQ to honor TTL for messages. As a consumer of this queue you should have a PublishSubscribeChannel with the setMinSubscribers(1) to make MessagingTemplate to throw a MessageDeliveryException when there is no subscribers yet. This way a JMS transaction will be rolled back and message will return to the queue for the next polling cycle.
The problem with in-memory QueueChannel that there is no transactional redelivery and message once polled from that queue is going to be lost.
Another option is similar to JMS (transactional) is a JdbcChannelMessageStore for the QueueChannel. Although this way we don't have a TTL functionality...
I have two IntegrationFlows defined that both uses this component. One reads from ftp and one read files from disk.
#Bean
public IntegrationFlow csvLineFlowDefinition() {
return IntegrationFlows.from(CHANNEL_NAME)
.filter(String.class, m -> {
// filter to remove column definition csv line
return !m.startsWith("ID");
})
.<String, MyPrettyObject>transform(csvLinePayload -> {
String[] array = csvLinePayload.split(",");
MyPrettyObject myPrettyObject = new MyPrettyObject();
myPrettyObject.setId(array[0]);
myPrettyObject.setType(array[1]);
return myPrettyObject;
})
.<MyPrettyObject, String>route(myPrettyObject -> myPrettyObject.getType(),
routeResult -> routeResult
.channelMapping("AA", "AA_CHANNEL")
.channelMapping("BB", "BB_CHANNEL")
.channelMapping("CC", "CC_CHANNEL"))
.get();
}
I would like these two IntegrationFlows only to fail if something is wrong with reading from ftp or reading files from the disk.
They have their own error channel defined.
I do not want an error in the transform of a csv line to MyPrettyObject to reach these two IntegrationFlows.
I have thought about dispatching the raw csv lines to a message queue and then i can define a specific error channel on the inbound consumer of this message queue.
However this seems a bit overkill.
I have tried to insert a ExpressionEvaluatingRequestHandlerAdvice for the transformer, but i'm not sure how to use it properly, and the messages does not reach the router or ERROR_CHANNEL_NAME
#Bean
public ExpressionEvaluatingRequestHandlerAdvice csvLineTransformerAdvice() {
ExpressionEvaluatingRequestHandlerAdvice expressionEvaluatingRequestHandlerAdvice = new ExpressionEvaluatingRequestHandlerAdvice();
expressionEvaluatingRequestHandlerAdvice.setFailureChannelName(ERROR_CHANNEL_NAME);
expressionEvaluatingRequestHandlerAdvice.setTrapException(true);
return expressionEvaluatingRequestHandlerAdvice;
}
.<String, MyPrettyObject>transform(csvLinePayload -> {
String[] array = csvLinePayload.split(",");
MyPrettyObject myPrettyObject = new MyPrettyObject();
myPrettyObject.setId(array[0]);
myPrettyObject.setType(array[1]);
return myPrettyObject;
}, t -> t.advice(csvLineTransformerAdvice()))
I'm afraid that "something wrong with reading" doesn't reach the error-channel because there is no message yet to deal with. So, isolating inbound channel adapter from the rest of the flow might not be a good idea. That is pretty normal for any downstream error to be propagated to the error-channel on the inbound channel adapter.
The ExpressionEvaluatingRequestHandlerAdvice is right way to go, but you should keep in mind that it works only for the transformer. The downstream flow isn't involved in that advice already.
In case of error the flow stops and it really can't reach the next endpoint because of error. Not sure what is your concerns there...