Strange behavior when returning ObjectNode from IntegrationFlows - spring-integration

When an ObjectNode is passed from the extractFramesFlow() and reaches the httpCallbackFlow(), HTTP request is successfully performed and JSON formatted payload is 'POST'ed to the "call_back" uri specified.
#Bean
public IntegrationFlow extractFramesFlow() {
return IntegrationFlows.from(extractFramesChannel())
.handle(ObjectNode.class, (payload, headers) -> {
payload = validateFields(payload);
String path = payload.get("path").asText();
try {
File moviePath = new File(path);
ArrayNode arrayNode = mapper.createArrayNode();
String imageType = payload.path("image_type").asText("JPG");
String prefix = payload.path("prefix").asText();
Tools.thumbnails(moviePath, payload.get("slice").asInt(), payload.get("scale").asInt(),
imageType, prefix, file -> arrayNode.add(file.toString()));
payload.set("files", arrayNode);
} catch (IOException e) {
e.printStackTrace();
}
return payload;
}).enrichHeaders(h-> h.header("errorChannel", "asyncErrorChannel", true))
.<ObjectNode, Boolean>route(p-> !p.hasNonNull("id"),
m->m.channelMapping("true","httpCallbackFlow.input")
.channelMapping("false","uploadToS3Channel")).get();
}
#Bean
public IntegrationFlow httpCallbackFlow() {
return f->f.handle(Http.<JsonNode>outboundChannelAdapter(m->m.getPayload().get("call_back").asText()));
}
However, when an ObjectNode is chained from the handleAsyncErrors() flow and reaches the same httpCallbackFlow(), we get an Exception which is caused by
org.springframework.web.client.RestClientException: Could not write request: no suitable HttpMessageConverter found for request type [com.fasterxml.jackson.databind.node.ObjectNode] and content type [application/x-java-serialized-object]
at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:811)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:594)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:572)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:493)
at org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler.handleRequestMessage(HttpRequestExecutingMessageHandler.java:382)
... 24 more
#Bean
public IntegrationFlow handleAsyncErrors() {
return IntegrationFlows.from(asyncErrorChannel())
.<MessagingException>handle((p, h) -> {
ObjectNode objectNode = mapper.createObjectNode();
objectNode.put("call_back", "http://some.test.uri");
return objectNode;
}).channel("httpCallbackFlow.input").get();
}
I don't know why we get this Exception handled by the same exact IntegrationFlow notwithstanding.

The message on the error flow has no contentType header.
It is an error message with the MessagingException payload; which has 2 properties; the cause and failedMessage.
Presumably you have a content type on the main flow message. You can set the content type with a header enricher, or add
.<MessagingException, Message<?>>transform(p -> p.getFailedMessage())
before your existing error handler, to restore the headers from the failed message.

Related

Spring Integration Default Response for Jms inboundGateway

Seeing the below exception when trying to send a default constructed response for Jms inboundGateway exception from the downstream call. We are extracting the failedMessage headers from the ErrorMessage and then setting the constructed response as payload. The replyChannel headers is matching with the initially logged message header
2023-01-26 20:34:32,623 [mqGatewayListenerContainer-1] WARN o.s.m.c.GenericMessagingTemplate$TemporaryReplyChannel - be776858594e7c79 Reply message received but the receiving thread has exited due to an exception while sending the request message:
ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: Failed to send or receive; nested exception is java.io.UncheckedIOException: java.net.SocketTimeoutException: Connect timed out, failedMessage=GenericMessage [payload=NOT_PRINTED, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#2454562d, b3=xxxxxxxxxxxx, nativeHeaders={}, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#2454562d, sourceTransacted=false, jms_correlationId=ID:xxxxxxxxxx, id=xxxxxxxxxx, jms_expiration=36000, timestamp=1674750867614}]
Code:
return IntegrationFlows.from(Jms.inboundGateway(mqGatewayListenerContainer)
.defaultReplyQueueName(replyQueue)
.replyChannel(mqReplyChannel)
.errorChannel(appErrorChannel)
.replyTimeout(mqReplyTimeoutSeconds * 1000L))
// log
.log(DEBUG, m -> "Request Headers: " + m.getHeaders() + ", Message: " + m.getPayload())
// transform with required response headers
.transform(Message.class, m -> MessageBuilder.withPayload(m.getPayload())
.setHeader(ERROR_CHANNEL, m.getHeaders().get(ERROR_CHANNEL))
.setHeader(REPLY_CHANNEL, m.getHeaders().get(REPLY_CHANNEL))
.setHeader(CORRELATION_ID, m.getHeaders().get(MESSAGE_ID))
.setHeader(EXPIRATION, mqReplyTimeoutSeconds * 1000L)
.setHeader(MSG_HDR_SOURCE_TRANSACTED, transacted)
.build())
return IntegrationFlows.from(appErrorChannel())
.publishSubscribeChannel(
pubSubSpec -> pubSubSpec.subscribe(sf -> sf.channel(globalErrorChannel))
.<MessagingException, Message<MessagingException>>
transform(AppMessageUtil::getFailedMessageWithoutHeadersAsPayload)
.transform(p -> "Failure")
.get();
public static Message<MessagingException> getFailedMessageAsPayload(final MessagingException messagingException) {
var failedMessage = messagingException.getFailedMessage();
var failedMessageHeaders = Objects.isNull(failedMessage) ? null : failedMessage.getHeaders();
return MessageBuilder.withPayload(messagingException)
.copyHeaders(failedMessageHeaders)
.build();
}
Since you perform the processing of the request message on the same thread, it is blocked on a send and therefore we just re-throw an exception as is:
try {
doSend(channel, requestMessage, sendTimeout);
}
catch (RuntimeException ex) {
tempReplyChannel.setSendFailed(true);
throw ex;
}
And as you see we mark that tempReplyChannel as failed on a send operation.
So, the replyChannel header correlated with that mqReplyChannel is out of use. If you get rid of it at all, then everything is OK. But you also cannot reply back an Exception since the framework treats it as an error to re-throw back to the listener container:
if (errorFlowReply != null && errorFlowReply.getPayload() instanceof Throwable) {
rethrow((Throwable) errorFlowReply.getPayload(), "error flow returned an Error Message");
}
So, here is a solution:
#SpringBootApplication
public class So75249125Application {
public static void main(String[] args) {
SpringApplication.run(So75249125Application.class, args);
}
#Bean
IntegrationFlow jmsFlow(ConnectionFactory connectionFactory) {
return IntegrationFlow.from(Jms.inboundGateway(connectionFactory)
.requestDestination("testDestination")
.errorChannel("appErrorChannel"))
.transform(payload -> {
throw new RuntimeException("intentional");
})
.get();
}
#Bean
IntegrationFlow errorFlow() {
return IntegrationFlow.from("appErrorChannel")
.transform(So75249125Application::getFailedMessageAsPayload)
.get();
}
public static Message<String> getFailedMessageAsPayload(MessagingException messagingException) {
var failedMessage = messagingException.getFailedMessage();
var failedMessageHeaders = failedMessage.getHeaders();
return MessageBuilder.withPayload("failed")
.copyHeaders(failedMessageHeaders)
.build();
}
}
and unit test:
#SpringBootTest
class So75249125ApplicationTests {
#Autowired
JmsTemplate jmsTemplate;
#Test
void errorFlowRepliesCorrectly() throws JMSException {
Message reply = this.jmsTemplate.sendAndReceive("testDestination", session -> session.createTextMessage("test"));
assertThat(reply.getBody(String.class)).isEqualTo("failed");
}
}
Or even better like this:
public static String getFailedMessageAsPayload(MessagingException messagingException) {
var failedMessage = messagingException.getFailedMessage();
return "Request for '" + failedMessage.getPayload() + "' has failed";
}
and this test:
#Test
void errorFlowRepliesCorrectly() throws JMSException {
String testData = "test";
Message reply = this.jmsTemplate.sendAndReceive("testDestination", session -> session.createTextMessage(testData));
assertThat(reply.getBody(String.class)).isEqualTo("Request for '" + testData + "' has failed");
}

Stop renaming file if data processing fails while streaming remote directory file

I am reading the file from remote directory using SFTP. I am able to get file by stream using outbound gateway, and move it to archive folder even.
I am processing the data in file but if there is some issue in data then I am throwing an error. I do not want to rename the file if there is any error thrown while processing the data, how can I achieve that. It will be very helpful if I can get some good practices for having error handler while using spring integration.
.handle(Sftp.outboundGateway(sftpSessionFactory(), GET, "payload.remoteDirectory + payload.filename").options(STREAM).temporaryFileSuffix("_reading"))
.handle(readData(),c->c.advice(afterReading()))
.enrichHeaders(h -> h
.headerExpression(FileHeaders.RENAME_TO, "headers[file_remoteDirectory] + 'archive/' + headers[file_remoteFile]")
.headerExpression(FileHeaders.REMOTE_FILE, "headers[file_remoteFile]")
.header(FileHeaders.REMOTE_DIRECTORY, "headers[file_remoteDirectory]"))
.handle(Sftp.outboundGateway(sftpSessionFactory(), MV, "headers[file_remoteDirectory]+headers[file_remoteFile]").renameExpression("headers['file_renameTo']"))
.get();
#Bean
public ExpressionEvaluatingRequestHandlerAdvice afterReading() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setSuccessChannelName("successReading.input");
advice.setOnSuccessExpressionString("payload + ' was successful streamed'");
advice.setFailureChannelName("failureReading.input");
advice.setOnFailureExpressionString("payload + ' was bad, with reason: ' + #exception.cause.message");
advice.setTrapException(true);
advice.setPropagateEvaluationFailures(true);
return advice;
}
#Bean
public IntegrationFlow successReading() {
return f -> f.log();
}
#Bean
public IntegrationFlow failureReading() {
return f -> f.log(ERROR);
}
public GenericHandler readData() {
return new GenericHandler() {
#Override
public Object handle(Object o, Map map) {
InputStream file = (InputStream) o;
String fileName = (String) map.get(REMOTE_FILE);
try {
// processing data
} catch (Exception e) {
return new SftpException(500, String.format("Error while processing the file %s because of Error: %s and reason %s", fileName, e.getMessage(), e.getCause()));
}
Closeable closeable = (Closeable) map.get(CLOSEABLE_RESOURCE);
if (closeable != null) {
try {
closeable.close();
file.close();
} catch (Exception e) {
logger.error(String.format("Session didn`t get closed after reading the stream data for file %s and error %s"), fileName, e.getMessage());
}
}
return map;
}
};
}
Updated
Add an ExpressionEvaluatingRequestHandlerAdvice to the .handler() endpoint .handle(readData(), e -> e.advice(...)).
The final supplied advice class is the o.s.i.handler.advice.ExpressionEvaluatingRequestHandlerAdvice. This advice is more general than the other two advices. It provides a mechanism to evaluate an expression on the original inbound message sent to the endpoint. Separate expressions are available to be evaluated, after either success or failure. Optionally, a message containing the evaluation result, together with the input message, can be sent to a message channel.

How to route using message headers in Spring Integration DSL Tcp

I have 2 server side services and I would like route messages to them using message headers, where remote clients put service identification into field type.
Is the code snippet, from server side config, the correct way? It throws cast exception indicating that route() see only payload, but not the message headers. Also all example in the Spring Integration manual shows only payload based decisioning.
#Bean
public IntegrationFlow serverFlow( // common flow for all my services, currently 2
TcpNetServerConnectionFactory serverConnectionFactory,
HeartbeatServer heartbeatServer,
FeedServer feedServer) {
return IntegrationFlows
.from(Tcp.inboundGateway(serverConnectionFactory))
.<Message<?>, String>route((m) -> m.getHeaders().get("type", String.class),
(routeSpec) -> routeSpec
.subFlowMapping("hearbeat", subflow -> subflow.handle(heartbeatServer::processRequest))
.subFlowMapping("feed", subflow -> subflow.handle(feedServer::consumeFeed)))
.get();
}
Client side config:
#Bean
public IntegrationFlow heartbeatClientFlow(
TcpNetClientConnectionFactory clientConnectionFactory,
HeartbeatClient heartbeatClient) {
return IntegrationFlows.from(heartbeatClient::send, e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(5))))
.enrichHeaders(c -> c.header("type", "heartbeat"))
.log()
.handle(outboundGateway(clientConnectionFactory))
.handle(heartbeatClient::receive)
.get();
}
#Bean
public IntegrationFlow feedClientFlow(
TcpNetClientConnectionFactory clientConnectionFactory) {
return IntegrationFlows.from(FeedClient.MessageGateway.class)
.enrichHeaders(c -> c.header("type", "feed"))
.log()
.handle(outboundGateway(clientConnectionFactory))
.get();
}
And as usual here is the full demo project code, ClientConfig and ServerConfig.
There is no standard way to send headers over raw TCP. You need to encode them into the payload somehow (and extract them on the server side).
The framework provides a mechanism to do this for you, but it requires extra configuration.
See the documentation.
Specifically...
The MapJsonSerializer uses a Jackson ObjectMapper to convert between a Map and JSON. You can use this serializer in conjunction with a MessageConvertingTcpMessageMapper and a MapMessageConverter to transfer selected headers and the payload in JSON.
I'll try to find some time to create an example of how to use it.
But, of course, you can roll your own encoding/decoding.
EDIT
Here's an example configuration to use JSON to convey message headers over TCP...
#SpringBootApplication
public class TcpWithHeadersApplication {
public static void main(String[] args) {
SpringApplication.run(TcpWithHeadersApplication.class, args);
}
// Client side
public interface TcpExchanger {
public String exchange(String data, #Header("type") String type);
}
#Bean
public IntegrationFlow client(#Value("${tcp.port:1234}") int port) {
return IntegrationFlows.from(TcpExchanger.class)
.handle(Tcp.outboundGateway(Tcp.netClient("localhost", port)
.deserializer(jsonMapping())
.serializer(jsonMapping())
.mapper(mapper())))
.get();
}
// Server side
#Bean
public IntegrationFlow server(#Value("${tcp.port:1234}") int port) {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(port)
.deserializer(jsonMapping())
.serializer(jsonMapping())
.mapper(mapper())))
.log(Level.INFO, "exampleLogger", "'Received type header:' + headers['type']")
.route("headers['type']", r -> r
.subFlowMapping("upper",
subFlow -> subFlow.transform(String.class, p -> p.toUpperCase()))
.subFlowMapping("lower",
subFlow -> subFlow.transform(String.class, p -> p.toLowerCase())))
.get();
}
// Common
#Bean
public MessageConvertingTcpMessageMapper mapper() {
MapMessageConverter converter = new MapMessageConverter();
converter.setHeaderNames("type");
return new MessageConvertingTcpMessageMapper(converter);
}
#Bean
public MapJsonSerializer jsonMapping() {
return new MapJsonSerializer();
}
// Console
#Bean
#DependsOn("client")
public ApplicationRunner runner(TcpExchanger exchanger,
ConfigurableApplicationContext context) {
return args -> {
System.out.println("Enter some text; if it starts with a lower case character,\n"
+ "it will be uppercased by the server; otherwise it will be lowercased;\n"
+ "enter 'quit' to end");
Scanner scanner = new Scanner(System.in);
String request = scanner.nextLine();
while (!"quit".equals(request.toLowerCase())) {
if (StringUtils.hasText(request)) {
String result = exchanger.exchange(request,
Character.isLowerCase(request.charAt(0)) ? "upper" : "lower");
System.out.println(result);
}
request = scanner.nextLine();
}
scanner.close();
context.close();
};
}
}

Spring-Integration Webflux exception handling

If an exception occurs in a spring-integration webflux flow, the exception itself (with stacktrace) is sent back to the caller as payload through MessagePublishingErrorHandler, which uses an error channel from the "errorChannel" header, not the default error channel.
How can I set up an error handler similar to WebExceptionHandler? I want to produce an Http status code and possibly a DefaultErrorAttributes object as response.
Simply defining a flow that starts from the errorChannel doesn't work, the error message won't end up there. I tried to define my own fluxErrorChannel, but it appears that it is also not used as error channel, the errors do not end up in my errorFlow:
#Bean
public IntegrationFlow fooRestFlow() {
return IntegrationFlows.from(
WebFlux.inboundGateway("/foo")
.requestMapping(r -> r.methods(HttpMethod.POST))
.requestPayloadType(Map.class)
.errorChannel(fluxErrorChannel()))
.channel(bazFlow().getInputChannel())
.get();
}
#Bean
public MessageChannel fluxErrorChannel() {
return MessageChannels.flux().get();
}
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from(fluxErrorChannel())
.transform(source -> source)
.enrichHeaders(h -> h.header(HttpHeaders.STATUS_CODE, HttpStatus.BAD_GATEWAY))
.get();
}
#Bean
public IntegrationFlow bazFlow() {
return f -> f.split(Map.class, map -> map.get("items"))
.channel(MessageChannels.flux())
.<String>handle((p, h) -> throw new RuntimeException())
.aggregate();
}
UPDATE
In MessagingGatewaySupport.doSendAndReceiveMessageReactive my error channel defined on the WebFlux.inboundGateway is never used to set the error channel, rather the error channel is always the replyChannel which is being created here:
FutureReplyChannel replyChannel = new FutureReplyChannel();
Message<?> requestMessage = MutableMessageBuilder.fromMessage(message)
.setReplyChannel(replyChannel)
.setHeader(this.messagingTemplate.getSendTimeoutHeader(), null)
.setHeader(this.messagingTemplate.getReceiveTimeoutHeader(), null)
.setErrorChannel(replyChannel)
.build();
The error channel is ultimately being reset to the originalErrorChannelHandler in Mono.fromFuture, but that error channel is ˋnullˋ in my case. Also, the onErrorResume lambda is never invoked:
return Mono.fromFuture(replyChannel.messageFuture)
.doOnSubscribe(s -> {
if (!error && this.countsEnabled) {
this.messageCount.incrementAndGet();
}
})
.<Message<?>>map(replyMessage ->
MessageBuilder.fromMessage(replyMessage)
.setHeader(MessageHeaders.REPLY_CHANNEL, originalReplyChannelHeader)
.setHeader(MessageHeaders.ERROR_CHANNEL, originalErrorChannelHeader)
.build())
.onErrorResume(t -> error ? Mono.error(t) : handleSendError(requestMessage, t));
How is this intended to work?
It's a bug; the ErrorMessage created for the exception by the error handler is sent to the errorChannel header (which has to be the replyChannel so the gateway gets the result). The gateway should then invoke the error flow (if present) and return the result of that.
https://jira.spring.io/browse/INT-4541

JMS Header not getting stored into Spring Integration message header

Have an incoming message from ActiveMQ queue and the message is being delivered properly. I need to access the JMS header value x-cutoffrule in my spring integration flow, but the value of cutoffrule in the handle section always is coming as null. My code is below:
#Bean
public JmsHeaderMapper sampleJmsHeaderMapper() {
return new DefaultJmsHeaderMapper() {
public Map<String, Object> toHeaders(javax.jms.Message jmsMessage) {
Map<String, Object> headers = super.toHeaders(jmsMessage);
try {
headers.put("cutoffrule", jmsMessage.getStringProperty("x-cutoffrule"));
} catch (JMSException e) {
e.printStackTrace();
}
return headers;
}
};
}
#Bean
public IntegrationFlow jmsMessageDrivenFlow(JmsHeaderMapper sampleJmsHeaderMapper ) {
return IntegrationFlows
.from(
Jms.messageDriverChannelAdapter(jmsMessagingTemplate.getConnectionFactory())
.destination(integrationProps.getIncomingRequestQueue())
.errorChannel(errorChannel())
.setHeaderMapper( sampleJmsHeaderMapper )
)
.handle((payload, headers) -> {
incomingPayload = payload;
logger.debug("cutoffrule"+ headers.get("cutoffrule"));
return payload;
})
.handle(message -> {
logger.debug("Message was succcessfully processed");
})
.get();
}
I thought the DefaultJmsHeaderMapper will map all JMS headers into the spring integration message. What am I missing?
The best way to understand what's wrong it to debug the code.
Or, at least log everything.
The best place for you is that your DefaultJmsHeaderMapper extension.
So, the DefaultJmsHeaderMapper maps all incoming properties. But it does that with the getObjectProperty() not getStringProperty(), like in your code:
Enumeration<?> jmsPropertyNames = jmsMessage.getPropertyNames();
if (jmsPropertyNames != null) {
while (jmsPropertyNames.hasMoreElements()) {
String propertyName = jmsPropertyNames.nextElement().toString();
try {
String headerName = this.toHeaderName(propertyName);
headers.put(headerName, jmsMessage.getObjectProperty(propertyName));
}
catch (Exception e) {
if (logger.isWarnEnabled()) {
logger.warn("error occurred while mapping JMS property '"
+ propertyName + "' to Message header", e);
}
}
}
}
So, your x-cutoffrule should be mapped exactly into the x-cutoffrule header.
See Andriy's comment, too.

Resources