Spring Integration HTTP outbound gateway retry based on reply content - spring-integration

I'm using an API that works in 2 steps:
It starts processing of a document in async way where it provides you an id that you use for step 2
It provides an endpoint where you can get the results but only when they are ready. So basically it will always give you a 200 response with some details like the status of the processing.
So the question is how can I implement a custom "success" criteria for the HTTP outbound gateway. I would also like to combine it with a RetryAdvice which I already have implemented.
I've tried the following but first of all the message's payload that is provided in the HandleMessageAdvice is empty, and secondly the retry is not triggered:
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET), c -> c.advice(this.advices.retryAdvice())
.handleMessageAdvice(new AbstractHandleMessageAdvice() {
#Override
protected Object doInvoke(MethodInvocation invocation, Message<?> message) throws Throwable {
String body = (String) message.getPayload();
if (StringUtils.isEmpty(body))
throw new RuntimeException("Still analyzing");
JSONObject document = new JSONObject(body);
if (document.has("analyzeResult"))
return message;
else
throw new RuntimeException("Still analyzing");
}
}))
I've found this answer from Artem from 4 years back but first of all I didn't find the reply channel method on the outbound gateway and secondly not sure if this scenario has already been improved in the newer version of Spring Integaration: http outbound retry with conditions (For checker condition).
UPDATE
Following Artem's suggestion I have the following:
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET), c -> c.advice(advices.verifyReplySuccess())
.advice(advices.retryUntilRequestCompleteAdvice()))
And the advice:
#Bean
public Advice verifyReplySuccess() {
return new AbstractRequestHandlerAdvice() {
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) {
try {
Object payload = ((MessageBuilder) callback.execute()).build().getPayload();
String body = (String) ((ResponseEntity) payload).getBody();
JSONObject document = new JSONObject(body);
if (document.has("analyzeResult"))
return message;
} catch (JSONException e) {
throw new RuntimeException(e);
}
throw new RuntimeException("Still analyzing");
}
};
}
But now when I debug the doInvoke method, the body of the payload is null. It's strange as when I execute the same GET request using Postman, the body is correctly returned. Any idea?
The body from response using Postman looks like this:
{
"status": "succeeded",
"createdDateTime": "2020-09-01T10:55:52Z",
"lastUpdatedDateTime": "2020-09-01T10:55:57Z",
"analyzeResult": {
"version": "3.0.0",
"readResults": [
{
"page": 1,........
Here is the payload that I get from the outbound gateway using callback:
<200,[Transfer-Encoding:"chunked", Content-Type:"application/json; charset=utf-8", x-envoy-upstream-service-time:"27", CSP-Billing-Usage:"CognitiveServices.ComputerVision.Transaction=1", apim-request-id:"a503c72f-deae-4299-9e32-625d831cfd91", Strict-Transport-Security:"max-age=31536000; includeSubDomains; preload", x-content-type-options:"nosniff", Date:"Tue, 01 Sep 2020 19:48:36 GMT"]>

There is indeed no request and reply channel options in Java DSL because you simply wrap that handle() into channel() configuration or just chain endpoints in the flow natural way and they are going to exchange messages using implicit direct channels in between. You can look into Java DSL IntegrationFlow as a <chain> in the XML configuration.
Your advice configuration is a bit wrong: you need declare your custom advice as a first in a chain, so when exception is thrown from there a retry one is going to handle it.
You should also consider to implement an AbstractRequestHandlerAdvice to align it with the RequestHandlerRetryAdvice logic.
You implement there a doInvoke(), call ExecutionCallback.execute() and analyze the result to return as is or throw a desired exception. A result of that call for HttpRequestExecutingMessageHandler is going to be an AbstractIntegrationMessageBuilder and probably a ResponseEntity as a payload to check for your further logic.

Following Artem's suggestion I came up with the following (additional trick was to set the expectedResponseType to String as otherwise using the ResponseEntity the body was empty):
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET).expectedResponseType(String.class),
c -> c.advice(advices.retryUntilRequestCompleteAdvice())
.advice(advices.verifyReplySuccess()))
And the advice:
#Bean
public Advice verifyReplySuccess() {
return new AbstractRequestHandlerAdvice() {
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) {
Object payload = ((MessageBuilder) callback.execute()).build().getPayload();
if (((String) payload).contains("analyzeResult"))
return payload;
else
throw new RuntimeException("Still analyzing");
}
};
}

Related

How to use custom error channel to produce custom error response?

I am trying to build a simple API using Spring Integration. For any exceptions or errors thrown in the API, I want to provide custom error responses to client. I am trying with the flow structure below:
#Bean
public IntegrationFlow mainFlow() {
return IntegrationFlows.from(WebFlux.inboundGateway(path)
.errorChannel(customErrorChannel())
)
.enrichHeaders(h -> h.header(MessageHeaders.ERROR_CHANNEL, "customErrorChannel"))
.transform(p -> {
if(<some validation fails>)
throw new RuntimeException("Error!");
return "Ok Response";
})
.get();
}
#Bean
public PublishSubscribeChannel customErrorChannel() {
return MessageChannels.publishSubscribe().get();
}
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("customErrorChannel")
.transform(p -> {
return "Error Response";
})
.get();
}
For success cases, "Ok Response" is provided to the client. But when exception is thrown in transformer, provided response is default error response from the framework with stack trace. How can I use errorFlow to produce final response for exceptions?
I can use CustomErrorWebExceptionHandler as global exception handler, but that is not my intention.
It turns out that the .errorChannel(customErrorChannel()) is out of use in case of reactive channel adapter like that WebFlux.inboundGateway(). Please, raise a GH issue and we'll think what and how we can do.
As a workaround I suggest you to take a look into an ExpressionEvaluatingRequestHandlerAdvice for that transform(): https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain.
This way you'll catch all the errors in the transformer and can send them to your customErrorChannel for handling.
There is no reason in the .enrichHeaders(h -> h.header(MessageHeaders.ERROR_CHANNEL, "customErrorChannel")) though. The .errorChannel(customErrorChannel()) should be enough. But when we have a fix already.
UPDATE
Turns out as a workaround you can add .channel(MessageChannels.flux()) before your transform() and the error should be process through your errorChannel.

How could I transform my object from my channel (spring-integration, DSL)?

I have got an Integrationflow which returns me a Message and now I would like to validate it in a second channel and after sending it back I would like to transform it.
But the transformation failed.
#Bean
public IntegrationFlow httpUserGetUserByUsername() {
return IntegrationFlows.from(httpGetGateUsersGetUserByUsername())
.channel("http.users.getUserByUsername").handle("userEndpoint", "getUserByUsername")
.channel("http.users.checkUserAuth").handle("userEndpoint","checkUserByUsername")
.transform(User u -> new UserResponseHelper(u))
.get();
}
public void checkUserByUsername(Message<User> msg) {
MessageChannel replayChannel = (MessageChannel) msg.getHeaders().getReplyChannel();
User u = msg.getPayload();
if (Authorization.isAllowByUsername(u.getUsername())) {
replayChannel.send(MessageBuilder.withPayload(u).build());
}else{
replayChannel.send(MessageBuilder.withPayload(new ResponseEntity(new ApiResponse(HttpStatus.FORBIDDEN,false,"You are not allow to get this ressource"), HttpStatus.FORBIDDEN)).build());
}
}
I get the user with all 8 properties and what i expect ist a user with 4 props
OK. I see what is your problem. You really have just missed the fact that your .transform(User u -> new UserResponseHelper(u)) is not called at all. So, that your "transformation failed" has confused me.
So, what is going on here:
Your void checkUserByUsername(Message<User> msg) returns void. Therefore there is nothing to wrap into an output Message. And from here nothing is going to trigger the next .transform() in your flow.
What you see for your REST service, that everything from that checkUserByUsername(Message<User> msg) is sent to the replyChannel header - the place where Inbound Gateway waits for reply.
In case of validation failure you send some custom ApiResponse. Why just don't throw an Exception, e.g. ResponseStatusException? This is going to be handled properly on the REST layer and returned as an error to the REST client.
I suggest you to really return that u from this checkUserByUsername() method and throw an exception otherwise. This way the User object is going to come to your .transform(User u -> new UserResponseHelper(u)) - and all good!
You have just confused yourself that void doesn't trigger transform and replyChannel is just for sending reply directly to the initiator gateway.

Http outbound gateway is manipulating the header value

We are using Spring-Integration in our project. I am experiencing a weird problem with http:outbound-gateway. We need to pass the following headers for executing a rest service.
1)Accept=application/vnd.dsths.services-v1+xml
2)Content-Type=application/xml
The weird part is that the response returned is not always unique, In dev environment, xml response(Content-Type=application/vnd.dsths.services-v1+xml) is returned while in client environment, json response(Content-Type=application/vnd.dsths.services-v1+json) is returned. I have verified the log files by turning on DEBUG and found that the org.springframework.web.client.RestTemplate is Setting request Accept header to [text/plain, application/json, application/*+json, */` * ].
2017-07-10 16:17:11,563 DEBUG [org.springframework.web.client.RestTemplate] (ajp-/10.226.55.163:8009-1) Setting request Accept header to [text/plain, application/json, application/*+json, */*]
I could able to overcome this problem by overriding the value of accept=*/* to accept=application/vnd.dsths.services-v1+xml in the client environment(Please note that this header is not the actual "Accept" header).
The question here is why http:outbound-gateway is behaving oddly and manipulating the header value? Why the Spring Integration is not able to identify the difference between the headers and "accept" and "Accept"? Is my fix correct one?
Not sure what is the question about difference, but according RFC-2616 HTTP headers are case-insensitive: Are HTTP headers case-sensitive?.
And DefaultHttpHeaderMapper follow that recommendations:
private void setHttpHeader(HttpHeaders target, String name, Object value) {
if (ACCEPT.equalsIgnoreCase(name)) {
if (value instanceof Collection<?>) {
What you show is the part of Spring MVC already, far away from Spring Integration. See RestTemplate code:
public void doWithRequest(ClientHttpRequest request) throws IOException {
if (this.responseType != null) {
Class<?> responseClass = null;
if (this.responseType instanceof Class) {
responseClass = (Class<?>) this.responseType;
}
List<MediaType> allSupportedMediaTypes = new ArrayList<>();
for (HttpMessageConverter<?> converter : getMessageConverters()) {
if (responseClass != null) {
if (converter.canRead(responseClass, null)) {
allSupportedMediaTypes.addAll(getSupportedMediaTypes(converter));
}
}
else if (converter instanceof GenericHttpMessageConverter) {
GenericHttpMessageConverter<?> genericConverter = (GenericHttpMessageConverter<?>) converter;
if (genericConverter.canRead(this.responseType, null, null)) {
allSupportedMediaTypes.addAll(getSupportedMediaTypes(converter));
}
}
}
if (!allSupportedMediaTypes.isEmpty()) {
MediaType.sortBySpecificity(allSupportedMediaTypes);
if (logger.isDebugEnabled()) {
logger.debug("Setting request Accept header to " + allSupportedMediaTypes);
}
request.getHeaders().setAccept(allSupportedMediaTypes);
}
}
}
As you see the logic is based on the provided HttpMessageConverter and it is, honestly, is correct. The Accept header is exactly what the client can handle from the server.
If you don't like such a behavior and you are sure in your client, you can inject RestTemplate to the http:outbound-gateway but already with only desired HttpMessageConverter.

How to do response payload logging and conversion in spring integration.?

I have set expectedResponseType(MyClass.class). So OutboundGateway is converting the message into my response class type and returning to me. I want to log the payload as well for debugging purpose along with the conversion.
How to do this response payload logging and conversion.?
I could do it by expecting the response as String and later convert into my class using marshallers. Is there any simpler way that can be used for all my outbound gateways?
The expectedResponseType(MyClass.class) is translated to the
httpResponse = this.restTemplate.exchange(realUri, httpMethod, httpRequest, (Class<?>) expectedResponseType);
where the last one does this:
public ResponseEntityResponseExtractor(Type responseType) {
if (responseType != null && Void.class != responseType) {
this.delegate = new HttpMessageConverterExtractor<T>(responseType,
getMessageConverters(), logger);
}
else {
this.delegate = null;
}
}
As you see it is copying its own logger to the HttpMessageConverterExtractor.
So, I think you can achieve some good result for your logging requirements switching on DEBUG (or even TRACE) for the org.springframework.web.client.RestTemplate category.
From other side you always can extend the RestTemplate a bit to make some hooks into it.
From the Spring Integration perspective we can do nothing. Because the whole hard conversion work is done in the RestTemplate.

Best way to automatically add stomp native headers to headers

Our javascript websocket clients adds "custom" headers to all STOMP messages.
My project handles websocket endpoints using spring-websocket #Controller.
#MessageMapping(value = "/mymessages")
public void save(#Payload ToBeSaved payload, #Headers MessageHeaders headers) {
service.save(toMsg(payload, headers));
}
protected <P> Message<P> toMsg(P payload, MessageHeaders headers) {
return MessageBuilder.createMessage(payload, headers);
}
The controller modifies the payload and then passes the new payload and original websocket headers (including the custom ones) to a spring-integration #MessagingGateway.
The underlying IntegrationFlow tries to access the "custom" headers by accessing the message headers with the SPLExpression headers['custom'].
Unfortunately headers['custom'] is always null because custom is actually contained in the nativeHeaders.
I haven't found a way to tell IntegrationFlow to look into nativeHeaders.
Is there a way in spring-websocket to copy all native headers as normal headers ?
Thanks in advance
The spring-websocket can do nothing for your on the matter. It isn't its responsibility.
If you would really like to have access to something in the nativeHeaders, you should do that manually.
For your particular case that SpEL may look like:
headers['nativeHeaders']['custom']
Because nativeHeaders is a Map as well.
From other side you can use <header-enricher> in your down stream flow to pop all those nativeHeaders to top level.
And one more point: since Spring Integration 4.2 we provide native support for STOMP adapters. And there is a StompHeaderMapper which does exactly what you want and the code there looks like:
else if (StompHeaderAccessor.NATIVE_HEADERS.equals(name)) {
MultiValueMap<String, String> multiValueMap =
headers.get(StompHeaderAccessor.NATIVE_HEADERS, MultiValueMap.class);
for (Map.Entry<String, List<String>> entry1 : multiValueMap.entrySet()) {
name = entry1.getKey();
if (shouldMapHeader(name, this.outboundHeaderNames)) {
String value = entry1.getValue().get(0);
if (StringUtils.hasText(value)) {
setStompHeader(target, name, value);
}
}
}
}

Resources