I am new to spring integration.
I have a flow on which I need to perform an http or a tcp call depending on some conditions.
The problem I am focused on is related to the http call.
The rest endpoint called needs an accessToken as header parameter for authentication that is provided by a spring service that has 2 methods getCurrentAccessToken() and refreshAccessToken(). I want to call the method refresh accessToken only when the currentAccessToken is expired.
What I would like to do is to add the following logic when performing the call to the rest api:
If the token is expired the rest endpoint returns a 401 and I would like to intercept in the flow this error and retry the request by adding a refreshed access token.
#Bean
public IntegrationFlow clientIn(AbstractServerConnectionFactory server,
AbstractClientConnectionFactory client, LogService logService) {
return IntegrationFlows.from(Tcp.inboundAdapter(client)
.enrichHeaders(t -> t.headerFunction(IpHeaders.CONNECTION_ID, message -> this.client, true))
.log(msg -> "client: " + logService.log(msg))
.<byte[], Boolean>route(this::shouldForwardToHttp,
mapping -> mapping.subFlowMapping(true, sf -> sf
.enrichHeaders(t -> t.header("Content-Type", MimeTypeUtils.APPLICATION_JSON_VALUE))
.<byte[], RequestMessage>transform(this::buildRequestFromMessage)
.<RequestMessage, HttpEntity>transform(this::getHttpEntity)
.handle(Http.outboundGateway(restUrl).httpMethod(HttpMethod.POST)
.expectedResponseType(ResponseMessage.class))
.<ResponseMessage, byte[]>transform(p -> this.transformResponse(p))
.handle(Tcp.outboundAdapter(client))).subFlowMapping(false,
t -> t.handle(Tcp.outboundAdapter(server).retryInterval(1000))))
.get();
}
HttpEntity getHttpEntity(RequestMessage request) {
MultiValueMap<String, String> mv = new HttpHeaders();
mv.add("accessToken", tokenProvider.getCurrentAccessToken());
HttpEntity entity = new HttpEntity(request, mv);
return entity;
}
I have tried by adding a requestHandlerRetry advice and redirecting it to a recoveryChannel, but I was not able to return something to the caller flow in order to get the response with the status code and retry the call with the new accessToken.
Any idea on how I can implement this?
I don't think you need a retry advice since you definitely are simulating it via catching that 401 exception and calling a service back with refreshed token. Sounds more like recursion. To achieve it properly I would suggest to take a look into an ExpressionEvaluatingRequestHandlerAdvice: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain. Yes, it is similar to the retry one and it also has that failureChannel, but there is no built-in retry since we are going to simulate it calling the same endpoint again and again when necessary.
To simplify a recursion logic, I would extract that .handle(Http.outboundGateway(restUrl).httpMethod(HttpMethod.POST) .expectedResponseType(ResponseMessage.class)) into a separate flow and use a gateway() with an input channel for that flow in the main flow instead.
A failureChannel sub-flow should re-route its message back to the input of the gateway flow.
What is the most important part in this logic is to carry on all the original request message headers which includes a required for the gateway logic replyChannel.
See more docs about gateways: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#gateway.
When an ExpressionEvaluatingRequestHandlerAdvice sends a message to the failureChannel, it comes as an ErrorMessage with a MessageHandlingExpressionEvaluatingAdviceException as a payload. The message which causes a failure and has all the required headers is there in the getFailedMessage() property. So, you take that message, request for fresh token, add it into headers of a new message based on that original. In the end you send this new message to the input channel of the IntegrationFlow for an HTTP request. When all is good, the result of the HTTP call is going to be forwarded to the mentioned replyChannel from headers and in therefore to the main flow for next steps.
Related
I am trying to build a Spring Integration Flow with DSL where a part of the flow calls an existing service that will process data asynchronously and return the response on a channel. The service call returns a task ID that can be used as a correlation ID to obtain the correct response message on the channel.
I am unsure how to build a flow (which components to use) that will call the service (I assume with a service activator), then take the returned ID and then wait for a message on a different channel that has that ID in the correlation ID header (maybe some sort of aggregator?). I have googled and cannot find anything that seems similar.
Also, my flow will receive a request object that I would like to pair up with the response object to pass along the flow after the response is received.
Request -> service call -> returns task ID ---->|
| |---- (Request+Response) --> More processing
| (async) |
---------> Response ----------->|
for task ID
on task complete channel
You are correct. The best way to solve your task is really an aggregator pattern:
https://www.enterpriseintegrationpatterns.com/Aggregator.html
https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#aggregator
So, you probably need to use a header enricher to instead of a service activator to obtain that task ID in the reply and store it in a header for future correlation of this request and some reply later with the same task ID. Or if you have a property on the request object for this task ID, you can use a content enricher instead: https://docs.spring.io/spring-integration/docs/current/reference/html/message-transformation.html#content-enricher
Then you send this request object with the task ID to an aggregator where this task ID must be used as a correlation key. The group size of course is just 2 - request and reply.
Your async service must send a completion to the same aggregator's input channel with. When aggregator encounters a proper correlation key, it will complete the group of two messages and send a single one to its output channel.
UPDATE
The aggregator we are talking about must have its own input channel, so you can send a request with task ID for correlation and then from your async service a reply must be sent to the same channel. With Java DSL it is a matter of exposing that input channel for your convenience:
#Bean
IntegrationFlow aggregatorFlow() {
return f -> f
.aggregate(...)
.channel("correlatedReplyChannel")
}
This flow implicitly starts with a channel like aggregatorFlow.input. So, you use this name in a channel() definition of request and reply flows as the last EIP-method in their definitions.
I'm new to Spring Integration. I have a simple flow which send request to external resource with several attemps.
IntegrationFlows.from(MY_CHANNEL)
.handle(myOutboundGateway, e -> e.advice(myRetryAdvice))
.wireTap(logResponse())
.get();
What I need to do is to take some action (saving data to a database) when calling external resource (after retrying) is not succesful (http status code is not 200 OK). How can I achieve that in my flow?
When all the attempts of the retry are exhausted, the RecoveryCallback is called.
See some sample here: How to get HTTP status in advice recovery callback. In that RecoveryCallback you can just return null and send a message to some channel for that storing to DB logic.
Another way is to have extra advice on top of that retry instead of RecoveryCallback. See its docs: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#expression-advice. This way when all the attempts are done, the exception is going to be bubbled and caught by that ExpressionEvaluatingRequestHandlerAdvice and its failureChannel. Pay attention to the trapException = true, so the error doesn't go back to the flow. Only to that failureChannel for your DB logic.
I have a simple IntegrationFlow sending SOAP messages with MarshallingWebServiceOutboundGateway. The deal is that, normally, the target service responds with empty message. It can of course return soap fault, but, on success, we expect an empty response. Now the problem is that if we get an empty response, SI recognizes it as "no response" and does not execute steps that are after the gateway in the IntegrationFlow. This happens even if gateway is set with setIgnoreEmptyResponses(false). According to documentation, it only works if it received an empty String, but response returned from the service is represented as null in my case.
I want to be able to continue the flow no matter if external service responds with empty or non-empty response. What would be the best way to achieve that?
EDIT1: It feels especially odd, because it stops execution of the IntegrationFlow even if both handlers are subscribed to publishSubscribeChannel (instead of being directly on the main flow). I would expect all subscribers to be executed unless the previous one produced an exception, but this is not what happens.
MarshallingWebServiceOutboundGateway outboundGateway = new MarshallingWebServiceOutboundGateway(
gatewaysUtils.buildOutboundUriToX(webServiceProperties.getServicePath()),
jaxb2Marshaller, messageFactory);
outboundGateway.setIgnoreEmptyResponses(false);
IntegrationFlows.from(channelName())
.transform(getKafkaMarshaller()::deserialize)
.handle(getOutboundGateway())
.handle(messageStatusRegistrator.registerMessageStatus())
.get();
EDIT2: Having tried the publishSubscribeChannel again, it worked this time around. It feels a bit hacky to me, but apparently it does work without any side-effects (for now). Fixed flow below. One thing to be careful about is scenario, when the service returns a non-empty, non-error response, which can potentially cause "output-channel or replychannel header available" if you do not provide the header or replyChannel explicitly.
IntegrationFlows.from(channelName())
.transform(getKafkaMarshaller()::deserialize)
.publishSubscribeChannel(channel -> channel
.subscribe(subflow -> subflow
.handle(getOutboundGateway()))
.subscribe(subflow -> subflow
.handle(messageStatusRegistrator::registerMessageStatus)))
.get();
I have a flow that is similar to
IntegrationFlows.from(
Http.inboundGateway("/events")
.requestMapping(requestMappingSpec -> {
requestMappingSpec.methods(HttpMethod.POST);
requestMappingSpec.consumes(MediaType.APPLICATION_JSON_VALUE);
requestMappingSpec.produces(MediaType.APPLICATION_JSON_VALUE);
})
.requestPayloadType(PushEvent.class)
.errorChannel(ERROR_CHANNEL))
.channel(ReleaseFlow.REQUEST_CHANNEL)
.enrichHeaders(h -> h
.header(HttpHeaders.STATUS_CODE, HttpStatus.ACCEPTED))
.get();
When submitting multiple requests, a request will be processed by the flow attached to the REQUEST_CHANNEL and the following request will be processed by just the enrichedHeaders. My understanding is that the endpoints in this example should be processed serially ...
A request arrives at the /events endpoint
The request is processed by the flow listening to REQUEST_CHANNEL
The response from the previous flow will then have its headers enriched
The flow ends and the response is returned to the remote requestor
I appreciate your help in understanding why request n is processed by the channel (and not enrichHeaders()), request n + 1 is being processed by enrichHeaders() (and not the flow listening to the REQUEST_CHANNEL), request n + 2 processed by the channel (and not enrichHeaders()), ...
UPDATE 1
I am new to Spring Integration, but thought it was appropriate to collect events from a GitHub server and then create a release using an external service. The integration service would be responsible for determining the appropriate workflow based upon the data associated to the commit. The endpoint in question would receive a push event and forward it to the flow attached to the subscribable request channel (REQUEST_CHANNEL). This second flow will make a number of outbound requests to collect the appropriate release template and construct and start the pipeline.
UPDATE 2
I have not developed the second flow completely at this point, but here is a first version that simply performs a transformation based upon data associated with the commit.
return f -> f
.route(branchRouter(), mapping -> mapping
.subFlowMapping(true, t -> t
.transform(pushEventToEventTransformer()))
.subFlowMapping(false, n -> n
.transform(skip -> "")));
When the code has been submitted to a "monitored" branch the actions described in the first update will be performed. I am attempting to build the flows incrementally given my limited knowledge of the framework.
Subscribable channels are point-to-point by default, which means if there are two subscribers, messages will be distributed in round-robin fashion.
If you have another flow " ... attached to the REQUEST_CHANNEL " then you have two subscribers - that flow and the header-enricher.
Perhaps if you can describe what you are trying to do we can help.
With the header enricher after the channel, all that happens is the headers are enriched and the inbound message is returned to the gateway.
Perhaps you want this... ?
IntegrationFlows.from(
Http.inboundGateway("/events")
.requestMapping(requestMappingSpec -> {
requestMappingSpec.methods(HttpMethod.POST);
requestMappingSpec.consumes(MediaType.APPLICATION_JSON_VALUE);
requestMappingSpec.produces(MediaType.APPLICATION_JSON_VALUE);
})
.requestPayloadType(PushEvent.class)
.errorChannel(ERROR_CHANNEL))
.enrichHeaders(h -> h
.header(HttpHeaders.STATUS_CODE, HttpStatus.ACCEPTED))
.channel(ReleaseFlow.REQUEST_CHANNEL)
.get();
Which means all messages with the enriched headers will be sent to the channel.
EDIT
If you want the message to go to both places, you need to use a publish-subscribe channel.
I am using
MessageProducerSupport messageProducer =
Jms.messageDriverChannelAdapter(jmsConnectionFactory, TransactedMessageListenerContainer.class)
.destination(queue)
.get();
to consume messages from ActiveMQ queue.
This is first part of my IntegrationFlow and then multiple stages occur (transform, route, handle..) within transaction
It is there to handle messages from upstream
In order to get the ACK from Spring integration pipeline I used Jms.inboundGateway(jmsConnectionFactory, TransactedMessageListenerContainer.class) which doesn't break existing flow and everything works
When I set replyTo header of upstream message, I would assume Spring Integration would send the object of the last stage of IntegrationFlow which was successful back to replyTo queue
Is my approach correct?
Is it possible to achieve such use-case?
Yes, that's correct and should work by its (Messaging Gateway) premise.
The Jms.inboundGateway() is based on the ChannelPublishingJmsMessageListener with the expectReply = true and there is a code:
private Destination getReplyDestination(javax.jms.Message request, Session session) throws JMSException {
Destination replyTo = request.getJMSReplyTo();
....
return replyTo;
}
to obtain a replyTo from the request.
Everything that works well, if your last MessageHandler in the flow is a AbstractReplyProducingMessageHandler and really returns something to be produced to the replyChannel from headers.
If you aren't sure in your case, so share the end of your flow and the place where would you like to send a reply.