My flow is like this
private IntegrationFlow myChannel() {
return f -> f
...
.handle("myHandler", "myMethod")
...
}
How to resolve the handler myHandler and the method myMethod dynamically from the headers?
Add a .router() with subflows for each header value.
We need to understand first of all what is the purpose of such a business logic.
I think we can achieve your requirements with this code:
.handle((p, h) ->
new MethodInvokingMessageProcessor(h.get("myHandler"), h.get("myMethod", String.class)
.processMessage(new GenericMessage<>(p, h))))
Related
I'm trying to implement the following flow:
1) files are read from inbound adapter
2) they are send to different flows using publish-subscribe channel with applied sequence
3) file is moved after all the subscriber flows are ready
This is the main flow
return IntegrationFlows
.from(Files.inboundAdapter(inboundOutDirectory)
.regexFilter(pattern)
.useWatchService(true)
.watchEvents(FileReadingMessageSource.WatchEventType.CREATE),
e -> e.poller(Pollers.fixedDelay(period)
.taskExecutor(Executors.newFixedThreadPool(poolSize))
.maxMessagesPerPoll(maxMessagesPerPoll)))
.publishSubscribeChannel(s -> s
.applySequence(true)
.subscribe(f -> f
.transform(Files.toStringTransformer())
.<String>handle((p, h) -> {
return "something"
}
})
.channel("consolidateFlow.input"))
.subscribe(f -> f
.transform(Files.toStringTransformer())
.handle(Http.outboundGateway(testUri)
.httpMethod(HttpMethod.GET)
.uriVariable("text", "payload") .expectedResponseType(String.class))
.<String>handle((p, h) -> {
return "something";
})
.channel("consolidateFlow.input")))
.get();
And the aggregation:
public IntegrationFlow consolidateFlow()
return flow -> flow
.aggregate()
.<List<String>>handle((p, h) -> "something").log()
}
}
Using the following code in the main flow after publish-subscribe
.handle(Files.outboundGateway(this.inboundProcessedDirectory).deleteSourceFiles(true))
ends up with
Caused by: org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
If I go with this the consolidation/aggregation flow won't be reached at all.
.handle(Files.outboundAdapter(this.inboundProcessedDirectory))
Any idea how I could solve it? Currently I'm moving the file after the aggregation by reading the original file name from the header but it doesn't seem to be the right solution.
I was also thinking about applying spec/advice to the inbound adapter with success logic to move the file but not sure whether that's the right approach.
EDIT1
As suggested by Artem, I've added another subscriber to the publish-subscribe as follows:
...
.channel("consolidateNlpFlow.input"))
.subscribe(f -> f
.handle(Files.outboundAdapter(this.inboundProcessedDirectory).deleteSourceFiles(true))
...
The files is moved properly, but the consolidateFlow is not being executed at all. Any idea?
I've also tried adding the channel to the new flow .channel("consolidateNlpFlow.input") but it didn't change the behavior.
Your problem that a consolidateFlow is not able to return result into the main flow. Just because there is anything gateway-like. You do there an explicit .channel("consolidateFlow.input") which means there is not going to be way back.
That's for the issue you have so far.
Regarding a possible solution.
According to your configuration both your subscribers in the publishSubscribeChannel are performed on the same thread, one by one. So, it is going to be very easy for you to add one more subscriber with that Files.outboundAdapter() and deleteSourceFiles(true). This one is going to be called already after existing subscribers.
I has integration flow for polling data from database. I set up message source which return list of object, this list I want to pass to method handle in subFlow.
It's code for this goals, but I get a compilation error: incompatible types Message to List.
#Bean
public IntegrationFlow integrationFlow(
DataSource dataSource,
MessageHandler amqpHandler,
PersonService personService,
PersonChecker personChecker) {
return IntegrationFlows
.from(getMessageSource(personService::getPersons), e -> e.poller(getPollerSpec()))
.wireTap(subFlow -> subFlow.handle(personChecker::checkPerson))
.split()
.publishSubscribeChannel(pubSub -> pubSub
.subscribe(flow -> flow.bridge()
.transform(Transformers.toJson())
.handle(amqpHandler))
.subscribe(flow -> flow.bridge()
.handle(personService::markAsSent)))
.get();
}
I know about solution to pass service and name of method handle(personChecker, checkPerson), but it's not suitable for me.
Is exists possibility to pass in wireTap subflow in method handle list with objects Person instead Message message?
.handle((p, h) -> personService.checkPerson(p))
A spring integration based converter consumes the messages from one system, checks, converts and sends it to the other one.
Should the target system be down, we stop the inbound adapters, but would also like to persist locally or forward the currently "in-flight" converted messages. For that would simply like to reroute the messages from the normal output channel to some "backup"-channel dynamically.
In the docs I have found only the option to route the messages based on their headers ( so on some step before in flow I would have to add those dynamically once the targer system is not availbale), or based on the payload type, which is not really my case. The case with adding dynamically some header, and then filtering it out down the pipe, or during de-/serializing still seems not the best approach for me. I would like rather to be able to turn a switch(on some internal Event) that would then reroute those "in-flight" messages to the "backup"-channel.
What would be a best SI approach to achive this? Thanks!
The router could not only be based on the the payload type or some header. You really can have a general POJO method invocation to return a channel, its name or some routing key which is mapped. That POJO method indeed can check some internal system state and produce this or that routing key.
So, you may have something like this in the router configuration:
.route(myRouter())
where your myRouter is something like this:
#Bean
MyRouter myRouter() {
return;
}
and its internal code might be like this:
public class MyRouter {
#Autowired
private SystemState systemState;
String route(Object payload) {
return this.systemState.isActive() ? "successChannel" : "backupChannel";
}
}
The same can be achieved a simple lambda definition:
.<Object, Boolean>route(p -> systemState().isActive(),
m -> m.channelMapping(true, "sucessChannel")
.channelMapping(false, "backupChannel"))
Also...
private final AtomicBoolean switcher = new AtomicBoolean();
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(() -> "foo", e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(5))))
.route(s -> switcher.get() ? "foo" : "bar")
.get();
}
I have a list of items which I want to retrieve and return as fast as possible.
For each item I also need to retrieve details, they may be returned a few seconds later.
I could of course create two different routes with HTTP gateways and request first the list, then the details. However, I then have to wait until all details have arrived. I want to send back the list immediately and then the details as soon as I get them.
UPDATE
Following Artem Bilan's advice my flow returns a Flux as payload which merges the list of items as a Mono and the processed items as a Flux.
Note that the example below simulates detail processing of the items by calling toUpperCase; my real use case requires routing and outgoing calls to get the details for each item:
#Bean
public IntegrationFlow sseFlow() {
return IntegrationFlows
.from(WebFlux.inboundGateway("/strings/sse")
.requestMapping(m -> m.produces(MediaType.TEXT_EVENT_STREAM_VALUE))
.mappedResponseHeaders("*"))
.enrichHeaders(Collections.singletonMap("aHeader", new String[]{"foo", "bar"}))
.transform("headers.aHeader")
.<String[]>handle((p, h) -> {
return Flux.merge(
Mono.just(p),
Flux.fromArray(p)
.map(t -> {
return t.toUpperCase();
// return detailsResolver.resolveDetail(t);
}));
})
.get();
}
That comes closer to my goal. When I request data from this flow using curl, I get the list of items immediately and the processed items slightly later:
λ curl http://localhost:8080/strings/sse
data:["foo","bar"]
data:FOO
data:BAR
While simply converting the string to uppercase works fine, I have difficulty to make an outgoing call for details using WebFlux.outboundGateway. The detailsResolver in the commented out code above is defined as follows:
#MessagingGateway
public interface DetailsResolver {
#Gateway(requestChannel = "itemDetailsFlow.input")
Object resolveDetail(String item);
}
#Bean
IntegrationFlow itemDetailsFlow() {
return f -> f.handle(WebFlux.<String>outboundGateway(m ->
UriComponentsBuilder.fromUriString("http://localhost:3003/rest/path/")
.path(m.getPayload())
.build()
.toUri())
.httpMethod(HttpMethod.GET)
.expectedResponseType(JsonNode.class)
.replyPayloadToFlux(false));
}
When I comment in the detailsResolver call and comment out t.toUpperCase, the outboundGateway seems to be set up properly (the log says Subscriber present, Demand signaled) but never gets a response (doesn't reach a breakpoint in ExchangeFunctions.exchange#91).
I have ensured that the DetailsResolver itself is working by getting it as a bean from the context and invoking its method - that gives me a JsonNode response.
What can be the reason?
Yes, I wouldn't use toReactivePublsiher() there because you have a context of the current request. You need fluxes per request. I would use something like Flux.merge(Publisher<? extends I>... sources), where the first Flux is for items and the second is for details per item (something like Tuple2).
For this purpose you really can use something like this:
IntegrationFlows
.from(WebFlux.inboundGateway("/sse")
.requestMapping(m -> m.produces(MediaType.TEXT_EVENT_STREAM_VALUE)))
And your downstream flow should produce Flux as a payload for reply.
I have a sample like this in test cases:
#Bean
public IntegrationFlow sseFlow() {
return IntegrationFlows
.from(WebFlux.inboundGateway("/sse")
.requestMapping(m -> m.produces(MediaType.TEXT_EVENT_STREAM_VALUE))
.mappedResponseHeaders("*"))
.enrichHeaders(Collections.singletonMap("aHeader", new String[] { "foo", "bar", "baz" }))
.handle((p, h) -> Flux.fromArray((String[]) h.get("aHeader")))
.get();
}
I'm trying to consume entirely a paged resource as follow, however my aproach is raising a StackOverflowException.
Any clue abount this? Or a different aproach?
Example: https://gist.github.com/daniel-frank/a88fa4553ed34c348528f51d33c3733b
OK. I see now. Let me simplify your recursive code to show the problem:
private IntegrationFlow getPageFlow() {
return f -> f
.publishSubscribeChannel(ps -> ps
.subscribe(this.nextPageFlow())
);
}
private IntegrationFlow nextPageFlow() {
return f -> f
.publishSubscribeChannel(ps -> ps
.subscribe(this.getPageFlow())
);
}
So, technically we have this structure in the memory:
getPageFlow
nextPageFlow
getPageFlow
nextPageFlow
getPageFlow
and so on.
Another problem here that each .subscribe(this.nextPageFlow()) creates a new instance of the IntegrationFlow meanwhile logically you expect only one.
I understand that you can't declare beans in the IntegrationFlowAdapter impl, but that won't have with the StackOverflowException anyway.
What I see as a problem in your approach is a lack of the MessageChannel abstraction.
You use publishSubscribeChannel everywhere, meanwhile you could just distinguish the logic by the explicit channel definition in your flow.
To break the recursion and keep the code as closer to your solution as possible I'd make like this:
private IntegrationFlow getPageFlow() {
return f -> f
.channel("pageServiceChannel")
.handle(Http
.outboundGateway("https://jobs.github.com/positions.json?description={description}&page={page}")
...
private IntegrationFlow nextPageFlow() {
return f -> f
.filter("!payload.isEmpty()")
.enrichHeaders(e -> e.headerExpression("page", "headers.getOrDefault('page', 0) + 1", true))
.channel("pageServiceChannel");
}
Of course you still have a recursion, but that will be already at run time, logical.