Proper syntax to send a message to the control bus channel in an integration flow - spring-integration

Using the reference documentation and this answer, I set up an integration flow with an inbound JMS adapter, that I can stop/start using a control bus:
Control bus:
#Configuration
#RequiredArgsConstructor
public class ControlBus {
#Bean
public IntegrationFlow controlBusFlow() {
return IntegrationFlows.from(controlChannel()).log().controlBus().get();
}
#Bean
public MessageChannel controlChannel() {
return MessageChannels.direct().get();
}
}
Integration flow:
#Component
#RequiredArgsConstructor // lombok
public class Admin {
private final MessageChannel controlChannel;
#Bean
public IntegrationFlow adminFlow() {
return IntegrationFlows
.from(Http.inboundGateway("/admin")
.replyChannel("adminReply")
.requestMapping(r -> r.methods(DELETE, POST)))
.route("headers.http_requestMethod", r -> r
.subFlowMapping(DELETE.toString(),
f -> f.handle((p, h) -> controlChannel.send(new GenericMessage<>("#fileInboundAdapter.stop()"))))
.subFlowMapping(POST.toString(),
f -> f.handle((p, h) -> controlChannel.send(new GenericMessage<>("#fileInboundAdapter.start()"))))
.channel("adminReply")
.get();
}
}
It works fine, the fileInboundAdapter bean (defined in another flow) starts or stops as expected:
2022-10-26 17:31:01.356 INFO 11200 --- [nio-8082-exec-1] o.s.integration.handler.LoggingHandler : GenericMessage [payload=#fileInboundAdapter.stop(), headers={id=0b273024-8ca9-e383-bcfc-c487f14c044b, timestamp=1666798261355}]
2022-10-26 17:31:01.358 INFO 11200 --- [nio-8082-exec-1] o.s.i.e.SourcePollingChannelAdapter : stopped bean 'fileInboundAdapter'
But I'm curious if I can change the syntax of the router to something like :
.route("headers.http_requestMethod",
r -> r.subFlowMapping(DELETE.toString(),
f -> f.handle((p, h) -> "#fileInboundAdapter.stop()")
.channel("controlChannel"))
.subFlowMapping(POST.toString(),
f -> f.handle((p, h) -> "#fileInboundAdapter.start()")
.channel("controlChannel"))
I've tried using the "controlChannel" name or the injected channel, to no avail, the message is sometimes passed, sometimes not, I guess I haven't properly understood the use of the channel method.
If I use the message channel name, "controlChannel", here's what I get in the log (I need 3 attempts before the bean is stopped)
2022-10-26 17:36:16.814 INFO 14768 --- [nio-8082-exec-2] o.s.integration.handler.LoggingHandler : GenericMessage [payload=#fileInboundAdapter.stop(), headers={content-length=337, http_requestMethod=DELETE, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1ea2cdcc, accept=*/*, authorization=Basic VXNlcjE6cGFzc3dvcmQ=, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1ea2cdcc, host=localhost:8082, http_requestUrl=http://localhost:8082/ldd-controller/admin, connection=keep-alive, id=909e81d5-490b-f6aa-9e59-0dc5807091c4, http_userPrincipal=UsernamePasswordAuthenticationToken [Principal=User1, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=0:0:0:0:0:0:0:1, SessionId=null], Granted Authorities=[Administrateur]], contentType=application/json;charset=UTF-8, accept-encoding=gzip, deflate, br, user-agent=PostmanRuntime/7.29.2, timestamp=1666798576813}]
2022-10-26 17:36:48.330 INFO 14768 --- [nio-8082-exec-3] o.s.integration.handler.LoggingHandler : GenericMessage [payload=#fileInboundAdapter.stop(), headers={content-length=337, http_requestMethod=DELETE, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#4d68b976, accept=*/*, authorization=Basic VXNlcjE6cGFzc3dvcmQ=, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#4d68b976, host=localhost:8082, http_requestUrl=http://localhost:8082/ldd-controller/admin, connection=keep-alive, id=d7608dc3-809d-7160-398d-f8e329d28e22, http_userPrincipal=UsernamePasswordAuthenticationToken [Principal=User1, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=0:0:0:0:0:0:0:1, SessionId=null], Granted Authorities=[Administrateur]], contentType=application/json;charset=UTF-8, accept-encoding=gzip, deflate, br, user-agent=PostmanRuntime/7.29.2, timestamp=1666798608330}]
2022-10-26 17:36:54.957 INFO 14768 --- [nio-8082-exec-4] o.s.integration.handler.LoggingHandler : GenericMessage [payload=#fileInboundAdapter.stop(), headers={content-length=337, http_requestMethod=DELETE, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#7037fa6a, accept=*/*, authorization=Basic VXNlcjE6cGFzc3dvcmQ=, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#7037fa6a, host=localhost:8082, http_requestUrl=http://localhost:8082/ldd-controller/admin, connection=keep-alive, id=e16b32f9-42cb-ef06-41ac-837efb8de9d9, http_userPrincipal=UsernamePasswordAuthenticationToken [Principal=User1, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=0:0:0:0:0:0:0:1, SessionId=null], Granted Authorities=[Administrateur]], contentType=application/json;charset=UTF-8, accept-encoding=gzip, deflate, br, user-agent=PostmanRuntime/7.29.2, timestamp=1666798614957}]
2022-10-26 17:36:54.960 INFO 14768 --- [nio-8082-exec-4] o.s.i.e.SourcePollingChannelAdapter : stopped bean 'fileInboundAdapter'
On the third attempt (the one that works), I also get a "No reply received within timeout" response on the http client that send the original request.
I also need 3 requests if I want to start the bean again.
I get exactly the same result when injecting the channel.

With a configuration like f -> f.handle((p, h) -> controlChannel.send(new GenericMessage<>("#fileInboundAdapter.start()"))) you got a reply from this handler as a true - result of the controlChannel.send() call.
With the .channel("controlChannel") you don't get a reply because a call fileInboundAdapter.stop() returns void.
Not sure why you don't get steady results with that router, but I don't think that you are intended for a reply from there anyway.
I'd suggest to revise your configuration with a publishSubscribeChannel(Consumer<PublishSubscribeSpec> publishSubscribeChannelConfigurer) where one of the subscriber sub-flows would go to your router. And another would answer back to the adminReply. Although we need to understand what really you'd like to reply to that HTTP call:
.publishSubscribeChannel(c -> c
.subscribe(sf -> sf
.route("headers.http_requestMethod",
r -> r.subFlowMapping(DELETE.toString(),
f -> f.handle((p, h) -> "#fileInboundAdapter.stop()")
.channel("controlChannel"))
.subFlowMapping(POST.toString(),
f -> f.handle((p, h) -> "#fileInboundAdapter.start()")
.channel("controlChannel")))
)
.channel("adminReply")
However it looks more like a plain handle() instead since you do not route to different channels:
.handle((p, h) -> DELETE.toString().equals(h.get("http_requestMethod") ? "#fileInboundAdapter.stop()" : "#fileInboundAdapter.start()"))
.channel("controlChannel")
but still it has to be a part of publishSubscribe sub-flow since you are not going to get a reply from that control bus call.

Related

In Spring Integrration,DSL how do you specify a subscription to a published channel?

When using the Spring Integration DSL builder pattern, often it will fill in the needed channels between elements "automagically". However, sometimes it does not.
At the high level, the wrapping application keeps metadata in a database for creating and destroying flows dynamically as needed across platforms we have (potentially) never seen before. For this reason, the flows are not suitable to instantiate using static notations such as #Bean, but must be dynamically created and destroyed, and registered/deregistered in the spring context at runtime.
I have a published message channel used in the dynamically created main flow, and a channel in the dynamically created subflow, but I can't see how to subscribe to the mainPublishChannel from the subFlow.
This leaves me pushing messages into the channel, but with no subscriptions nothing happens.
Thanks in advance.
Some prior research (not an exhaustive listing:
https://github.com/spring-projects/spring-integration-flow
https://dzone.com/articles/spring-integration-building
https://xpadro.com/2014/05/spring-integration-4-0-a-complete-xml-free-example.html
Spring integration gateway "Dispatcher has no subscribers"
Spring Integration - How to debug 'Dispatcher has no Subscribers'?
Log snip
task-scheduler-1 2020-12-31 00:25:32,526 INFO o.s.i.g.GatewayProxyFactoryBean - started b653ca1c-038d-4567-bd4e-4c16ecc502a3.org.springframework.integration.config.ConsumerEndpointFactoryBean#3#gpfb
task-scheduler-1 2020-12-31 00:25:32,538 DEBUG o.s.i.c.PublishSubscribeChannel - preSend on channel 'b653ca1c-038d-4567-bd4e-4c16ecc502a3.mainPublishChannel', message: GenericMessage [payload=[{... timestamp=1609395932538}]
task-scheduler-1 2020-12-31 00:25:32,539 DEBUG o.s.i.d.BroadcastingDispatcher - No subscribers, default behavior is ignore
task-scheduler-1 2020-12-31 00:25:32,539 DEBUG o.s.i.c.PublishSubscribeChannel - postSend (sent=true) on channel 'b653ca1c-038d-4567-bd4e-4c16ecc502a3.mainPublishChannel', message: GenericMessage [payload=[{aaa=ee}], headers={aaa=ee, sequenceNumber=1, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#7771e96a, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#7771e96a, sequenceSize=2, yyy=2020-12-24 11:15:30.915278, correlationId=0eef0e4e-768c-90db-fa7b-2d1767335b26, timestamp=1609395932538}]
Code Snip:
String channelId=getId().toString()+'.'+"mainPublishChannel";
MessageChannel channel = MessageChannels.publishSubscribe(channelId, stepTaskExecutor).get();
final IntegrationFlowBuilder bldr = IntegrationFlows
.from(setupAdapter,
c -> c.poller(Pollers.fixedRate(pollerFixedRate, TimeUnit.MILLISECONDS).maxMessagesPerPoll(1)))
.enrichHeaders(h -> h.headerExpression("xxx", "payload[0].get(\"xxx\")")
.headerExpression("yyy", "payload[0].get(\"yyy\")"))
.split(tableSplitter)
.gateway(channel)
.routeToRecipients(r -> this.buildRecipientListRouterSpecForRules(r, rules, channel)
)
.aggregate()
.handle(cleanupAdapter)
;
...
snip
...
private RecipientListRouterSpec buildRecipientListRouterSpecForRules(RecipientListRouterSpec recipientListSpec,
Collection<RuleMetadata> rules, MessageChannel publishedChannel) {
// ??? How to subscribe this to publishedChannel??
recipientListSpec
.recipient(MessageChannels.publishSubscribe(this.getId().toString()+'.'+"mainReceiveChannel", stepTaskExecutor).get());
rules.forEach(
rule -> recipientListSpec.recipientFlow(getFilterExpression(rule), f -> createFlowDefForRule(f, rule)));
recipientListSpec
.ignoreSendFailures(true)
.defaultOutputToParentFlow();
return recipientListSpec;
}
The publishedChannel must be passed to the child flow as the input channel
return flowDef
.channel(receiveChannel) // <---- This is the reference to the main publish channel in the child flow, which allows the builder to create the subscription
.log()
.handle(inboundAdapter)
... snip ...
;

Move file from inbound adapter after publish subscribe flow

I'm trying to implement the following flow:
1) files are read from inbound adapter
2) they are send to different flows using publish-subscribe channel with applied sequence
3) file is moved after all the subscriber flows are ready
This is the main flow
return IntegrationFlows
.from(Files.inboundAdapter(inboundOutDirectory)
.regexFilter(pattern)
.useWatchService(true)
.watchEvents(FileReadingMessageSource.WatchEventType.CREATE),
e -> e.poller(Pollers.fixedDelay(period)
.taskExecutor(Executors.newFixedThreadPool(poolSize))
.maxMessagesPerPoll(maxMessagesPerPoll)))
.publishSubscribeChannel(s -> s
.applySequence(true)
.subscribe(f -> f
.transform(Files.toStringTransformer())
.<String>handle((p, h) -> {
return "something"
}
})
.channel("consolidateFlow.input"))
.subscribe(f -> f
.transform(Files.toStringTransformer())
.handle(Http.outboundGateway(testUri)
.httpMethod(HttpMethod.GET)
.uriVariable("text", "payload") .expectedResponseType(String.class))
.<String>handle((p, h) -> {
return "something";
})
.channel("consolidateFlow.input")))
.get();
And the aggregation:
public IntegrationFlow consolidateFlow()
return flow -> flow
.aggregate()
.<List<String>>handle((p, h) -> "something").log()
}
}
Using the following code in the main flow after publish-subscribe
.handle(Files.outboundGateway(this.inboundProcessedDirectory).deleteSourceFiles(true))
ends up with
Caused by: org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
If I go with this the consolidation/aggregation flow won't be reached at all.
.handle(Files.outboundAdapter(this.inboundProcessedDirectory))
Any idea how I could solve it? Currently I'm moving the file after the aggregation by reading the original file name from the header but it doesn't seem to be the right solution.
I was also thinking about applying spec/advice to the inbound adapter with success logic to move the file but not sure whether that's the right approach.
EDIT1
As suggested by Artem, I've added another subscriber to the publish-subscribe as follows:
...
.channel("consolidateNlpFlow.input"))
.subscribe(f -> f
.handle(Files.outboundAdapter(this.inboundProcessedDirectory).deleteSourceFiles(true))
...
The files is moved properly, but the consolidateFlow is not being executed at all. Any idea?
I've also tried adding the channel to the new flow .channel("consolidateNlpFlow.input") but it didn't change the behavior.
Your problem that a consolidateFlow is not able to return result into the main flow. Just because there is anything gateway-like. You do there an explicit .channel("consolidateFlow.input") which means there is not going to be way back.
That's for the issue you have so far.
Regarding a possible solution.
According to your configuration both your subscribers in the publishSubscribeChannel are performed on the same thread, one by one. So, it is going to be very easy for you to add one more subscriber with that Files.outboundAdapter() and deleteSourceFiles(true). This one is going to be called already after existing subscribers.

How to parameterize an object in integration flow?

I has integration flow for polling data from database. I set up message source which return list of object, this list I want to pass to method handle in subFlow.
It's code for this goals, but I get a compilation error: incompatible types Message to List.
#Bean
public IntegrationFlow integrationFlow(
DataSource dataSource,
MessageHandler amqpHandler,
PersonService personService,
PersonChecker personChecker) {
return IntegrationFlows
.from(getMessageSource(personService::getPersons), e -> e.poller(getPollerSpec()))
.wireTap(subFlow -> subFlow.handle(personChecker::checkPerson))
.split()
.publishSubscribeChannel(pubSub -> pubSub
.subscribe(flow -> flow.bridge()
.transform(Transformers.toJson())
.handle(amqpHandler))
.subscribe(flow -> flow.bridge()
.handle(personService::markAsSent)))
.get();
}
I know about solution to pass service and name of method handle(personChecker, checkPerson), but it's not suitable for me.
Is exists possibility to pass in wireTap subflow in method handle list with objects Person instead Message message?
.handle((p, h) -> personService.checkPerson(p))

Enriching in parallel after a split

This is a continuation of the shopping cart sample, where we have an external API that allows checkout from a shopping cart. To recap, we have a flow where we create an empty shopping, add line item(s) and finally checkout. All the operations above, happen as enrichments through HTTP calls to an external service. We would like to add line items concurrently (as part of the add line items) call. Our current configuration looks like this:
#Bean
public IntegrationFlow fullCheckoutFlow() {
return f -> f.channel("inputChannel")
.transform(fromJson(ShoppingCart.class))
.enrich(e -> e.requestChannel(SHOPPING_CART_CHANNEL))
.split(ShoppingCart.class, ShoppingCart::getLineItems)
.enrich(e -> e.requestChannel(ADD_LINE_ITEM_CHANNEL))
.aggregate(aggregator -> aggregator
.outputProcessor(g -> g.getMessages()
.stream()
.map(m -> (LineItem) m.getPayload())
.map(LineItem::getName)
.collect(joining(", "))))
.enrich(e -> e.requestChannel(CHECKOUT_CHANNEL))
.<String>handle((p, h) -> Message.called("We have " + p + " line items!!"));
}
#Bean
public IntegrationFlow addLineItem(Executor executor) {
return f -> f.channel(MessageChannels.executor(ADD_LINE_ITEM_CHANNEL, executor).get())
.handle(outboundGateway("http://localhost:8080/api/add-line-item", restTemplate())
.httpMethod(POST)
.expectedResponseType(String.class));
}
#Bean
public Executor executor(Tracer tracer, TraceKeys traceKeys, SpanNamer spanNamer) {
return new TraceableExecutorService(newFixedThreadPool(10), tracer, traceKeys, spanNamer);
}
To add line items in parallel, we are using an executor channel. However, they still seem to be getting processed sequentially when seen in zipkin:
What are we doing wrong? The source for the whole project is on github for reference.
Thanks!
First of all the main feature of Spring Integration is MessageChannel, but it still isn't clear to me why people are missing .channel() operator in between endpoint definitions.
I mean that for your case it must be like:
.split(ShoppingCart.class, ShoppingCart::getLineItems)
.channel(c -> c.executor(executor()))
.enrich(e -> e.requestChannel(ADD_LINE_ITEM_CHANNEL))
Now about your particular problem.
Look, ContentEnricher (.enrich()) is request-reply component: http://docs.spring.io/spring-integration/reference/html/messaging-transformation-chapter.html#payload-enricher.
Therefore it sends request to its requestChannel and waits for reply. And it is done independently of the requestChannel type.
I raw Java we can demonstrate such a behavior with this code snippet:
for (Object item: items) {
Data data = sendAndReceive(item);
}
where you should see that ADD_LINE_ITEM_CHANNEL as an ExecutorChannel doesn't have much value because we are blocked within loop for the reply anyway.
A .split() does exactly similar loop, but since by default it is with the DirectChannel, an iteration is done in the same thread. Therefore each next item waits for the reply for the previous.
That's why you definitely should parallel exactly as an input for the .enrich(), just after .split().

How do I specify default output channel on routeToRecipients using Spring Integration Java DSL 1.0.0.M3

Since upgrading to M3 of spring-integration java dsl I'm seeing the following error on any flow using a recipient list router:
org.springframework.messaging.MessageDeliveryException: no channel resolved by router and no default output channel defined
It's not clear how to actually specify this in M3. There is no output channel option on the endpoint configurer and nothing on the RecipientListRouterSpec. Any suggestions?
According to the https://jira.spring.io/browse/INTEXT-113 there is no more reason to specify .defaultOutputChannel(), because the next .channel() (or implicit) is used for that purpose. That's because that defaultOutputChannel exactly plays the role of standard outputChannel. Therefore you have now more formal integration flow:
#Bean
public IntegrationFlow recipientListFlow() {
return IntegrationFlows.from("recipientListInput")
.<String, String>transform(p -> p.replaceFirst("Payload", ""))
.routeToRecipients(r -> r.recipient("foo-channel", "'foo' == payload")
.recipient("bar-channel", m ->
m.getHeaders().containsKey("recipient")
&& (boolean) m.getHeaders().get("recipient")))
.channel("defaultOutputChannel")
.handle(m -> ...)
.get();
}
Where .channel("defaultOutputChannel") can be omitted.

Resources