Is there a default output channel if DSL flow ends with endpoin? - spring-integration

The last element in the code for the following DSL flow is Service Activator (.handle method).
Is there a default output direct channel to which I can subscribe here? If I understand things correctly, the output channel must be present
I know I can add .channel("name") at the end but the question is what if it's not written explicitly.
Here is the code:
#SpringBootApplication
#IntegrationComponentScan
public class QueueChannelResearch {
#Bean
public IntegrationFlow lambdaFlow() {
return f -> f.channel(c -> c.queue(50))
.handle(System.out::println);
}
public static void main(String[] args) {
ConfigurableApplicationContext ctx = SpringApplication.run(QueueChannelResearch.class, args);
MessageChannel inputChannel = ctx.getBean("lambdaFlow.input", MessageChannel.class);
for (int i = 0; i < 1000; i++) {
inputChannel.send(MessageBuilder.withPayload("w" + i)
.build());
}
ctx.close();
}
Another question is about QueueChannel. The program hangs if comment handle() and completes if uncomment it. Does that mean that handle() add a default Poller before it?
return f -> f.channel(c -> c.queue(50));
// .handle(System.out::println);

No, that doesn't work that way.
Just recall that integration flow is a filter-pipes architecture and result of the current step is going to be sent to next one. Since you use .handle(System.out::println) there is no output from that println() method call therefore nothing is returned to build a Message to sent to the next channel if any. So, the flow stops here. The void return type or null returned value is a signal for service activator to stop the flow. Consider your .handle(System.out::println) as an <outbound-channel-adapter> in the XML configuration.
And yes: there is no any default channels, unless you define one via replyChannel header in advance. But again: your service method must return something valuable.
The output from service activator is optional, that's why we didn't introduce extra operator for the Outbound Channel Adapter.
The question about QueueChannel would be better to handle in the separate SO thread. There is no default poller unless you declare one as a PollerMetadata.DEFAULT_POLLER. You might use some library which delcares that one for you.

Related

Spring Integration - Stop Inbound Adapter after files have been copied

I am doing a local file copy using Spring integration. My class is presented later in this post.
I initiate the file copy by issuing an adapter.start() from another class. That works fine, and the adapter (localFileTransferAdapter) runs once (based on the FireOnceTrigger) copying all the files I expect. But I then want to stop the adapter after all the files have been copied.
What can I do to detect that the adapter has copied the files so I can then stop the adapter? The FireOnceTrigger will never fire again - but the adapter still shows running when I query it from the class I use to start the adapter. I could wait some number of seconds and stop the adapter - but it could stop the adapter in the middle of copying files if there are many large files to copy - which I don't want to happen.
I've reviewed How to stop polling after a message is received? Spring Integration but it does not appear to match my use case.
Thanks in advance for for any assistance.
public class LocalFileTransfer {
#Value("${source.directory:c:/source}")
private String sourceDirectory;
#Value("${target.directory:c:/target}")
private String targetDirectory;
#Bean
public MessageSource<File> sourceDirectory() {
FileReadingMessageSource messageSource = new FileReadingMessageSource();
messageSource.setDirectory(new File(sourceDirectory));
return messageSource;
}
#Bean
public IntegrationFlow fileMover() {
return IntegrationFlows.from(sourceDirectory(),
c -> c.autoStartup(false)
.id("localFileTransferAdapter")
.poller(Pollers.trigger(new FireOnceTrigger())
.maxMessagesPerPoll(-1)))
.filter(source -> ((File) source).getName().endsWith(".txt"))
.log(LoggingHandler.Level.ERROR, "localfile.category", m -> m.getPayload())
.log(LoggingHandler.Level.ERROR, "localfile.category", m -> m.getHeaders())
.handle(targetDirectory())
.get();
}
#Bean
public MessageHandler targetDirectory() {
FileWritingMessageHandler handler = new FileWritingMessageHandler(new File(targetDirectory));
handler.setFileExistsMode(FileExistsMode.REPLACE);
handler.setExpectReply(false);
return handler;
}
}
You are on the right track with the FireOnceTrigger and maxMessagesPerPoll(-1). Also that answer with an AbstractMessageSourceAdvice sample fully fits to your scenario.
In the MessageSourceMutator.afterReceive() you just check for the null of result argument and call a stop() of that localFileTransferAdapter.
Since everything happens on the same thread, you are safe to call stop just when you meet a null: no need to introduce some delay and the channel adapter doesn't exist until it finishes producing messages in that one poll cycle.

Spring Integration: reuse MessageProducer definition

I have an outbound gateway for soap calls (MarshallingWebServiceOutboundGateway) with elaborate setup. I need to use that gateway definition from multiple flows.
The question spring-integration: MessageProducer may only be referenced once is somewhat similar, but this question is about the proper use of the spring bean scope prototype for spring integration collaborators.
I have a separate config file which sets up the gateway and its dependencies:
#Bean
public MarshallingWebServiceOutboundGateway myServiceGateway() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("blah.*");
MarshallingWebServiceOutboundGateway gateway = new MarshallingWebServiceOutboundGateway(
serviceEndpoint, marshaller, messageFactory);
gateway.setMessageSender(messageSender);
gateway.setRequestCallback(messageCallback);
return gateway;
}
This is how I initially tried to wire up the outbound gateway from two different flows in two different config files.
In one config file:
#Bean
public IntegrationFlow flow1() {
MarshallingWebServiceOutboundGateway myServiceGateway = context.getBean("myServiceGateway", MarshallingWebServiceOutboundGateway.class);
return IntegrationFlows
.from(Http.inboundGateway("/res1")
.requestMapping(r -> r.methods(HttpMethod.GET))
.transform(soapRequestTransformer)
.handle(myServiceGateway) // wrong: cannot be same bean
.transform(widgetTransformer)
.get();
}
In a separate config file:
#Bean
public IntegrationFlow flow2() {
MarshallingWebServiceOutboundGateway myServiceGateway = context.getBean("myServiceGateway", MarshallingWebServiceOutboundGateway.class);
return IntegrationFlows
.from(Http.inboundGateway("/res2")
.requestMapping(r -> r.methods(HttpMethod.GET))
.transform(soapRequestTransformer)
.handle(myServiceGateway) // wrong: cannot be same bean
.transform(widgetTransformer)
.handle(servicePojo)
.get();
}
This is a problem because - as I understand it - myServiceGateway cannot be the same instance, since that instance has only one outbound channel and cannot belong to two different flows.
In the related question spring-integration: MessageProducer may only be referenced once, #artem-bilan advised not to create the outbound gateway in an #Bean method, rather to use a plain method which creates new instances for every call.
That works, but it is inconvenient in my case. I need to reuse the outbound gateway from several flows in different config files and I would have to copy the code to create the gateway into each config file. Also, the gateway dependencies inflate my Configuration file constructors, making Sonar bail.
Since the error message coming out of IntegrationFlowDefinition.checkReuse() says A reply MessageProducer may only be referenced once (myServiceGateway) - use #Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE) on #Bean definition. I wanted to give the scope prototype another try.
So I try to make spring integration look up a prototype gateway from the context by name, hoping to get a different gateway instance in flow1 and flow2:
.handle(context.getBean("myServiceGateway",
MarshallingWebServiceOutboundGateway.class))
And I annotated the outbound gateway #Bean definition with
#Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE)
But I can see that the myServiceGateway() method is only invoked once, despite the prototype scope, and application startup still fails with the error message which advises to use the prototype scope - quite confusing, actually ;-)
Based on Mystery around Spring Integration and prototype scope I also tried:
#Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE, proxyMode = ScopedProxyMode.TARGET_CLASS)
The application starts, but the responses never reach the step after the gateway, the widgetTransformer. (Even more strange, exactly the widgetTransformer is skipped: in flow1 the outcome is the untransformed gateway response and in flow2 the untransformed messages hit the step after the widgetTransformer, i.e. the servicePojo). Making a proxy out of a message producer seems not to be a good idea.
I really want to get to the bottom of this. Is the exception message wrong which asks to use the prototype scope or am I just getting it wrong? How can I avoid to repeat the bean definition for message producers if I need several such producers which are all set up the same way?
Using spring-integration 5.0.9.
I am not entirely sure why the #Scope is not working, but here is a work-around...
#SpringBootApplication
public class So52453934Application {
public static void main(String[] args) {
SpringApplication.run(So52453934Application.class, args);
}
#Autowired
private HandlerConfig config;
#Bean
public IntegrationFlow flow1() {
return f -> f.handle(this.config.myHandler())
.handle(System.out::println);
}
#Bean
public IntegrationFlow flow2() {
return f -> f.handle(this.config.myHandler())
.handle(System.out::println);
}
#Bean
public ApplicationRunner runner() {
return args -> {
context.getBean("flow1.input", MessageChannel.class).send(new GenericMessage<>("foo"));
context.getBean("flow2.input", MessageChannel.class).send(new GenericMessage<>("bar"));
};
}
}
#Configuration
class HandlerConfig {
public AbstractReplyProducingMessageHandler myHandler() {
return new AbstractReplyProducingMessageHandler() {
#Override
protected Object handleRequestMessage(Message<?> requestMessage) {
return ((String) requestMessage.getPayload()).toUpperCase();
}
};
}
}
i.e. do as #artem suggested, but inject the bean with the factory method.

Splitter aborts during exception with out processing subsequent messages

I have a requirement to split the messages and process one by one. If any of the messages fails, I would like to report it to error channel and resume processing the next available messages
I am using spring cloud aws stream starter with 1.0.0-SNAPSHOT
I wrote a sample program using splitter
#Bean
public MessageChannel channelSplitOne() {
return new DirectChannel();
}
#StreamListener(INTERNAL_CHANNEL)
public void channelOne(String message) {
if (message.equals("l")) {
throw new RuntimeException("Error due to l");
}
System.out.println("Internal: " + message);
}
#Splitter(inputChannel = Sink.INPUT, outputChannel = INTERNAL_CHANNEL)
public List<Message> extractItems(Message<String> input) {
return Arrays.stream(input.getPayload().split(""))
.map(s -> MessageBuilder.withPayload(s).copyHeaders(input.getHeaders()).build())
.collect(Collectors.toList());
}
When I send the message as Hello, the exxpectation is that
'h','e','o' shall be processed, but 'l' shall be reported as error.
But here the after 'l', the processing is not resumed.
Is there any way to achieve this.
You can do that, but with the #ServiceActivator instead of #StreamListener. The first one has adviceChain option where you can inject an ExpressionEvaluatingRequestHandlerAdvice: https://docs.spring.io/spring-integration/docs/5.0.4.RELEASE/reference/html/messaging-endpoints-chapter.html#expression-advice.
The problem that the splitter is like a regular loop in Java, so to continue after error we need to add somehow a try...catch there. But that’s already not a splitter responsibility. Therefore we have to move such a logic into the place we have a error problem.

create sftp reply-channel to reply with error or messages that was unsuccessfully sent

I am using java dsl to configure sfp outbound flow.
Gateway:
#MessagingGateway
public interface SftpGateway {
#Gateway(requestChannel = "sftp-channel")
void sendFiles(List<Message> messages);
}
Config:
#Bean
public IntegrationFlow sftpFlow(DefaultSftpSessionFactory sftpSessionFactory) {
return IntegrationFlows
.from("sftp-channel")
.split()
.handle(Sftp.outboundAdapter(sftpSessionFactory, FileExistsMode.REPLACE)
.useTemporaryFileName(false)
.remoteDirectory(REMOTE_DIR_TO_CREATE).autoCreateDirectory(true)).get();
}
#Bean
public DefaultSftpSessionFactory sftpSessionFactory() {
...
}
How can i configure flow to make my gateway reply with Messages that were failed?
In other words i want my gateway to be able to return list of messages which were failed, not void.
I marked gateway with
#MessagingGateway(errorChannel = "errorChannel")
and wrote error channel
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("errorChannel").handle(new GenericHandler<MessagingException>() {
public Message handle(MessagingException payload, Map headers) {
System.out.println(payload.getFailedMessage().getHeaders());
return payload.getFailedMessage();
}
})
.get();
}
#Bean
public MessageChannel errorChannel() {
return MessageChannels.direct().get();
}
and in case of some errors(i.e. no connection to SFTP) i get only one error (payload of first message in list).
Where should i put Advice to aggregate all messages?
This is not the question to Spring Integration Java DSL.
This is mostly a design and architecture task.
Currently you don't have any choice because you use Sftp.outboundAdapter() which is one-way, therefore without any reply. And your SftpGateway is ready for that behavior with the void return type.
If you have a downstream errorr, you can only throw them or catch and send to some error-channel.
According to your request of:
i want my gateway to be able to return list of messages which were failed, not void.
I'd say it depends. Actually it is just return from your gateway. So, if you return an empty list into gateway that may mean that there is no errors.
Since Java doesn't provide multi-return capabilities we don't have choice unless do something in our stream which builds that single message to return. As we decided list of failed messages.
Since you have there .split(), you should look into .aggregate() to build a single reply.
Aggregator correlates with the Splitter enough easy, via default applySequence = true.
To send to aggregator I'd suggest to take a look into ExpressionEvaluatingRequestHandlerAdvice on the Sftp.outboundAdapter() endpoint (second param of the .handle()). With that you should send both good and bad messages to the same .aggregate() flow. Than you can iterate a result list to clean up it from the good result. The result after that can be send to the SftpGateway using replyChannel header.
I understand that it sounds a bit complicated, but what you want doesn't exist out-of-the-box. Need to think and play yourself to figure out what can be reached.

Spring Integration 4 asynchronous request/response

I am trying to write a simple message flow using Spring Integration v4's DSL APIs which would look like this:
-> in.ch -> Processing -> JmsGatewayOut -> JMS_OUT_QUEUE
Gateway
<- out.ch <- Processing <- JmsGatewayIn <- JMS_IN_QUEUE
With the request/response being asynchronous, when I inject a message via the initial Gateway, the message goes all the way to JMS_OUT_QUEUE. Beyond this message flow, a reply message is put back into JMS_IN_QUEUE which it is then picked up by JmsGatewayIn. At this point, the message is Processed and placed into out.ch (I know the response gets to out.ch because I have a logger interceptor there which logs the message being placed there) but, the Gateway never receives the response.
Instead of a response, the system outside of this message flow which picked up the message from JMS_OUT_QUEUE and placed the response in JMS_IN_QUEUE, receives a javax.jms.MessageFormatException: MQJMS1061: Unable to deserialize object on its own JmsOutboundgateway (I think it is failing to deserialize a jms reply object from looking at the logs).
I have clearly not got something configured correctly but I don't know exactly what. Does anyone know what I am missing?
Working with spring-integration-core-4.0.3.RELEASE, spring-integration-jms-4.0.3.RELEASE, spring-integration-java-dsl-1.0.0.M2, spring-jms-4.0.6.RELEASE.
My Gateway is configured as follows:
#MessagingGateway
public interface WsGateway {
#Gateway(requestChannel = "in.ch", replyChannel = "out.ch",
replyTimeout = 45000)
AResponse process(ARequest request);
}
My Integration flow is configured as follows:
#Configuration
#EnableIntegration
#IntegrationComponentScan
#ComponentScan
public class IntegrationConfig {
#Bean(name = "in.ch")
public DirectChannel inCh() {
return new DirectChannel();
}
#Bean(name = "out.ch")
public DirectChannel outCh() {
return new DirectChannel();
}
#Autowired
private MQQueueConnectionFactory mqConnectionFactory;
#Bean
public IntegrationFlow requestFlow() {
return IntegrationFlows.from("in.ch")
.handle("processor", "processARequest")
.handle(Jms.outboundGateway(mqConnectionFactory)
.requestDestination("JMS_OUT_QUEUE")
.correlationKey("JMSCorrelationID")
.get();
}
#Bean
public IntegrationFlow responseFlow() {
return IntegrationFlows.from(Jms.inboundGateway(mqConnectionFactory)
.destination("JMS_IN_QUEUE"))
.handle("processor", "processAResponse")
.channel("out.ch")
.get();
}
}
Thanks for any help on this,
PM.
First of all your configuration is bad:
Since you start the flow from WsGateway#process you really should wait reply there.
The gateway's request/reply capability is based on TemporaryReplyChannel, which is placed to the headers as non-serializable value.
As long as you wait rely on that gateway, actually there is no reason to provide the replyChannel, if you aren't going to do some publish-subscribe logic on the reply.
As you send message to the JMS queue, you should understand that consumer part might be a separete remote application. And the last one might know nothing about your out.ch.
The JMS request/reply capability is really based on JMSCorrelationID, but it isn't enough. The one more thing here is a ReplyTo JMS header. Hence, if you are going to send reply from the consumer you should really just rely on the JmsGatewayIn stuff.
So I'd change your code to this:
#MessagingGateway
public interface WsGateway {
#Gateway(requestChannel = "in.ch", replyTimeout = 45000)
AResponse process(ARequest request);
}
#Configuration
#EnableIntegration
#IntegrationComponentScan
#ComponentScan
public class IntegrationConfig {
#Bean(name = "in.ch")
public DirectChannel inCh() {
return new DirectChannel();
}
#Autowired
private MQQueueConnectionFactory mqConnectionFactory;
#Bean
public IntegrationFlow requestFlow() {
return IntegrationFlows.from("in.ch")
.handle("processor", "processARequest")
.handle(Jms.outboundGateway(mqConnectionFactory)
.requestDestination("JMS_OUT_QUEUE")
.replyDestination("JMS_IN_QUEUE"))
.handle("processor", "processAResponse")
.get();
}
}
Let me know, if it is appropriate for you or try to explian why you use two-way gateways for one one-way cases. Maybe Jms.outboundAdapter() and Jms.inboundAdapter() are more good for you?
UPDATE
How to use <header-channels-to-string> from Java DSL:
.enrichHeaders(e -> e.headerChannelsToString())

Resources