Spring Integration Wiretap with Annotations (4.3 Spring Integration) - spring-integration

I have a Spring Integration (4.3) application and i have a JMS receiver which sends the received message to a ServiceActivator.
I am trying to Wiretap a channel, and send the JMS message to Logger (which in this case is Service Activator as well)
I get a strange behavior with the code below.
Every odd message sent (first, third) - it only goes to the logger.
Every even message sent (second, fourth, etc) - only goes to the ServiceActivator that is supposed to do some work and in the future sent the message downstream.
Here are logs
2018-07-23 16:14:43.278 INFO 16532 --- [ container-1] zzz : MSG1
Received via barChannel: MSG2
How can i change my code so all messages go both to the logger and the ServiceActivator that is supposed to do the work?
Thanks a lot in advance
#Bean
public ConnectionFactory jmsConnectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
#Bean
public JmsMessageDrivenEndpoint inbound() {
JmsMessageDrivenEndpoint jmsMessageDrivenEndpoint = new JmsMessageDrivenEndpoint(container(), listener());
jmsMessageDrivenEndpoint.setSessionAcknowledgeMode("transacted");
jmsMessageDrivenEndpoint.setAutoStartup(true);
return jmsMessageDrivenEndpoint;
}
#Bean
public DefaultMessageListenerContainer container() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConnectionFactory());
container.setDestinationName("foo.bar");
return container;
}
#Bean
public ChannelPublishingJmsMessageListener listener() {
ChannelPublishingJmsMessageListener listener = new ChannelPublishingJmsMessageListener();
listener.setRequestChannelName("myLogChannel");
return listener;
}
#ServiceActivator(inputChannel = "myLogChannel")
public void bar(String in) {
System.out.println("Received via barChannel: " + in);
}
#Bean
public MessageChannel TappingChannel(MessageChannel myLogChannel) {
DirectChannel d = new DirectChannel();
d.addInterceptor(new WireTap("myLogChannel"));
return d;
}
#Bean
#ServiceActivator(inputChannel = "myLogChannel")
public MessageHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler(LoggingHandler.Level.INFO.name());
loggingHandler.setLoggerName("zzz");
return loggingHandler;
}
#Bean
public MessageChannel myLogChannel() {
return new DirectChannel();
}
EDIT: Here is the original code that works
#Bean
public MessageChannel toRouter() {
return new DirectChannel();
}
#Bean
public ConnectionFactory jmsConnectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
#Bean
public JmsMessageDrivenEndpoint inbound(ConnectionFactory jmsConnectionFactory) {
return new JmsMessageDrivenEndpoint(container(jmsConnectionFactory), listener());
}
#Bean
public DefaultMessageListenerContainer container(ConnectionFactory jmsConnectionFactory) {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConnectionFactory);
container.setDestinationName("foo.bar");
return container;
}
#Bean
public ChannelPublishingJmsMessageListener listener() {
ChannelPublishingJmsMessageListener listener = new ChannelPublishingJmsMessageListener();
listener.setRequestChannelName("toRouter");
return listener;
}
#ServiceActivator(inputChannel = "toRouter")
public void bar(String in) {
System.out.println("Received via barChannel: " + in);
}
I want to do the Wiretap logging on this code

You have a mess in your config.
First of all you have two #ServiceActivator for the same myLogChannel DirectChannel.
By default it uses a round-robin dispatching strategy. That's why you see that odd/even behavior between your two consumers on the same channel.
You separate TappingChannel doesn't bring any value and it is out of use at all. No one sends message to this channel: you send to the listener.setRequestChannelName("myLogChannel"); directly though... Therefore a WireTap isn't performed at all as well.
I'm not sure how to help you since it isn't clear what you are going to use as a main channel and what should be used from the wire-tap.
UPDATE
So, you need to configure that new WireTap("myLogChannel") on this toRouter channel.

Related

Using EmbeddedKafkaBroker with spring integration and spring kafka

I want to use EmbeddedKafkaBroker to test my flow that involves KafkaMessageDrivenChannelAdapter,
it looks like consumer starts correclty , subscribed to topic but handler is not triggered after pushing message to EmbeddedKafkaBroker.
#SpringBootTest(properties = {"...."}, classes = {....class})
#EmbeddedKafka
class IntTests {
#BeforeAll
static void setup() {
embeddedKafka = new EmbeddedKafkaBroker(1, true, TOPIC);
embeddedKafka.kafkaPorts(57412);
embeddedKafka.afterPropertiesSet();
}
#Test
void testit() throws InterruptedException {
String ip = embeddedKafka.getBrokersAsString();
Map<String, Object> configs = new HashMap<>(KafkaTestUtils.producerProps(embeddedKafka));
Producer<String, String> producer = new DefaultKafkaProducerFactory<>(configs, new StringSerializer(), new StringSerializer()).createProducer();
// Act
producer.send(new ProducerRecord<>(TOPIC, "key", "{\"name\":\"Test\"}"));
producer.flush();
....
}
...
}
And the main class:
#Configuration
public class Kafka {
#Bean
public KafkaMessageDrivenChannelAdapter<String, String> adapter(KafkaMessageListenerContainer<String, String> container) {
KafkaMessageDrivenChannelAdapter<String, String> kafkaMessageDrivenChannelAdapter =..
kafkaMessageDrivenChannelAdapter.setOutputChannelName("kafkaChannel");
}
#Bean
public KafkaMessageListenerContainer<String, String> container() {
ContainerProperties properties = new ContainerProperties(TOPIC);
KafkaMessageListenerContainer<String, String> kafkaContainer = ...;
return kafkaContainer;
}
#Bean
public ConsumerFactory<String, String> consumerFactory() {
Map<String, Object> props = new HashMap<>();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:57412");
props.put(ConsumerConfig.GROUP_ID_CONFIG, "group12");
...
return new DefaultKafkaConsumerFactory<>(props);
}
#Bean
public PublishSubscribeChannel kafkaChannel() {
return new PublishSubscribeChannel ();
}
#Bean
#ServiceActivator(inputChannel = "kafkaChannel")
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
}
};
}
...
}
in log I do see:
clients.consumer.KafkaConsumer : [Consumer clientId=consumer-group12-1, groupId=group12] Subscribed to topic(s): TOPIC
ThreadPoolTaskScheduler : Initializing ExecutorService
KafkaMessageDrivenChannelAdapter : started bean 'adapter'; defined in: 'com.example.demo.demo.Kafka';
Having embeddedKafka = new EmbeddedKafkaBroker(1, true, TOPIC); and #EmbeddedKafka, you essentially start two separate Kafka clusters. See ports option of the #EmbeddedKafka if you want to change a random port for embedded broker. But at the same time it is better to rely in what Spring Boot provides for us with its auto-configuration.
See documentation for more info: https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-embedded-kafka. Pay attention to the bootstrapServersProperty = "spring.kafka.bootstrap-servers" property.
UPDATE
In your test you have this #SpringBootTest(classes = {Kafka.class}). When I remove that classes attribute, everything has started to work. The problem that your config class is not auto-configuration aware, therefore you don't have Spring Integration initialized properly and the message is not consumed from the channel. There might be some other affect. But still: better to rely on the auto-configuration, so let your test to see that #SpringBootApplication annotation.

ServiceActivator does not receive message from ImapIdleChannelAdapter

ServiceActivator does not receive messages from ImapIdleChannelAdapter...
JavaMail logs successful FETCH, but MIME messages do not get delivered to SA endpoint... I want to understand what is wrong in my code.
A7 FETCH 1:35 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1 FETCH (ENVELOPE ("Fri....
Code snippet below:
`
#Autowired
EmailConfig emailCfg;
#Bean
public SubscribableChannel mailChannel() {
return MessageChannels.direct().get();
}
#Bean
public ImapIdleChannelAdapter getMailAdapter() {
ImapMailReceiver mailReceiver = new ImapMailReceiver(emailCfg.getImapUrl());
mailReceiver.setJavaMailProperties(javaMailProperties());
mailReceiver.setShouldDeleteMessages(false);
mailReceiver.setShouldMarkMessagesAsRead(true);
ImapIdleChannelAdapter imapIdleChannelAdapter = new ImapIdleChannelAdapter(mailReceiver);
imapIdleChannelAdapter.setOutputChannel(mailChannel());
imapIdleChannelAdapter.setAutoStartup(true);
imapIdleChannelAdapter.afterPropertiesSet();
return imapIdleChannelAdapter;
}
#ServiceActivator(inputChannel = "mailChannel")
public void receive(String mail) {
log.warn(mail);
}
private Properties javaMailProperties() {
Properties javaMailProperties = new Properties();
javaMailProperties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
javaMailProperties.setProperty("mail.imap.socketFactory.fallback", "false");
javaMailProperties.setProperty("mail.store.protocol", "imaps");
javaMailProperties.setProperty("mail.debug", "true");
javaMailProperties.setProperty("mail.imap.ssl", "true");
return javaMailProperties;
}
`
The problem was due to wrong bean initialization. Full version that works OK:
#Slf4j
#Configuration
#EnableIntegration
public class MyMailAdapter {
#Autowired
EmailConfig emailCfg;
#Bean
public SubscribableChannel mailChannel() {
log.info("Channel ready");
return MessageChannels.direct().get();
}
#Bean
public ImapMailReceiver receiver() {
ImapMailReceiver mailReceiver = new ImapMailReceiver(emailCfg.getImapUrl());
mailReceiver.setJavaMailProperties(javaMailProperties());
mailReceiver.setShouldDeleteMessages(false);
mailReceiver.setShouldMarkMessagesAsRead(true);
return mailReceiver;
}
#Bean
public ImapIdleChannelAdapter adapter() {
ImapIdleChannelAdapter imapIdleChannelAdapter = new ImapIdleChannelAdapter(receiver());
imapIdleChannelAdapter.setOutputChannel(mailChannel());
imapIdleChannelAdapter.afterPropertiesSet();
return imapIdleChannelAdapter;
}
#ServiceActivator(inputChannel = "mailChannel")
public void receive(Message<MimeMessage> mail) throws MessagingException {
log.info(mail.getPayload().toString());
}
private Properties javaMailProperties() {
Properties javaMailProperties = new Properties();
javaMailProperties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
javaMailProperties.setProperty("mail.imap.socketFactory.fallback", "false");
javaMailProperties.setProperty("mail.store.protocol", "imaps");
javaMailProperties.setProperty("mail.debug", "true");
javaMailProperties.setProperty("mail.imap.ssl", "true");
return javaMailProperties;
}
}
I don't know what's exactly wrong with your code but I will suggest you few approches that could help you.
Firstly I suggest you to use java DSL in java based configuration. It will provide you nice way to directly specific flow of your integration application (and avoid simply mistakes). For example for spliiter and service activator:
#Bean
public IntegrationFlow yourFlow(AbstractMessageSplitter splitter,
MessageHandler handler) {
return
IntegrationFlows
.from(CHANNEL)
.split(splitter)
.handle(handler).get();
}
Secondly it's generally bad idea to directly specify message type to String. Try something like this (why String?):
#ServiceActivator(inputChannel = "mailChannel")
public void receive(Message<?> message) {
/* (String) message.getPayload() */
}
Maybe it's not a case but let's check it.

Spring Integration - Handling stale sftp sessions

I have implemented the following scenario:
A queueChannel holding Messages in form of byte[]
A MessageHandler, polling the queue channel and uploading files over sftp
A Transformer, listening to errorChannel and sending extracted payload from the failed message back to the queueChannel (thought of as an error handler to handle failed messages so nothing gets lost)
If the sftp server is online, everything works as expected.
If the sftp server is down, the errormessage, that arrives as the transformer is:
org.springframework.messaging.MessagingException: Failed to obtain pooled item; nested exception is java.lang.IllegalStateException: failed to create SFTP Session
The transformer cannot do anything with this, since the payload's failedMessage is null and throws an exception itself. The transformer looses the message.
How can I configure my flow to make the tranformer get the right message with the corresponding payload of the unsucsesfully uploaded file?
My Configuration:
#Bean
public MessageChannel toSftpChannel() {
final QueueChannel channel = new QueueChannel();
channel.setLoggingEnabled(true);
return new QueueChannel();
}
#Bean
public MessageChannel toSplitter() {
return new PublishSubscribeChannel();
}
#Bean
#ServiceActivator(inputChannel = "toSftpChannel", poller = #Poller(fixedDelay = "10000", maxMessagesPerPoll = "1"))
public MessageHandler handler() {
final SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory());
handler.setRemoteDirectoryExpression(new LiteralExpression(sftpRemoteDirectory));
handler.setFileNameGenerator(message -> {
if (message.getPayload() instanceof byte[]) {
return (String) message.getHeaders().get("name");
} else {
throw new IllegalArgumentException("byte[] expected in Payload!");
}
});
return handler;
}
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
final DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
final Properties jschProps = new Properties();
jschProps.put("StrictHostKeyChecking", "no");
jschProps.put("PreferredAuthentications", "publickey,password");
factory.setSessionConfig(jschProps);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
if (sftpPrivateKey != null) {
factory.setPrivateKey(sftpPrivateKey);
factory.setPrivateKeyPassphrase(sftpPrivateKeyPassphrase);
} else {
factory.setPassword(sftpPasword);
}
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<>(factory);
}
#Bean
#Splitter(inputChannel = "toSplitter")
public DmsDocumentMessageSplitter splitter() {
final DmsDocumentMessageSplitter splitter = new DmsDocumentMessageSplitter();
splitter.setOutputChannelName("toSftpChannel");
return splitter;
}
#Transformer(inputChannel = "errorChannel", outputChannel = "toSftpChannel")
public Message<?> errorChannelHandler(ErrorMessage errorMessage) throws RuntimeException {
Message<?> failedMessage = ((MessagingException) errorMessage.getPayload())
.getFailedMessage();
return MessageBuilder.withPayload(failedMessage)
.copyHeadersIfAbsent(failedMessage.getHeaders())
.build();
}
#MessagingGateway
public interface UploadGateway {
#Gateway(requestChannel = "toSplitter")
void upload(#Payload List<byte[]> payload, #Header("header") DmsDocumentUploadRequestHeader header);
}
Thanks..
Update
#Bean(PollerMetadata.DEFAULT_POLLER)
#Transactional(propagation = Propagation.REQUIRED, isolation = Isolation.READ_COMMITTED)
PollerMetadata poller() {
return Pollers
.fixedRate(5000)
.maxMessagesPerPoll(1)
.receiveTimeout(500)
.taskExecutor(taskExecutor())
.transactionSynchronizationFactory(transactionSynchronizationFactory())
.get();
}
#Bean
#ServiceActivator(inputChannel = "toMessageStore", poller = #Poller(PollerMetadata.DEFAULT_POLLER))
public BridgeHandler bridge() {
BridgeHandler bridgeHandler = new BridgeHandler();
bridgeHandler.setOutputChannelName("toSftpChannel");
return bridgeHandler;
}
The null failedMessage is a bug; reproduced INT-4421.
I would not recommend using a QueueChannel for this scenario. If you use a direct channel, you can configure a retry advice to attempt redeliveries. when the retries are exhausted (if so configured), the exception will be thrown back to the calling thread.
Add the advice to the SftpMessageHandler's adviceChain property.
EDIT
You can work around the "missing" failedMessage by inserting a bridge between the pollable channel and the sftp adapter:
#Bean
#ServiceActivator(inputChannel = "toSftpChannel", poller = #Poller(fixedDelay = "5000", maxMessagesPerPoll = "1"))
public BridgeHandler bridge() {
BridgeHandler bridgeHandler = new BridgeHandler();
bridgeHandler.setOutputChannelName("toRealSftpChannel");
return bridgeHandler;
}
#Bean
#ServiceActivator(inputChannel = "toRealSftpChannel")
public MessageHandler handler() {
final SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory());
handler.setRemoteDirectoryExpression(new LiteralExpression("foo"));
handler.setFileNameGenerator(message -> {
if (message.getPayload() instanceof byte[]) {
return (String) message.getHeaders().get("name");
}
else {
throw new IllegalArgumentException("byte[] expected in Payload!");
}
});
return handler;
}

How to configure a trigger in spring integration flow to get value from a method invoking message source?

How to configure a trigger in spring integration flow to get value from a method invoking message source and start it in another flow ?
#Bean
public IntegrationFlow integrationFlow() {
return IntegrationFlows.from(messageSource,channelSpec -> channelSpec.poller(Pollers.trigger(new SomeTrigger())).handle(...).get()
}
#Bean
public MessageSource<?> messageSource() {
MethodInvokingMessageSource source = new MethodInvokingMessageSource();
source.setObject(new Random());
source.setMethod("nextInt");
}
#Bean
public IntegrationFlow someOtherFlow() {
return IntegrationFlows.from("messageChannel")
///some logic to trigger and get the value of random int
}
The MessageSource has receive() method, so you can do just this:
#Bean
public MessageSource<?> randomIntSource() {
MethodInvokingMessageSource source = new MethodInvokingMessageSource();
source.setObject(new Random());
source.setMethodName("nextInt");
return source;
}
#Bean
public IntegrationFlow someOtherFlow() {
return IntegrationFlows.from("messageChannel")
.handle(randomIntSource(), "receive")
.handle(System.out::println)
.get();
}
pay attention to the .handle(randomIntSource(), "receive").

Spring integration DSL creating Sftp Inbound Adapter in java 1.7

I am creating a sftp inbound flow in spring dsl but when I am trying to test this in junit the mesage is coming null. I think the sftp ibound adapter is not starting properly, but not sure. So I am not able to go further. Can any ony one please provide any pointer to it as I am not able to proceed.
This is my flow....
#Bean
public IntegrationFlow sftpInboundFlow() {
return IntegrationFlows
.from(Sftp.inboundAdapter(this.sftpSessionFactory)
.preserveTimestamp(true).remoteDirectory(remDir)
.regexFilter(".*\\.txt$")
.localFilenameExpression("#this.toUpperCase()")
.localDirectory(new File(localDir))
.remoteFileSeparator("/"),
new Consumer<SourcePollingChannelAdapterSpec>() {
#Override
public void accept(
SourcePollingChannelAdapterSpec e) {
e.id("sftpInboundAdapter")
.autoStartup(true)
.poller(Pollers.fixedRate(1000)
.maxMessagesPerPoll(1));
}
})
.channel(MessageChannels.queue("sftpInboundResultChannel"))
.get();
}
This is my session Factory...
#Autowired
private DefaultSftpSessionFactory sftpSessionFactory;
#Bean
public DefaultSftpSessionFactory sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(
true);
factory.setHost("111.11.12.143");
factory.setPort(22);
factory.setUser("sftp");
factory.setPassword("*******");
return factory;
}
This is my Pollable channel..
#Autowired
private PollableChannel sftpInboundResultChannel;
#Bean
public PollableChannel sftpChannel() {
return new QueueChannel();
}
And this is my Test method..
#Test
public void testSftpInboundFlow() {
Message<?> message = ((PollableChannel) sftpInboundResultChannel).receive(1000); //Not receiving any message
System.out.println("message====" + message);
Object payload = message.getPayload();
File file = (File) payload;
message = this.sftpInboundResultChannel.receive(1000);
file = (File) message.getPayload();
}

Resources