How to configure SFTP Outbound Gateway using Java Config? - spring-integration

I would like to use the SFTP Outbound Gateway to get a file via SFTP but I only find examples using XML config. How can this be done using Java config?
Update (Thanks to Artem Bilan help)
MyConfiguration class:
#Configuration
public class MyConfiguration {
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory sftpSessionFactory = new DefaultSftpSessionFactory();
sftpSessionFactory.setHost("myhost");
sftpSessionFactory.setPort(22);
sftpSessionFactory.setUser("uname");
sftpSessionFactory.setPassword("pass");
sftpSessionFactory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(sftpSessionFactory);
}
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "get", "#getPayload() == '/home/samadmin/test.endf'");
sftpOutboundGateway.setLocalDirectory(new File("C:/test/gateway/"));
return sftpOutboundGateway;
}
}
My application class:
#SpringBootApplication
#EnableIntegration
public class TestIntegrationApplication {
public static void main(String[] args) {
SpringApplication.run(TestIntegrationApplication.class, args);
}
}
Configuration succeeds now but no SFTP occurs. Need to figure out how to request SFTP.

Quoting Reference Manual:
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
return new SftpOutboundGateway(ftpSessionFactory(), "ls");
}
Also pay attention to the Java DSL sample in the next section there.
EDIT
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "get", "payload");
sftpOutboundGateway.setLocalDirectory(new File("C:/test/gateway/"));
return sftpOutboundGateway;
}
In case of GET SFTP command the expression ctor arg maybe like above - just a reference to the Message.getPayload() for all incoming messages.
In this case you should send to the sftpChannel a Message like:
new GenericMessage<>("/home/samadmin/test.endf");
So, that /home/samadmin/test.endf is a payload of that Message. When it arrives to the SftpOutboundGateway, that expression is evaluated against that message and getPayload() is called by SpEL. So, the GET command will be preformed with the desired path to the remote file.
An other message may have fully different path to some other file.

Related

Spring Integration Wiretap with Annotations (4.3 Spring Integration)

I have a Spring Integration (4.3) application and i have a JMS receiver which sends the received message to a ServiceActivator.
I am trying to Wiretap a channel, and send the JMS message to Logger (which in this case is Service Activator as well)
I get a strange behavior with the code below.
Every odd message sent (first, third) - it only goes to the logger.
Every even message sent (second, fourth, etc) - only goes to the ServiceActivator that is supposed to do some work and in the future sent the message downstream.
Here are logs
2018-07-23 16:14:43.278 INFO 16532 --- [ container-1] zzz : MSG1
Received via barChannel: MSG2
How can i change my code so all messages go both to the logger and the ServiceActivator that is supposed to do the work?
Thanks a lot in advance
#Bean
public ConnectionFactory jmsConnectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
#Bean
public JmsMessageDrivenEndpoint inbound() {
JmsMessageDrivenEndpoint jmsMessageDrivenEndpoint = new JmsMessageDrivenEndpoint(container(), listener());
jmsMessageDrivenEndpoint.setSessionAcknowledgeMode("transacted");
jmsMessageDrivenEndpoint.setAutoStartup(true);
return jmsMessageDrivenEndpoint;
}
#Bean
public DefaultMessageListenerContainer container() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConnectionFactory());
container.setDestinationName("foo.bar");
return container;
}
#Bean
public ChannelPublishingJmsMessageListener listener() {
ChannelPublishingJmsMessageListener listener = new ChannelPublishingJmsMessageListener();
listener.setRequestChannelName("myLogChannel");
return listener;
}
#ServiceActivator(inputChannel = "myLogChannel")
public void bar(String in) {
System.out.println("Received via barChannel: " + in);
}
#Bean
public MessageChannel TappingChannel(MessageChannel myLogChannel) {
DirectChannel d = new DirectChannel();
d.addInterceptor(new WireTap("myLogChannel"));
return d;
}
#Bean
#ServiceActivator(inputChannel = "myLogChannel")
public MessageHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler(LoggingHandler.Level.INFO.name());
loggingHandler.setLoggerName("zzz");
return loggingHandler;
}
#Bean
public MessageChannel myLogChannel() {
return new DirectChannel();
}
EDIT: Here is the original code that works
#Bean
public MessageChannel toRouter() {
return new DirectChannel();
}
#Bean
public ConnectionFactory jmsConnectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
#Bean
public JmsMessageDrivenEndpoint inbound(ConnectionFactory jmsConnectionFactory) {
return new JmsMessageDrivenEndpoint(container(jmsConnectionFactory), listener());
}
#Bean
public DefaultMessageListenerContainer container(ConnectionFactory jmsConnectionFactory) {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(jmsConnectionFactory);
container.setDestinationName("foo.bar");
return container;
}
#Bean
public ChannelPublishingJmsMessageListener listener() {
ChannelPublishingJmsMessageListener listener = new ChannelPublishingJmsMessageListener();
listener.setRequestChannelName("toRouter");
return listener;
}
#ServiceActivator(inputChannel = "toRouter")
public void bar(String in) {
System.out.println("Received via barChannel: " + in);
}
I want to do the Wiretap logging on this code
You have a mess in your config.
First of all you have two #ServiceActivator for the same myLogChannel DirectChannel.
By default it uses a round-robin dispatching strategy. That's why you see that odd/even behavior between your two consumers on the same channel.
You separate TappingChannel doesn't bring any value and it is out of use at all. No one sends message to this channel: you send to the listener.setRequestChannelName("myLogChannel"); directly though... Therefore a WireTap isn't performed at all as well.
I'm not sure how to help you since it isn't clear what you are going to use as a main channel and what should be used from the wire-tap.
UPDATE
So, you need to configure that new WireTap("myLogChannel") on this toRouter channel.

ServiceActivator does not receive message from ImapIdleChannelAdapter

ServiceActivator does not receive messages from ImapIdleChannelAdapter...
JavaMail logs successful FETCH, but MIME messages do not get delivered to SA endpoint... I want to understand what is wrong in my code.
A7 FETCH 1:35 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1 FETCH (ENVELOPE ("Fri....
Code snippet below:
`
#Autowired
EmailConfig emailCfg;
#Bean
public SubscribableChannel mailChannel() {
return MessageChannels.direct().get();
}
#Bean
public ImapIdleChannelAdapter getMailAdapter() {
ImapMailReceiver mailReceiver = new ImapMailReceiver(emailCfg.getImapUrl());
mailReceiver.setJavaMailProperties(javaMailProperties());
mailReceiver.setShouldDeleteMessages(false);
mailReceiver.setShouldMarkMessagesAsRead(true);
ImapIdleChannelAdapter imapIdleChannelAdapter = new ImapIdleChannelAdapter(mailReceiver);
imapIdleChannelAdapter.setOutputChannel(mailChannel());
imapIdleChannelAdapter.setAutoStartup(true);
imapIdleChannelAdapter.afterPropertiesSet();
return imapIdleChannelAdapter;
}
#ServiceActivator(inputChannel = "mailChannel")
public void receive(String mail) {
log.warn(mail);
}
private Properties javaMailProperties() {
Properties javaMailProperties = new Properties();
javaMailProperties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
javaMailProperties.setProperty("mail.imap.socketFactory.fallback", "false");
javaMailProperties.setProperty("mail.store.protocol", "imaps");
javaMailProperties.setProperty("mail.debug", "true");
javaMailProperties.setProperty("mail.imap.ssl", "true");
return javaMailProperties;
}
`
The problem was due to wrong bean initialization. Full version that works OK:
#Slf4j
#Configuration
#EnableIntegration
public class MyMailAdapter {
#Autowired
EmailConfig emailCfg;
#Bean
public SubscribableChannel mailChannel() {
log.info("Channel ready");
return MessageChannels.direct().get();
}
#Bean
public ImapMailReceiver receiver() {
ImapMailReceiver mailReceiver = new ImapMailReceiver(emailCfg.getImapUrl());
mailReceiver.setJavaMailProperties(javaMailProperties());
mailReceiver.setShouldDeleteMessages(false);
mailReceiver.setShouldMarkMessagesAsRead(true);
return mailReceiver;
}
#Bean
public ImapIdleChannelAdapter adapter() {
ImapIdleChannelAdapter imapIdleChannelAdapter = new ImapIdleChannelAdapter(receiver());
imapIdleChannelAdapter.setOutputChannel(mailChannel());
imapIdleChannelAdapter.afterPropertiesSet();
return imapIdleChannelAdapter;
}
#ServiceActivator(inputChannel = "mailChannel")
public void receive(Message<MimeMessage> mail) throws MessagingException {
log.info(mail.getPayload().toString());
}
private Properties javaMailProperties() {
Properties javaMailProperties = new Properties();
javaMailProperties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
javaMailProperties.setProperty("mail.imap.socketFactory.fallback", "false");
javaMailProperties.setProperty("mail.store.protocol", "imaps");
javaMailProperties.setProperty("mail.debug", "true");
javaMailProperties.setProperty("mail.imap.ssl", "true");
return javaMailProperties;
}
}
I don't know what's exactly wrong with your code but I will suggest you few approches that could help you.
Firstly I suggest you to use java DSL in java based configuration. It will provide you nice way to directly specific flow of your integration application (and avoid simply mistakes). For example for spliiter and service activator:
#Bean
public IntegrationFlow yourFlow(AbstractMessageSplitter splitter,
MessageHandler handler) {
return
IntegrationFlows
.from(CHANNEL)
.split(splitter)
.handle(handler).get();
}
Secondly it's generally bad idea to directly specify message type to String. Try something like this (why String?):
#ServiceActivator(inputChannel = "mailChannel")
public void receive(Message<?> message) {
/* (String) message.getPayload() */
}
Maybe it's not a case but let's check it.

how to use SFTP Outbound Gateway 'mget' command to download files?

I want to use the 'mget' command to download files from sftp server. here's my java config:
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
....
return new CachingSessionFactory<>(factory);
}
#Bean(name = "lsGateway")
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handlerLs() {
// call 'mget' command to download all the files in server folder
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "mget", "payload");
sftpOutboundGateway.setLocalDirectory(new File("/local/path"));
return sftpOutboundGateway;
}
gateway interface:
#MessagingGateway
public interface OutboundGatewayOption {
#Gateway(requestChannel = "sftpChannel")
List<File> mget(String dir);
}
execute download:
#Component
public class Step1Tasklet implements Tasklet {
#Autowired
private OutboundGatewayOption gatewayOption;
#Override
public RepeatStatus execute(StepContribution stepContribution,
ChunkContext chunkContext) throws Exception {
// download files in server folder
List<File> files = gatewayOption.mget("/ftp/server/path/");
return RepeatStatus.FINISHED;
}
}
I got this exception :
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'sftpChannel' available
I've google around but cannot found the issue ,is anyone kindly help please!
No bean named 'sftpChannel'
Means you don't have sftpChannel MessageChannel bean yet.
I think the problem comes from the #MessagingGateway bean definition.
What you need to do is just declare that sftpChannel bean:
#Bean
public MessageChannel sftpChannel() {
return new DirectChannel();
}
Right, in the newest Spring Integration this problem has been fixed and MessageChannel is resolved from its name lately on demand.

spring integration sftp error: Dispatcher has no subscribers

I'm trying to use the outbound gateway to download files from sftp server,
my config:
#IntegrationComponentScan
#EnableIntegration
#Configuration
public class FtpConfig {
#Bean(name = "myGateway")
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handlerLs() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "mget", "payload");
sftpOutboundGateway.setLocalDirectory(new File("/Users/xxx/Documents/"));
return sftpOutboundGateway;
}
#MessagingGateway
public interface OutboundGatewayOption {
#Gateway(requestChannel = "sftpChannel")
List<File> mget(String dir);
}
#Bean
public MessageChannel sftpChannel() {
return new DirectChannel();
}
}
and the execute bean:
#Service
public class DownloadService implements InitializingBean{
#Autowired
FtpConfig.OutboundGatewayOption gatewayOption;
#Override
public void afterPropertiesSet() throws Exception {
List<File> files = gatewayOption.mget("/sftp/server/path");
}
}
and I got this exception:org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application.sftpChannel'.;
Qestion :how can I add the 'subscribers' ?
You can't perform messaging operator in the afterPropertiesSet(). That's too early : some beans might not be initialized yet. And that exception confirms the problem.
You have to implement SmartLifecicle instead and do the same in the start().

spring integration : solutions/tips on connect multiple sftp server?

My spring batch project needs to download files from multiple sftp servers.
the sftp host/port/filePath is config in application.properties file. I consider using the spring integration 'sftp out-bound gateway' to connect these servers and download files. but Im don't how to do this kind of configuration(I'm using java config, ) and make it work? i guess I need some way to define multiple session factory according to the number of sftp server info config in application.properties file.
properties file:
sftp.host=host1,host2
sftp.user=user1,user2
sftp.pwd=pwd1,pwd2
config class:
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory1() {
...
}
#Bean(name = "myGateway1")
#ServiceActivator(inputChannel = "sftpChannel1")
public MessageHandler handler1() {
...
}
#MessagingGateway
public interface DownloadGateway1 {
#Gateway(requestChannel = "sftpChannel1")
List<File> start(String dir);
}
#Bean(name="sftpChannel1")
public MessageChannel sftpChannel1() {
return new DirectChannel();
}
Right, the server is specified in the session factory, not the gateway. The framework does provide a delegating session factory, allowing it to be selected from one of the configured factories for each message sent to the gateway. See Delegating Session Factory.
EDIT
Here's an example:
#SpringBootApplication
public class So46721822Application {
public static void main(String[] args) {
SpringApplication.run(So46721822Application.class, args);
}
#Value("${sftp.name}")
private String[] names;
#Value("${sftp.host}")
private String[] hosts;
#Value("${sftp.user}")
private String[] users;
#Value("${sftp.pwd}")
private String[] pwds;
#Autowired
private DelegatingSessionFactory<?> sessionFactory;
#Autowired
private SftpGateway gateway;
#Bean
public ApplicationRunner runner() {
return args -> {
try {
this.sessionFactory.setThreadKey("one"); // use factory "one"
this.gateway.send(new File("/tmp/f.txt"));
}
finally {
this.sessionFactory.clearThreadKey();
}
};
}
#Bean
public DelegatingSessionFactory<LsEntry> sessionFactory() {
Map<Object, SessionFactory<LsEntry>> factories = new LinkedHashMap<>();
for (int i = 0; i < this.names.length; i++) {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(this.hosts[i]);
factory.setUser(this.users[i]);
factory.setPassword(this.pwds[i]);
factories.put(this.names[i], factory);
}
// use the first SF as the default
return new DelegatingSessionFactory<LsEntry>(factories, factories.values().iterator().next());
}
#ServiceActivator(inputChannel = "toSftp")
#Bean
public SftpMessageHandler handler() {
SftpMessageHandler handler = new SftpMessageHandler(sessionFactory());
handler.setRemoteDirectoryExpression(new LiteralExpression("foo"));
return handler;
}
#MessagingGateway(defaultRequestChannel = "toSftp")
public interface SftpGateway {
void send(File file);
}
}
with properties...
sftp.name=one,two
sftp.host=host1,host2
sftp.user=user1,user2
sftp.pwd=pwd1,pwd2

Resources