I want to use the 'mget' command to download files from sftp server. here's my java config:
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
....
return new CachingSessionFactory<>(factory);
}
#Bean(name = "lsGateway")
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handlerLs() {
// call 'mget' command to download all the files in server folder
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "mget", "payload");
sftpOutboundGateway.setLocalDirectory(new File("/local/path"));
return sftpOutboundGateway;
}
gateway interface:
#MessagingGateway
public interface OutboundGatewayOption {
#Gateway(requestChannel = "sftpChannel")
List<File> mget(String dir);
}
execute download:
#Component
public class Step1Tasklet implements Tasklet {
#Autowired
private OutboundGatewayOption gatewayOption;
#Override
public RepeatStatus execute(StepContribution stepContribution,
ChunkContext chunkContext) throws Exception {
// download files in server folder
List<File> files = gatewayOption.mget("/ftp/server/path/");
return RepeatStatus.FINISHED;
}
}
I got this exception :
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'sftpChannel' available
I've google around but cannot found the issue ,is anyone kindly help please!
No bean named 'sftpChannel'
Means you don't have sftpChannel MessageChannel bean yet.
I think the problem comes from the #MessagingGateway bean definition.
What you need to do is just declare that sftpChannel bean:
#Bean
public MessageChannel sftpChannel() {
return new DirectChannel();
}
Right, in the newest Spring Integration this problem has been fixed and MessageChannel is resolved from its name lately on demand.
Related
We use spring-batch-integration to process .json files from directory. It's necessary to stop processing after failure, to find out a problem (to change file with problem or other solutions) and then continue. Current configuration continue polling after error. How to change it? or maybe there are different approach for such scenario.
#Configuration
#IntegrationComponentScan
#EnableIntegration
public class IntegrationConfig {
private #Autowired Job job;
#Bean
#ServiceActivator(inputChannel = "jobChannel",
outputChannel = "errorChannel")
protected JobLaunchingMessageHandler launcher(JobLauncher jobLauncher) {
return new JobLaunchingMessageHandler(jobLauncher);
}
#Bean
public MessageChannel fileInputChannel() {
return new DirectChannel();
}
#Bean
#InboundChannelAdapter(value = "fileInputChannel",
poller = #Poller(fixedDelay = "5000"))
public MessageSource<File> fileReadingMessageSource() {
FileReadingMessageSource source = new FileReadingMessageSource();
source.setDirectory(new File("C:/Temp/myfiles/"));
source.setFilter(new SimplePatternFileListFilter("*.json"));
source.setScanEachPoll(true);
source.setUseWatchService(true);
return source;
}
#Transformer(inputChannel = "fileInputChannel",
outputChannel = "jobChannel")
public JobLaunchRequest transform(File aFile) {
String fileName = aFile.getAbsolutePath();
JobParameters jobParameters =
new JobParametersBuilder().addString("input.file.name", fileName)
.addDate("dateTime", new Date()).toJobParameters();
JobLaunchRequest request = new JobLaunchRequest(job, jobParameters);
return request;
}
}
example was from this article
neither outputChannel = "nullChannel" nor outputChannel = "errorChannel" helps
You need to stop the inbound channel adapter.
You can autowire a SourcePollingChannelAdapter which is registered by the #InboundChannelAdapter annotation.
When you detect the failure, call stop() on the adapter.
I added
#Bean
#DependsOn("fileInputChannel")
#ServiceActivator(inputChannel = "errorChannel",
outputChannel = "nullChanel")
protected ErrorLogger errorLogger(JobLauncher jobLauncher) {
return new ErrorLogger();
}
and
public class ErrorLogger {
private static final Logger logger =
LoggerFactory.getLogger(ErrorLogger.class);
#Autowired
private SourcePollingChannelAdapter fileInputChannel;
#ServiceActivator
public void logError(Message<JobExecution> message) {
JobExecution msgex=message.getPayload();
if (msgex.getStatus() == BatchStatus.FAILED) {
logger.error("Exception " +
msgex.getExitStatus().getExitDescription());
fileInputChannel.stop();
}
}
}
But I get autowiring error in ErrorLogger
Unsatisfied dependency expressed through field 'fileInputChannel'; nested
exception is org.springframework.beans.factory.NoSuchBeanDefinitionException:
No qualifying bean of type
'org.springframework.integration.endpoint.SourcePollingChannelAdapter'
available:
Seems like initialization order issue despite of #DependsOn("fileInputChannel"), because i can autoware it in separate controller without error.
It works only with
#Autowired(required = false)
private SourcePollingChannelAdapter fileInputChannel;
I'm trying to use the outbound gateway to download files from sftp server,
my config:
#IntegrationComponentScan
#EnableIntegration
#Configuration
public class FtpConfig {
#Bean(name = "myGateway")
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handlerLs() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "mget", "payload");
sftpOutboundGateway.setLocalDirectory(new File("/Users/xxx/Documents/"));
return sftpOutboundGateway;
}
#MessagingGateway
public interface OutboundGatewayOption {
#Gateway(requestChannel = "sftpChannel")
List<File> mget(String dir);
}
#Bean
public MessageChannel sftpChannel() {
return new DirectChannel();
}
}
and the execute bean:
#Service
public class DownloadService implements InitializingBean{
#Autowired
FtpConfig.OutboundGatewayOption gatewayOption;
#Override
public void afterPropertiesSet() throws Exception {
List<File> files = gatewayOption.mget("/sftp/server/path");
}
}
and I got this exception:org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application.sftpChannel'.;
Qestion :how can I add the 'subscribers' ?
You can't perform messaging operator in the afterPropertiesSet(). That's too early : some beans might not be initialized yet. And that exception confirms the problem.
You have to implement SmartLifecicle instead and do the same in the start().
My spring batch project needs to download files from multiple sftp servers.
the sftp host/port/filePath is config in application.properties file. I consider using the spring integration 'sftp out-bound gateway' to connect these servers and download files. but Im don't how to do this kind of configuration(I'm using java config, ) and make it work? i guess I need some way to define multiple session factory according to the number of sftp server info config in application.properties file.
properties file:
sftp.host=host1,host2
sftp.user=user1,user2
sftp.pwd=pwd1,pwd2
config class:
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory1() {
...
}
#Bean(name = "myGateway1")
#ServiceActivator(inputChannel = "sftpChannel1")
public MessageHandler handler1() {
...
}
#MessagingGateway
public interface DownloadGateway1 {
#Gateway(requestChannel = "sftpChannel1")
List<File> start(String dir);
}
#Bean(name="sftpChannel1")
public MessageChannel sftpChannel1() {
return new DirectChannel();
}
Right, the server is specified in the session factory, not the gateway. The framework does provide a delegating session factory, allowing it to be selected from one of the configured factories for each message sent to the gateway. See Delegating Session Factory.
EDIT
Here's an example:
#SpringBootApplication
public class So46721822Application {
public static void main(String[] args) {
SpringApplication.run(So46721822Application.class, args);
}
#Value("${sftp.name}")
private String[] names;
#Value("${sftp.host}")
private String[] hosts;
#Value("${sftp.user}")
private String[] users;
#Value("${sftp.pwd}")
private String[] pwds;
#Autowired
private DelegatingSessionFactory<?> sessionFactory;
#Autowired
private SftpGateway gateway;
#Bean
public ApplicationRunner runner() {
return args -> {
try {
this.sessionFactory.setThreadKey("one"); // use factory "one"
this.gateway.send(new File("/tmp/f.txt"));
}
finally {
this.sessionFactory.clearThreadKey();
}
};
}
#Bean
public DelegatingSessionFactory<LsEntry> sessionFactory() {
Map<Object, SessionFactory<LsEntry>> factories = new LinkedHashMap<>();
for (int i = 0; i < this.names.length; i++) {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(this.hosts[i]);
factory.setUser(this.users[i]);
factory.setPassword(this.pwds[i]);
factories.put(this.names[i], factory);
}
// use the first SF as the default
return new DelegatingSessionFactory<LsEntry>(factories, factories.values().iterator().next());
}
#ServiceActivator(inputChannel = "toSftp")
#Bean
public SftpMessageHandler handler() {
SftpMessageHandler handler = new SftpMessageHandler(sessionFactory());
handler.setRemoteDirectoryExpression(new LiteralExpression("foo"));
return handler;
}
#MessagingGateway(defaultRequestChannel = "toSftp")
public interface SftpGateway {
void send(File file);
}
}
with properties...
sftp.name=one,two
sftp.host=host1,host2
sftp.user=user1,user2
sftp.pwd=pwd1,pwd2
I would like to use the SFTP Outbound Gateway to get a file via SFTP but I only find examples using XML config. How can this be done using Java config?
Update (Thanks to Artem Bilan help)
MyConfiguration class:
#Configuration
public class MyConfiguration {
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory sftpSessionFactory = new DefaultSftpSessionFactory();
sftpSessionFactory.setHost("myhost");
sftpSessionFactory.setPort(22);
sftpSessionFactory.setUser("uname");
sftpSessionFactory.setPassword("pass");
sftpSessionFactory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(sftpSessionFactory);
}
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "get", "#getPayload() == '/home/samadmin/test.endf'");
sftpOutboundGateway.setLocalDirectory(new File("C:/test/gateway/"));
return sftpOutboundGateway;
}
}
My application class:
#SpringBootApplication
#EnableIntegration
public class TestIntegrationApplication {
public static void main(String[] args) {
SpringApplication.run(TestIntegrationApplication.class, args);
}
}
Configuration succeeds now but no SFTP occurs. Need to figure out how to request SFTP.
Quoting Reference Manual:
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
return new SftpOutboundGateway(ftpSessionFactory(), "ls");
}
Also pay attention to the Java DSL sample in the next section there.
EDIT
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), "get", "payload");
sftpOutboundGateway.setLocalDirectory(new File("C:/test/gateway/"));
return sftpOutboundGateway;
}
In case of GET SFTP command the expression ctor arg maybe like above - just a reference to the Message.getPayload() for all incoming messages.
In this case you should send to the sftpChannel a Message like:
new GenericMessage<>("/home/samadmin/test.endf");
So, that /home/samadmin/test.endf is a payload of that Message. When it arrives to the SftpOutboundGateway, that expression is evaluated against that message and getPayload() is called by SpEL. So, the GET command will be preformed with the desired path to the remote file.
An other message may have fully different path to some other file.
We are planning to migrate our code from Spring integration XML to DSL. In XML Version, we are using channel name pattern to do tracing.
For Eg: If channel name has *_EL_*, we intercept the channel and do some logging.
How to do this kind or more simpler in Java dsl.
The #GlobalChannelInterceptor is for you. And it is a part of Spring Integration Core.
So, you must do something like this:
#Bean
public MessageChannel bar() {
return new DirectChannel();
}
#Bean
#GlobalChannelInterceptor(patterns = "*_EL_*")
public WireTap baz() {
return new WireTap(this.bar());
}
I mean specify the ChannelInterceptor #Bean and mark it with that annotation to make pattern-based interceptions.
UPDATE
The sample test-case which demonstrate the work for #GlobalChannelInterceptor for the auto-created channel from DSL flows:
#ContextConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
#DirtiesContext
public class SO31573744Tests {
#Autowired
private TestGateway testGateway;
#Autowired
private PollableChannel intercepted;
#Test
public void testIt() {
this.testGateway.testIt("foo");
Message<?> receive = this.intercepted.receive(1000);
assertNotNull(receive);
assertEquals("foo", receive.getPayload());
}
#MessagingGateway
public interface TestGateway {
#Gateway(requestChannel = "testChannel")
void testIt(String payload);
}
#Configuration
#EnableIntegration
#IntegrationComponentScan
public static class ContextConfiguration {
#Bean
public IntegrationFlow testFlow() {
return IntegrationFlows.from("testChannel")
.channel("nullChannel")
.get();
}
#Bean
public PollableChannel intercepted() {
return new QueueChannel();
}
#Bean
#GlobalChannelInterceptor(patterns = "*Ch*")
public WireTap wireTap() {
return new WireTap(intercepted());
}
}
}