Is there any working example for file writing support of the Spring Integration DSL? I cannot find anything about DSL implementation.(e.g. a handle() step in the integration flow, etc.)
Thanks.
There is a sample in the Reference Manual:
#Bean
public IntegrationFlow fileWritingFlow() {
return IntegrationFlows.from("fileWritingInput")
.enrichHeaders(h -> h.header(FileHeaders.FILENAME, "foo.txt")
.header("directory", new File(tmpDir.getRoot(), "fileWritingFlow")))
.handle(Files.outboundAdapter(m -> m.getHeaders().get("directory")))
.channel(MessageChannels.queue("fileWritingResultChannel"))
.get();
}
The File-Split-FTP may give you some insights too.
Related
Making the plunge into Java from .NET for a long time.
I am looking for is an example on how to periodically download a file, read the text from it and then take some action based on the read using Springs Integration library and the annotation based approach.
I want to pull GTFS formatted zip file from a transit provider. This provider produces a simple text file with a timestamp to indicate the last publishing time.
Specifically the producers of the data publish a text file at:
https://someserver.com/gfts/published.txt
This file has a simple timestamp to indicate when the last time their data file was published.
Then there is the data:
https://someserver.com/gfts/schedule.zip
I have tried to find some examples on how to go about polling the "published" file. Basically I want to periodically download the file and check the timestamp to determine if the schedule should be downloaded.
Most of the examples I have seen are using the XML based configuration with spring - and I barely am holding onto the annotation based. I have also seen examples of downloading a file using FTP / SFTP.
I need to use http AND I also need to include Basic Authorization (in the header).
This is as far as I have gotten. I am not sure how to go about wiring this up?
From the Spring Integration docs - this is how I am supposed to declare an outbound gateway (I think that is what I need?)
The question is now what? I need that HttpRequestExecutingMessageHandler to save the stream (file) a local file so I can read the contents and take some other action?
#Configuration
#EnableIntegration
public class GtfsConfiguration {
#Bean
public MessageChannel fileUpdateChannel () {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "fileUpdateChannel", polling = #Poller(fixedDelay="5000")
public HttpRequestExecutingMessageHandler fileUpdateGateway() {
HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler("https://someserver.com/gtfs/raw/published.txt");
handler.setHttpMethod(HttpMethod.GET);
handler.setExpectedResponseType(byte[].class);
return handler;
}
}
If you need to download such a file periodically, you need to use a "fake" Inbound Channel Adapter, for example:
#Bean
#InboundChannelAdapter(value = "fileUpdateChannel"
poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public String downloadFileSchedule() {
return () -> "";
}
The #ServiceActivator for the HttpRequestExecutingMessageHandler is going to be called every second. You don't need to have there a #Poller on the #ServiceActivator. It isn't going to do anything by itself. Plus your fileUpdateChannel is a DirectChannel, not QueueChannel.
I don't think you need to save a downloaded file locally. I even would say that handler.setExpectedResponseType(String.class); is fully enough to get a file content as a reply message payload for downstream analyze.
The easiest way to configure a Basic Authorization is with the Apache Commons HTTP Client:
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials
= new UsernamePasswordCredentials("user1", "user1Pass");
provider.setCredentials(AuthScope.ANY, credentials);
HttpClient client = HttpClientBuilder.create()
.setDefaultCredentialsProvider(provider)
.build();
and use this in the HttpComponentsClientHttpRequestFactory, which you then should inject into the mentioned HttpRequestExecutingMessageHandler via its setRequestFactory(ClientHttpRequestFactory requestFactory).
I am using SpringBoot 2.0 with Spring Integration 5.0.3 and have an issue with my HTTP.inboundGateway. My goal is to validate the JSON posted to the gateway, because the request pojo consists of mandatory fields.
#Bean
public IntegrationFlow notifyUpdateVehicleFlow() {
return IntegrationFlows.from(Http.inboundGateway("/update")
.requestMapping(r -> r.methods(HttpMethod.POST))
.requestPayloadType(RequestPojo.class)
.requestChannel(updateChannel())
.replyChannel(updateReplyChannel()))
.get();
}
Is there an easy way to validate fields in the pojo have been set? What I have already tested is using #NotNull SpringValidation but it seems not to be supported with Spring Integration.
Greetings,
smoothny
There is no such a functionality in the Spring Integration. You can use .filter() downstream that Http.inboundGateway() and really perform Validator.validate() from there on the payload.
If you think it must be done somehow on the Http.inboundGateway() and you have strong requirements and clean description, feel free to raise a JIRA on the matter and we will discuss what can be done from the Framework perspective.
In the current release 1.0.2 of spring integration dsl I can see some of the basic channels are not present like ReST/HTTP, TCP/UDP, JDBC, MQTT, etc.
Just wanted to know whether this protocols/channel are in roadmap or it has been excluded deliberately.
PS: I might be sounding stupid with posted question but just wanted to know the reason.
From one side you should understand that it is enough big work to address them all. For example HTTP module is on our radar for the 1.1 release.
From other side the Spring Integration Java DSL is just an edition to the existing Spring Java & Annotation configuration, so any #Bean definition is valid there, too.
With those desired protocols you can go ahead and configure their components as #Bean and refer them from the .handle() or .from() EIP-methods.
For example:
#Bean
public MessageSource<Object> jdbcMessageSource() {
return new JdbcPollingChannelAdapter(this.dataSource, "SELECT * FROM foo");
}
#Bean
public IntegrationFlow myFlow() {
return IntegrationFlows.from(jdbcMessageSource())
.split(...)
.transform(...)
.handle(new MqttPahoMessageHandler("tcp://localhost:1883", "si-test-out"))
.get();
}
Can we use somehow "subflows" in Spring Integration?
I have many different processes which would use the same "subflow". These processes have always the same part which would be good to be put into a separate file.
What would be the corrent way to implement these flows?
I tried to find a solution to use subflows in Spring Integration but I could not find anything.
One simple technique is to put the subflow in a separate file with "well-known" input and output channels (the subflow starts with one channel and ends with another). Then, simply <import/> the subflow and send/consume to/from the input/output channel.
Or, instead of an import you can use the Java DSL to define the subflow and add it to your application contexts that need the subflow...
#Configuration
public class MySubflowDefinition {
#Bean
public IntegrationFlow subflow() {
return IntegrationFlows.from("someInChannel")
.transform(...)
...
.channel("someOutChannel")
.get();
}
}
For a more formal "subflow" definition, see the spring-integration-flow extension. This solution also allows the same subflow to be invoked from multiple places in the same application context.
spring-integration-java-dsl and spring-integration-flow are both available in the spring repo and maven central with (currently) versions 1.0.0.RELEASE.
I have a simple application that injects another component
#ComponentScan
#EnableAutoConfiguration
#Configuration
class Application {
static void main(String[] args) {
SpringApplication.run(Application, args)
}
#Bean
AuthorizationServerTokenServices tokenServices() {
return MY THING HERE
}
}
I'd like a quick/minimal way to new this up and grab the item springboot wires up (for tokenServices in this example). I'm trying to get at this to verify some configuration/settings/etc using TestNG
I should also say that I"m not using any xml to configure this (using gradle/groovy/springboot)
You can easily introduce conditional bean with the help of Spring profiles.
In your case the code would look like:
#Configuration
#Profile("tokenService")
public TestTokenServiceConfig {
#Primary
#Bean
AuthorizationServerTokenServices tokenServices() {
//implementation
}
}
The custom implementation you supply in this class will only be used by Spring in case the profile tokenService is active. The use of #Primary is needed in order to make Spring use the specified bean instead of any others present in the application context.
Also note that since you are going to be using the custom service in a test environment, you could easily mock the implementation using Mockito (or whatever other mocking framework you prefer)
And the actual integration test would be something like:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = Application.class)
#ActiveProfiles("tokenService")
class YourIntegrationTest {
#Autowired
AuthorizationServerTokenServices tokenServices;
//test
}