I am using Spring Batch Integration as I have to perform certain works before invoking the batch.
I have done the work using Spring Integration and invoke the batch using job-launching-gateway and it works like a charm.
Now the question is how to pass the data from Spring Integration to Batch?
Suggestions please.
The message sent to the JobLaunchingGateway has a JobLaunchRequest payload.
When you build the request in your Spring Integration flow, you can add JobParameters which are then used within the batch job.
EDIT
For custom objects you would have to do it indirectly. For example via a Map.
If you are using XML configuration, you can use a:
<util:map id="jobParams"> ... </...>
Stuff your object into the map under some key, pass the key as a JobParameter, #Autowire the map into your batch job and retrieve/remove the object using the key.
Related
I need to inject a callContext object as thread Local in spring integration flow. A traceId which is received in jms header should be extracted n set in the callContext object so that it gets printed in log which is configured at project level.
IntegrationFlows.from(Jms.messageDrivenChannelAdapter(connectionFactory).destination(topicName))
.log(INFO, m-> “message received for: + ((Order)m.getPayload()).getOrderId())
.handle(orderService)
.get();
You can just do it in your orderService, or add another service just before it.
I would say a Function for log() operator provides you full control over what you would like to log in the message. Of course, also avoiding overhead with the ThreadLocal. But if you still would like to use it, I would suggest a .wireTap() before that log(), so you can store a value in your ThreadLocal (MDC?) without effecting the main flow.
Only the problem that you have to remember to clean that callContext somehow to avoid thread local pollution.
I have set up a simple Spring Integration flow which is composed of such steps:
poll a rest api periodically then
do some processing on the payload
and land it on a Kafka topic.
Please observe the code below:
#Component
public class MyIntegrationFlow extends IntegrationFlowAdapter {
#Override
protected IntegrationFlowDefinition<?> buildFlow() {
return from(() -> List.of("pathVariable1", "pathVariable2"), c -> c.poller(Pollers.fixedDelay(5, TimeUnit.SECONDS)))
.split()
.handle(httpRequest(), c -> c.advice(new RequestHandlerRetryAdvice()))
.transform(Tranformers.fromJson(Foo.class))
.filter(payload -> payload.isValid())
.log()
.transform(Tranformers.toJson())
.channel(Source.OUTPUT); // output channel for kafka topic
}
private HttpMessageHandlerSpec httpRequest() {
return Http.outboundGateway("http://somehost:8080/{pathVariable}")
.httpMethod(GET)
.uriVariable("pathVariable", Message::getPayload)
.expectedResponseType(String.class);
}
}
This works brilliantly, however, I am struggling to come up with some good tests.
How am I supposed to mock the external REST API?
How am I supposed to test that the retry policy does kick in and the desired number of http requests are made?
How do I change the MessageSource of the flow (list of path vars) that is polled periodically?
How do I check if the payload has successfully made it to the Kafka topic?
Too much questions and some of them requires too broad explanation. Anyway I think you can start from Spring Integration Testing Framework and its documentation.
How am I supposed to mock the external REST API?
I think you can just consider to use a Mock MVC from Spring Framework and its MockMvcClientHttpRequestFactory to inject into the HttpRequestExecutingMessageHandler based on the HttpMessageHandlerSpec.
retry policy does kick
Well, I guess the same mocked MVC endpoint can verify how many times it has been called and fail for first several times to initiate that retry.
How do I change the MessageSource
This is exactly a part of Spring Integration Testing Framework with its MockIntegration.mockMessageSource() and MockIntegrationContext: https://docs.spring.io/spring-integration/docs/5.1.6.RELEASE/reference/html/#mockintegration
made it to the Kafka topic?
Or you the mentioned MockIntegration.mockMessageHandler() to verify that endpoint for Kafka is called. Or use an Embedded Kafka from Spring Kafka project: https://docs.spring.io/spring-kafka/docs/2.2.7.RELEASE/reference/html/#embedded-kafka-annotation
I have a spring batch project. I would like to mock up the JobBuilderFactory for junit test. When I do a when().thenReturn I get that the JobBuilderFactory is null.
You can find a complete example of how to test a Spring Batch job in the End-To-End Testing of Batch Jobs section of the reference documentation. As you will see, there is no need to mock the JobBuilderFactory.
Scenario: I have 3 spring cloud streaming apps
1'st: unmarshalls XML payload into JAXB object
2'nd: Converts JAXB payload into our domain POJO
3'rd: Validate domain object
I am trying to test the 3'rd app. I have included the 1'st and 2'nd applications as test dependencies. I have added:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-test-support</artifactId>
<scope>test</scope>
</dependency>
Now I have about 20 xml files with various validation scenario's. The first test works fine. I am able pick up the expected message of the channel using:
final Message<PaymentInstruction> mceMessage =
(Message<PaymentInstruction>) collector.forChannel(
validationBindings.mce()).take();
The 2'nd test that is run is where I have an issue. The test just sits at 'take'.
I have done some digging in the spring-integration-core-4.3.8.jar and have traced the issue to org.springframework.integration.dispatcher.AbstractDispatcher:
#Override
public synchronized boolean addHandler(MessageHandler handler) {
Assert.notNull(handler, "handler must not be null");
Assert.isTrue(this.handlers.size() < this.maxSubscribers, "Maximum subscribers exceeded");
boolean added = this.handlers.add(handler);
if (this.handlers.size() == 1) {
this.theOneHandler = handler;
}
else {
this.theOneHandler = null;
}
return added;
}
There is a handler that was added for the first test, so it assigns null to 'this.theOneHandler'
My options are:
Refactor the code in the other 2 projects so that I can do the unmarshalling and creating of my domain object without the need for the spring cloud app code
I can create an individual unit test class per test case, however I'd rather not go this route as the startup time for spring boot will be high that will be multiplied by the number of test cases
Do I have some missing configuration that would allow me to have these multiple handlers or am I breaking the way I want to use spring cloud streaming?
Environment:
Java 8 Update 131
org.springframework.cloud:spring-cloud-dependencies:Dalston.RELEASE
org.springframework.boot:spring-boot-dependencies:1.5.2.RELEASE
theOneHandler is a dispatching optimization used when a channel has only one subscriber.
It sounds like you are binding all your services to the same channel, which is not what you want.
Bottom line is you can't do what you are trying to do because each of the services are using the same channel name (presumably input).
You would need to load each service in a separate application context and wire them together with bridges.
EDIT
You could also test them using an aggregated application.
I have a ws:outbound-gateway in place pointing to a org.springframework.ws.transport.jms.JmsMessageSender class in order to push a Soap message into the queue.
The output message has been generated okay and published into the queue normally with the following JMS properties on it: SOAPJMS_soapAction, SOAPJMS_contentLength, SOAPJMS_contentType, etc.
My question is: how can I add a custom JMS property as part of the JMS properties generated by default? Is this possible? I'm using Spring Integration 4.3.5.RELEASE.
The JmsMessageSender can be supplied with the MessagePostProcessor.
The you can supply any desired JMS property on target Message.