I see many examples related to spring-integration-file. But I am looking for an example application where it uses spring-integration-jpa to pull data from database using Inbound Channel Adapter and create a Java object out of it.
Any help is much appreciated.
There is a basic JPA sample in the official Spring Integration Samples repository: https://github.com/spring-projects/spring-integration-samples/tree/master/basic/jpa.
The simple Java DSL sample for Inbound Channel Adapter might look like this:
#Bean
public IntegrationFlow pollingAdapterFlow(EntityManagerFactory entityManagerFactory) {
return IntegrationFlows
.from(Jpa.inboundAdapter(entityManagerFactory)
.entityClass(StudentDomain.class)
.maxResults(1)
.expectSingleResult(true),
e -> e.poller(p -> p.trigger(new OnlyOnceTrigger())))
.channel(c -> c.queue("pollingResults"))
.get();
}
Related
I have set up a simple Spring Integration flow which is composed of such steps:
poll a rest api periodically then
do some processing on the payload
and land it on a Kafka topic.
Please observe the code below:
#Component
public class MyIntegrationFlow extends IntegrationFlowAdapter {
#Override
protected IntegrationFlowDefinition<?> buildFlow() {
return from(() -> List.of("pathVariable1", "pathVariable2"), c -> c.poller(Pollers.fixedDelay(5, TimeUnit.SECONDS)))
.split()
.handle(httpRequest(), c -> c.advice(new RequestHandlerRetryAdvice()))
.transform(Tranformers.fromJson(Foo.class))
.filter(payload -> payload.isValid())
.log()
.transform(Tranformers.toJson())
.channel(Source.OUTPUT); // output channel for kafka topic
}
private HttpMessageHandlerSpec httpRequest() {
return Http.outboundGateway("http://somehost:8080/{pathVariable}")
.httpMethod(GET)
.uriVariable("pathVariable", Message::getPayload)
.expectedResponseType(String.class);
}
}
This works brilliantly, however, I am struggling to come up with some good tests.
How am I supposed to mock the external REST API?
How am I supposed to test that the retry policy does kick in and the desired number of http requests are made?
How do I change the MessageSource of the flow (list of path vars) that is polled periodically?
How do I check if the payload has successfully made it to the Kafka topic?
Too much questions and some of them requires too broad explanation. Anyway I think you can start from Spring Integration Testing Framework and its documentation.
How am I supposed to mock the external REST API?
I think you can just consider to use a Mock MVC from Spring Framework and its MockMvcClientHttpRequestFactory to inject into the HttpRequestExecutingMessageHandler based on the HttpMessageHandlerSpec.
retry policy does kick
Well, I guess the same mocked MVC endpoint can verify how many times it has been called and fail for first several times to initiate that retry.
How do I change the MessageSource
This is exactly a part of Spring Integration Testing Framework with its MockIntegration.mockMessageSource() and MockIntegrationContext: https://docs.spring.io/spring-integration/docs/5.1.6.RELEASE/reference/html/#mockintegration
made it to the Kafka topic?
Or you the mentioned MockIntegration.mockMessageHandler() to verify that endpoint for Kafka is called. Or use an Embedded Kafka from Spring Kafka project: https://docs.spring.io/spring-kafka/docs/2.2.7.RELEASE/reference/html/#embedded-kafka-annotation
I am using following to define my integration flow:
#Bean
public IntegrationFlow pollingFlow(MessageSource<Object> jdbcMessageSource) {
return IntegrationFlows.from(jdbcMessageSource,
c -> c.poller(Pollers.fixedRate(250, TimeUnit.MILLISECONDS)
.maxMessagesPerPoll(1)
.transactional()))
.split()
.channel(taskSourceChannel())
.get();
}
I would like to make call to service activator that reads from taskSourceChannel as transactional. Also, I want to use following with my transaction.
#Bean
public TransactionSynchronizationFactory transactionSynchronizationFactory() {
ExpressionEvaluatingTransactionSynchronizationProcessor syncProcessor
= new ExpressionEvaluatingTransactionSynchronizationProcessor();
syncProcessor.setAfterCommitChannel(successChannel());
syncProcessor.setAfterRollbackChannel(failureChannel());
return new DefaultTransactionSynchronizationFactory(syncProcessor);
}
The taskSourceChannel is an executor channel.
#Bean
public MessageChannel taskSourceChannel() {
return new ExecutorChannel(executor());
}
How can I add transaction support after split while using TransactionSynchronizationFactory. I don't want to make polling transacational. The only solution I can think of is putting transactional on activator but that won't solve my problem. I would like to make it applicable to any service activator uses this channel.
You question is not so clear, but you definitely need to consider to add transaction into the service activator. Although you don't show what is the subscriber for that taskSourceChannel, but you need to think do not have several subscribers on it.
Nevertheless I think your point is to apply TX into the service activator on this taskSourceChannel and everything after that one.
For this purpose Spring Integration provides a TransactionHandleMessageAdvice. See more info the Reference Manual: https://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html#tx-handle-message-advice.
The TransactionSynchronizationFactory is only used from the AbstractPollingEndpoint implementations. However you can still utilize it in your transactional context relying on the TransactionSynchronizationManager.registerSynchronization().
I need to throttle the movement of messages between some JMS (activeMQ) queues to ensure I dont overrun an external service used during message processing.
I have done this in the past with Camel but given that this project is otherwise entirely Spring based, I figured I'd give spring-integration a whirl.
I am happy to see that in 5.0.7 the Java DSL is in core and would really like to use it instead of xml.
But...I cant seem to find good/current docs for using the DSL to do even simple things like create the input and output messageChannels for JMS.
Could anyone point me to any current example of using the java DSL to create channels that I can use to consume and produce messages with...and then later use in a bridge with some throttling applied?
Well, looks like our JMS chapter in the Reference Manual leaks of the Java DSL samples, similar to what we have so far with AMQP, for example:
https://docs.spring.io/spring-integration/docs/5.0.7.RELEASE/reference/html/amqp.html#_configuring_with_the_java_dsl_2
https://docs.spring.io/spring-integration/docs/5.0.7.RELEASE/reference/html/amqp.html#_configuring_with_the_java_dsl_4
I believe we can add similar paragraphs into the JMS chapter as well. Please, raise a JIRA on the matter and we will address it soon. Meanwhile I suggest you to open a org.springframework.integration.jms.dsl.Jms factory for appropriate builder to use.
On the other hand I can suggest you to take a look into the existing test-case of some possible configurations: https://github.com/spring-projects/spring-integration/blob/master/spring-integration-jms/src/test/java/org/springframework/integration/jms/dsl/JmsTests.java
For example to read from the queue you need a configuration like this:
#Bean
public IntegrationFlow jmsMessageDrivenFlow() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(jmsConnectionFactory(), DefaultMessageListenerContainer.class)
.outputChannel(jmsMessageDrivenInputChannel())
.destination("jmsMessageDriven")
.configureListenerContainer(c -> c.clientId("foo")))
.<String, String>transform(String::toLowerCase)
.channel(jmsOutboundInboundReplyChannel())
.get();
}
To send to the JMS you need something like this:
#Bean
public IntegrationFlow jmsOutboundFlow() {
return f -> f
.handle(Jms.outboundAdapter(jmsConnectionFactory())
.destinationExpression("headers." + SimpMessageHeaderAccessor.DESTINATION_HEADER)
.configureJmsTemplate(t -> t.id("jmsOutboundFlowTemplate")));
}
I am refactoring some of my Spring Integration RSS feed code which uses feed inbound channel adaptor to a microservice. I want the feeds to be stored internally in a mongodb database (in case of failures, audit etc) and also to write the feed (in JSON) to a kafka topic for onward processing.
How do I do this with Spring Integration? Do I need a pub/subscribe queue with two handlers?
Any example code using Java DSL would be most helpful.
Not sure if you need pub/sub for such a use-case, but definitely you can use QueueChannel in between Feed Inbound Channel Adapter and Kafka Outbound Channel Adapter. That QueueChannel can really be supplied with the MongoDbChannelMessageStore for persistence in case of failures.
The Java DSL sample is like this:
#Bean
public IntegrationFlow feedFlow(MongoDbChannelMessageStore messageStore) {
return IntegrationFlows
.from(Feed.inboundAdapter(...),
e -> e.poller(p -> p.fixedDelay(1000)))
.channel(c -> c.queue(messageStore, "entries"))
.handle(Kafka.outboundChannelAdapter(...),
e -> e.poller(p -> p.fixedDelay(1000)))
.get();
}
https://docs.spring.io/spring-integration/docs/5.0.3.RELEASE/reference/html/feed.html#feed-java-configuration
https://docs.spring.io/spring-integration/docs/5.0.3.RELEASE/reference/html/mongodb.html#mongodb-priority-channel-message-store
https://docs.spring.io/spring-kafka/docs/2.1.4.RELEASE/reference/html/_spring_integration.html#si-outbound
Do we have support for Outbound Channel Adapter in Spring Integration with annotation based? Please share link for example which supports.
See Programming Tips in the Reference Manual, where we have a sample for XMPP chat message sending:
#Bean
#ServiceActivator(inputChannel = "input")
public MessageHandler sendChatMessageHandler(XMPPConnection xmppConnection) {
ChatMessageSendingMessageHandler handler = new ChatMessageSendingMessageHandler(xmppConnection);
DefaultXmppHeaderMapper xmppHeaderMapper = new DefaultXmppHeaderMapper();
xmppHeaderMapper.setRequestHeaderNames("*");
handler.setHeaderMapper(xmppHeaderMapper);
return handler;
}
The same technique is applied for the FtpMessageHandler.
Also see Annotation Support there as well.