How to handle/assemble segments of a MQ message when using Spring Integration JMS Jms.messageDrivenChannelAdapter. ? I did find some reference on how to do in when using MQ API .
https://medium.com/#marcus_j/consuming-segmented-ibm-mq-messages-in-java-cbdee4a9ad85
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_ALL_SEGMENTS_AVAILABLE
| MQC.MQGMO_SYNCPOINT
| MQC.MQGMO_COMPLETE_MSG;
gmo.matchOptions = MQC.MQMO_NONE;
gmo.waitInterval = 10000;
MQMessage message = new MQMessage();
queue.get(message, gmo);
// Do your stuff with the message
message.clearMessage();
queue.close();
manager.disconnect();
Per my understanding i will have to pass the appropriate MQGetMessageOptions , to be able to ask the queue manager to reassemble the message if it has been segmented. Could not find any reference on how to pass these options when using spring JMS .
Unfortunately IBM MQ does not support Message segmentation in JMS:
This feature is not supported on IBM® MQ for z/OS® or by applications using IBM MQ classes for JMS.
Spring Integration doesn't provide channel adapter implementation for IBM MQ API.
On one hand there is a JMS bridge API over IBM MQ: https://developer.ibm.com/components/ibm-mq/tutorials/mq-develop-mq-jms/
On the other hand (as you already noticed), Spring Integration provides channel adapters for JMS: https://docs.spring.io/spring-integration/docs/current/reference/html/jms.html#jms
So, I would try first to use official IBM JMS client and then reach IBM support for de-segmentation option in their JMS client.
There is indeed nothing what Spring Integration can do.
Although you always can write your own MessageProducerSupport to perform that IBM MQ-specific logic. You need to implement a doStart() and call sendMessage() from there when you done an assembling an IBM MQ message and so on.
Whilst it is true that you won't be able to use out of the box code to apply the gmo options on a #JmsListener or a JmsTemplate, if you have included the MQ JMS Spring Boot Starter as a dependancy -
<dependency>
<groupId>com.ibm.mq</groupId>
<artifactId>mq-jms-spring-boot-starter</artifactId>
<version>2.4.1</version>
</dependency>
Then you do have access to all the MQ Classes that you use in your example. So you can craft your own objects and methods and use them in your Spring application. Although they won't be very JMS Spring like, and the connections won't be managed by Spring.
The difficulty is that get message options are applied at the get, and not on the connection. To override options set on the connection you could have created your own customised connection factory bean, but not for gmo options.
Related
How can I consume Server Sent Events with Spring Integration? I am aware Spring supports SSE with Webflux, but how to convert the incoming Flux into separate Message instances? And possibly wrap this code into some Spring-Integration-Lifecycle-aware component (MessageProducerSupport?)
WebClient client = WebClient.create("http://myhost:8080/sse");
ParameterizedTypeReference<ServerSentEvent<String>> type
= new ParameterizedTypeReference<ServerSentEvent<String>>() {};
Flux<ServerSentEvent<String>> eventStream = client.get()
.uri("/stream-sse")
.retrieve()
.bodyToFlux(type);
eventStream.subscribe(
content -> ;/* here I believe the message should be produced/sent to a channel */ );
See Spring Integration WebFlux Outbound Gateway: https://docs.spring.io/spring-integration/docs/current/reference/html/webflux.html#webflux-outbound:
The setExpectedResponseType(Class<?>) or setExpectedResponseTypeExpression(Expression) identifies the target type of the response body element conversion. If replyPayloadToFlux is set to true, the response body is converted to a Flux with the provided expectedResponseType for each element, and this Flux is sent as the payload downstream. Afterwards, you can use a splitter to iterate over this Flux in a reactive manner.
WebFlux.outboundGateway("http://myhost:8080/sse/stream-sse")
.httpMethod(HttpMethod.GET)
.replyPayloadToFlux(true)
.setExpectedResponseTypeExpression(new ParameterizedTypeReference<ServerSentEvent<String>>() {})
To make it start working just after an application is ready, yo can implement an ApplicationRunner to send a "void" message into a channel for the flow with that WebFlux.outboundGateway(). I don't think we need a special, dedicated component just for SSE requesting and producing. The combination of existing components is fully enough.
I have set up a simple Spring Integration flow which is composed of such steps:
poll a rest api periodically then
do some processing on the payload
and land it on a Kafka topic.
Please observe the code below:
#Component
public class MyIntegrationFlow extends IntegrationFlowAdapter {
#Override
protected IntegrationFlowDefinition<?> buildFlow() {
return from(() -> List.of("pathVariable1", "pathVariable2"), c -> c.poller(Pollers.fixedDelay(5, TimeUnit.SECONDS)))
.split()
.handle(httpRequest(), c -> c.advice(new RequestHandlerRetryAdvice()))
.transform(Tranformers.fromJson(Foo.class))
.filter(payload -> payload.isValid())
.log()
.transform(Tranformers.toJson())
.channel(Source.OUTPUT); // output channel for kafka topic
}
private HttpMessageHandlerSpec httpRequest() {
return Http.outboundGateway("http://somehost:8080/{pathVariable}")
.httpMethod(GET)
.uriVariable("pathVariable", Message::getPayload)
.expectedResponseType(String.class);
}
}
This works brilliantly, however, I am struggling to come up with some good tests.
How am I supposed to mock the external REST API?
How am I supposed to test that the retry policy does kick in and the desired number of http requests are made?
How do I change the MessageSource of the flow (list of path vars) that is polled periodically?
How do I check if the payload has successfully made it to the Kafka topic?
Too much questions and some of them requires too broad explanation. Anyway I think you can start from Spring Integration Testing Framework and its documentation.
How am I supposed to mock the external REST API?
I think you can just consider to use a Mock MVC from Spring Framework and its MockMvcClientHttpRequestFactory to inject into the HttpRequestExecutingMessageHandler based on the HttpMessageHandlerSpec.
retry policy does kick
Well, I guess the same mocked MVC endpoint can verify how many times it has been called and fail for first several times to initiate that retry.
How do I change the MessageSource
This is exactly a part of Spring Integration Testing Framework with its MockIntegration.mockMessageSource() and MockIntegrationContext: https://docs.spring.io/spring-integration/docs/5.1.6.RELEASE/reference/html/#mockintegration
made it to the Kafka topic?
Or you the mentioned MockIntegration.mockMessageHandler() to verify that endpoint for Kafka is called. Or use an Embedded Kafka from Spring Kafka project: https://docs.spring.io/spring-kafka/docs/2.2.7.RELEASE/reference/html/#embedded-kafka-annotation
I have a ws:outbound-gateway in place pointing to a org.springframework.ws.transport.jms.JmsMessageSender class in order to push a Soap message into the queue.
The output message has been generated okay and published into the queue normally with the following JMS properties on it: SOAPJMS_soapAction, SOAPJMS_contentLength, SOAPJMS_contentType, etc.
My question is: how can I add a custom JMS property as part of the JMS properties generated by default? Is this possible? I'm using Spring Integration 4.3.5.RELEASE.
The JmsMessageSender can be supplied with the MessagePostProcessor.
The you can supply any desired JMS property on target Message.
I'm currently implementing a flow on a Spring Integration-based (ver. 3.0.1.RELEASE) application that requires to store messages on a JMS queue to be picked up later.
For that, I've been trying to use a Spring Integration JMS Inbound Channel Adapter with a custom selector, and then picking up the message from the queue by changing the JMS selector of the JMSDestinationPollingSource to some matching ID included as a header property.
One of the requirements for this is that I cannot add a new service or a JAVA method, so I've been trying to sort it out using a Control Bus, but keep receiving the same error when I send the message to set the messageSelector to something different.
Inbound Channel Adapter definition:
<int-jms:inbound-channel-adapter id="inboundAdapter"
channel="inboundChannel"
destinationName="bufferQueue"
connection-factory="connectionFactory"
selector="matchingID = 'NO VALUE'">
<int:poller fixed-delay="1000"/>
</int-jms:inbound-channel-adapter>
Message:
#'inboundAdapter.source'.setMessageSelector("matchingID = 'VALUE'")
Error:
EvaluationException: The method 'public void org.springframework.integration.jms.JmsDestinationPollingSource.setMessageSelector(java.lang.String)' is not supported by this command processor. If usign the Control Bus, consider adding #ManagedOperation or #ManagedAttribute.
Which, AFAIK, means that the JmsDestinationPollingSource class is not Control Bus manageable, as it's not passing the ControlBusMethodFilter.
Is this approach nonviable, or is there something I'm missing? Is there any way to set the selector dynamically using SI XML configuration files only?
First of all it is strange to use Java tool and don't allow to write code on Java...
But that is your choice, or as you said requirements.
Change the employer! ;-)
That's correct: Control Bus allows only #ManagedOperation and #ManagedAttribute method. Since JmsDestinationPollingSource.setMessageSelector. We can make it like that. But does it make so much sense if we can reach it a bit different approach?
<int:outbound-channel-adapter id="changeSelectorChannel"
ref"inboundAdapter.source method="setMessageSelector"/>
where a new selector expression should be as a payload of the Message to this channel.
I need to integrate MQ feature in my ServiceStack application. I have registered the Message Handler in AppHost. The handler for my ServiceStack request(Post) will publish the message to the MQ broker. I have created the consumer for that message. This is all working fine.
Now what I need is to have multiple threads available to consume the message and process that. I have read that ServiceStack implementation of rabbitMQ provides the feature to specify multiple threads for an operation:
https://github.com/ServiceStack/ServiceStack/wiki/Rabbit-MQ#allocating-multiple-threads-for-specific-operations
But I am not able to specify these threads. I have registered the handler as
container.Register(c => new RabbitMqServer());
var mqServer = container.Resolve();
mqServer.RegisterHandler(ServiceController.ExecuteMessage, noOfThreads: 4);
But it gives me error as RegisterHandler does not have parameter 'noOfThreads'.
I am running the 4.0.24.0 version for ServiceStack RabbitMQ. Is there something else that I am missing here?
The noOfThreads is only available on the RabbitMqServer and not a feature of the generic IMessageService. You need to cast the IMessageService you get back from the Container.Resolve() to a RabbitMqServer.
container.Register<IMessageService>(c => new RabbitMqServer());
var mqServer = (RabbitMqServer)container.Resolve<IMessageService>();
mqServer.RegisterHandler<CallBatchMessage>(ServiceController.ExecuteMessage, noOfThreads: 4);