I'm using an AMQP-Backed channel in my workflow and I would like to handle ACK manually.
I though this could be done as in the AMQP Inbound Channel where you get reference of the AMQP Client Channel in the Message Header but I do not find the Header AmqpHeaders.CHANNEL in the message. Here is how I've setup my AmqpChannelFactoryBean :
#Bean(name = AMQP_BACKED_CHANNEL)
public AmqpChannelFactoryBean pubSub(ConnectionFactory connectionFactory) {
AmqpChannelFactoryBean factoryBean = new AmqpChannelFactoryBean();
factoryBean.setConnectionFactory(connectionFactory);
factoryBean.setQueueName(AMQP_BACKED_CHANNEL);
factoryBean.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factoryBean.setPubSub(false);
factoryBean.setExtractPayload(true);
return factoryBean;
}
My feeling is that I should not use the same approach as in AMQP Inbound Channel but cannot find documentation out there. Anyone can help, please?
Little update: Not sure if this is the "proper" way to do it but inspired by the comment of Artem Bilam in this post Spring AMQP Integration - Consumer Manual Acknowledgement I've resolved using a MethodBeforeAdvice. Basically my MethodBeforeAdvice is applied to the invokeListener() method of the SimpleMessageListenerContainer created by the AmqpChannelFactoryBean therefore I can get hold of the Amqp Client Channel and amqp headers and the game is done! Below is my code modified (just a bit concise for the sake of a clearer reading):
#Bean(name = AMQP_BACKED_CHANNEL)
public AmqpChannelFactoryBean amqpBackedChannel(ConnectionFactory connectionFactory) {
AmqpChannelFactoryBean factoryBean = new AmqpChannelFactoryBean();
factoryBean.setConnectionFactory(connectionFactory);
factoryBean.setQueueName(AMQP_BACKED_CHANNEL);
factoryBean.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factoryBean.setPubSub(false);
factoryBean.setExtractPayload(true);
factoryBean.setConcurrentConsumers(5);
MethodBeforeAdvice methodBeforeAdvice = new MethodBeforeAdvice() {
#Override public void before(Method method, Object[] args, Object target)
throws Throwable {
Channel amqpClientChannel = (Channel) args[0];
Message amqpCoreMessage = (Message) args[1];
Map<String, Object>
amqpCoreMessageHeaders = amqpCoreMessage.getMessageProperties().getHeaders();
amqpCoreMessageHeaders.put(AmqpHeaders.CHANNEL,amqpClientChannel);
}
};
factoryBean.setAdviceChain(new Advice[]{methodBeforeAdvice});
return factoryBean;
}
Related
Continuation of Error handling - no output-channel or replyChannel header available,
I am returning ResponseEntity from the transformer of ExpressionEvaluatingRequestHandlerAdvice failureChannel. when I debug I can see
ResponseEntity<Object> response = <409 CONFLICT Conflict,com.practice.integration.commons.error.AdapterErrorResponse#4d5a370b,[]> and its body(AdapterErrorResponse POJO) has HttpStatus status, List<AdapterError> errors which has populated correct value that I want and as per Artem Bilan's suggestion for preserving request message headers I am sending that response as MessageBuilder.withPayload(response).copyHeaders(message.getPayload().getFailedMessage().getHeaders()).build()
and I have also configured output channel on the transformer but it still does not show the above response as a part of http response payload, output channel I have is same as reply channel of the inbound gateway. could you please help here?
and I have one more external call following the above, there also I have used different transformer to handle exception and I am sending similar ResponseEntity from there , it works fine there and send response to the reply channel of the inbound gateway. Only difference is I am not using ExpressionEvaluatingRequestHandlerAdvice for the second outbound gateway.
Do you think I should do something extra with handling response using ExpressionEvaluatingRequestHandlerAdvice or am I missing anything on the first outbound gateway?
You probably didn't do this: ExpressionEvaluatingRequestHandlerAdvice.setTrapException(true);
Here is a working test, it is not HTTP based, but approach is exactly the same for any inbound request-reply gateway:
#SpringJUnitConfig
public class So74658669Tests {
#Autowired
InputGateway inputGateway;
#Test
void errorHandlerResultPropagatedBackToGateway() {
assertThat(this.inputGateway.sendAndReceive("test"))
.isEqualTo("Request failed for: test");
}
#Configuration
#EnableIntegration
#Import(InputGateway.class)
public static class TestConfiguration {
#Bean
MessageChannel outputChannel() {
return new DirectChannel();
}
#ServiceActivator(inputChannel = "inputChannel", outputChannel = "outputChannel", adviceChain = "requestHandlerAdvice")
String requestAndReply(String payload) {
throw new RuntimeException("failure");
}
#Bean
ExpressionEvaluatingRequestHandlerAdvice requestHandlerAdvice() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setFailureChannelName("errorHandlerChannel");
advice.setTrapException(true);
return advice;
}
#Transformer(inputChannel = "errorHandlerChannel", outputChannel = "outputChannel")
Message<String> errorHandler(Message<MessagingException> errorMessage) {
return MessageBuilder.withPayload("Request failed for: " + errorMessage.getPayload().getFailedMessage().getPayload())
.copyHeaders(errorMessage.getPayload().getFailedMessage().getHeaders())
.build();
}
}
#MessagingGateway
interface InputGateway {
#Gateway(requestChannel = "inputChannel", replyChannel = "outputChannel")
String sendAndReceive(String payload);
}
}
By the way there is no need in that outputChannel at all if you don't do any extra work on reply. The framework just find a replyChannel header and sends reply message directly to the input gateway.
There is a IBM MQ topic which accepts two types of messages Orders and Shipments.
I have a Springboot subscriber app which is interested in subscribing only the Shipment message type.
Below is how I am routing the channel. If the inbound message is neither of above types it will be thrown to errorChannel that I have.
Here if I do not have orderChannel app will throw an error saying no proper channel for the inbound message.
How do I silently ignore the messages of type order here?
#Bean
#ServiceActivator(inputChannel = "routerChannel")
public HeaderValueRouter router() throws Exception {
HeaderValueRouter router = new HeaderValueRouter(messageType);
router.setChannelMapping(shipment, "shipmentChannel");
router.setChannelMapping(order, "orderChannel");
router.setDefaultOutputChannel(invalidHeaderValueChannel);
return router;
}
Currently I have the below code snippet which I need to have just to avoid the error when there was a Order message.
#ServiceActivator(inputChannel = "orderChannel")
public void getInboundOrderMessage(Message<?> message) throws Exception {
logger.info("Inbound Order message...");
String payload = (String) message.getPayload();
logger.info("Order Header: {}, payload: \n{}", pMessage.getHeaders(), payload);
}
Below is how I have the MsgDrivenChannelAdapter defined
#MessageEndpoint
public class MsgDrivenChannelAdapter {
private AbstractMessageListenerContainer messageListenerContainer;
private DirectChannel inboundErrorChannel;
private DirectChannel routerChannel;
public MsgDrivenChannelAdapter(AbstractMessageListenerContainer pMessageListenerContainer,
DirectChannel pInboundErrorChannel,
DirectChannel pRouterChannel) {
this.messageListenerContainer = pMessageListenerContainer;
this.inboundErrorChannel = pInboundErrorChannel;
this.routerChannel = pRouterChannel;
}
#Bean
public IntegrationFlow jmsInboundFlow() throws Exception {
return IntegrationFlows.from(Jms.messageDrivenChannelAdapter(messageListenerContainer)
.errorChannel(inboundErrorChannel))
.channel(routerChannel)
.get();
}
}
Is there anyway I can avoid this? thanks in advance
See this option on the router:
/**
* When true (default), if a resolved channel key does not exist in the channel map,
* the key itself is used as the channel name, which we will attempt to resolve to a
* channel. Set to false to disable this feature. This could be useful to prevent
* malicious actors from generating a message that could cause the message to be
* routed to an unexpected channel, such as one upstream of the router, which would
* cause a stack overflow.
* #param channelKeyFallback false to disable the fall back.
* #since 5.2
*/
public void setChannelKeyFallback(boolean channelKeyFallback) {
So, it does not fallback to the order as a channel name.
Then it will return as null from the mapping and the logic goes like this:
if (!sent) {
getDefaultOutputChannel();
if (this.defaultOutputChannel != null) {
this.messagingTemplate.send(this.defaultOutputChannel, message);
}
else {
throw new MessageDeliveryException(message, "No channel resolved by router '" + this
+ "' and no 'defaultOutputChannel' defined.");
}
}
If you want just to ignore it and don't want to have that MessageDeliveryException, configure a defaultOutputChannel as a nullChannel.
But better to consider a messageSelector for the listener container, so it does not pull messages from a topic which it is not interested in.
This is how I did it to make it work as I wanted. I used filter to fetch only the particular message type
#Bean
public IntegrationFlow jmsInboundFlow() throws Exception {
return IntegrationFlows.from(Jms.messageDrivenChannelAdapter(messageListenerContainer)
.errorChannel(inboundErrorChannel))
.filter(Message.class, m -> m.getHeaders().get("message_type").equals("shipment"))
.channel(routerChannel)
.get();
}
this is a follow-up question to Spring Integration AWS RabbitMQ Kinesis
I have the following configuration. I am noticing that when I send a message to the input channel named kinesisSendChannel for the first time, the aggregator and release strategy is getting invoked and messages are sent to Kinesis Streams. I put debug breakpoints at different places and could verify this behavior. But when I again publish messages to the same input channel the release strategy and the outbound processor are not getting invoked and messages are not sent to the Kinesis. I am not sure why the aggregator flow is getting invoked only the first time and not for subsequent messages. For testing purpose , the TimeoutCountSequenceSizeReleaseStrategy is set with count as 1 & time as 60 seconds. There is no specific MessageStore used. Could you help identify the issue?
#Bean(name = "kinesisSendChannel")
public MessageChannel kinesisSendChannel() {
return MessageChannels.direct().get();
}
#Bean(name = "resultChannel")
public MessageChannel resultChannel() {
return MessageChannels.direct().get();
}
#Bean
#ServiceActivator(inputChannel = "kinesisSendChannel")
public MessageHandler aggregator(TestMessageProcessor messageProcessor,
MessageChannel resultChannel,
TimeoutCountSequenceSizeReleaseStrategy timeoutCountSequenceSizeReleaseStrategy) {
AggregatingMessageHandler handler = new AggregatingMessageHandler(messageProcessor);
handler.setCorrelationStrategy(new ExpressionEvaluatingCorrelationStrategy("headers['foo']"));
handler.setReleaseStrategy(timeoutCountSequenceSizeReleaseStrategy);
handler.setOutputProcessor(messageProcessor);
handler.setOutputChannel(resultChannel);
return handler;
}
#Bean
#ServiceActivator(inputChannel = "resultChannel")
public MessageHandler kinesisMessageHandler1(#Qualifier("successChannel") MessageChannel successChannel,
#Qualifier("errorChannel") MessageChannel errorChannel, final AmazonKinesisAsync amazonKinesis) {
KinesisMessageHandler kinesisMessageHandler = new KinesisMessageHandler(amazonKinesis);
kinesisMessageHandler.setSync(true);
kinesisMessageHandler.setOutputChannel(successChannel);
kinesisMessageHandler.setFailureChannel(errorChannel);
return kinesisMessageHandler;
}
public class TestMessageProcessor extends AbstractAggregatingMessageGroupProcessor {
#Override
protected Object aggregatePayloads(MessageGroup group, Map<String, Object> defaultHeaders) {
final PutRecordsRequest putRecordsRequest = new PutRecordsRequest().withStreamName("test-stream");
final List<PutRecordsRequestEntry> putRecordsRequestEntry = group.getMessages().stream()
.map(message -> (PutRecordsRequestEntry) message.getPayload()).collect(Collectors.toList());
putRecordsRequest.withRecords(putRecordsRequestEntry);
return putRecordsRequestEntry;
}
}
I believe the problem is here handler.setCorrelationStrategy(new ExpressionEvaluatingCorrelationStrategy("headers['foo']"));. All your messages come with the same foo header. So, all of them form the same message group. As long as you release group and don’t remove it, all the new messages are going to be discarded.
Please, revise aggregator documentation to make yourself familiar with all the possible behavior : https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#aggregator
I have a simple MQTT Client that outputs received messages via IntegrationFlow:
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
MqttConnectOptions options = new MqttConnectOptions();
options.setServerURIs(new String[] { "tcp://test.mosquitto.org:1883" });
factory.setConnectionOptions(options);
return factory;
}
public MessageProducerSupport mqttInbound() {
MqttPahoMessageDrivenChannelAdapter adapter = new MqttPahoMessageDrivenChannelAdapter(
"myConsumer",
mqttClientFactory(),
"/test/#");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
return adapter;
}
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p + ", received from MQTT")
.handle(logger())
.get();
}
private LoggingHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler("INFO");
loggingHandler.setLoggerName("siSample");
return loggingHandler;
}
I need to pipe all received messages into a Flux though for further processing.
public Flux<String> mqttChannel() {
...
return mqttFlux;
}
How can I do that? The loggingHandler receives all messages from the IntegrationFlow. Couldn't my Flux get it's input in a similar fashion - by passing it somehow to IntegrationFlows handle function?
MQTT Example code is take from https://github.com/spring-projects/spring-integration-samples/blob/master/basic/mqtt/src/main/java/org/springframework/integration/samples/mqtt/Application.java
Attempt: Following Artem Bilans advise I'm now trying to use toReactivePublisher to convert my inbound IntegrationFlow to Flux.
public Flux<String> mqttChannel() {
Publisher<Message<Object>> flow = IntegrationFlows.from(mqttInbound())
.toReactivePublisher();
Flux<String> mqttFlux = Flux.from(flow)
.log()
.map(i -> "TESTING: Received a MQTT message");
return mqttFlux;
}
Running the example i get following error:
10:14:39.541 [MQTT Call: myConsumer] ERROR o.s.i.m.i.MqttPahoMessageDrivenChannelAdapter - Unhandled exception for GenericMessage [payload=OFF,26.70,65.00,663,-62,192.168.2.100,0.026,25,4,6,7,933,278,27,4,1,0,1580496218,730573600,1800000,1980000,1580496218,730573600,10800000,11880000, headers={mqtt_receivedRetained=true, mqtt_id=0, mqtt_duplicate=false, id=3f7565aa-ff4f-c389-d8a9-712d4f06f1cb, mqtt_receivedTopic=/083B7036697886C41D2DF2FD919143EE/MasterBedroom/Sensor/, mqtt_receivedQos=0, timestamp=1602231279537}]
Conclusion: as soon as the first message arrives, it's handled wrong and an exception is thrown.
Please, read this doc: https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/reactive-streams.html#reactive-streams
It is not clear what you would like to achieve with that "my flux" and how that could look, but for your current configuration there are a couple of solutions.
You can use a FluxMessageChannel which is already a Publisher, so you can simply use Flux.from() and subscriber to that for consuming data produced by the mentioned MqttPahoMessageDrivenChannelAdapter.
Another way is to use a toReactivePublisher() on the IntegrationFlowBuilder to expose the whole flow as a reactive Publsiher source. In this case, of course, you can't use the LoggingHandler because it is a one-way and makes your flow ending exactly here. You may consider to use a log() operator instead though: https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/dsl.html#java-dsl-log
By the way the FluxMessageChannel is publish-subscribe, so you can have it in the flow for those logs and also have it externally for Flux.from() subscription. All the subscribers to this channel are going to get the same message.
A POJO Message.java is to be Converted to JSON(JSON is to be sent to pubsub Topic,using Spring Integration MessageChannels.),using following:
#Bean
#Transformer(inputChannel = "pubsubOutputChannel", outputChannel = "handleOutChannel")
public ObjectToJsonTransformer transformOut() {
return new ObjectToJsonTransformer();
}
#MessagingGateway(defaultRequestChannel = "pubsubOutputChannel")
public interface PubsubOutboundGateway {
void sendToPubsub(Messages msg);
}
#Bean
#ServiceActivator(inputChannel = "handleOutChannel")
public MessageHandler messageSender(PubSubOperations pubsubTemplate) {
return new PubSubMessageHandler(pubsubTemplate, "TestTopic");
}
When i call sendToPubsub() with an instance of Message.java with required properties set,i get an error "Null".
Is serviceActivator not able to receive the required data?
Any suggestions to fix this?.
Yes, it can't do that because you just don't tell it to do that.
Your gateway is configured for this:
#MessagingGateway(defaultRequestChannel = "handleOutChannel")
But that is not an input channel for the ObjectToJsonTransformer. So, whatever you send over that gateway is going directly to the messageSender service activator.
Try to configure your gateway like this:
#MessagingGateway(defaultRequestChannel = "pubsubOutputChannel")