I am trying to use AMQP-backed message channels in my Spring Integration app, but I think I am fundamentally misunderstanding something, specifically around the Message<?> interface and how instances of GenericMessage<?> are written to and read from, a RabbitMQ queue.
Given I have a Spring Integration app containing the following domain model object:
#Immutable
class Foo {
String name
long quantity
}
and I declare an AMQP backed message channel called fooChannel as follows:
#Bean
public AmqpChannelFactoryBean deliveryPlacementChannel(CachingConnectionFactory connectionFactory) {
AmqpChannelFactoryBean factoryBean = new AmqpChannelFactoryBean(true)
factoryBean.setConnectionFactory(connectionFactory)
factoryBean.setQueueName("foo")
factoryBean.beanName = 'fooChannel'
factoryBean.setPubSub(false)
factoryBean
}
When I initially tried to send a message to my fooChannel I received a java.io.NotSerializableException. I realised this to be caused by the fact that the RabbitTemplate used by my AMQP-backed fooChannel was using a org.springframework.amqp.support.converter.SimpleMessageConverter which can only work with Strings, Serializable instances, or byte arrays, of which my Foo model is none of those things.
Therefore, I thought that I should use a org.springframework.amqp.support.converter.Jackson2JsonMessageConverter to ensure my Foo model is properly converted to/from and AMQP message. However, it appears that the type of the message that is being added to the RabbitMQ queue which is backing my fooChannel is of type org.springframework.messaging.support.GenericMessage. This means that when my AMQP backed fooChannel tries to consume messages from the RabbitMQ queue it receives the following exception:
Caused by: com.fasterxml.jackson.databind.JsonMappingException: No suitable constructor found for type [simple type, class org.springframework.messaging.support.GenericMessage]: can not instantiate from JSON object (missing default constructor or creator, or perhaps need to add/enable type information?)
From looking at the GenericMessage class I see that is designed to be immutable, which clearly explains why the Jackson2JsonMessageConverter can't convert from JSON to the GenericMessage type. However, I am unsure what I should be doing in order to allow my fooChannel to be backed by AMQP and have the conversion of my Spring Integration messages containing my Foo model work correctly?
In terms of the flow of my application I have the following Transformer component which consumes Bar models from the (non-AMQP backed) barChannel and places Foo models on the fooChannel as follows:
#Transformer(inputChannel = 'barChannel', outputChannel = 'fooChannel')
public Foo transform(Bar bar) {
//transform logic removed for brevity
new Foo(name: 'Foo1', quantity: 1)
}
I then have a ServiceActivator component which I wish to have consume from my fooChannel as follows:
#ServiceActivator(inputChannel = 'fooChannel')
void consumeFoos(Foo foo){
// Do something with foo
}
I am using spring-integration-core:4.2.5.RELEASE and spring-integration-amqp:4.2.5.RELEASE.
Can anyone please advise where I am going wrong with the configuration of my Spring Integration application?
If any further information is needed to in order to better clarify my question or problem, please let me know. Thanks
Yes - amqp-backed channels are currently limited to Java serializable objects.
We should provide an option to map the Message<?> to a Spring AMQP Message (like the channel adapters do) rather than...
this.amqpTemplate.convertAndSend(this.getExchangeName(), this.getRoutingKey(), message);
...which converts the entire message.
You could use a pair of channel adapters (outbound/inbound) instead of a channel.
Since you are using Java config, you could wrap the adapter pair in a new MessageChannel implementation.
I opened a JIRA Issue for this.
Related
I have been working on a "paved road" for setting up asynchronous messaging between two micro services using AMQP. We want to promote the use of separate domain objects for each service, which means that each service must define their own copy of any objects passed across the queue.
We are using Jackson2JsonMessageConverter on both the producer and the consumer side and we are using the Java DSL to wire the flows to/from the queues.
I am sure there is a way to do this, but it is escaping me: I want the consumer side to ignore the __TypeID__ header that is passed from the producer, as the consumer may have a different representation of that event (and it will likely be in in a different java package).
It appears there was work done such that if using the annotation #RabbitListener, an inferredArgumentTypeargument is derived and will override the header information. This is exactly what I would like to do, but I would like to use the Java DSL to do it. I have not yet found a clean way in which to do this and maybe I am just missing something obvious. It seems it would be fairly straight forward to derive the type when using the following DSL:
return IntegrationFlows
.from(
Amqp.inboundAdapter(factory, queueRemoteTaskStatus())
.concurrentConsumers(10)
.errorHandler(errorHandler)
.messageConverter(messageConverter)
)
.channel(channelRemoteTaskStatusIn())
.handle(listener, "handleRemoteTaskStatus")
.get();
However, this results in a ClassNotFound exception. The only way I have found to get around this, so far, is to set a custom message converter, which requires explicit definition of the type.
public class ForcedTypeJsonMessageConverter extends Jackson2JsonMessageConverter {
ForcedTypeJsonMessageConverter(final Class<?> forcedType) {
setClassMapper(new ClassMapper() {
#Override
public void fromClass(Class<?> clazz, MessageProperties properties) {
//this class is only used for inbound marshalling.
}
#Override
public Class<?> toClass(MessageProperties properties) {
return forcedType;
}
});
}
}
I would really like this to be derived, so the developer does not have to really deal with this.
Is there an easier way to do this?
The simplest way is to configure the Jackson converter's DefaultJackson2JavaTypeMapper with TypeIdMapping (setIdClassMapping()).
On the sending system, map foo:com.one.Foo and on the receiving system map foo:com.two.Foo.
Then, the __TypeId__ header gets foo and the receiving system will map it to its representation of a Foo.
EDIT
Another option would be to add an afterReceiveMessagePostProcessor to the inbound channel adapter's listener container - it could change the __TypeId__ header.
I have a stateless REST API build on Spring Boot 1.4.2. I want to log all the API calls into elk. Requests and responses data (headers, parameters, payload) need to be logged as well. I don't want to log them 1:1 - I want to filter out sensitive data etc.
I made an aspect that is intercepting my #RestController's methods invocation. I created custom annotation for method's parameter that should be logged (I use it on payloads annotated as well by #RequestBody) following this article and it gave me access to my data transfer objects in my #Around advice. I dont care about their type - I would like to call logger.debug(logObject) and send this log to logstash.
As far as I understand log message should be send as JSON with JSONLayout set in Log4j2 appender to ease things on the logstash side. So I serialize my logObject into JSON log message but during this and this only serialization I want to filter sensitive data out. I can not use transient because my controller depends on the same field.
Can I somehow create an #IgnoreForLogging annotation, that will be detected only by my custom Gson serializer that I use within logging advice and will be ignored within standard Spring's infrastructure? Is my logging into logstash approach even correct (I am trying to set it up for the first time)?
I can't believe I missed that in documentation. Here is the link
My custom annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface IgnoreForLogging {
}
Strategy for serializing objects:
public class LoggingExclusionStrategy implements ExclusionStrategy {
#Override
public boolean shouldSkipField(FieldAttributes fieldAttributes) {
return fieldAttributes.getAnnotation(IgnoreForLogging.class) != null;
}
#Override
public boolean shouldSkipClass(Class<?> aClass) {
return false;
}
}
Serializing log message in aspect class:
Gson gson = new GsonBuilder()
.setExclusionStrategies(new LoggingExclusionStrategy())
.create();
String json = gson.toJson(logObject);
This way Spring internally uses default serializer that doesn't know about #IgnoreForLogging and I can take advantage of my annotation in other places.
I'm trying to implement an annotation driven event bus (e.g. Guava Event Bus) using spring integration.
I have a PublishSubscribeChannel where I publish my events and the idea is to use methods annotated with #ServiceActivator as event handlers.
Each method can have a different signature based on the event (payload) they need to handle.
What I noticed is that when an event is published, all instances of ServiceActivatingHandler created by the ServiceActivatorAnnotationPostProcessor are called and an exception for each method that has a signature that does not match the payload. E.g.
Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1004E:(pos 8): Method call: Method handle(model.api.ServiceAvailableEvent) cannot be found on service.eai.TestServiceActivatorImpl2 type
Is there a way to define a #ServiceActivator method only for specific payload types?
That's correct, all the subscribers for the PublishSubscribeChannel accept the same message. And if there is no any chance to convert incoming payload into expected method argument type, we get that exception.
If you would like to filter unexpected types, you definitely have to use #Filter before your #ServiceActivator. In other words you do the same as now, but make your flow(s) a bit complex with front filters as subscribers to that PublishSubscribeChannel.
You even can rely on the existing PayloadTypeSelector:
#Bean
#Filter(inputChannel = "publishSubscribeChannel", outputChannel="service1")
public MessageSelector payloadTypeSelector() {
return new PayloadTypeSelector(...);
}
Or, yes, just simple POJO method which checks the payload type and marked with the same #Filter.
I guess your next question will be like: why doesn't #ServiceActivator ignore those types which aren't unsuitable for the target method?
Just don't mix concerns. Service Activator is for Message handling in the target business logic. For filtering and skipping we have a difefrent EI pattern - filter.
I am new to Spring Integration and want to know if there exist a component in Spring Integration which will help me in joining the results of 2 channels. I know the answer would be aggregator, however I don't want messages to be merged into 1 message but they should flow to the downstream component when messages from both the channels have arrived. It is sort of Cyclic Barrier case.
Thank You
Adi
The default behavior of the aggregator is to aggregate the released group of messages into a single message with a collection in the payload.
You can add a splitter after the aggregator to bust them up again.
However, if the aggregator output MessageGroupProcessor produces a Collection<Message<?> then each message is released one after the other.
You can't wire in a custom MessageGroupProcessor using the XML configuration but you can do so when configuring in Java...
AggregatingMessageHandler aggregator = new AggregatingMessageHandler(new FooProcessor());
...
private class FooProcessor implements MessageGroupProcessor {
#Override
public Object processMessageGroup(MessageGroup group) {
return group.getMessages();
}
}
Given a MessageChannel or Message object, how is it possible to get from one of them the name of the underlying JMS Queue which the message was received on ?
Here is the scenario:
Several jms:message-driven-channel-adapter instances are defined in the xml. The destination-name of each adapter uses SEL to receive from different queues. This SEL is dynamic, and is not possible to know these queue names ahead of time. All channel adapters output to the same internal Spring Integration channel.
I want to add the actual underlying queue name which the message was received on to the header of the message.
The idea is to setup a ChannelInterceptor for either the channel-adapters or the internal channel. The postReceive() method has both the Message and MessageChannel as arguments. Using either of these, is it possible to get the name of the underlying Queue name which the message came in on?
Thanks
Looks like you need to extend a bit DefaultJmsHeaderMapper:
class DestinationJmsHeaderMapper extends DefaultJmsHeaderMapper {
public Map<String, Object> toHeaders(javax.jms.Message jmsMessage) {
Map<String, Object> headers = super.toHeaders(jmsMessage);
headers.put("JMS_DESTINATION", ((Queue) jmsMessage.getJMSDestination()).getQueueName());
}
}
And inject it to your <jms:message-driven-channel-adapter>s
This is how we did it:
<int:header-enricher>
<int:header name="JMS_DESTINATION" expression="payload.JMSDestination.queueName"/>
</int:header-enricher>
It requires extract-payload="false" in your <jms:message-driven-channel-adapter>.
P.S. The answer of Artem is missing the return statement (I do not have enough reputations to comment).