In my code: spring-integration, a scatter gather channel is there and the scatter channel using a task-executor to call two micro-services parallely and gather the response in gather channel. I used logger channel to log request response in the scatter gather channel. I can see the response from two services came to gather channel but stuck there not going to next channel. I suspected release strategy of gather channel may not working but same is working if I don't use thread(task executor). Suggest something. Thanks.
I think you have in between some persistent message store, so replyChannel header is removed during message serialization.
I suggest you to use <header-enricher> with the header-channels-to-string option to convert the replyChannel to string representation.
Meanwhile you're free to raise a JIRA about this issue in the Scatter-Gather. And we can make that string conversion as out-of-the-box feature, like it is right now there for the gatherResultChannel:
PollableChannel gatherResultChannel = new QueueChannel();
Object gatherResultChannelName = this.replyChannelRegistry.channelToChannelName(gatherResultChannel);
Message<?> scatterMessage = getMessageBuilderFactory()
.fromMessage(requestMessage)
.setHeader(GATHER_RESULT_CHANNEL, gatherResultChannelName)
.setReplyChannel(this.gatherChannel)
.build();
Related
Currently I'm trying to put a message to queue with json format. Below it's my code snippet but it does not work.
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, NOTE_INCOMING_QUEUE)
.concurrentConsumers(2))
.transform(new JsonToObjectTransformer(Note.class))
.handle(Note.class, (note, header) -> {
// doing something
return note;
})
.channel(Amqp.channel(connectionFactory).queueName(NOTE_OCRED_QUEUE).messageConverter(
new MappingJackson2MessageConverter()))
.get();
The message was putted in queue as application/x-java-serialized-object.
Two problems:
AMQP-backed channels are intended for persistence, not for simply sending messages to RabbitMQ; by default the whole message is serialized (using the RabbitTemplate's converter, not the channel's).
Message converters on channels are only used on channels for converting DataTypes, not for serialization.
Use an outbound channel adapter...
.handle(Amqp.outboundAdapter(rabbitTemplate).routingKey(NOTE_OCRED_QUEUE));
Where the RabbitTemplate is configured with a Jackson2JsonMessageConverter.
I am trying to learn Spring Integration. As a test project to explore all the features. My project goes like this
AMQP Queue -> Header Router -> Transformer -> OutBound SOAP WS Call -> Log Results -> End
1.) Start the flow by Listening to a RabbitMq Queue. The RabbitMq Message that are sent will have a String Value (Example: 32) and a Header like {"operation" "CelsiusToFahrenheit"}.
2.) The RabbitMq message is then transformed to the corresponding Java Object based on the RabbitMQ Header Value ("operation").
3.) Then I am using a MarshallingWebServiceOutboundGateway to invoke the SOAP WebService # (http://www.w3schools.com/xml/tempconvert.asmx). When I do that I am getting an Error.
ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: error occurred in message handler [wsOutboundGateway]; nested exception is org.springframework.ws.soap.client.SoapFaultClientException: Server did not recognize the value of HTTP Header SOAPAction: ., headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#601affec, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#601affec, id=7f6624cc-e7a4-37f6-0a1f-578dc78c4afa, timestamp=1482356790717}]
As you can see the SOAPAction was Blank and the server did NOT like it.
If I hard code the SOAPAction like this
public MessageHandler wsOutboundGateway()
{
MarshallingWebServiceOutboundGateway temperatureConversionWebServiceGateway = new MarshallingWebServiceOutboundGateway(WEBSERVICE_URL, jaxb2Marshaller(), jaxb2Marshaller());
temperatureConversionWebServiceGateway.setRequestCallback(new SoapActionCallback("http://www.w3schools.com/xml/CelsiusToFahrenheit"));
temperatureConversionWebServiceGateway.setOutputChannelName("webServiceResponseChannel");
return temperatureConversionWebServiceGateway;
}
it works!
However, the endpoint exposes 2 Services (2 Possible Actions). See below.
FahrenheitToCelsius soapAction="http://www.w3schools.com/xml/FahrenheitToCelsius"
CelsiusToFahrenheit soapAction="http://www.w3schools.com/xml/CelsiusToFahrenheit"
I want to be able to set the SoapAction at run time based on the RabbitMQ Header Value ("operation"). How do I do it?
On a Side note, I have done a few different experiments to solve this problem but none seems to have yielded the results I am looking for. One of the solutions I tried was to set the SoapAction as part of the Transformation process (Converting the Input from Rabbit MQ message to the corresponding Java Type based on the RabbitMQ Header Value) and send that request to the Outbound Gateway (See Code below).
#Bean
public IntegrationFlow generateCelsiusToFahrenheitRequestFlow() {
Map<String, Object> headers = new HashMap<>();
headers.put("SOAPAction", "http://www.w3schools.com/xml/CelsiusToFahrenheit");
return IntegrationFlows.from(celsiusToFahrenheitChannel)
.transform(celsiusToFahrenheitTransformer)
.enrichHeaders(headers)
.channel(webServiceRequestChannel)
.get();
}
After I do that and tap the 'webServiceRequestChannel', I see the SoapAction Header that I added previously (See Below).
GenericMessage [payload=CelsiusToFahrenheit [celsius=32], headers={SOAPAction=http://www.w3schools.com/xml/CelsiusToFahrenheit, amqp_receivedDeliveryMode=NON_PERSISTENT,........
However, when the MarshallingWebServiceOutboundGateway picked up the request and makes the outbound call to the SOAP WebService, the "SOAPAction: " is still Blank. I am not sure why the Soap Action got blanked out. What am I missing?
Sorry if I have not followed conventions when Writing / Formatting my question. This is my first time here on Stack Overflow. Any inputs or suggestions to address this issues will be greatly appriciated.
Yeah, the mistake here that the header name must be exactly WebServiceHeaders.SOAP_ACTION, where its name is ws_soapAction. Only now a DefaultSoapHeaderMapper is able to map it into request WebserviceMessage:
public DefaultSoapHeaderMapper() {
super(WebServiceHeaders.PREFIX, STANDARD_HEADER_NAMES, Collections.<String>emptyList());
}
I wrote a simple flow for AMQP inbound messages with Json payloads, something like
IntegrationFlows
.from(Amqp.inboundGateway(connectionFactory, new Queue("qin"))
.errorChannel(Amqp.channel("dlx", connectionFactory))
)
.handle(new MessageTransformingHandler(m -> {
Object result = null;
try {
result = (...)
} catch (Exception e) {
throw new MessageTransformationException(m, e.getMessage());
}
(...)
}))
.transform(Transformers.toJson(...))
.handle(Amqp.outboundAdapter(new RabbitTemplate(connectionFactory))
.routingKey("qout"))
.get();
}
This works perfectly OK, except when there's errors! As it is now I do get the error in DLX but in content_type: application/x-java-serialized-object and it is required to be application/json.
I could do this by having the error channel specify 2 converters
.amqpMessageConverter(...)
.messageConverter(...)
but the problem is that I have to implement then myself which is not easy because I have to deal with converting messages to ampqmessages, plus the business objects, plues the error object and text, and so on...
So I was thinking if I couldn't have a adapter in front of the error channel that at least took care of message->amqpmessage conversion (hopefully the payloads as well).
I also tried having a errorHandler instead of a errorChannel but the problems are the same.
Any sugestion?
Thanks in advance.
EDITED
Many thanks for your reply. However I'm struggling with it. After many tries and errors, I finally think I understand the solution (to use a "intermediary" channel so I can handle the message before send it to Amqp?) but I still can't get it to work. I have now
.errorChannel(MessageChannels.direct("amqpErrorChannel").get())
and the a flow listening to that channel
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("amqpErrorChannel")
.handle(new MessageTransformingHandler(m ->(...)
but I still have a error
MessageDeliveryException: Dispatcher has no subscribers for channel
'amqpErrorChannel'.
Any pointers to what I'm doing wrong?
Cheers.
Yes, you can have .transform() or any other adapter in front of (Amqp.channel("dlx", connectionFactory). Actually .errorChannel() is just a hook to send error to the error handling flow. So, you can use there any simple Spring Integration channel (not an AMQP one) and build any complex error handling logic.
Correct, in the end of that flow you can send a result message (after a bunch of transformation, enrichment etc.) to the AMQP dlx, but for this purpose the simple one-way Amqp.outboundAdapter() would be enough.
To be honest Amqp.channel() is two-way and that really would be better that you have a subscriber for it. But your case is one-way, so you should use Amqp.outboundAdapter() there instead.
I am building a system with Spring Integration that processes all lines in a file as records. Because some of the String records are malformed I have multiple paths through the application via a Splitter and Aggregator combination (I'm building the Aggregator as we speak).
Further, some of the records are so malformed that they are effectively errors. However I have a requirement that all records must be processed therefore I must identify and log gross malformation errors separately and finish processing the file. In other words, I can not fail to process the file but instead must only log errors.
Aggregator
I intend to do achieve the goal of processing grossly malformed records by modifying the headers on the incoming message and passing the message on-ward to the Aggregator which can search for the existence of such a parameter. I'll effectively be hand coding in some error handling situations to my processors and aggregator.
My Release Strategy for the Aggregator will be when all messages are processed.
Code Extract
This code comes from a blog entry by Matt Vickery. He constructs an entirely new message (using MessageBuilder and transferring headers) whereas I will just add something to the Message headers. He includes this code in a gateway which subsequently transfers the Message onto the Aggregator.
public Message<AvsResponse> service(Message<AvsRequest> message) {
Assert.notNull(message, MISSING_MANDATORY_ARG);
Assert.notNull(message.getPayload(), MISSING_MANDATORY_ARG);
MessageHeaders requestMessageHeaders = message.getHeaders();
Message<AvsResponse> responseMessage = null;
try {
logger.debug("Entering AVS Gateway");
responseMessage = avsGateway.send(message);
if (responseMessage == null)
responseMessage = buildNewResponse(requestMessageHeaders,
AvsResponseType.NULL_RESULT);
logger.debug("Exited AVS Gateway");
return responseMessage;
}
catch (Exception e) {
return buildNewResponse(responseMessage, requestMessageHeaders,
AvsResponseType.EXCEPTION_RESULT, e);
}
}
Confusion (...at least, that which I know about)
My questions are as follows:
When I have such a release strategy (all messages processed), is that the best way to ensure all messages get through to the Aggregator?
When using an Aggregator it seems like in practical cases, it would be very common to need access to the Message in some previous step, as opposed to just passing and processing simple POJOs. Would that be true or is there something I should be doing to simplify my design so I can avoid Message
I came across a blog entry by Matt Vickery showing how he achieves what seems to be similar with an Aggregator. I'm using his work as a guide.
P.S. Per Artem Bilan's advice, I'm avoiding creating my own messages and letting SI turn them into Messages
There is no difference for Aggregator if payload is valid or not. Its general purpose is to build a List (by default) of payloads to one Message. And it does it via some sequenceDetails from MessageHeaders. It is first.
If you use Splitter, it is responsible to enrich each produced Message with default sequenceDetails. So, if you have this configuration:
<splitter/>
<aggregator/>
And if your inbound payload is List, you end up with List after aggregator as well.
I assume, that your Splitter just produces String payloads from File lines.
Then you pass each Message to some service/transformer.
The result of that you may pass to the Aggregator.
But as you say some of payloads are not valid and your processor fails with an Exception.
So, how about just try...catch within that POJO method and return some payload with error indicator, e.g. simple String "Oops!".
As I described before: the result of POJO method will be pushed to payload of the Message by Framework. And what is magic, that sequenceDetails will be there in the MessageHeaders too.
I don't see reason to write some custom ReleaseStrategy for this task, or even any other Aggregator's strategies...
Let me know, what you don't understand.
UPDATE
To add some error-indicator to message headers and don't throw Exception, it really will be simpler to build a new Message from code, not via some error-channel flow:
try {
return [GOOD_RESULT];
}
catch(Exception e) {
return MessageBuilder.withPayload(payload).setHeader("ERROR", e.getMessage()).build();
}
But in this case you should use <service-activator> instead of <transformer>, because the last one doesn't copy headers from inbound Message. And you really need them - setHeader for aggregator.
I am working on an AS3 project in FDT6. I am using the lastest FLEX 4.6 and AIR 3.7.
I have a worker.swf file that is embedded into the main application to do threading work with.
I am using the MessageChannel class to pass information between the two.
In my main class I have defined
private var mainToWorker:MessageChannel;
private var workerToMain:MessageChannel;
mainToWorker = Worker.current.createMessageChannel(worker);
workerToMain = worker.createMessageChannel(Worker.current);
on the mainToWorker I only ever send messages. In these messages I send a byte array of information. The information is an object that contains a 'command' property and a 'props' property. Basically acting like a function call. The command is a function name and the props is an object that contains data for that function.
mainToWorkerMutex.lock();
mainToWorker.send(ByteArrayUtils.ObjectToByteArray({command:"DoSomething", props:{propA:1,propB:7}}));
mainToWorkerMutex.unlock();
The same occurs for the workerToMain var except I only send byte data that contains the 'message' and 'props' parameters.
workerToMainMutex.lock();
workerToMain.send(ByteArrayUtils.ObjectToByteArray({command:"complete", props:{return:"result"}}));
workerToMainMutex.unlock();
As a sanity check I make sure that the message channels are getting what they should.
It is working fine when I build it in FDT, however when it is built using an ANT script through flash builder I am sometimes getting the 'command' events coming back through in the workerToMain channel.
I am sending quite a lot of data through the message channel. Is it possible that I am overloading it and causing a buffer overflow into the other message channel somehow? How could that only be happening in FB?
I have checked my code many times and I am sure there is nothing in my own code that is sending that message back.
I had similar issue. When sending many bytearrays using channels sometimes things i received was not things i've actually sended. I had 4 channels (message channel to worker, message channel to main, data channel to worker, data channel to main).
I've noticed that data channel to main was affecting message channel to worker. When i turned off data channel to main - message channel to worker stared working just fine :D...
They have a big issue there with sending byte arrays it seems.
But what helped me was using shareable (at first it was not shareable) bytearray for communication via channels, but only for communication, as soon as i am receiving such bytearray i'm copying it to another byte array and parsing a copy.
This removed the problem (made quite hard stress tests there)...
Cheers
P.S. I'm also using static functions (like your ByteArrayUtils) to create bytearray's used for communication, but it seems fine, even made tests using non static functions.
So, it looks like I have found the issue. Looks like it's the ByteArray that is doing it.
ByteArray.toString() is basically sometimes mangles your data meaning you can't really trust it.
http://www.actionscript.org/forums/showthread.php3?t=155067
If you read the comment by "Jim Freer" he mentions how strings sometimes do this.
My solution was to switch to using a JSON encoded string instead of ByteArray data in the message channel. The reason I was using bytearray data to begin with is because I wanted to preserve class definition information, which JSON doesn't do.