spring integration modify error message payload - spring-integration

For error handling in my spring integration flow I want to catch exceptions in a service activator which receives its input from a aggregator so its working on a collection of messages. When the exception is thrown though the full collection is sent as the message payload. Instead I want to put the actual item that threw the exception as the content of the error message.
public Collection<File> move(Collection<File> files){
...
//process all files
for(File file : files){
if(file.getName().equals("file-2.done")){
throw new RuntimeException("SOMETHING WENT WRONG");
}
... process the file
}
My exception handler expects to retrieve the file that causes the error
File file = (File) message.getPayload().getFailedMessage().getPayload();
but in this case a collection is send as the payload not a single file. Any help would be appreciated.

The framework doesn't know what happens inside your move method.
You could do something like...
public classs MyFileFailureException extends RuntimeException {
private final File file;
public MyFileFailureException(String msg, File file) {
super(msg);
this.file = file;
}
public File getFailedFile() {return this.file}
}
Then in move()...
throw new MyFileFailureException("message", file);
Then, access it with...
message.getPayload().getCause().getFailedFile().

Related

How to calculate checksum/digest of a transferred file in spring-integration

How/where can I compute md5 digest for a file I need to transfer to a samba location in spring-integration in order to validate it against the digest I receive at the beginning of the flow. I get the file from a rest service and I have to make sure file is safely landing to samba location. The middle flow looks like this: (the digest to be compared against is stored somewhere in the messages)
GenericHandler smbUploader;
HttpRequestExecutingMessageHandler httpDownloader;
from(inbound()) //here I receive a notification with url where to download file + a checksum to be validated against
...
.handle(httpDownloader) //here I get file effectively
.handle(smbUploader) //here I upload the file to samba
...
and httpDownloader is defined like this:
public HttpRequestExecutingMessageHandler httpDownloader(){
HttpRequestExecutingMessageHandler h = new HttpRequestExecutingMessageHandler ("payload.url");
h.setExpectedResponseType(String.class);
h.setHttpMethod(GET);
return h;
}
and smbUploader is defined like this:
public GenericHandler smbUploader (MessageHandler smbMessageHandler){
return new GenericHandler<Message>(){
#Override
public Message handle(Message m, MessageHeaders h){
smbMessageHandler.handleMessage(m);
return m;
}
}
and smbMessageHandler is defined like this:
public MessageHandler smbMessageHandler (SmbRemoteFileTemplate template, FileNameGenerator g){
SmbMessageHandler h = new smbMessageHandler (template, REPLACE);
h.setAutoCreateDirectory(true);
h.setRemoteDirectoryExpression(getExpression("headers['msg'].smbFolder"));
h.setFileNameGenerator(g);
return h;
}
the inbound (starting the flow) is defined like this:
public HttpRequestHandlerEndpointSpec inbound(){
return Http.inboundChannelAdapter ("/notification")
.requestMapping(m->m.methods(POST))
.requestPayloadType(String.class)
.validator(notificationValidator);
}
First of all you should store a digest in the message headers in the beginning of the flow.
Then you need to write a service method to calculate a checksum of the file you got downloaded. And insert a new handle() in between:
.handle(httpDownloader) //here I get file effectively
.handle(smbUploader) //here I upload the file to samba
to call your service method. The input for that method must be a whole Message, so you got access to the downloaded file in the payload and digest in the headers. The result of this method could be just your file to proceed into an SMB handler for uploading.
How to calculate a checksum you can find in this SO thread: Getting a File's MD5 Checksum in Java

Spring Integration HTTP outbound gateway retry based on reply content

I'm using an API that works in 2 steps:
It starts processing of a document in async way where it provides you an id that you use for step 2
It provides an endpoint where you can get the results but only when they are ready. So basically it will always give you a 200 response with some details like the status of the processing.
So the question is how can I implement a custom "success" criteria for the HTTP outbound gateway. I would also like to combine it with a RetryAdvice which I already have implemented.
I've tried the following but first of all the message's payload that is provided in the HandleMessageAdvice is empty, and secondly the retry is not triggered:
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET), c -> c.advice(this.advices.retryAdvice())
.handleMessageAdvice(new AbstractHandleMessageAdvice() {
#Override
protected Object doInvoke(MethodInvocation invocation, Message<?> message) throws Throwable {
String body = (String) message.getPayload();
if (StringUtils.isEmpty(body))
throw new RuntimeException("Still analyzing");
JSONObject document = new JSONObject(body);
if (document.has("analyzeResult"))
return message;
else
throw new RuntimeException("Still analyzing");
}
}))
I've found this answer from Artem from 4 years back but first of all I didn't find the reply channel method on the outbound gateway and secondly not sure if this scenario has already been improved in the newer version of Spring Integaration: http outbound retry with conditions (For checker condition).
UPDATE
Following Artem's suggestion I have the following:
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET), c -> c.advice(advices.verifyReplySuccess())
.advice(advices.retryUntilRequestCompleteAdvice()))
And the advice:
#Bean
public Advice verifyReplySuccess() {
return new AbstractRequestHandlerAdvice() {
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) {
try {
Object payload = ((MessageBuilder) callback.execute()).build().getPayload();
String body = (String) ((ResponseEntity) payload).getBody();
JSONObject document = new JSONObject(body);
if (document.has("analyzeResult"))
return message;
} catch (JSONException e) {
throw new RuntimeException(e);
}
throw new RuntimeException("Still analyzing");
}
};
}
But now when I debug the doInvoke method, the body of the payload is null. It's strange as when I execute the same GET request using Postman, the body is correctly returned. Any idea?
The body from response using Postman looks like this:
{
"status": "succeeded",
"createdDateTime": "2020-09-01T10:55:52Z",
"lastUpdatedDateTime": "2020-09-01T10:55:57Z",
"analyzeResult": {
"version": "3.0.0",
"readResults": [
{
"page": 1,........
Here is the payload that I get from the outbound gateway using callback:
<200,[Transfer-Encoding:"chunked", Content-Type:"application/json; charset=utf-8", x-envoy-upstream-service-time:"27", CSP-Billing-Usage:"CognitiveServices.ComputerVision.Transaction=1", apim-request-id:"a503c72f-deae-4299-9e32-625d831cfd91", Strict-Transport-Security:"max-age=31536000; includeSubDomains; preload", x-content-type-options:"nosniff", Date:"Tue, 01 Sep 2020 19:48:36 GMT"]>
There is indeed no request and reply channel options in Java DSL because you simply wrap that handle() into channel() configuration or just chain endpoints in the flow natural way and they are going to exchange messages using implicit direct channels in between. You can look into Java DSL IntegrationFlow as a <chain> in the XML configuration.
Your advice configuration is a bit wrong: you need declare your custom advice as a first in a chain, so when exception is thrown from there a retry one is going to handle it.
You should also consider to implement an AbstractRequestHandlerAdvice to align it with the RequestHandlerRetryAdvice logic.
You implement there a doInvoke(), call ExecutionCallback.execute() and analyze the result to return as is or throw a desired exception. A result of that call for HttpRequestExecutingMessageHandler is going to be an AbstractIntegrationMessageBuilder and probably a ResponseEntity as a payload to check for your further logic.
Following Artem's suggestion I came up with the following (additional trick was to set the expectedResponseType to String as otherwise using the ResponseEntity the body was empty):
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("Ocp-Apim-Subscription-Key")
.httpMethod(HttpMethod.GET).expectedResponseType(String.class),
c -> c.advice(advices.retryUntilRequestCompleteAdvice())
.advice(advices.verifyReplySuccess()))
And the advice:
#Bean
public Advice verifyReplySuccess() {
return new AbstractRequestHandlerAdvice() {
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) {
Object payload = ((MessageBuilder) callback.execute()).build().getPayload();
if (((String) payload).contains("analyzeResult"))
return payload;
else
throw new RuntimeException("Still analyzing");
}
};
}

Sending JSON String through http.post on REST API

I am having some trouble sending a JSON String through a http.post to the backend. I am working on a project that it already has its structure and everything.
I created a PaymentResource.java file acting as the REST controller for this functionality. I have also created a PaymentService. When calling from the TypeScript file I call like this
this.http.post(SERVER_API_URL + 'path/anotherpath',request);
This is my PaymentResource file
#RestController
#RequestMapping("/path")
public class PaymentResource {
private final Logger log = LoggerFactory.getLogger(PaymentResource.class);
public PaymentResource(){
}
#PostMapping("/anotherpath")
#ResponseStatus(value = HttpStatus.OK)
public void tokenize(#RequestBody String body) {
log.logger("here");
}
}
The string tokenize method never gets executed because I can not see the log.
Any help is appreciated it!
Thank you!
I figured it out. I was missing the subscribe on the component.ts:
this.paymentService.sendInfo().subscribe((response) => {
console.log(response.ok);
});
and an Observable on the service.ts file.

cometd bayeux client can't get subscribed acknowledgement?

Actually i am running cometd-demo server in my local using maven jetty run shown in the doc https://docs.cometd.org/current/reference/ and trying to subscribe and publish something in a broadcast channel. Using Groovy script shown below,
ClientSessionChannel.MessageListener mylistener = new Mylistener();
def myurl = "http://localhost:8080/cometd/"
MyHttpClient httpClient = new MyHttpClient();
httpClient.start()
Map<String, Object> options = new HashMap<String, Object>();
ClientTransport transport = new LongPollingTransport(options, httpClient);
BayeuxClient client = new BayeuxClient(myurl, transport)
println 'client started on URL : '+ client.getURL()
client.handshake ( new ClientSessionChannel.MessageListener() {
public void onMessage(ClientSessionChannel channel, Message message) {
if (message.isSuccessful()) {
println 'Handshake Message : ' + message
}
}
})
boolean handshakecheck = client.waitFor(1000, BayeuxClient.State.CONNECTED);
println 'Handshake check : '+ handshakecheck
client.batch( new Runnable() {
public void run() {
client.getChannel("/foo/hello").subscribe(
new ClientSessionChannel.MessageListener() {
public void onMessage(ClientSessionChannel channel,
Message message) {
println "subscribed : "+ message
}
})
}
});
The program Output :
client started on URL : http://localhost:8080/cometd/
Handshake Message : [minimumVersion:1.0, clientId:fv0ozxw8cb5e11vtlwpacm7afp, supportedConnectionTypes:[websocket, long-polling, callback-polling], advice:[reconnect:retry, interval:0, maxInterval:10000, timeout:20000], channel:/meta/handshake, id:1, version:1.0, successful:true]
Handshake check : true
Here I can't get the subscribed message as in the code. But in server log It prints like shown below,
2018-02-12 20:30:32,687 qtp2069584894-17 [ INFO][examples.CometDDemoServlet] Monitored Subscribe from fv0ozxw8cb5e11vtlwpacm7afp,last=0,expire=0 for /foo/hello
Update 1:
Also i can't subscribe with callback method, i get the message as [channel:/meta/subscribe, id:4, subscription:/foo/hello, error:403:denied_by_not_granting:create_denied, successful:false]. I don't know what i am doing wrong ? I am just following the documentation steps. Thanks in advance.
The ClientSessionChannel.MessageListener that you pass to the subscribe(...) method will be invoked whenever a message will be published on channel /foo/hello.
Your program never publishes a message on that channel, so the listener is never invoked, therefore in your code subscribed is never printed.
You want to double check what version of the subscribe() method you want to use, as there are 2 versions.
The single parameter version takes a listener, while the two parameter version takes a listener and a callback.
Guessing from your code, you want the subscribed log line be in the callback not in the listener, so you just need to change your code to use the two parameter version of the subscribe() method.
Also, pay attention to the fact that if the JVM exits at the end of your groovy script, then that client will be gone and will never receive any message.

Blob does not exist when using BeginUploadFromStream

After I use CloudBlob.BeginUploadFromStream() method to upload a file, I later get a StorageClientException with StorageErrorCode.ResourceNotFound when trying to retrieve the file for a download. If I upload the same file using CloudBlob.UploadFromStream() method, then the blob DOES exist and i can download it.
here's my download code:
var client = _storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference(BLOB_CONTAINER_DOCUMENTS_ADDRESS);
container.CreateIfNotExist();
string blobName = id.ToString();
var newBlob = container.GetBlobReference(blobName);
if (newBlob.Exists())
{
var stream = newBlob.OpenRead();
return stream;
}
else
{
throw new Exception("Blob does not exist!");
}
Exists is an extension method. I'm getting the StorageClientException with the error code ResourceNotFound when I use the BeginUploadFromStream() method
public static bool Exists(this CloudBlob blob)
{
try
{
blob.FetchAttributes();
return true;
}
catch (StorageClientException e)
{
if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
{
return false;
}
else
{
throw;
}
}
}
And my call to upload
var blob = container.GetBlobReference(blobName);
This will NOT throw an exception when i later check if the blob exists
blob.UploadFromStream(fileStream);
This will
AsyncCallback uploadCompleted = new AsyncCallback(OnUploadCompleted);
blob.BeginUploadFromStream(fileStream, uploadCompleted, documentId);
EDIT
As suggested, i didn't have a call to EndUploadFromStream() method. Here is my updated call to upload:
blob.BeginUploadFromStream(fileStream, uploadCompleted, blob);
And my handler
private void OnUploadCompleted(IAsyncResult result)
{
var blob = (CloudBlob) result.AsyncState;
blob.EndUploadFromStream(result);
}
Running this, the EndUploadFromStream() method throws a WebException with the msg: "The request was aborted: The request was canceled." The InnerException is "Cannot close stream until all bytes are written."
Anyone have any idea what's going on here?
BeginUploadFromStream uploads the blob asynchronously, so your method proceeds while the blob uploads on a thread in the background. If the blob hasn't finished uploading -- or if Azure hasn't been told that the upload has completed -- you won't see the blob in storage. Only blobs uploaded through successfully completed transactions are visible.
Could you post the code for OnUploadCompleted?
It looks at first glance as if either the blob is still uploading -- or you've forgotten to call EndUploadFromStream() in your OnUploadCompleted method.
What it sounds like is happening is IIS is cancelling the thread that is being initiated to make the BeginUploadFromStream. Since the storage API is really just manipulating a bunch of REST calls under the hood you can think of these storage calls as web service calls and not like traditional IO.
Check out this topic on HttpKeepAlives, this might solve your problem but as the article pointed out it may impact performance of your site. So you may want to add logic to only enable the keep alive for the requests that are performing the upload.
http://www.jaxidian.org/update/2007/05/05/8/

Resources