ServiceStack: AppHost.OnRequestEndCallbacks handler gets called twice when an exception is thrown outside of a service - servicestack

I have a snippet in the OnEndRequestCallbacks block of the app host that records a row in an audit table for business purposes. The unexpected behavior is that when a request fails for some reason outside of a service (exception thrown), the endrequest event gets hit twice, and so 2 requests are audited instead of 1. This specific error is happening in the JwtAuthProvider when the token is expired. The exception funnels through as you would expect, but hits the end request callbacks handler twice. I added a snippet in the UncaughtExceptionHandlers block to write the error and end the request, but that doesn't seem to work, and 'two' requests are logged anyway. This was happening when I was getting exceptions in a Request Filter, but adding the snippet to write an error to the response and end the request worked in fixing the duplicates there. The same approach does not seem to work in the global handler in the same way.

This should now be resolved from this commit.
This change is available from v5.6.1+ that's now available on MyGet.

Related

How do I register 400 errors in Azure Function Apps as failures in Application Insights?

I want to treat 4xx HTTP responses from a function app (e.g. a 400 response after sending a HTTP request to my function app) as failures in application insights. The function app is being called by another service I control so a 4xx response probably means an implementation error and so I'd like to capture that to ultimately run an alert on it (so I can get an email instead of checking into Azure everytime).
If possible, I'd like it to appear here:
If not, are there any alternative approaches that might fit my use case?
Unless an unhandled exception occurs the function runtime will mark the invocation as succesful, whether the status code is actually denoting an error or not. Since this behavior is defined by the runtime there are 2 things you can do: throw an exception in the code of the function and/or remove exception handling so the invocation is marked as not succesful.
Since you ultimately want to create an alert, you better alert on this specific http status code using a "Custom log search" alert
requests
| where toint(resultCode) >= 400

Taking an action when HTTP response status is not 200

I'm new to Spring Integration. I have a simple flow which send request to external resource with several attemps.
IntegrationFlows.from(MY_CHANNEL)
.handle(myOutboundGateway, e -> e.advice(myRetryAdvice))
.wireTap(logResponse())
.get();
What I need to do is to take some action (saving data to a database) when calling external resource (after retrying) is not succesful (http status code is not 200 OK). How can I achieve that in my flow?
When all the attempts of the retry are exhausted, the RecoveryCallback is called.
See some sample here: How to get HTTP status in advice recovery callback. In that RecoveryCallback you can just return null and send a message to some channel for that storing to DB logic.
Another way is to have extra advice on top of that retry instead of RecoveryCallback. See its docs: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#expression-advice. This way when all the attempts are done, the exception is going to be bubbled and caught by that ExpressionEvaluatingRequestHandlerAdvice and its failureChannel. Pay attention to the trapException = true, so the error doesn't go back to the flow. Only to that failureChannel for your DB logic.

Azure Function Timer Trigger & API management - Manual execution returns 404

I have a function app with:
a few functions triggered by a Timer Trigger
and some triggered by the HTTP Trigger.
I have also an Azure API Management service set up for the function app, where the HTTP Triggered functions have their endpoints defined.
I am trying to trigger one of my timer triggered functions manually as per the guide here https://learn.microsoft.com/en-us/azure/azure-functions/functions-manually-run-non-http
I am however getting a 404 result in Postman, despite the seemingly correct URL and x-functions-key.
The function:
The key:
The request:
I also noticed that:
if I don't include the x-functions-key header, then I get 401 Unauthorized result
and if I include an incorrect key, then I get 403 Forbidden.
Could it be related to the API management service being set up for the function app?
How can I troubleshoot this further?
I have managed to solve it.
It turns out that Azure Functions timer trigger requires six parts cron expression (I was only aware of the five part style)
Without that, it does not work - sadly this is not easily noticeable in the UI.
I have realized that by investigating Application Insights logs:
The function page shows that everything is fine:
Changing the CRON format has fixed the 404 issue and I started getting 202 Accepted response.
As a bonus note, I have to add:
Even though the response was 202 Accepted, the triggering didn't work correctly, because my function return type was Task<IActionResult> which is not accepted for timer triggered functions.
Again, only ApplicationInsights showed that anything is wrong:
The 'MonkeyUserRandom' function is in error: Microsoft.Azure.WebJobs.Host: Error indexing method 'MonkeyUserRandom'. Microsoft.Azure.WebJobs.Host: Cannot bind parameter '$return' to type IActionResult&. Make sure the parameter Type is supported by the binding. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
That's a bonus tip for a 'manual triggering of non-http function does not work'.
I test it in my side, it works fine. Please refer to the below screenshot:
Please check if you request https://xxx.azurewebsites.net/admin/functions/TimerTrigger1 but not https://xxx.azurewebsites.net/admin/functions/TimerTrigger. Note it's "TimerTrigger1".
I requst with ..../TimerTrigger at first test because the document shows us QueueTrigger, and it response 404.

The remote server returned an error: (404) Not Found While deleting message from queue

We are using Azure Queue for our printing job but when deleting message from queue by queue.DeleteMessage(message), the method throws below exception.
The remote server returned an error: (404) Not Found
Above exception was handled but still looking for workaround.
Can anyone please suggest how to fix it.
Thanks,
Sneh
According to this article, we can find that:
After a client retrieves a message with the Get Messages operation,
the client is expected to process and delete the message. To delete
the message, you must have two items of data returned in the response
body of the Get Messages operation:
The message ID, an opaque GUID value that identifies the message in the queue.
A valid pop receipt, an opaque value that indicates that the message has been retrieved.
If a message with a matching pop receipt is not found, the service returns error code 404 (Not Found). And Pop receipts remain valid until one of the following events occurs:
The message has expired.
The message has been deleted using the last pop receipt received
either from Get Messages or Update Message.
The invisibility timeout has elapsed and the message has been
dequeued by a Get Messages request. When the invisibility timeout
elapses, the message becomes visible again. If it is retrieved by
another Get Messages request, the returned pop receipt can be used
to delete or update the message.
The message has been updated with a new visibility timeout. When the
message is updated, a new pop receipt will be returned.
I ran into this issue today and the root cause was ownership issues between two different queues. We had setup two queues, one to hold our message awaiting processing and one for messages that had errored out. The problem came with the logic of how the message was moved between queues.
If our processing failed, we would perform the following logic:
_errorQueue.AddMessage(msg);
_queue.DeleteMessage(msg);
The DeleteMessage would also return a (404) Not Found because the msg had been moved to the errorQueue. There were two solutions that I found to this issue:
1. Switch Logic
If you switch the logic than the msg will be deleted before being added to the errorQueue which will avoid the ownership swap.
_queue.DeleteMessage(msg);
_errorQueue.AddMessage(msg);
2. Insert Copy of Message
Solution #1 has the potential to lose the message if something happens between deletion and insertion (small chance but a chance nonetheless). The solution I went with inserted a copy of the msg with the same payload so it didn't run into this ownership issue because it was a different object.
_errorQueue.AddMessage(new CloudQueueMessage(msg.AsString));
_queue.DeleteMessage(msg);
Debugging Tip
One useful tip I encountered while debugging it making sure the exception your catching isn't the default Exception. Catch the StorageException instead to get access to Azure Storage related error information.
try
{
_queue.DeleteMessage(msg);
}
catch (StorageException ex) //use this instead of base Exception
{
var info = ex.RequestInformation; //has useful information
}
If can provide more information to help you debug your real issue.

SoapHttpClientProtocol automatically retry after exception?

I am just curious about this. I am making a change in this project, that is using NetSuite web service, and sometimes it throws a SoapException at random, "Only one request may be made against a session at a time".
private NetSuiteService _service;
SessionResponse response = _service.login(passport); //if SoapException, retries??
Status status = response.status;
Reference.cs:
public partial class NetSuiteService :
System.Web.Services.Protocols.SoapHttpClientProtocol
My question is: If I am in debug mode, I trace this, and I hit F5, and it seems to automatically retry after exception is thrown (the code keeps running, with no try catch block implemented, no while loop) until successful (status.isSuccess == true). When I run it in release mode, as a windows service, the log shows it stops running after exception is thrown.
How is this possible? Is it better to catch this exception in a try catch block and retry?
NS Server refuses a request if its already processing one from the same user.
If you want to make sure that your request succeeds than you have to catch this exception and retry.
This was not the experience I had. We thought this related to netsuite sessions but turned out to be nothing to do with that at all and in fact was not even hitting netsuite (according to netsuite log)​​. Turned out we were trying to execute too many commands in a single request and it totally refused to send it to netsuite. Never seen this error before, may be it is a new thing with the new version!

Resources