I want to treat 4xx HTTP responses from a function app (e.g. a 400 response after sending a HTTP request to my function app) as failures in application insights. The function app is being called by another service I control so a 4xx response probably means an implementation error and so I'd like to capture that to ultimately run an alert on it (so I can get an email instead of checking into Azure everytime).
If possible, I'd like it to appear here:
If not, are there any alternative approaches that might fit my use case?
Unless an unhandled exception occurs the function runtime will mark the invocation as succesful, whether the status code is actually denoting an error or not. Since this behavior is defined by the runtime there are 2 things you can do: throw an exception in the code of the function and/or remove exception handling so the invocation is marked as not succesful.
Since you ultimately want to create an alert, you better alert on this specific http status code using a "Custom log search" alert
requests
| where toint(resultCode) >= 400
Related
I have Azure durable function run by timer trigger, which runs another function (UploadActivity) that does some http call to the external to Azure REST service. We know for sure that small percentage of all UploadActivity invocations end up in http error and exception risen, the rest are exception-free and upload some data to the remote http resource. Interesting finding I got is that Azure Insight's 'requests' collection contains only failed requests, and no successful one recorded
// gives no results
requests
| where success == "True"
// gives no results
requests
| where success <> "False"
// gives results
requests
| where success == "False"
I can't realize why. Here are some attributes of one of returned request with success=='False' if it helps to find why
operation_Name:
UploadActivity
appName:
/subscriptions/1b3e7d9e-e73b-4061-bde1-628b728b43b7/resourcegroups/myazuretest-rg/providers/microsoft.insights/components/myazuretest-ai
sdkVersion:
azurefunctions: 4.0.1.16815
'resource' is defined in Azure as http call to http-triggered function, but I have no http triggered functions in my app which makes things even more confusing, I think maybe these requests belong to Azure Insights calls, that could be also built based on Azure Functions
For a timer triggered function it is normal that there are no records in the requests collection of Application Insights. If it would be an http triggered function you would have 1. Only the request that triggers the function is recorded as a request in Application Insights. A timer trigger does not respond to a request.
Once the function is triggered all http requests (and all kind of other communication like calls to service busses etc.) executed by that function will be recorded as a dependency in the dependencies collection. This is by design and is how Application Insight works.
We call REST based APIs hosted by Azure Functions and fail to implement a consistent error handling supporting App Insights and wonder what can be done about it:
If we don't handle exceptions of the function, then App Insights
reports a 'failure', but the service returns only the the error code to the caller, but no error content:
Hence, the client receives a 500 and thats it.
If we handle the exception and log it (to AppInsights) then App Insights stops reporting a 'failure' hence monitoring on function level is broken. We can query for the exception, but they are out-of-context (i.e. we can see the exception by a custom query only) and we don't know which function is impacted actually.
How to marry up the two needs:
Let the function fail so that AppInsights reports the failure (and monitor can alert)
Return a bit more meaningful error message to the caller than 500.
Example on how it looks in AppInsights:
Exception is visible on the Exceptions tab, but the underlying operation has not failed
UPDATE:
According to Microsoft, App Insight Failures are exclusive to unhandled exceptions. Still, open whether there is a way to at least pass-through an error message.
If don't handle exceptions of the function, then App Insights reports a 'failure', returns the error code, but no error content. Hence, the client receives a 500 and thats it.
App Insights doesn't return anything, so what do you mean with returns the error code?
If we handle the exception, LOG it (to AppInsights), return simple error message in code 500, then App Insights doesn't log this as 'failure' hence monitoring is not possible.
Can you show how you do the logging? Because as soon as you log an exception using App Insights you should see it under failures.
This should work:
try
{
...
}
catch(Exception e)
{
logger.LogError(e, e.Message);
return httpRequest.CreateResponse(HttpStatusCode.InternalServerError, e.Message);
}
I have a function app with:
a few functions triggered by a Timer Trigger
and some triggered by the HTTP Trigger.
I have also an Azure API Management service set up for the function app, where the HTTP Triggered functions have their endpoints defined.
I am trying to trigger one of my timer triggered functions manually as per the guide here https://learn.microsoft.com/en-us/azure/azure-functions/functions-manually-run-non-http
I am however getting a 404 result in Postman, despite the seemingly correct URL and x-functions-key.
The function:
The key:
The request:
I also noticed that:
if I don't include the x-functions-key header, then I get 401 Unauthorized result
and if I include an incorrect key, then I get 403 Forbidden.
Could it be related to the API management service being set up for the function app?
How can I troubleshoot this further?
I have managed to solve it.
It turns out that Azure Functions timer trigger requires six parts cron expression (I was only aware of the five part style)
Without that, it does not work - sadly this is not easily noticeable in the UI.
I have realized that by investigating Application Insights logs:
The function page shows that everything is fine:
Changing the CRON format has fixed the 404 issue and I started getting 202 Accepted response.
As a bonus note, I have to add:
Even though the response was 202 Accepted, the triggering didn't work correctly, because my function return type was Task<IActionResult> which is not accepted for timer triggered functions.
Again, only ApplicationInsights showed that anything is wrong:
The 'MonkeyUserRandom' function is in error: Microsoft.Azure.WebJobs.Host: Error indexing method 'MonkeyUserRandom'. Microsoft.Azure.WebJobs.Host: Cannot bind parameter '$return' to type IActionResult&. Make sure the parameter Type is supported by the binding. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
That's a bonus tip for a 'manual triggering of non-http function does not work'.
I test it in my side, it works fine. Please refer to the below screenshot:
Please check if you request https://xxx.azurewebsites.net/admin/functions/TimerTrigger1 but not https://xxx.azurewebsites.net/admin/functions/TimerTrigger. Note it's "TimerTrigger1".
I requst with ..../TimerTrigger at first test because the document shows us QueueTrigger, and it response 404.
I cannot find a way to fail a http call to a nodejs azure function, and include a custom error response.
Calling context.done() allows for a custom response (but not indicated as a failure in Application Insights)
Calling context.done(true, XXX) does create a failure, but returns a generic error to the user (no matter what I put in XXX):
{"id":"b6ca6fb0-686a-4a9c-8c66-356b6db51848","requestId":"7599d94b-d3f2-48fe-80cd-e067bb1c8557","statusCode":500,"errorCode":0,"message":"An error has occurred. For more information, please check the logs for error ID b6ca6fb0-686a-4a9c-8c66-356b6db51848"}
This is just the latest headache I have ran into in trying to get a fast web api running on Azure funcs. If you cant track errors, than it should hardly be called "Application Insights". Any ideas?
Success will be true, but resultCode will be set to your value.
Try an AppInsights query like this:
// Get all errors
requests
| where toint(resultCode) >= 400
| limit 10
[Update]
The Id value in Requests is the 'function instance id', which uniquely identifies that invocation.
There is also a 'traces' table that contains the logging messages from your azure function. You can join between requests and traces via the operation_Id.
requests
| where toint(resultCode) >= 400
| take 10
| join (traces) on operation_Id
| project id, message, operation_Id
The response body is not automatically logged to AppInsights. You'll need to add some explicit log statements to capture that.
Why not use context.res to return a customer response for an HTTP trigger function?
I want to do something when/if an insert operation on Azure Table Storage fails. Assume that I want to return false from the below code when I receive an error. _table is of type CloudTable and the code below works.
public bool InsertEntity(TableEntity entity)
{
var insertOperation = TableOperation.Insert(entity);
var result = _table.Execute(insertOperation);
return (result.HttpStatusCode == (int)System.Net.HttpStatusCode.OK);
}
I get the result 203 when the operation succeeds. But there are other possible results like "200 OK".
How can I write a piece of code that will allow me to understand from the status code that something went wrong?
Using the .NET SDK, any situation that needs to be handled will throw an exception. i.e. Any status code that is not 2xx will cause an exception.
To handle situations where something went wrong, I don't have to manually check the status code of the result for every request. All I have to do is to write exception handling code. Like below:
try
{
var result = _table.Execute(insertOperation);
}
catch (Exception)
{
Log("Something went wrong in table operation.");
}
From this page:
REST API operations for Azure storage services return standard HTTP
status codes, as defined in the HTTP/1.1 Status Code Definitions.
So every successful operation against table service will return 2XX status code. To find out about the exact code returned, I would recommend checking out each operation on the REST API Documentation page. For example, Create Table operation returns 201 status code if the operation is successful.
Similarly, for errors in table service you will get error code in 400 range (that would mean you provided incorrect data e.g. 409 (Conflict) error if you're trying to create a table which already exists) or in 500 range (for example, table service is unavailable). You can find the list of all Table Service Error Codes here: https://msdn.microsoft.com/en-us/library/azure/dd179438.aspx.
Basically, any return in 2xx is "OK". In this example:
https://msdn.microsoft.com/en-us/library/system.net.httpstatuscode%28v=vs.110%29.aspx
203 Non-Authoritative Information:
Indicates that the returned metainformation is from a cached copy
instead of the
origin server and therefore may be incorrect.
This Azure white paper elaborates further:
http://go.microsoft.com/fwlink/?LinkId=153401
9.6.5 Error handling and reporting
The REST API is designed to look like a standard HTTP server interacting with existing HTTP clients
(e.g., browsers, HTTP client libraries, proxies, caches, and so on).
To ensure the HTTP clients handle errors properly, we map each Windows
Azure Table error to an HTTP status code.
HTTP status codes are less expressive than Windows Azure Table error
codes and contain less information about the error. Although the HTTP
status codes contain less information about the error, clients that
understand HTTP will usually handle the error correctly.
Therefore, when handling errors or reporting Windows Azure Table
errors to end users, use the Windows Azure Table error code along with
the HTTP status code as it contains more information about the error.
Additionally, when debugging your application, you should also consult
the human readable element of the XML error
response.
These links are also useful:
Microsoft Azure: Status and Error Codes
Clean way to catch errors from Azure Table (other than string match?)
If you are using Azure Storage SDK accessing Azure Table Storage, the SDK would throw a StorageException on the client side for unexpected Http Status Codes returned from the table storage service. To extract the actual HttpStatusCode you would need to wrap your code in a try {} catch(StorageException ex){} block. And then parse the actual exception object to extract the HttpStatusCode embedded in it.
Have a look at Azure Storage Exception parser I implemented in Nuget:
https://www.nuget.org/packages/AzureStorageExceptionParser/
This extracts HttpStatusCode and many other useful fields from Azure StorageExceptions. You can use the same library accross table, blob, queue clients etc. as they all follow the same StorageException pattern.
Note that there will be some exceptions thrown by the Azure Storage SDK that are not StorageExceptions, those are mostly client side request validation type of exceptions and naturally they do not contain any HttpStatusCode. (Hence you would need to have a catch for specifically StorageExceptions to extract HttpStatusCode s).
As a separate note, Azure Storage SDK has a fairly robust retry mechanism for failed requests. Below is the snippet from SDK source code where they decide if the failed response is retrieable or not.
https://github.com/Azure/azure-storage-net/blob/master/Lib/Common/RetryPolicies/ExponentialRetry.cs
if ((statusCode >= 300 && statusCode < 500 && statusCode != 408)
|| statusCode == 501 // Not Implemented
|| statusCode == 505 // Version Not Supported
|| lastException.Message == SR.BlobTypeMismatch)
{
return false; //aka. do not Retry if w are here otherwise Retry if within max retry count..
}