When my system is running some time I got the connection error so I want to remove it from my Application Insign
It is possible If I want to remove the exception and trace come from EventProcessorHost error. You can see my insign log as below.
The only way is that you can use app insights Purge api to delete logs from Exceptions table and Traces table.
But the limitation is that you cannot specify such detailed filters, like the messages are from EventProcessorHost etc.
And the delete operation will be competed in 7 days in background, you should know these limitaions when using this api.
If the question was "how do i not collect these in the future", I believe the information you are looking for is here:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-monitoring?tabs=cmd#configure-categories-and-log-levels
summary:
Log configuration in host.json
The host.json file configures how much logging a function app sends to Application Insights. For each category, you indicate the minimum log level to send
there are a lot of samples in the link above to turn on and off things of various levels, sources, sampling, batching and is probably too much to paste here and keep up to date
Related
We are looking at implementing AppInsights for our non-web application. One of the things that we want to monitor for is processes that may be "hung" for more than N number of seconds or minutes. I have been unable to find something built in that does this. The closest thing I have seen or thought of would be to log 2 custom events for the start and end of a process, and then have an alert for a custom log that queries events with no matching "end" event after N minutes.
Is there another way to monitor for hung processes using AppInsights that I am not seeing? Thanks for any help.
If you choose to use application insights, here is the suggestion just for your reference(but if you have another better solution, you can ignore this):
As per this post, you can leverage heartbeat feature, details as below:
if this application runs more than several seconds, you can leverage heartbeat
feature - it sends metric every N minutes/seconds (configurable) and the absence of such
metric will indicate that application is no longer actively running. However, if
Application Insights thread survives, then heartbeat will still be reported.
You can rely on presense/absense of the telemetry from this app in general as well as
couple custom events as you outlined above - Azure Monitor allows to set an alert on
analytics query, so you'll be able to craft a query that returns nothing in case of
application issues and set an alert on 0 count returned by such a query.
I have an Azure Functions application which once in a while "freezes" and stops processing messages and timed events.
When this happens I do not see anything in the logs (AppInsight), neither exceptions nor any kind of unfamiliar traces.
The application has following functions:
One processing messages from a Service Bus topic subscription (belonging to another application)
One processing from an internal storage queue
One timer based function triggered every half hour
Four HTTP endpoints
Our production app runs fine. This is due to an internal dashboard (on big screen in the office), which polls one of the HTTP endpoints every 5 minutes, there by keeping it alive.
Our test, stage and preproduction apps stop after a while, stopping to process messages and timer events.
This question is more or less the same as my previous question, but the without error message that was in focus then. Much fewer error messages now, as our deployment has been fixed.
A more detailed analysis can be found in the GitHub issue.
On a consumption plan, all triggers are registered in the host, so that these can be handled, leading to my functions being called at the right time. This part of the host also handles scalability.
I had two bugs:
Wrong deployment. Do zip based deployment as described in the Docs.
Malformed host.json. Comments in JSON are not right, although it does work in most circumstances in Azure Functions. But not all.
The sites now works as expected, both concerning availability and scalability.
Thanks to the people in the Azure Functions team (Ling Toh, Fabio Cavalcante, David Ebbo) for helping me out with this.
I understand that I could use the clienttrackid and setting this in a header, but I'm unsure of what is handle by application insights / azure and what I need to to manually. This is the case (I would like to see logs from ServiceA, FunctionA, ServiceB as related events) :
Clientapp calls ServiceA
ServicesA adds a message to a queue
FunctionA triggers by the queue, and calls ServiceB
Do I need to add the tracking id to the message I add to the queue? Or is everything handled automagically?
Thanks
Larsi
There is an Application Insights pattern for correlation - see this link
However, often a business transaction spans the scope of many services and technologies and it is useful to be able to correlate across these. Define correlation ID's at the business transaction level and then flow this correlation ID across the entire solution, some of the solution may include Application Insights, data stores and other logging and diagnostics. Unfortunately this is a manual process and takes some thinking through but the benefits in tracking and debugging quickly outweigh the additional time spent on this "plumbing".
I have timer-triggered Azure functions running in production, but now I want to be notified if the function fails.
In my case, access to various connected services can cause crashes, and there are many to troubleshoot. The crash is the type of error I need notification for.
When the function does fail, the log entry indicates failure, so I wonder if there is a hook in the system that would allow me to cause the system to generate a notification.
I know that blob and queue bindings, for instance, support the creation of poison queue entries, but timer trigger binding doesn't say anything about any trigger outputs of that nature.
I see that functions can pass their $return status as input to other functions, but that operation is not explained in depth in the docs. Also, in that case, I need to write another function to process the error status, and I was looking for something built-in.
I Have inquired with #AzureSupport on this, but their answer had nothing to do with Azure Functions, instead referring me to DLL notification hooks, then recommending I file on uservoice.
I'm sure there must be people here who have implemented some sort of error status notification. I prefer a solution that doesn't require code.
The recommended way to monitor and alert on failures is to use AppInsights which integrates fully with Azure Functions now
https://blogs.msdn.microsoft.com/appserviceteam/2017/04/06/azure-functions-application-insights/
Since all the logs are available in AppInsights it's easy to monitor for failures and setup alerts based on your own criteria.
However, if you only care about alerting and not things like monitoring etc, you could use Azure Monitor instead: https://learn.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-get-started
When the function does fail, the log entry indicates failure, so I wonder if there is a hook in the system that would allow me to cause the system to generate a notification.
...
I prefer a solution that doesn't require code.
This is a zero-code solution:
I poked #AzureFunctions once before on this topic, and a suggested response was to use Application Insights. It can handle the alerts upon failure and also can use webhooks.
See the Azure Functions App-Insights documentation on how to link your function app to App Insights. Then set up any alerts you want.
Unfortunately this hook doesn't exist.
Can you switch from a timer trigger to a queue trigger?
You can get retries (if you want them), and after the specified number of attempts the message is sent to a poison queue.
To schedule executions you can add queue messages with a visibility timeout to match your schedule.
In order to get alerts on failure you have two options:
A timer trigger than scans the execution logs (via SFTP) for failures.
Wrap the whole function in a try/catch block and in the catch block write a few lines to send you an email with the error details.
Hope this helps.
No code:
Go to your azure cloud account
From the menu select Monitor
Then select Add New Rule
Then Select your condition, action and add the alert details.
We were just trying out the Azure Storage Analytics Service, and something very unusual caught our attention.
The transaction count for the diagnostics storage account ( the account to which the Diagnostics Service writes it's data) was extremely high. We are talking about 600~ transaction per hour, all of which are GetBlob() operations, and all of them ended with error ( ClientOtherError is equal to the total number of operations ). Further investigation revealed that each instance running which has Diagnostics turned on, produces 300~ transactions per hour ( we has 2 instances, thus the 600). Continuing the investigation, looking at the $logs that the Analytics Service is producing revealed what really going on :
The log is filled with lots of calls to an xml file that's not exists. The log file itself is very cluttered but it's very clear that most of the calls are searching for
https://****.blob.core.windows.net/mam/MACommand.xml and also /mam/MACommanda.xml and /mam/MACommandb.xml
all those calls have an error of 404.
This issue is a real problem for us, and we have no idea what causing it.
Has anyone encountered this issue ?
(edit: Forgot to mention, the Diagnostics Service is not logging anything - scheduledTransferPeriod is zero for all the categories)
Those transactions are an expected behavior since SDK 1.6.
See full explanation here:
http://social.msdn.microsoft.com/Forums/en-US/windowsazuretroubleshooting/thread/2e2f46dd-638a-4af1-b8ac-cfd7659a3171