I am wondering if it is possible to combine two alert conditions, for example MessageCount and FieldContent, to send an alarm callback when there is more than 5 messages in a minute that have specific field content.
Is it even possible to make a plugin to do that?
Thanks in advance.
Related
I'm consuming messages from readStream, but in messages I dont have any time based column. In my scenario, I want to aggregate all messages I received in every last 10 mins.
How can I achieve this in spark ? I saw tumbling window function but it works only on timestamp column which is already coming in message.
Any suggestion please ?
Thanks
I'm trying to use application insights to keep track of a counter of number of active streams in my application. I have 2 goals to achieve:
Show the current (or at least recent) number of active streams in a dashboard
Activate a kind of warning if the number exceeds a certain limit.
These streams can be quite long lived, and sometimes brief. So the number can sometimes change say 100 times a second, and sometimes remain unchanged for many hours.
I have been trying to track this active streams count as an application insights metric.
I'm incrementing a counter in my application when a new stream opens, and decrementing when one closes. On each change I use the telemetry client something like this
var myMetric = myTelemetryClient.GetMetric("Metricname");
myMetric.TrackValue(myCount);
When I query my metric values with Kusto, I see that because of these clusters of activity within a 10 sec period, my metric values get aggregated. For the purposes of my alarm, I can live with that, as I can look at the max value of the aggregate. But I can't present a dashboard of the number of active streams, as I have no way of knowing the number of active streams between my measurement points. I know the min value, max and average, but I don't know the last value of the aggregate period, and since it can be somewhere between 0 and 1000, its no help.
So the solution I have doesn't serve my needs, I thought of a couple of changes:
Adding a scheduled pump to my counter component, which will send the current counter value, once every say 5 minutes. But I don't like that I then have to add a thread for each of these counters.
Adding a timer to send the current value once, 5 minutes after the last change. Countdown gets reset each time the counter changes. This has the same problem as above, and does an excessive amount of work to reset the counter when it could be changing thousands of times a second.
In the end, I don't think my needs are all that exotic, so I wonder if I'm using app insights incorrectly.
Is there some way I can change the metric's behavior to suit my purposes? I appreciate that it's pre-aggregating before sending data in order to reduce ingest costs, but it's preventing me from solving a simple problem.
Is a metric even the right way to do this? Are there alternative approaches within app insights?
You can use TrackMetric instead of the GetMetric ceremony to track individual values withouth aggregation. From the docs:
Microsoft.ApplicationInsights.TelemetryClient.TrackMetric is not the preferred method for sending metrics. Metrics should always be pre-aggregated across a time period before being sent. Use one of the GetMetric(..) overloads to get a metric object for accessing SDK pre-aggregation capabilities. If you are implementing your own pre-aggregation logic, you can use the TrackMetric() method to send the resulting aggregates.
But you can also use events as described next:
If your application requires sending a separate telemetry item at every occasion without aggregation across time, you likely have a use case for event telemetry; see TelemetryClient.TrackEvent (Microsoft.ApplicationInsights.DataContracts.EventTelemetry).
I have a logic app that is fetching data from an API endpoint. The API is using pagination and has an limit of 50 objects per request and then provides an link for the next 50 objects until it gets all the objects, however I have no idea on how many objects there will be for each request. My flow is briefly described down below:
First make an initial HTTP request against the endpoint
Parsing the response HTTP Body to be able to use the nextLink url provided.
Until loop with the conditon to run until nextLink is equal to null.
In the until loop I have an action for Set Variable that get Set to a new URL for each request made with a new pagination in the end of the url: "&_offset=100"
The issue with the until loop is that you can set limits for count and timeout as you can see here. As I have no clue on how many pagination there will be I am expecting this loop to run until the condition specified is met. However, I have tried specify some different values listed below:
Count = 1 - Resulted in just 1 run
Count = empty - Resulted in it running for an hour (approx 3300 loops), as specified by the Timeout value.
Count = 60 - Resulted in it running for 60 times
I have researched on how many pagination this specific request has and it turns out it has 290 paginations. My expectations is that this until loop will run until nextLink is equal to null which will be after 290 loops. But I wonder if there is any possibiliy to specify a dynamic value for Count in the until action?
I am expecting the UNTIL action to run as many time as needed based on how many pagination there is, that is atleast what I suppose it should do because if I need to specify a value for how many times it needs to run then this action is pretty useless. Hopefully there is someone in here that maybe have faced the same issue.
Best regards
As far as I know, "Until" action requires us to define at least one limit to prevent endless loops.
For your problem, you can just define a count which is large enough to allow your endpoints show all of the pages. If you want to specify a dynamic value for the count, you need to meet two conditions:
You have to be able to access total number of pages (if your endpoint provides a url to get it).
The count set in "Until" action can only reference trigger inputs, trigger outputs and parameters.
According to the statement in your question, I guess you can't meet these two conditions. So I think we can just set a count which is large enough.
I have an Azure Function App with Azure Functions that I individually want to monitor with the following rule: If an Azure Function didn't execute for N amount of minutes, send out an email/notification.
I am wondering if this is possible with the Application Insights Alerts, which does provide signal logic for the count on an individual Azure Function basis. But this count is never 0, in the graphs it appears that any count < 0 is not seen as a number. It displays as --, as you can see in the graph for my test function below:
testfunction chart (don't have enough reputation to post images)
The peak on the chart is seen as a 3, but if I use the condition "Whenever the testfunction Count is Less than 1" then the alert is never triggered.
Changing the aggregation granularity doesn't really do much, since the signal logic doesn't ever seem to record a count of 0, or any count smaller than 1.
There are lots of (slightly) more inconvenient ways to do this type of monitoring, but it seemed very possible with the nice built-in Azure Application Insights Alerts and I'd like to use that if at all possible.
Am I trying to misuse Application Insights Alerts or is there something obvious that I'm not getting? I would think it should be possible to have monitoring rules based on a lack of executions.
you might have to do this with log/query alerts instead. If you're doing metric based alerts, some of those don't send 0's as data. so if nothing happened during a time range, there's no 0's to alert on, since nothing is submitting 0, 0, 0, 0.
instead, you'd create alerts based on queries: https://learn.microsoft.com/en-us/azure/azure-monitor/platform/alerts-unified-log
the doc has this exact scenario listed:
In some cases, you may want to create an alert in the absence of an event.
For example, a process may log regular events to indicate that it's working properly. If it doesn't log one of these events within a particular time period, then an alert should be created. In this case, you would set the threshold to less than 1. [emphasis added, this is your scenario, correct]?
Example of Number of Records type log alert
Consider a scenario where you want to know when your web-based App gives a response to users with code 500 (that is) Internal Server Error. You would create an alert rule with the following details:
Query: requests | where resultCode == "500"
Time period: 30 minutes
Alert frequency: five minutes
Threshold value: Greater than 0
in that example the query would end up being something like requests | where timespan < ago(30m) | where resultCode == "500" because of the time period set. (the query itself can then filter that time range/result set down however you want)
so for yours, you'd probably just do requests with no where condition at all, and whatever time period and frequency you have, and "less than one" as the threshold.
you could make much more complicated queries as well, to filter out test data, etc.
one thing to watch out for is that I believe log alerts will fire an alert every time the frequency elapses. so if you had a requests < 1 alert set up for every 5 minutes, and your function had no calls for 2 hours, the alert is going to fire every 5 minutes, sending you 40 emails or whatever. maybe you want that :)
I have ELK installed, and all works fine. I have one index that always receives logs from Logstash.
Sometimes, Logstash stops working (every second month or so), and nothing comes to the index.
I was wondering is there a way to query the index (some interval), if it does not have any entries to produce some kind of event, which I will handle.
For example, query that index every 10 mins, and if there are no logs, then create an event.
I assume you are looking for ELK's internal tools. There is the Elasticsearch Xpack plugin that gives watchers and notifications. But if that's not a requirement, you can write a nodeJS server that querys the last 5 minutes or so, and you can write the exact notification you need.
I hope I could help.