I have a problem with Prometheus retaining data. I exporters that export metrics to Prometheus and i use prom dash to represent the data. But when i restart Prometheus i lose all the data of my metrics.
And the other problem i have is ,I have configured prometheus to have trigger alerts and i am using alertmanager to send notifications. I have configure alertmanager to send notification of alert to my Email and Flowdock. But I Receive them on flowdock but not on my mail.
Any help would be great!
But when i restart Prometheus i lose all the data of my metrics.
Prometheus stores data in the directory configured by the -storage.local.path flag which defaults to data. Make sure that's not being deleted between runs.
I have configure alertmanager to send notification of alert to my Email and Flowdock. But I Receive them on flowdock but not on my mail.
Can you share your configuration? Have you configured a SMTP server and -notification.smtp.smarthost per http://prometheus.io/docs/alerting/alertmanager/#email?
Related
I am running a couple of VM's inside Azure portal and I have my own private besu nodes running on them. I have my metrics set up inside the Prometheus but I was hoping to hook it up securely to Grafana but I tried everything and I can't. So the next thing is to see can I get the metrics available through prometheus into azure monitor, specifically into log analytics?
the aim is to get the sync status, and the highest block number on each node, into log analytics so we can see what each is doing. That way we know, on a quick look, the status of each node and by extension, the condition of the private chain. What worries me is that although I have alerts if blocks stop being created or nodes lose peers we cannot see it quickly.
Prometheus is one option to give us those stats. If we can get data from prometheus into log anaytics that would solve the problem.
Can anyone help me to how I can go about it or any links. All I am seeing is for containers but I want for my VM's
I have created an Alert Rule in Event Grid for Dropped Events, alert works fine and I get the email, but neither in the Alert email or when I go and look at it in the Azure portal it shows me the actual error, like which subscription(s) had the failure and the error text (in my case I actually stopped one of the Functions that was subscribing one of the events to test the Alert)
In the Diagnostic Settings of Event Grid, I have set the logs to be sent to Log Analytics workspace but even after this I see no logs for the corresponding failures, am I missing anything? Is there an easy way to see the logs against a particular Alert?
Azure alerts proactively notify you when important conditions are found in your monitoring data. Log alerts allow users to use a Log Analytics query to evaluate resources logs every set frequency, and fire an alert based on the results.
You can search for the functions app logs under log analytics, check this Monitoring Azure Functions with Azure Monitor Logs documentation for the steps to do it.
Azure monitor processes terabytes of customer's log from across the world, which can also cause logs ingestion latency. It can be solved by optimizing the log alert queries.
I have an application installed on a VM that publishes a page via OData.
I need to be able to trigger a high urgency alert based on one specific value published by the page.
I am little aware of Application Insights to monitor applications in Azure. But I am not sure if there is a way to read the data from the API and trigger an alert.
What options do I have to accomplish this in Azure?
Pls allow me to share my idea here.
creating app insights alert
enable app insights for azure vm
First, adding app insights to your application can made AI to capture your requests, dependencies, logs and some other messages, including calling API.
In my thoughts, I can log the response message of the API and set alert for some specific keywords. I created an azure function, and it will call MS graph API when triggered, and log the response. This function has integrated AI, so I created an alert by kql:
traces
| where timestamp > ago(30m) and message contains "xxx"
and set alert rule based on number of results greater than 0. When the alert was triggered, it will send email to my mailbox to mention me.
This means any time my API returned the response contains specific words, I will receive an email about it.
I'm having some problems getting the auto-shutdown feature for a VM to work when I enable the email notification feature as shown int the following image:
If I remove the Send notification option, then the machine will be shutdown. However, if I active the notification, I won't receive any email and the machine won't be shutdown.
Am I missing something?
Why not? On my side, it works as expected. But there is one thing you need to take care of. When you enable the auto-shutdown and set the notification with an e-mail, the Scheduled shutdown time should be later than the current time at least half an hour.
Here is the example on my side:
I have 2 web apps in Azure, one for UAT and another for Production. Both have Application Insights telemetry installed and active on them for performance monitoring.
I currently am receiving the Weekly Application insights digest email, which summarizes the telemetry for the week, as expected. However both environments are included in the one email.
I do require to switch off this feature for the UAT web app. I have already disabled all alerts.
I have read the following post, Disable digest email in Application insights, but this is a request to unsubscribe from the email and not to disable for an environment.
Disabled Alerts
I'm afraid it's not possible to explicitly disable AI(likewise it's not possible to disable storage account/event hub and so on).
If this is not a case you can stop sending the data to your AI instance, but still I'm not sure whether this is going to help you(by stop sending the data I mean - disable whole telemetry I avoid initializing it from your code).
Unfortunately for now there is no way to selectively unsubscribe digest emails per application. We are planning to come up with a solution, but we cannot put a date on it yet.
Thanks,