Under Event Grid system topic I don't see "Logs" option to view the diagnostic settings logs. How do I view it using Log Analytics workspace ? The tables are not seen under Log Analytics workspace as well.
To view the logs of Azure Event Grid System Topic, you have to enable the diagnostic settings under monitoring section of the Azure Event Grid System Topic Menu Page.
You have to select the type of logs required and where to send those logs for accessibility.
I don't see "Logs" option to view the diagnostic settings logs.
You can View the logs in the Storage account that you sent from the diagnostic setting and one of the sample examples is available in below MS Document.
Refer to this MS Doc for more information on the Azure Event Grid System Topic Logs Configuration.
Related
I have created an Alert Rule in Event Grid for Dropped Events, alert works fine and I get the email, but neither in the Alert email or when I go and look at it in the Azure portal it shows me the actual error, like which subscription(s) had the failure and the error text (in my case I actually stopped one of the Functions that was subscribing one of the events to test the Alert)
In the Diagnostic Settings of Event Grid, I have set the logs to be sent to Log Analytics workspace but even after this I see no logs for the corresponding failures, am I missing anything? Is there an easy way to see the logs against a particular Alert?
Azure alerts proactively notify you when important conditions are found in your monitoring data. Log alerts allow users to use a Log Analytics query to evaluate resources logs every set frequency, and fire an alert based on the results.
You can search for the functions app logs under log analytics, check this Monitoring Azure Functions with Azure Monitor Logs documentation for the steps to do it.
Azure monitor processes terabytes of customer's log from across the world, which can also cause logs ingestion latency. It can be solved by optimizing the log alert queries.
Topic.
I want to setup an alert when someone on my team updates a configuration in a function app. I could not find this in the docs. Is this possible?
have a look at the Azure App Service as an Event Grid source
you can use an azure portal to subscribe for the eventType Microsoft.Web.AppUpdated and handle this event message in your properly subscriber, for instance: send the email, etc.
Azure Portal is down so can not provide more detailed answer right now. Will try to update it later.
You can "create log alert" if you're able to find a log message in Application Insights. You'll have to setup Function to send logs to Log Analytics.
Something like:
Go to your Function's logs (Logs at left bottom in Portal or your Log Analytics Workspace).
Find the right table that has Function App Activity events/logs.
Write appropriate query.
Click on Create Alert button above the query editor and follow steps.
If you've configured it to run say every 30 mins over last 30 min window, then Azure will just run that query on your logs every 30 minutes and if there are any hits (or not, depends on your alert configuration) it'll create an alert.
If the configuration of an AppService (or Function App) is changed, an event is created in the Activity Log.
You can send these logs to Log Analytics and create an alert based on the entries.
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log
Send the Activity log to a Log Analytics workspace to enable the features of Azure Monitor Logs which includes the following:
...
Use log alerts with Activity entries allowing for more complex alerting logic.
I have a hard time here understanding what a log analytics workspace is. As I have a requirement to monitor 7 application insigths out of many. Send emails if some of them throws exceptions. I can see in alerts you can only select a single application insight. And I dont want to create 7 alerts. So my plan was to create a analytics workspace. I havent found anyway to bind application insights to a workspace. Is that possible? I can see you in alerts can choose a analytics workspace as source, but what does that mean? Sounds like you some how can group data in that specific workspace?
I can see when I create a new Application Insight service i have the option to choose a workspace, but what with existing?
So as per your requirement, you can create the Workspace-based Application Insights resources, or connect the existing classical AI to azure log analytics. And of course, these 6 AI should connect to the same azure log analytics.
After you have the the Workspace-based Application Insights resources, the logs are logged into both the AI and the azure log analytics. So when you create an alert rule, select the azure log analytics, and create a custom log search alert rule. That can totally meet your requirement.
And you should also understand the table scheme between AI and Azure log Analytics. For example, in AI, the exceptions are logged in exceptions table; but if it's connected to azure log analytics, then in log analytics, the exceptions are in AppSystemEvents table.
We are currently investigating methods in getting our Security log data out of our Azure VM's and into our SIEM for analysis.
Currently I have been able to get the logs from the VM to log analytics work-space but I'm no sure how to get them from the log analytics workspace to the eventhub to then pull down the events.
Has anyone faced a similar challenge before / how did you overcome this challenge?
I'm currently pull the data into a Log analytics workspace
Welcome to Stackoverflow!
Azure diagnostic logs can be streamed in near real time to any application using the built-in “Export to Event Hubs” option in the Portal, or by enabling the Event Hub Authorization Rule ID in a diagnostic setting via the Azure PowerShell Cmdlets or Azure CLI.
What you can do with diagnostics logs and Event Hubs:
Here are just a few ways you might use the streaming capability for Diagnostic Logs:
Stream logs to 3rd party logging and telemetry systems – You can stream all of your diagnostic logs to a single event hub to pipe log data to a third-party SIEM or log analytics tool.
View service health by streaming “hot path” data to Power BI – Using Event Hubs, Stream Analytics, and Power BI, you can easily transform your diagnostics data in to near real-time insights on your Azure services.
Build a custom telemetry and logging platform – If you already have a custom-built telemetry platform or are just thinking about building one, the highly scalable publish-subscribe nature of Event Hubs allows you to flexibly ingest diagnostic logs.
After data is displayed in the event hub, you can access and read the data in two ways:
Configure a supported SIEM tool. To read data from the event hub, most tools require the event hub connection string and certain permissions to your Azure subscription. Third-party tools with Azure Monitor integration included.
For more details, refer "Stream Azure Diagnostic Logs to an event hub" and "How to integrate Azure Monitor with SIEM tools".
Hope this helps.
You can’t pull the VM data from log analytics to an event hub, you can use windows/Linux diagnostic extensions to route data to an event hub.
Stream Azure monitoring data to an event hub for consumption by an external tool
I have questions on using Azure Log Analytics for API Management. I cannot find more information on Azure Log Analytics with API Management.
Questions
What data is Log Analytics based on by default?
Do I need to specify or turn on what data Log Analytics is based on? Screenshot of Log Analytics tool are below
What data is in each field in red box under Log and Metric is on the screenshot below?
Below are two ways to add data to Log Analytics with APIM.
API within APIM
API under APIM instance -> Settings -> Azure Monitor -> enable it ->
Once it is enabled, it seems all the API instances under the APIM instance will use the settings. That is, data from ALL API instances instead of the one API instance will be added to Log Analytics.
Is this correct?
Under Diagnostics setting
APIM instance -> Diagnostics settings -> Add diagnostic setting
This will show up the screenshot above.
Below is all tables shown under Log Analytics tool
How to integrate Azure API Management with Azure Application Insights has no info on Log Analytics.
Get started with Azure Monitor Log Analytics
Great questions!
AFAIK the answer to your question #1 is, log data (telemetry) is collected by Azure Monitor which is stored in a Log Analytis workspace, which is based on Azure Data Explorer. For more information, please refer https://learn.microsoft.com/en-us/azure/azure-monitor/log-query/log-query-overview and https://learn.microsoft.com/en-us/azure/azure-monitor/.
In your case when you try to turn on Log Analytics for APIM then APIM related log data (telemetry) is stored in your Log Analytics workspace repository. To know that APIM related log data and its metrics and dimensions, please refer https://learn.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported#microsoftapimanagementservice
AFAIK the answer to your question #2 is yes you need to specify or turn on what all APIM related Log Analytics data needs to be stored in your Log Analytics workspace repository.
IMHO the answer to your question #3 is covered in the answer of question #1.
To understand the APIM log metrics better, I suggest you to go to your APIM resource in Azure portal and go to Metrics tile and create new charts for all the available metrics like capacity, gateway requests, eventhub events. Also you may refer https://learn.microsoft.com/en-us/azure/api-management/api-management-capacity and https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-log-event-hubs
Hope this helps!!