In Azure's Portal you can view instance specific metrics per site if you go to a resource, select Metrics per instance (Apps), select the tab Site Metrics and then click an individual instance (starting with RD00... in the screenshot below):
I'd like to get this data (per instance, including the instance name RD00...) using some REST API call. I've looked at Azure's Resource Manager and their Metrics API, but couldn't find a way to get this data.
Is this possible, and, if so, how/where can I get this data?
I've looked at Azure's Resource Manager and their Metrics API, but couldn't find a way to get this data.
Based on the supported metrics with Azure Monitor of websites, Azure Metrics API only supports total and average type metrics for Azure Web App. We can't get per instance metrics by Azure Metrics API.
If you turn on the Web server logging in Azure portal, you will get the detail request data from /LogFiles/http/RawLogs/ folder using FTP. You could download the log and generate the metrics according the log.
Following is a record of raw logs. The ARRAffinity property will specify which instance is used to handle user request.
2017-04-27 08:51:32 AMOR-WEBAPP-TESTMSBUILD GET /home/index X-ARR-LOG-ID=bbdf4e53-3b96-4884-829c-cf82554abcc7 80 - 167.220.255.28 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/51.0.2704.79+Safari/537.36+Edge/14.14393 ARRAffinity=8f8ac6c076f7a9e2132f2eea1ff0fc61836fde1fef8c5525da0e81359003c9e8;+_ga=GA1.3.411824075.1493282866;+_gat=1 - amor-webapp-testmsbuild.azurewebsites.net 200 0 0 2607 1145 10095
ARRAffinity=8f8ac6c076f7a9e2132f2eea1ff0fc61836fde1fef8c5525da0e81359003c9e8
Related
I have multiple Azure Cognitive Search services, distributed among different subscriptions. I would like to monitor the usage of each service, what should include:
storage: current/quota
no. indexes: current/quota
no. indexers: current/quota
no. data sources: current/quota
as I can access them through Overview/Usage pane in Azure portal or through Management RestAPI.
I would like to push this data to Grafana monitoring but I have some problems around that:
I am not able to fetch this type of data from Metrics
diagnostic settings does not allow to export this type of data (only metrics and operation logs)
Since it's possible to access the usage data through RestAPI I was thinking about creating Function App, that will ping each search service to collect the data and then push it to Log Analytics, which I can then use in Grafana. Maybe I can have one function app per subscription and use RBAC to grant access to search services, but still I don't like to have one app that have access to multiple search services.
How can I push the data from RestAPI to Log Analytics/Grafana other than using a function app?
As of now, it looks like there is no other option other than pushing this data via additional function to your destination.
We have provisioned the instance of the Azure app gateway (Standard v2 East AU region) and has enabled the diagnostics settings of it to dump all metrics and logs to the log analytics workspace and this seem to be working fine, however we wanted to additional insights of the request and hence have scaled up the tier and enabled the WAF v2 (as shown in the image below).
Now based on this documentation here https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-diagnostics#diagnostic-logging and after waiting for some time, we expected that the firewall logs will be automatically populating in the same log analytics workspace however this does not seem to work and they are simply not populated there.
Note that we can see the "ApplicationGatewayAccessLog" logs and below query is evident of the same AzureDiagnostics | distinct Category that returns only one category i.e. "ApplicationGatewayAccessLog"
Does anyone know if we are missing something or have any input?
Sometimes, the output is not the same when you explore data from Application Gateway ---logs and from your specific Log Analytics workspace---logs. You cam compare these results on your side. See this issue.
In this case, you should have finished some access actions to your Application Gateway and trigger the firewall access log collection before the data can be collected by the Azure monitoring. Though document stated Firewall logs are collected every 60 seconds. Sometimes, the data delays(even more than 2 days) to be logged in the logs and your located region also impacts on the data display time. From this blog, you can see hourly log of firewall actions on the WAF.
For more information, you can use Log Analytics to examine Application Gateway Web Application Firewall Logs.
We've logs(W3CIISLogs) on Log analytics workspace for websites hosted on VMs. Similarly we have app insights enabled for websites hosted on App service. Now we want to access telemetry data of both type of websites thru single interface, either via app insights or via Log analytics. Just wondering if it's possible and what's the best way.
With Azure Monitor you can now query not only across multiple Log Analytics workspaces, but also data from a specific Application Insights app in the same resource group, another resource group, or another subscription. This provides you with a system-wide view of your data. You can only perform these types of queries in Log Analytics.
Querying across Log Analytics workspaces and from Application Insights - reference another workspace in your query, use the workspace identifier and for an app from Application Insights, use the app identifier.
Cross-resource query limits:
The number of Application Insights resources that you can include in
a single query is limited to 100.
Cross-resource query is not supported in View Designer. You can Author a query in Log
Analytics and pin it to Azure dashboard and visualize a log search.
Cross-resource query in log alerts is supported in the new
scheduledQueryRules API. By default, Azure Monitor uses the legacy
Log Analytics Alert API for creating new log alert rules from Azure
portal, unless you switch from legacy Log Alerts API. After the
switch, the new API becomes the default for new alert rules in Azure
portal and it lets you create cross-resource query log alerts rules.
You can create cross-resource query log alert rules without making
the switch by using the ARM template for scheduledQueryRules API –
but this alert rule is manageable though scheduledQueryRules API and
not from Azure portal.
Documentation Reference - Cross-Resource Log queries in Azure Monitor
Hope the above information helps.
I have questions on using Azure Log Analytics for API Management. I cannot find more information on Azure Log Analytics with API Management.
Questions
What data is Log Analytics based on by default?
Do I need to specify or turn on what data Log Analytics is based on? Screenshot of Log Analytics tool are below
What data is in each field in red box under Log and Metric is on the screenshot below?
Below are two ways to add data to Log Analytics with APIM.
API within APIM
API under APIM instance -> Settings -> Azure Monitor -> enable it ->
Once it is enabled, it seems all the API instances under the APIM instance will use the settings. That is, data from ALL API instances instead of the one API instance will be added to Log Analytics.
Is this correct?
Under Diagnostics setting
APIM instance -> Diagnostics settings -> Add diagnostic setting
This will show up the screenshot above.
Below is all tables shown under Log Analytics tool
How to integrate Azure API Management with Azure Application Insights has no info on Log Analytics.
Get started with Azure Monitor Log Analytics
Great questions!
AFAIK the answer to your question #1 is, log data (telemetry) is collected by Azure Monitor which is stored in a Log Analytis workspace, which is based on Azure Data Explorer. For more information, please refer https://learn.microsoft.com/en-us/azure/azure-monitor/log-query/log-query-overview and https://learn.microsoft.com/en-us/azure/azure-monitor/.
In your case when you try to turn on Log Analytics for APIM then APIM related log data (telemetry) is stored in your Log Analytics workspace repository. To know that APIM related log data and its metrics and dimensions, please refer https://learn.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported#microsoftapimanagementservice
AFAIK the answer to your question #2 is yes you need to specify or turn on what all APIM related Log Analytics data needs to be stored in your Log Analytics workspace repository.
IMHO the answer to your question #3 is covered in the answer of question #1.
To understand the APIM log metrics better, I suggest you to go to your APIM resource in Azure portal and go to Metrics tile and create new charts for all the available metrics like capacity, gateway requests, eventhub events. Also you may refer https://learn.microsoft.com/en-us/azure/api-management/api-management-capacity and https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-log-event-hubs
Hope this helps!!
Trying to access metrics such as Requests/sec, Capacity, RUs, etc - programmatically. Have access to API tokens/etc. Not seeing a .NET management nuget package for DocumentDb
TIA
You can read metrics from the Azure Insights SDK. The .NET SDK i currently in preview https://www.nuget.org/packages/Microsoft.Azure.Insights
The Rest API documentation is here https://msdn.microsoft.com/en-us/library/azure/dn931939.aspx?f=255&MSPPError=-2147217396
To list all metrics you can call the following endpoint (you'll need to include the Bearer token in the authorization header)
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resource group of your documentDB}/providers/Microsoft.DocumentDb/databaseAccounts/{documentDB account name}/metricDefinitions?api-version=2015-04-08
This will list all available metric definitions. You can then use a query like this to read the induvidual metrics.
https://management.azure.com/subscriptions/{subecriptionId}/resourceGroups/{resource group}/providers/Microsoft.DocumentDb/databaseAccounts/{DocumentDB account}/metrics?api-version=2015-04-08&$filter=%28name.value%20eq%20%27Total%20Requests%27%29%20and%20timeGrain%20eq%20duration%27PT5M%27%20and%20startTime%20eq%202016-05-28T20%3A26%3A00.0000000Z%20and%20endTime%20eq%202016-05-29T20%3A26%3A00.0000000Z
For more info on reading metrics, see https://blogs.msdn.microsoft.com/cloud_solution_architect/2016/02/23/retrieving-resource-metrics-via-the-azure-insights-api/