I have two identical resource groups for dev and qa.
They have the same services and same configurations.
At some point, I found that appinsight in QA contains a lot of performance counters which are produced by the app service plan.
I've tried to compare their configuration, but they look similar.
What can cause such different behaviour?
According to Document it says,
At the set sample interval, Azure Monitor gathers data from all deployed performance counters on all agents. For the time period defined by your log analytics workspace, the raw data is accessible in all log query views and has not been aggregated.
According to Document it says,
Performance counters show how the system resources are performing. Performance counter data ingestion depends on your environment size and usage. In most cases, performance counters should make up 80 to 99% of your data ingestion for Azure Monitor for Azure Virtual Desktop.
References:
https://learn.microsoft.com/en-us/dotnet/framework/debug-trace-profile/performance-counters
Related
Azure availability report is based on the number of heartbeat alert generated in log analytics workspace. Therefore, low availability in the report doesn't really mean that a VM was unavailable due to issues in a given month. It could be different reasons eg. was switched off/deallocated or only created in the last few days in a given month etc.
any logic to improve this any KQL or new azure solution.
Have you gone through looking for VM insights, does that provide you the information you required?
VM insights can help deliver predictable performance and availability of vital applications by identifying performance bottlenecks and network issues and can also help you understand whether an issue is related to other dependencies.
You can create a dashboard with different possible options of kusto queries that depend on what you are looking for.
Here are the few other examples of Kusto Query
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/examples
https://learn.microsoft.com/en-us/azure/azure-monitor/insights/solution-agenthealth#sample-log-searches
You can even have these additional readings of all possible ways of the information you required.
Visualizing data from Azure Monitor
Monitoring Azure virtual machines with Azure Monitor
Create and share dashboards of Log Analytics data
Quickstart: Monitor an Azure virtual machine with Azure Monitor
Collect data from an Azure virtual machine with Azure Monitor
Overview of VM insights
Does azure monitor's continuous performance monitoring and sending the logs to Workspace have impact on the server at all? Looked for documentation around this and could not find any. Any information about how it internally works on sending the information or links would be appreciated.
If you have Azure Monitor agents installed, it will collect the data as a process of the operating system.
Normally, the agent will consume relatively small share of the VM performance such as CPU, memory and disk. However, the performance consumption is based on the different types of the data collecting. As some solutions will be relied on the agent to collect the data, it is likely that the more solutions you are using, the more VM performance resource will be consumed.
Regarding " continuous performance monitoring and sending the logs to Workspace", the agent will only retrieve the data from the VM such as metrics and logs. It just uses little resources.
We have legacy applications that currently write out various run time metrics (SQL calls run time, api / http request run times etc) to local SQL DB.
format:( source, event, data, executionduration)
We are moving away from storing those in local SQL DB, and are now publishing those same metrics to azure event hub.
Looking for a good place to store those metrics for the purpose of monitoring the health of the application. Simple solution would be to store in some DB and build custom application to visualize the data in custom ways.
We are also considering using Azure Monitor for this purpose via data collector API (https://learn.microsoft.com/en-us/azure/azure-monitor/platform/data-collector-api)
QUESTION: Are there any issues with azure monitor that would prevent us from achieving this type of health monitoring?
Details
each event is small (few hundred characters)
expecting ~ 10 million events per day
retention of 1-2 days is enough
ability to aggregate old events per source per event is important (to have historical run time information)
Thank you
You can do some simple graphs and with the Log Analytics query language, you can do just about any form of data analytics you need.
Here's a pretty good article on Monitor Visualizations.
learn.microsoft.com/en-us/azure/azure-monitor/log-query/charts
We run a software application on azure for one of our customers. The customer want to see the performance of the systems. This consist of two parts. One is the metric information of the servers and they also want to see some information I want to provide by custom logging.
My plan is to give the customer access to the portal and only allow him access to the metric information and the custom tables.
It seems to me that by assigning a role to the customer I should be able to block all the other possibilities.
Does someone can me tell which actions I have to allow/forbid to achieve this? Or were I can find the information for this?
Solution #1
Instead of giving Read access to the virtual machine which may breaks security policy, I'd recommend to go with Azure Log Analytics (ref: https://learn.microsoft.com/en-us/azure/log-analytics/log-analytics-overview
) workspace. That said, you will need to create a workspace which collects and stores server metrics (ref: https://learn.microsoft.com/en-us/azure/log-analytics/log-analytics-quick-collect-windows-computer) and other custom metrics.
Your customer will be given access to the workspace only which he can see all metrics in a dashboard. If there is a need for log filtering, you can use Log Analytics query language (ref: https://learn.microsoft.com/en-us/azure/log-analytics/log-analytics-log-search-transition)
Log Analytics is a paid service. You are given free up to 10 workspaces per subscription. The workspace is considered an Azure resource so the limit follows by subscription limit, which means you can create up to 800 workspaces per a resource group. A subscription can allow 800 * 800 (for reference if you would like to do capacity planning for your workspace-based solution). For Log Analytics pricing, read here (https://azure.microsoft.com/en-us/pricing/details/log-analytics/).
Log Analytics is a good choice as its value proportion is to offer your customer intuitive dashboard to monitor their virtual machine performance, and to offer Near Real Time monitoring. And this solution is a cloud native compatibility.
There is a management solution which offers a bundle of VM capacity and performance monitoring which you can try now https://learn.microsoft.com/en-us/azure/log-analytics/log-analytics-capacity
Solution #2
Log Analytics might not be your choice because it might add more Azure service and operational cost. If you need a cheaper cost, you would need to collect your virtual machine by Performance Counter which is a built-in feature in Windows OS. With Performance Counter you can export to Excel file, or visualize into Power BI or some custom chart.
Other Solutions
You can utilize Azure Monitor and API to get data, For example, this API https://learn.microsoft.com/en-us/rest/api/monitor/metricdefinitions/list. You would certainly need to visualize or format in some intuitive way to satisfy your customer. It can be a custom front-end web, or Power BI or even Excel with chart.
You can just query to Azure Blob Storage and use Stream Analytics combining with Power BI to visualize your data (https://thuansoldier.net/?p=7187).
There is not a single solution. This really depends on your existing resource capacity, financial stuff or so on.
The Azure management dashboard gives you the possibility to monitor metrics such as CPU utilization, network in/out, response time, among others.
But how can you measure consumption/availability of memory? I am running a web app that is memory intensive, and it is hard for me to gauge which instance types (or number of instances) I should provision without having an understanding of the memory situation across time.
Yes, my service is a web role on Azure cloud services, I am not talking about VMs (IaaS) here.
Thanks
In your Azure project, in the Roles folder you'll find a folder for each of your Roles. If you use the latest version of the SDK you'll find a file called diagnostics.wadcfg. This is where you'll be able to configure Performance Counters, like \Memory\Available Bytes. This file will also allow you to configure the sample rate (ex: every 30 sec) and the scheduled transfer period (how frequently the logs should be transferred to your Storage Account).
Then you can use a tool like the Azure Diagnostics Manager to view memory consumption over time.
More information: Using performance counters in Windows Azure
A way to do this from the Management Console:
On the Configure tab for your web role, in the monitoring section, change level to Verbose.
On the Monitor tab at the bottom, click Add Metrics
With monitoring set to Verbose, the available metrics should include Memory Available.