As part of the storage cost optimization, we need to extract the following information from Azure at the disk level for migrated servers so far.
Please suggest if any one have a PowerShell script.
I could only get the provisioned storage, but not the used storage.
VM name.
Disk name (maybe more than one).
Provisioned storage.
Used storage.
You should be able to do this with log analytics, if you connect the VMs to a Log Analytics Workspace and enable Guest OS Metrics / Performance Counters and then query the Log Analytics Workspace.
Here is a script to get you started:
Perf
| where ObjectName == "LogicalDisk" and CounterName == "% Free Space"
| summarize FreeSpace = min(CounterValue) by Computer, InstanceName
| where strlen(InstanceName) ==2 and InstanceName contains ":"
| sort by CounterName asc
Related
writing kusto queries to collect data from CustomLogs in azure from a particular resource group with Azure Monitoring Agent installed and Data Collection Rule created and performance counters to insert data in the log analytics workspace
I have a windows vm where i have inserted some data in c:
I write this kusto query to check for disk space utilization
Perf | where (ObjectName=='LogicalDisk' and CounterName=='% Free Space' and InstanceName=="_Total" )
I am getting records for that vm.
But if i change the instance name
Perf | where (ObjectName=='LogicalDisk' and CounterName=='% Free Space' and InstanceName!="_Total" )
I am getting no records. even though i have inserted file in c: of that virtual machine.
Can anyone help me to understand this behaviour?
I am trying to run the below log analytics query using Azure CLI. I have created the workspace required. However, I am getting the error as below. I have upgraded my CLI version to the latest one based on google recommendations but nothing seems to have worked so far. I have also ensured the workspace ID is the same as the one in Azure Workspace.
az monitor log-analytics -query -w xxxxxx --analytics-query "Perf | where ObjectName == 'LogicalDisk' "
'xxxxxx' is misspelled or not recognized by the system.
The eventual outcome I am looking for is to get a list of all VMs in all the subscriptions and their used disk space ( not the allocated disk space ).
Thanks in advance.
I got same error when I tried in my environment. So, I modified a bit and executed as below:
az monitor log-analytics query --analytics-query "Perf | where (ObjectName == 'LogicalDisk')" -w <workspace-ID>
Error is disappeared, but empty block is being retrieved. So, there is a need to upgrade the CLI & python versions to the latest releases. However, running the "upgrade" command in CLI shows that it is still in development.
I found an alternative to run azure Monitor Log Analytics query using "Kql query inside workspace logs":
Perf | where (ObjectName == "LogicalDisk" and CounterName == "Free Megabytes") | summarize arg_max(TimeGenerated, *) by Computer | sort by TimeGenerated desc
Reference:
MsDoc to check log queries
I have a log analytics workspace and 2 VMs connected to it. The VMs do not have Guest-OS enabled.
When I navigate to the Log Analytics --> Log blade and run the Azure provided query for "% Free Space" nothing shows up at all.
Do I need to enable Guest-OS for the VMs ? I thought this metric was out-of-the-box by Azure. What am I missing here ?
More observations:
VM1 and VM2 are connected to the log workspace. I enabled guest-os for VM2 ONLY thinking that this is needed. When I ran this Free Space query with log analytics workspace as the scope, I could see the data for VM1 also which was strange.
So I concluded that Guest-OS is not needed for this metric.
So I removed Guest-OS and removed WADPerformaceCounterTable from the storage too.
And now I dont see ANY data for the query
According to my test, if we want to monitor the servers available disk space using Azure Log Analytics, we need to have the Azure monitor agent installed on the VM’s you want to monitor and enable Performance counters in Azure log analysis.. For further details about it, please refer to the blog.
For example(I use windows VM for test)
Enable the Log Analytics VM Extension. For more details, please refer to here and here
Configuring Performance counters
Query
erf
| where ObjectName == "LogicalDisk" or // the object name used in Windows records
ObjectName == "Logical Disk" // the object name used in Linux records
| where CounterName == "Free Megabytes"
| summarize arg_max(TimeGenerated, *) by InstanceName // arg_max over TimeGenerated returns the latest record
| project TimeGenerated, InstanceName, CounterValue, Computer, _ResourceId
I need some help to understand if we can anyway pull a value through Azure log analytics if a VM is in on premise or on Azure Cloud ?
Right now i am querying IP ranges from the Heartbeat table in the log analytics and determining if it is on prem or Azure. But this approach does not work for me always as there are new IP ranges and if the VM's are on Express route Vnet. Is there a direct table data which can be pulled from Log Analytics.
I got the answer it was right there in the Heartbeat table in Log Analytics. Below is the Kusto query.
Heartbeat
| distinct Computer , ResourceProvider
| extend VMType = iff(ResourceProvider == 'Microsoft.Compute','AzureVM' , 'OnPremise' )
| project Computer , VMType
All Azure VMs will also emit resource Ids and it will be present in "Heartbeat" table and "_ResourceId" column. Below query should group Azure and Non-Azure VMs.
Heartbeat
| distinct Computer, _ResourceId
| extend Environment = iff(_ResourceId != "", "Azure", "Non-Azure")
Looking at the Metrics available through the azure Metrics API disk space, nor free memory, are available as metrics.
https://learn.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported#microsoftcomputevirtualmachines
Now I know I can view these metrics through the portal using this.
https://learn.microsoft.com/en-us/azure/cost-management/azure-vm-extended-metrics#enable-extended-metrics-in-the-azure-portal
But I'm specifically looking to query this data on a regular basis to alert me when any of my VMs disk space is nearly full (or memory is nearly full).
Is there any way of doing this?
If you ultimate goal is to
setup alerts and get notified when a threshold is met - then you can just accomplish by creating a log alert rule. For more information w.r.t it, please refer this document.
query an Azure VM's free disk / memory space metrics using API - then follow this API reference or this and this documents.
query an Azure VM's free disk / memory space metrics using PowerShell - then follow this cmdlet.
For all the above mentioned ways to work, you would have to first
create a Log Analytics workspace and install Log Analytics agent in your Azure VM (or in other words, enable the Log Analytics VM extension). To accomplish this using Azure portal, follow this document. To accomplish the same using Azure PowerShell / CLI, follow this or this document based on OS of your Azure VM.
once your Azure Log Analytics workspace starts collecting log data then go to Azure Portal -> Log Analytics workspaces -> Your Log Analytics workspace -> Logs and then type your kusto query to find free disk / memory space details of your VM. The queries will be something like shown below.
If your Azure VM is of Windows OS then query to find disk Total free space is:
Perf
| where ( ObjectName == "LogicalDisk" )
| where ( CounterName == "% Free Space" )
| where ( InstanceName == "_Total" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Windows OS then query to find disk C drive free space is:
Perf
| where ( ObjectName == "LogicalDisk" )
| where ( CounterName == "% Free Space" )
| where ( InstanceName == "C:" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Linux OS then query to find disk Total free space is:
Perf
| where ( ObjectName == "Logical Disk" )
| where ( CounterName == "% Free Space" )
| where ( InstanceName == "_Total" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Linux OS then query to find disk mounted on Root free space is:
Perf
| where ( ObjectName == "Logical Disk" )
| where ( CounterName == "% Free Space" )
| where ( InstanceName == "/" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Linux OS then query to find Available MBytes memory is:
Perf
| where ( ObjectName == "Memory" )
| where ( CounterName == "Available MBytes Memory" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Windows OS then query to Available MBytes is:
Perf
| where ( ObjectName == "Memory" )
| where ( CounterName == "Available MBytes" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
If your Azure VM is of Windows OS then query to find Committed Bytes In Use is:
Perf
| where ( ObjectName == "Memory" )
| where ( CounterName == "% Committed Bytes In Use" )
| summarize AggregatedValue= avg(CounterValue) by Computer, bin(TimeGenerated, 30s)
Note1: For all the above mentioned queries to work, make sure the respective performance counters are added in Azure Portal -> Log Analytics workspaces -> Your Log Analytics workspace -> Advanced settings -> Data -> Windows Performance Counters / Linux Performance Counters.
Note2: Using other performance counters, you can also fetch much more data like Disk read time, Disk write time, Idle time, current disk queue length, cache bytes, commited bytes, page faults, page reads, page writes, free inodes, etc.
Hope this helps! Cheers!
You can use powershell and execute scripts remotely:
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/run-command
and/or
https://techcommunity.microsoft.com/t5/ITOps-Talk-Blog/PowerShell-Basics-Connecting-to-VMs-with-Azure-PSRemoting/ba-p/428403
To get the metrics you want:
How to get free physical memory of remote computer using PowerShell
How to get disk capacity and free space of remote computer
If you collect disk space / free memory as custom metrics in Azure Monitor, you will be able to query them via the standard Azure Monitor Metrics REST API. For example, you can use Windows Azure Diagnostics (WAD) to collect these performance counters on Windows VM and send them as custom metrics.
https://learn.microsoft.com/en-us/azure/azure-monitor/platform/metrics-custom-overview
https://learn.microsoft.com/en-us/azure/azure-monitor/platform/rest-api-walkthrough