VM Insights - Not logging all running processes? - azure

I was wondering why VM Insights doesn't log all running processes on a VM when using the Service MAP & Dependency options. I dump this information into a log analytic workspace and run the KQL:
VMProcess |
where UserName =~'admin_local'
It shows me only a few processes that are running, but not everything, e.g. 'notepad.exe', even though I have Notepad open & running in the VM.
How would one achieve this?
Many thanks in advance!

When you use service dependencies or enable Monitoring for Azure VM’s from Azure Monitor section, Not all processes are logged. As every process writes to a performance counter that is not collected by Azure monitor or device map dependencies. Notepad.exe falls under performance counter logs in Azure Log analytics. So you need to select the required process from the data-source to enable logging of all the processes you need.
I tried enabling Azure VM logging via Azure Monitor with Service Map or Dependency but did not receive logs on all the processes. But when I explicitly added Notepad as a data source in the Windows Performance counter, I got the required notepad process.
Notepad is not visible in the VMProcesses or Service Map :-
Deploy a Log Analytics Workspace and select Agents management > Data Collection Rules >
Click on Add resources and add your VM
Collect the notepad process and other required processes data that are needed to be logged.
As Notepad is part of performance counter and not an Event create a custom rule for notepad performance counter like below :-
Select other processes for Logs :-
And Added Data collection rules for both Performance Counters and Event Logs like below :-
My VM got connected to the Log analytics workspace like below :-
after 30 mins, the selected custom log data is visible in the Logs section in the Perf table :-
You can also enable Performance counter logs from your existing Data collection rule from Azure Monitor like below :-
In order to get logs of all the processes you need to enable Logs from Performance counter and add custom logs if required by adding those logs in Data collection rule.

Related

Azure application gateway firewall logs not being populated to log analytics workspace

We have provisioned the instance of the Azure app gateway (Standard v2 East AU region) and has enabled the diagnostics settings of it to dump all metrics and logs to the log analytics workspace and this seem to be working fine, however we wanted to additional insights of the request and hence have scaled up the tier and enabled the WAF v2 (as shown in the image below).
Now based on this documentation here https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-diagnostics#diagnostic-logging and after waiting for some time, we expected that the firewall logs will be automatically populating in the same log analytics workspace however this does not seem to work and they are simply not populated there.
Note that we can see the "ApplicationGatewayAccessLog" logs and below query is evident of the same AzureDiagnostics | distinct Category that returns only one category i.e. "ApplicationGatewayAccessLog"
Does anyone know if we are missing something or have any input?
Sometimes, the output is not the same when you explore data from Application Gateway ---logs and from your specific Log Analytics workspace---logs. You cam compare these results on your side. See this issue.
In this case, you should have finished some access actions to your Application Gateway and trigger the firewall access log collection before the data can be collected by the Azure monitoring. Though document stated Firewall logs are collected every 60 seconds. Sometimes, the data delays(even more than 2 days) to be logged in the logs and your located region also impacts on the data display time. From this blog, you can see hourly log of firewall actions on the WAF.
For more information, you can use Log Analytics to examine Application Gateway Web Application Firewall Logs.

Best practices for azure webapp loging and automate log clearing policy (in the case log size is increasing )

I'm setting up Azure WebApp logging. My concern is that error logs are stored in webapp server level, the size increasing day by day from Elmah. Is there a best approach to maintaining the logs, both storing and automating archiving or deleting?
My web development is based on angular. Any suggestion for aggregating logs, like what kind of logs would be generated?
Yes, by default, logs are not automatically deleted (with the exception of Application Logging (Filesystem)). To automatically delete logs, set the Retention Period (Days) field. You could automate the deletion by leveraging KUDU Virtual File System (VFS) Rest API. For a sample script, checkout this discussion thread for a similar approach:
How can you delete all log files from an Azure WebApp using powershell?
Just to highlight, these are the logging that you could capture on WebApps:
• Detailed Error Logging
• Failed Request Tracing
• Web Server Logging
• Application logging - you can turn on the file system option temporarily for debugging purposes. This option turns off automatically in 12 hours. You can also turn on the blob storage option to select a blob container to write logs to.
For log directory information kindly refer to this document: https://learn.microsoft.com/azure/app-service/troubleshoot-diagnostic-logs

Azure VM diagnostics for SQL server not showing in log analytics

I have enabled sql server as part of performance counters. but when I a check log analytics or under metrics or Monitor. I see nothing. does it take time for data to come through? or some more setting is required.
AFAIK yes it does take some time of ~15 minutes. Before you verify SQL Perf related logs from Log Analytics, I would suggest you to double check whether SQLServer related performance counters are already added under YourLogAnalyticsWorkspaceName -> AdvancedSettings tile -> Data -> WindowsPerformanceCounters or not.
You may already be aware of this but as you are referring to VM level monitoring stuff so I would recommend you to read through this and this tutorials to understand about a new feature 'Virtual Machines (preview)' which is basically seen as a new tile under Azure Monitor.
Hope this helps!! Cheers!!

Possible to view azure performance diagnostics via Perfmon

I've setup a 150plus performance counters via diagnostics.wascfg file. The counters are appearing in wadperformance table.
When I logged on to azure VM and used Permon tool I could not see any of these counters setup. Please help me understand, how does it work?
One way to view this data is through 3rd party tools like Cerebrata's Azure Management Studio or Azure Diagnostics Manager (http://www.cerebrata.com). These tools essentially fetches the data from WADPerformanceCountersTable table and displays them in a Perfmon like UI.
If you want to view the data locally on your computer through Perfmon, do take a look at this blog post: http://blogs.msdn.com/b/developertofu/archive/2010/08/17/announcing-the-perfmon-friendly-azure-log-viewer-plug-in.aspx which talks about an extension to Microsoft's Windows Azure MMC (not sure if this tool is still supported). This tool again fetches the data from diagnostics table, converts them into a format Perfmon understands.
UPDATE
When I logged on to azure VM and used Permon tool I could not see any
of these counters setup. Please help me understand, how does it work?
Coming to your question, if I understand correctly your expectation is that when you launch perfmon you should see the counters which you have set already in the list. I don't think it's possible. When you configure Windows Azure Diagnostics (WAD) for capturing performance counters, basically you're telling WAD process to read values for the specified performance counters every "x" seconds/minutes and transfer this data into Windows Azure Storage every "y" minutes/hours. Perfmon is a client utility which has no idea about WAD. One possibility (though I have not tried it) is to launch Perfmon and configure the counters it needs to capture when your VM starts. That way when you RDP into your VM, you'll see Perfmon running and collecting the data for you.

Which kind of Azure diagnositcs log stores the data for webrole instance count change, start, stop, Upgrade etc.?

Which kind of Azure diagnositcs log stores the data for webrole instance count change, start, strop, Upgrade etc.? I need to store this logs for my application.
Windows Azure Role instance count data is not stored through Windows Azure Diagnostics log. Windows Azure Diagnostics is designed to store log information inside the instance related to resources, application so you can get such data from it.
However instance specific data such as Start, Stop, Ready etc is already stored in your VM outside Azure Diagnostics scope (mean you have nothing to do in your VM to get this data, it is logged by default). This data is stored directly by Windows Azure VM AppAgent and the size could be in several Megabytes so you sure can RDP to Azure VM and take a look at this log data. I will not suggest you to move this log data from Azure VM as it may not be any use for you.
To see such log you just need to RDP to your Azure VM and visit C:\logs folder and the log will be in file name i.e. WaAppAgent.###.log and opening these log files you can see the current instance status:
[00000011] [06/07/2012 12:01:03.01] WaAppAgent Heartbeat.
[00000011] [06/07/2012 12:01:03.01] Role ***.MainWebRole_IN_0 is reporting state Ready.
[00000012] [06/07/2012 12:01:04.32] Role ***.MainWebRole_IN_0 has current state Started, desired state Started, and goal state execution status StartSucceeded.
[00000011] [06/07/2012 12:01:08.01] WaAppAgent Heartbeat.
As long as your role is running such file are accumulating in several counts so while it is good to have this info however I just don't see that you are going to get any big value for it.
I have written a tool to summarize these logs to tell when the last time role was started or stopped so you can try here: http://azurevmassist.codeplex.com/

Resources