I want to be able to find specific logs from AKS container, that have diagnostic configured for storage account. I'm able to generate this kind of query from log analytics:
ContainerLog
| join kind = inner KubePodInventory on $left.ContainerID == $right.CointainerID
| where Namespace == "default" and LogEntry contains "error"
| project TimeGenerated, LogEntry, ContainerName
showing me container logs, yet I'm not able to find same output in actual log files, saved in storage account. Shouldn't both reciever services have same logs available? Here's the list of log types that aks cluster generates.
Here's the output from log analytics query:
Related
I have an Azure Container Instance running a docker image of Coldfusion application.
I also configured this Container Instance to Log Analytics Workspace.
Now I need the Coldfusion logs to show at Log Analytics Workspace. I am unable to get those logs.
Is there a way to get those logs at Log Analytics Workspace.
Assuming you have integrated the log analytic workspace after creation of azure container instance.
If this is the case it won't works for you, for storing logs of container instance we have to create the log analytic workspace while creating the azure container instance.
So you will need to delete and recreate the container instance.
You can refer this microsoft document for detailed information of how to store the logs in log analytic workspace.
you can also refer this link. for custom log.
Creating the log analytics workspace first and then providing the workspace ID and workspace key at container group creation worked fine for me (no need to create them both "at the same time"). Note, that it does take up to 10 minutes (according to the docs) for the ContainerInstanceLog_CL table to populate with your container's console logs.
Various programmatic ways to specify this at container creation, pertinent bit of C# client code shown below.
var containerGroupData = new ContainerGroupData(location, new[] { container }, ContainerInstanceOperatingSystemType.Linux);
var logAnalyticsWorkspaceId = ConfigurationManager.AppSettings["LogAnalyticsWorkspaceId"];
var logAnalyticsWorkspaceKey = ConfigurationManager.AppSettings["LogAnalyticsWorkspaceKey"];
containerGroupData.DiagnosticsLogAnalytics = new ContainerGroupLogAnalytics(logAnalyticsWorkspaceId, logAnalyticsWorkspaceKey);
I have configured the following Kusto query to gather info of Azure deployments in our subscription.
AzureActivity
| where ResourceProviderValue startswith "Microsoft.resources"
| where ActivitySubstatusValue contains "Created"
This query will display various information about new deployments across all resource providers. These are the columns that are included when running the query:
TenantId, SourceSystem, CallerIpAddress, CategoryValue, CorrelationId, Authorization, Authorization_d, Claims, Claims_d, Level, OperationNameValue, Properties, Properties_d, Caller, EventDataId, EventSubmissionTimestamp, HTTPRequest, OperationId, ResourceGroup, ResourceProviderValue, ActivityStatusValue, ActivitySubstatusValue, Hierarchy, TimeGenerated, SubscriptionId, OperationName, ActivityStatus, ActivitySubstatus, Category, ResourceId, ResourceProvider, Resource, Type, _ResourceId
however not the SKU of the deployed resource.
I want to monitor the deployed resources in a subscription, including SKU's. Is it possible to include the SKU of deployed resources of all resource providers in a Log Analytics query from Azure Activity logs?
We have tested this in our local environment , Below analysis are based on our observations.
The Activity log is a platform log in Azure that provides insight into subscription-level events. This includes such information as when a resource is modified or when a virtual machine is started.
Is it possible to include the SKU of deployed resources of all
resource providers in a Log Analytics query from Azure Activity logs?
No it is not possible,To validate this we have created a NetworkSecurityGroup & DataDisk in our subscription.
when we are trying to fetch those resources logs using AzureActivity table & applying a filter of ResourceProviderValue == "MICROSOFT.RESOURCES" the actual Names for those resources that are passed by the user while creating are not getting populated as shown below.
Instead of using ResourceProviderValue == "MICROSOFT.RESOURCES" if you use the resource specific resource provider of the resource that you are deploying then you will be able to pull the SKU of that resource.
Here Disk comes under Microsoft.Compute resource provider if we apply some additional filters to the above AzureActivity log we can pull the sku of the particular resource.
So, I can see create_or_update logs of my VM on activity logs. There is no filter just to get the create logs as much as I am aware.
So is there any way where I can just see the create logs of a VM using API or commands?
You can follow below steps to achieve your requirement
You need to enable diagnostic settings to activity logs.
refer https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log#send-to-log-analytics-workspace for enabling the diagnostic settings.
Once the Log analytics workspace is established, you can query the logs as
AzureActivity
| where OperationName == 'Create or Update Virtual Machine' and ActivitySubstatusValue == 'Created'
| order by TimeGenerated desc
above output will show only the Create operations. You can further filter it based on your requirement.
I'm looking for custom policy to connect and get activity/audit logs from Azure to Log Analytics workspace. There are not build in policy to this so it would need to be done with custom policy. Has anybody created or seen this kinda policy because I have not been able to find? Policy should be AuditIfNotExists and should take Log Analytics workspace as a parameter. I'm not policy specialist so finding policy, would help a lot.
There is no policy to set this up but it is possible to set it up. I got this information from this tutorial:
"The Azure policy compliance status is logged in the Azure subscription’s Activity logs. The Azure Log Analytics workspace can be configured to collect Azure Activity logs from any subscriptions in the same tenant. Azure Monitor alert rules can then be created to execute queries in the Log Analytics workspace on a schedule and generate alerts when non-compliant resources are detected by the query. "
Connect log Analytics workspace to desired subscription
Add Kusto queries in the workspace to get information needed:
Here is an example of a kusto query for Get a list of non-compliant resources from a single policy (using “audit-resources-without-tags-policyDef” definition as an example):
let policyDefId = 'audit-resources-without-tags-policyDef'; AzureActivity | where Category == 'Policy' and Level != 'Informational' | extend p=todynamic(Properties) | extend policies=todynamic(tostring(p.policies)) | mvexpand policy = policies | where policy.policyDefinitionName in (policyDefId) | distinct ResourceId
You can set up alerts using Azure Monitor with a custom log search
I have an Azure Load Balancer,
I have gone into the Load Balancer resource, Diagnostics logs and ticked 'Send to Log Analytics' and set to my Azure Log Analytics OMS Workspace.
But when I go into the Log Analytics and run:
AzureDiagnostics
| where Category == "LoadBalancerProbeHealthStatus" and TimeGenerated > ago(3d)
| project ResourceGroup, Resource, TimeGenerated, port_d, totalDipCount_d, dipDownCount_d, healthPercentage_d
I don't get any results, can anyone help my understand how to make this work?
Check to see if the Azure Diagnostics table in Log Analytics has reached the max 500 columns.
Can Run the following query to confirm:
AzureDiagnostics
| getschema
| summarize AggregatedValue = count(ColumnName)