I am trying to get left outer join or join with extend or mv-expand on Graph API to list all non-compliant disk with specific path (non-comp reason). In other words, extracting fields from policy and fields from resource e.g.-
resources
| where type == "microsoft.compute/disks" and subscriptionId == '3mmmmm333333####e35'
| join kind=leftouter (policyresources
| where type == "microsoft.policyinsights/policystates"
Some resource I am going through are https://www.youtube.com/watch?v=r_3Ydr6fCHQ and https://github.com/globalbao/azure-resource-graph/blob/master/README.md but the join statement to correlate resource query are not working with both outer or join. Maybe I need to narrow down to related item under the resource and resource policy?
Azure Management Talk: Azure Resource Graph Zero to Hero - YouTube
In this session, Microsoft Consultant Billy York will go over the basics of Azure Resource Graph, including how Kusto Query Language (KQL) is used and its li...
If anyone has similar report extraction code with leftouter join between policy with compute resource that would be helpful
KQL join query to extract compliance report from resource. To get some fields from resource and some from compliance related to resource
To achieve, KQL query to extract compliance disk resources:
By referring MSDoc, I've included the relevant part of the query and tried by changing the resource type
"resourceType = "microsoft.compute/disks" , I was able to receive the expected results.
PolicyResources
| where type =~ 'Microsoft.PolicyInsights/PolicyStates'
| extend complianceState = tostring(properties.complianceState)
| extend
resourceId = tostring(properties.resourceId),
resourceType = "microsoft.compute/disks",
policyAssignmentId = tostring(properties.policyAssignmentId),
policyDefinitionId = tostring(properties.policyDefinitionId),
policyDefinitionReferenceId = tostring(properties.policyDefinitionReferenceId),
stateWeight = iff(complianceState == 'NonCompliant', int(300), iff(complianceState == 'Compliant', int(200), iff(complianceState == 'Conflict', int(100), iff(complianceState == 'Exempt', int(50), int(0)))))
| summarize max(stateWeight) by resourceId, resourceType
| summarize counts = count() by resourceType, max_stateWeight
| summarize overallStateWeight = max(max_stateWeight),
nonCompliantCount = sumif(counts, max_stateWeight == 300),
compliantCount = sumif(counts, max_stateWeight == 200),
conflictCount = sumif(counts, max_stateWeight == 100),
exemptCount = sumif(counts, max_stateWeight == 50) by resourceType
| extend totalResources = todouble(nonCompliantCount + compliantCount + conflictCount + exemptCount)
| extend compliancePercentage = iff(totalResources == 0, todouble(100), 100 * todouble(compliantCount + exemptCount) / totalResources)
| project resourceType,
overAllComplianceState = iff(overallStateWeight == 300, 'noncompliant', iff(overallStateWeight == 200, 'compliant', iff(overallStateWeight == 100, 'conflict', iff(overallStateWeight == 50, 'exempt', 'notstarted')))),
compliancePercentage,
compliantCount,
nonCompliantCount,
conflictCount,
exemptCount
Output:
I've pulled up all the relevant fields here, but you can extend and project the fields you want based on your requirements.
Related
I am trying to get all data disks attached to a Azure VM from Azure Resource Graph Query. I am able to get specific data disk by specifying the index( for example properties.storageProfile.dataDisks[0].name for first disk) but but how do I get this dynamically when more than 1 data disks are attached.
Row per disk
resources
| where ['type'] == 'microsoft.compute/virtualmachines'
| mv-expand with_itemindex = i properties.storageProfile.dataDisks
| extend DataDiskName = properties_storageProfile_dataDisks.name
,DataDiskSizeGB = properties_storageProfile_dataDisks.diskSizeGB
,DataDiskSizeType = properties_storageProfile_dataDisks.managedDisk.storageAccountType
Column per disk
resources
| where ['type'] == 'microsoft.compute/virtualmachines'
| mv-expand with_itemindex=i dataDisk = properties.storageProfile.dataDisks
| extend dataDisk = pack_array(dataDisk.name, dataDisk.diskSizeGB, dataDisk.managedDisk.storageAccountType)
| summarize dataDisk0 = anyif(dataDisk, i == 0)
,dataDisk1 = anyif(dataDisk, i == 1)
,dataDisk2 = anyif(dataDisk, i == 2)
,dataDisk3 = anyif(dataDisk, i == 3)
,dataDisk4 = anyif(dataDisk, i == 4)
by id
I am very new with the sintaxis of Kusto query. My goal is to create a kusto query to retreive which Logic App has a system error and in which action the error was located. Additionally, I would like to know which connector, this failed action belongs. For example, If the action "Move Email" failed I would like to have the connector name, in this case, Office 365 Outlook or something similar in order to classify the action.
My query to achieve this goal was based on the Table "AzureDiagnostics":
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.LOGIC"
| where Category == "WorkflowRuntime"
| where status_s == "Failed"
| where code_s !has 'ActionFailed'
| where OperationName has "workflowActionCompleted" or OperationName has "workflowTriggerCompleted"
| extend ResourceName = coalesce(resource_actionName_s, resource_triggerName_s)
| extend ResourceCategory = substring(OperationName, 34, strlen(OperationName) - 43)
| project
LogicAppName = resource_workflowName_s,
ResourceCategory,
ResourceName,
LogicAppId = resource_runId_s,
ErrorCode = code_s,
ErrorMessage = error_message_s,
ErrorTime = format_datetime(startTime_t,'dd.MM.yyyy')
The connector name will give me the possibility to classify the failed logic apps and this way I can create a report to show which type of connector we are having issues.
Thanks in advance for your help or another workarround to classify the failed logic apps.
After reproducing from our end, One of the workarounds is that we can fetch the action name of the failed step along with the status using the below query.
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.LOGIC"
| where Category == "WorkflowRuntime"
| where status_s == "Failed"
| extend Status = code_s
| project
LogicAppName = resource_workflowName_s,
ResourceRunID = resource_runId_s,
Operation = OperationName,
ActionName = coalesce(resource_actionName_s, resource_triggerName_s),
Status
RESULTS:
Updated Answer
There is no direct way to get the connector's name. One of the workarounds would be using tracked properties to save the connector name and retrieve it through logs. Not a perfect way but this is one of the workarounds that achieves the requirement.
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.LOGIC"
| where OperationName == "Microsoft.Logic/workflows/workflowActionCompleted"
| where status_s == "Failed"
| extend Status = code_s
| project
LogicAppName = resource_workflowName_s,
ResourceRunID = resource_runId_s,
Operation = OperationName,
ActionName = coalesce(resource_actionName_s, resource_triggerName_s),
Status,
ConnectorName = trackedProperties_ConnectorName_s
Below is the flow in my logic app
Failed Run
In logs
I have this query that works in Azure logs when i set the scope to the specific application insights I want to use
let usg_events = dynamic(["*"]);
let mainTable = union pageViews, customEvents, requests
| where timestamp > ago(1d)
| where isempty(operation_SyntheticSource)
| extend name =replace("\n", "", name)
| where '*' in (usg_events) or name in (usg_events)
;
let queryTable = mainTable;
let cohortedTable = queryTable
| extend dimension =tostring(client_CountryOrRegion)
| extend dimension = iif(isempty(dimension), "<undefined>", dimension)
| summarize hll = hll(user_Id) by tostring(dimension)
| extend Users = dcount_hll(hll)
| order by Users desc
| serialize rank = row_number()
| extend dimension = iff(rank > 5, 'Other', dimension)
| summarize merged = hll_merge(hll) by tostring(dimension)
| project ["Country or region"] = dimension, Counts = dcount_hll(merged);
cohortedTable
but trying to use the same in grafana just gives an error.
"'union' operator: Failed to resolve table expression named 'pageViews'"
Which is the same i get in azure logs if i dont set the scope to the specific application insights resource. So my question is. how do i make it so grafana targets this specific scope inside the logs? The query jsut gets the countries of the users that log in
As far as I know, Currently, there is no option/feature to add Scope in Grafana.
The Scope is available only in the Azure Log Analytics Workspace.
If you want the Feature/Resolution, please raise a ticket in Grafana Community where all the issues are officially addressed.
I have a issue. I want to know how can I join two columns in one.
I want to join the "OS" and "sku" columns in one with the name "OS"
This is my KQL:
Kusto Query on Azure Resource Graph
Resources
| where type == "microsoft.compute/virtualmachines"
| extend OS = properties.storageProfile.imageReference.offer
| extend sku = properties.storageProfile.imageReference.sku
| project OS, sku, name, nic = (properties.networkProfile.networkInterfaces)
| mvexpand nic
| project OS, sku, name, nic_id = tostring(nic.id)
| join (
Resources
| where type == "microsoft.network/networkinterfaces"
| project nic_id = tostring(id), properties) on nic_id
| mvexpand ipconfig = (properties.ipConfigurations)
| extend subnet_resource_id = split(tostring(ipconfig.properties.subnet.id), '/'), ipAddress = ipconfig.properties.privateIPAddress
| order by name desc
| project vmName=(name), OS, sku, vnetName=subnet_resource_id[8], subnetName=subnet_resource_id[10], ipAddress
This is my result:
I need like this:
Can anyone help me, thanks so much.
I've tried to use the "union" operator, but I can't make it work.
I have used these reference link:
Azure Docs Link 1
Azure Docs Link 2
Azure Docs Link 3
If you want to combine two strings - you can use strcat() function:
Resources
| where type == "microsoft.compute/virtualmachines"
| extend OS = properties.storageProfile.imageReference.offer
| extend sku = properties.storageProfile.imageReference.sku
| project OS, sku, name, nic = (properties.networkProfile.networkInterfaces)
| mvexpand nic
| project OS, sku, name, nic_id = tostring(nic.id)
| join (
Resources
| where type == "microsoft.network/networkinterfaces"
| project nic_id = tostring(id), properties) on nic_id
| mvexpand ipconfig = (properties.ipConfigurations)
| extend subnet_resource_id = split(tostring(ipconfig.properties.subnet.id), '/'), ipAddress = ipconfig.properties.privateIPAddress
| order by name desc
| project vmName=(name), OS = strcat(OS, ' ', sku), vnetName=subnet_resource_id[8], subnetName=subnet_resource_id[10], ipAddress
We are showing SLA charts for URLs, VPN and VMs for that if there is any planned scheduled maintenance we want to exclude that timings in KQL SLA charts as its known downtime.
We are disabling Alerts via powershell during this time we are passing below columns to Loganalytics custom table.
"resourcename": "$resourcename",
"Alertstate": "Enabled",
"Scheduledmaintenance" : "stop",
"Environment" : "UAT",
"timestamp": "$TimeStampField",
Now we want to use join condition SLA charts queries with custom table data and exclude the time range in SLA charts during scheduled maintenance.
Adding query as per request
---------------------------
url_json_CL
| where Uri_s contains "xxxx"
| extend Availablity = iff(StatusCode_d ==200,1.000,0.000)
| extend urlhit = 1.000
| summarize PassCount = sum(Availablity), TestCount = sum(urlhit) by Uri_s ,ClientName_s
| extend AVLPERCENTAGE = ((PassCount / TestCount ) * 100)
| join kind=leftouter
( scheduledmaintenance2_CL
| where ResourceName_s == "VMname"
| where ScheduledMaintenance_s == "start"
| extend starttime = timestamp_t)
on ClientName_s
| join kind= leftouter
(scheduledmaintenance2_CL
| where ResourceName_s == "VMname"
| where ScheduledMaintenance_s == "stop"
| extend stoptime = timestamp_t )
on ClientName_s
| extend excludedtime=stoptime - starttime
| project ClientName_s, ResourceName_s, excludedtime, AVLPERCENTAGE , Uri_s
| top 3 by ClientName_s desc
You can perform cross-resource log queries in Azure Monitor
Using Application Insights explorer we can query Log analytics workspace custom tables as well.
workspace("/subscriptions/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx/resourcegroups/rgname/providers/Microsoft.OperationalInsights/workspaces/workspacename").Event | count
Using Log Analytics logs explorer you can query the Application Insights Availability Results
app("applicationinsightsinstancename").availabilityResults
You can use any of the above options to query the required tables and join the tables. Please refer to this documentation on joins.
Additional documentation reference.
Hope this helps.