How do I retrieve the lasttime a pipeline failed in Log analytics azure - azure

I need to retrieve the last run time a pipeline failed by using the below query but it doesn't work.
ADFPipelineRun
| where Status == "Failed" AND max(TimeGenerated)
So with the below example, I want to retrieve only the ones highlighted red on the screenshot. I have different pipelines and would like to retrieved only the last runtime of all the pipelines that failed.
For example in the above screenshot, I will like to retrieve only:
I need the Kusto query that can do that.

Please use the query below. It returns the expected result as per my testing:
ADFPipelineRun
| where Status == "Failed"
| summarize TimeGenerated= max(TimeGenerated) by PipelineName, Status 

Related

Trying to query Azure Resource Graph Explorer for NSGs with missing rules

The following query fails with 2 ParserFailure errors, both on line 5. At least that's where the query builder shows the red curly line.
The intention of this query is probably obvious to the Azure KQL initiates, but I'll explain nonetheless just to make sure it's clear.
This query should list all NSGs that do not have either one of the rules named "AllowThis" or "AllowThat".
Resources
| where type == "microsoft.network/networksecuritygroups"
| where isnotempty(properties.securityRules)
| where not(properties.securityRules
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat"))
| project NSGName = name
| order by NSGName asc
It would even be nicer if the table shows the actual missing rule(s) for the listed NSGs, but I have no idea where to start with that.
Does anyone have a working version of this type of query? Having to go through a lot of NSGs manually can't be the answer.
I have tried multiple variations of the query, but I couldn't find a single working version.
Below are my findings and observations from the query posted in question.
Lines 1 to 3 looks good and will give you list of NSG resources which has values for "securityRules" field.
For line number 4
| where not(properties.securityRules)
I am not sure what are you trying to achieve in this step. The not() takes bool values as mentioned in the documentation.
For line number 5
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat")
There is no need to use tolower() when you are using =~ as this supports case-insensitive match. Also under "securityRules" in NSG json object there is no field named as "ruleName", however there is a field "name". Please find the document for the same - Link. You can use the same documentation to check for the fields available to query NSG resource data.
When you are trying to write condition for "AllowThis" or "AllowThat" in Azure Resource Graph Explorer you should use the syntax properties.securityRules.name == "allowthis" or properties.securityRules.name == "allowthat"
If you write anything within quotes it will be taken as single string. Hence in your query "allowthis|allowthat" will be considered as a single string.

Nesting InTune data properly for a specific KQL query

I am using Windows Update for Business to pull in InTune data to track patching for my org. This data is stored in a Log Analytics Workspace and can be queried using KQL.
I am trying to write a specific KQL query that shows two categories with nested dropdowns.
Ideal Format for output of this query
The current query I have built is:
let _SnapshotTime = datetime(2023-01-18T06:00:00Z);
UCClientUpdateStatus
| where TimeGenerated == _SnapshotTime
| join (UCClient | where TimeGenerated == _SnapshotTime) on DeviceName
| summarize arg_max(TimeGenerated, *) by OSSecurityUpdateStatus, TargetKBNumber, DeviceName
This returns too much data and is not quite structured the way I'm looking for. Has anyone here had any luck with creating a KQL query that returns Intune patching data status in a format similar to above?

Passing dynamic content inside a SQL Query in lookup stage of Azure Data Factory

I'm using a lookup stage as a source to fetch some data and i want to pass that output as the input to the next lookup stage. I tried adding #activity('Step1').output.firstRow.Col and it failed with scalar variable was not being declared. Also tried with #{activity('Step1').output.firstRow.col} which failed and the log shows only default expressions supported. Please help if it is possible.
I have accomplished this using dataflow, but considering the performance i would like to know if it can be done in a pipeline.
Please try this:
Query:select * from dbo.test5 where col = '#{activity('Lookup1').output.firstRow.col}'
Output:

Stream Analytics UDF works in Test but not in Job

I need to parse a JSON data in Stream Analytics,
Below is the sample is am using,
SELECT
UDF.parseData(GetRecordPropertyValue(GetArrayElement(A.message,0), 'raw')).intent as 'rawData'
FROM
AppInsightMessages A
I can able to parse the intent from the field. This is a custom logging required.
However it is not working in Stream analytics job.
I am getting error like
Stream Analytics job has validation errors: Query compilation error: Expression is not supported: 'udf . parseData
Tried with CAST ing to string to record also. no luck.
What I am doing wrong ?
thanks in advance ..
Usually, this is due to trying to merge multiple stages into a single expression.
Please try splitting the processing to several steps:
With UDFStep AS (
SELECT
UDF.parseData(GetRecordPropertyValue(GetArrayElement(A.message,0), 'raw'))
FROM
AppInsightMessages A
)
SELECT intent as rawData
FROM UDFStep
BTW, you don't need to quote the 'rawData'.

Understanding Kusto

I am trying to understand Kusto (Log Analytics Query Language in Azure).
According to the documentation;
To retrieve , project name and resultsCode from the dependencies table, I need to enter the following:
dependencies
| project name, resultCode
The machines I have subscribed to do not have this table.
I am using the heartbeat table and trying to retrieve computer and category like so:
Heartbeat
| Category, Computer , IsGatewayInstalled
I however get the following error:
Query could not be parsed at 'Category' on line [2,2]
Token: Category Line: 2 Position: 2
This seems trivial and will appreciate any pointers on this.
the error you're getting is due to the fact there's no valid operator after the pipe (|), you should use the project operator before specifying the column names you want to retrieve

Resources