I have an issue with showing specific resources with azure kusto query.
what i want is to write a kusto query that show only database resources and server resources in azure.
i have written following query regarding Databases:
resources
| where type in ("microsoft.sql/servers/databases","microsoft.dbforpostgresql/servers","microsoft.azuredata/postgresinstances","microsoft.dbformariadb/servers","microsoft.dbformysql/flexibleservers","microsoft.dbformysql/servers","microsoft.dbforpostgresql/flexibleservers","microsoft.dbforpostgresql/servergroups","microsoft.kusto/clusters/databases","microsoft.sql/managedinstances/databases","microsoft.synapse/workspaces/sqldatabases","ravenhq.db/databases","microsoft.documentdb/databaseaccounts")
| summarize Amount=count() by type
But when i execute the query it shows me two Databases even though i only have create one, the extra one is a "master" which should not be included because there is only one resource in the resource group
i have also tried with the following query:
resources
| where type contains "database" | distinct type
| summarize Amount=count() by type
But then the issue is that it doesnt include all the db's that doesnt have the word "database" in the type name for example "microsoft.azuredata/postgresinstances"
so the question is, how do i write a query that shows ALL the databases on my dashboard.
The second part of the question which is similar to the previous with databases is how i show all the Servers.
I have tried with the following queries:
resources
| where split(type,"/")[array_length(split(type,"/"))] contains "servers"
it gave me no result even though i had a server.
then i tried:
resources
| where type contains "/server" | distinct type
| summarize Amount=count() by type
that didnt work because it also returned all the database resources cuntaining the work "server"
i have tried to look through microsofts documentation, but cannot figure out what to do.
If you don't want the master databases (which are the databases that store system level data in SQL databases, you can simply filter them out:
resources
| where type in ("microsoft.sql/servers/databases","microsoft.dbforpostgresql/servers","microsoft.azuredata/postgresinstances","microsoft.dbformariadb/servers","microsoft.dbformysql/flexibleservers","microsoft.dbformysql/servers","microsoft.dbforpostgresql/flexibleservers","microsoft.dbforpostgresql/servergroups","microsoft.kusto/clusters/databases","microsoft.sql/managedinstances/databases","microsoft.synapse/workspaces/sqldatabases","ravenhq.db/databases","microsoft.documentdb/databaseaccounts")
| where name type != "microsoft.sql/servers/databases" or name != "master"
| summarize Amount=count() by type
Regarding the 2nd question, this should work since the has operator will only match whole tokens (and a slash separates tokens):
resources | where type has "servers"
Related
The following query fails with 2 ParserFailure errors, both on line 5. At least that's where the query builder shows the red curly line.
The intention of this query is probably obvious to the Azure KQL initiates, but I'll explain nonetheless just to make sure it's clear.
This query should list all NSGs that do not have either one of the rules named "AllowThis" or "AllowThat".
Resources
| where type == "microsoft.network/networksecuritygroups"
| where isnotempty(properties.securityRules)
| where not(properties.securityRules
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat"))
| project NSGName = name
| order by NSGName asc
It would even be nicer if the table shows the actual missing rule(s) for the listed NSGs, but I have no idea where to start with that.
Does anyone have a working version of this type of query? Having to go through a lot of NSGs manually can't be the answer.
I have tried multiple variations of the query, but I couldn't find a single working version.
Below are my findings and observations from the query posted in question.
Lines 1 to 3 looks good and will give you list of NSG resources which has values for "securityRules" field.
For line number 4
| where not(properties.securityRules)
I am not sure what are you trying to achieve in this step. The not() takes bool values as mentioned in the documentation.
For line number 5
| where (tolower(tostring(properties.securityRules.ruleName)) =~ "allowthis|allowthat")
There is no need to use tolower() when you are using =~ as this supports case-insensitive match. Also under "securityRules" in NSG json object there is no field named as "ruleName", however there is a field "name". Please find the document for the same - Link. You can use the same documentation to check for the fields available to query NSG resource data.
When you are trying to write condition for "AllowThis" or "AllowThat" in Azure Resource Graph Explorer you should use the syntax properties.securityRules.name == "allowthis" or properties.securityRules.name == "allowthat"
If you write anything within quotes it will be taken as single string. Hence in your query "allowthis|allowthat" will be considered as a single string.
I am using Windows Update for Business to pull in InTune data to track patching for my org. This data is stored in a Log Analytics Workspace and can be queried using KQL.
I am trying to write a specific KQL query that shows two categories with nested dropdowns.
Ideal Format for output of this query
The current query I have built is:
let _SnapshotTime = datetime(2023-01-18T06:00:00Z);
UCClientUpdateStatus
| where TimeGenerated == _SnapshotTime
| join (UCClient | where TimeGenerated == _SnapshotTime) on DeviceName
| summarize arg_max(TimeGenerated, *) by OSSecurityUpdateStatus, TargetKBNumber, DeviceName
This returns too much data and is not quite structured the way I'm looking for. Has anyone here had any luck with creating a KQL query that returns Intune patching data status in a format similar to above?
I'm trying to create Computer group for Update management from log analytics saved search but can't figure it totally out.
When using this query, it will list all computer that has "test" in their name. Working fine, but not exactly what I'm looking for.
Heartbeat
| where Computer contains "test"
| distinct Computer
But when I want to add computers by their specific name, I get error. Any this how can I get result by using computers specific names?
[
Heartbeat
| where Computer contains "test" and "test2" and "test3"
| distinct Computer
Also tried with "or" "has" "==" but could not get it working.
Please try the in operator.
Sample query looks like below:
Heartbeat
| where Computer in ("test", "test2", "test3")
| distinct Computer
Where is a list of all (valid, built-in) Spark properties?
The list of Available Properties on the official Spark documentation does not include all (valid, built-in) properties for the current stable version of Spark (2.4.4 as of 2020-01-22). An example is spark.sql.shuffle.partitions, which defaults to 200. Unfortunately, properties like this one do not appear to be accessible via any of sparkConf.getAll(), sparkConf.toDebugString(), or sql.("SET -v").
Rather, built-in defaults appear to be accessible only by explicit name (i.e. sparkConf.get("foo")). However, this does not help me since the exact property name must be already known, and I need to survey properties that I don't already know about for debugging/optimization/support purposes.
you can use.
sql("SET -v").show(500,false)
Which will give you a near complete list not including the internal properties.
+-----------------------------------------------------------------+-------------------------------------------------+
|key |value |
+-----------------------------------------------------------------+-------------------------------------------------+
|spark.sql.adaptive.enabled |false |
|spark.sql.adaptive.shuffle.targetPostShuffleInputSize |67108864b |
|spark.sql.autoBroadcastJoinThreshold |10485760 |
|spark.sql.avro.compression.codec |snappy |
|spark.sql.avro.deflate.level |-1 |
...
I don't think this is the complete answer, but it can help. It will show more properties than your alternatives. At least will show options modified by some kind of middle ware, like Livy.
Set this parameter:
spark.logConf=true
Now all your session configuration will be saved in yarn log at level INFO. Do a yarn logs -applicattionID <your app id> and search for spark.app.name= to find your session properties.
Another problem is that you will see the properties values just after executing the job.
I am trying to understand Kusto (Log Analytics Query Language in Azure).
According to the documentation;
To retrieve , project name and resultsCode from the dependencies table, I need to enter the following:
dependencies
| project name, resultCode
The machines I have subscribed to do not have this table.
I am using the heartbeat table and trying to retrieve computer and category like so:
Heartbeat
| Category, Computer , IsGatewayInstalled
I however get the following error:
Query could not be parsed at 'Category' on line [2,2]
Token: Category Line: 2 Position: 2
This seems trivial and will appreciate any pointers on this.
the error you're getting is due to the fact there's no valid operator after the pipe (|), you should use the project operator before specifying the column names you want to retrieve