I'm new to KQL and I'm trying to list all Azure SQL Databases with the word "_old" in their name.
My protoquery is this and it works:
// Find "_old" Databases
Resources
| where type =~ 'microsoft.sql/servers/databases'
| where * contains "old"
| project resourceName = name,type,sku.name,sku.tier,tags.createdBy,tags.customerCode,tags.managedBy, resourceGroup, subscriptionId, location
But here the WHERE clause is bombing everywhere across all columns.
Is there a more fashionable way to search into Azure ideally with more words like:
_old
.old
_test
.test
_dev
.dev
I have to clean up unused resources and I have to search per resource name.
AS of today, Azure Resource Graph supports a quite limited subset of KQL. E.g. has_any is currently not supported.
If needed, you can uncomment the commented line to improve performance.
Resources
| where type == 'microsoft.sql/servers/databases'
// | where name has_cs "old" or name has_cs "dev" or name has_cs "test"
| parse-where kind=regex name with ".*[._]" suffix
| where suffix in ("old", "dev", "test")
Thank you #PeterBonz, the name is what I was missing.
My code now works this way:
// Find Unused Virtual Machines
Resources
| where type has "microsoft.compute/virtualmachines"
| where name contains "old"
or name contains "dev"
or name contains "test"
| project resourceName = name,type,sku.name,sku.tier,tags.createdBy,tags.customerCode,tags.managedBy, resourceGroup, subscriptionId, location
I also found the clause has_any but I couldn't make it work:
let ComputerTerms = pack_array('old', 'dev', 'test');
// Find Unused Virtual Machines
Resources
| where type has "microsoft.compute/virtualmachines"
| where name has_any (ComputerTerms)
| project resourceName = name,type,sku.name,sku.tier,tags.createdBy,tags.customerCode,tags.managedBy, resourceGroup, subscriptionId, location
Related
I need to get the category name from category id using kusto query.
First i have got the most searched url from the website using the below kusto query and ran it in app insights logs
Requests
| where resultCode==200
| where url contains "bCatID"
| summarize count=sum(itemCount) by url
| sort by count
| take 1
From the above query i got the result like
https://www.test.com/item.aspx?idItem=123456789&bCatID=1282
So for corresponding categoryid=1282 i need to get the category name using kusto
you can use the parse operator.
for example:
print input = 'https://www.test.com/item.aspx?idItem=123456789&bCatID=1282'
| parse input with 'https://www.test.com/item.aspx?idItem=123456789&bCatID='category_id:long
parse_urlquery
print url ="https://www.test.com/item.aspx?idItem=123456789&bCatID=1282"
| extend tolong(parse_urlquery(url)["Query Parameters"]["bCatID"])
url
Query Parameters_bCatID
https://www.test.com/item.aspx?idItem=123456789&bCatID=1282
1282
Fiddle
Feedback to the OP query
Not an answer
KQL is case sensitive. The name of the table in Azure Application Insights is requests (and not Requests).
resultCode is of type string (and not integer/long) and should be compared to "200" (and not 200)
bCatID is a token and therefore can be searched using has or even has_cs, which should be preferred over contains due to performance reasons.
URLs can be used with different parameters. It might make more sense to summarize only by the Host & Path parts + the bCatID query parameter.
count is a reserved word. It can be used as alias only if qualified: ["count"] or ['count'] (or better not used at all).
sort followed by take 1 can be replaced with the more elegant top 1 by ...
requests
| where resultCode == "200"
| project url = parse_url(url), itemCount
| summarize sum(itemCount) by tostring(url.Host), tostring(url.Path), tolong(url["Query Parameters"]["bCatID"])
| top 1 by sum_itemCount
I need to be able to compare 2 rows of IP Subnets and tell if there is overlap.
For example:
In Row 1 I have a /24 and I need to check if this /24 is existing in Row 2 (either via the /24 or via the supernet /21 for instance)
so:
ROW 1: 192.168.2.0/24
ROW 2: 192.168.0.0/21
Result -> Row 1 exists in Row 2
I am not sure how to do this in Excel
Anybody any idea?
If you want to do it in powershell, you may use this script:
Clear-Host
#Import csv file with IPs (Delimiter ";")
$rowlist = Import-Csv -Path "C:\rows_directory\rowlist.csv" -Delimiter ";"
$row1 = $rowlist | Select-Object -ExpandProperty "row1"
$row2 = $rowlist | Select-Object -ExpandProperty "row2"
foreach($string in $row1) {
if($string -in $row2) {
Write-Output "ROW1: $string exist in ROW2"
}
}
I filled file with:
And result was:
ROW1: 123 exist in ROW2
For this I would create a function to find the base address (as UInt32 type) of the IP address in the concerned Subnet:
Function Get-IPBase($Address) {
$IP, $SubNet = $Address.Split('/', 2)
$Bytes = ([IPAddress]$IP).GetAddressBytes()
if ([BitConverter]::IsLittleEndian) { [Array]::Reverse($Bytes) }
[BitConverter]::ToUInt32($bytes, 0) -BAnd -BNot ([UInt32][Math]::Pow(2, $SubNet) - 1)
}
Example of what the function returns:
Get-IPBase 192.168.2.0/24
3221225472
Get-IPBase 192.168.0.0/24
3221225472
Than, do a self-join, using this Join-Object script/Join-Object Module (see also: In Powershell, what's the best way to join two tables into one?):
Import-CSV .\My.csv |Join -On { Get-IPBase $_.Row1 } -Eq { Get-IPBase $_.Row2 }
Please add more details to your question (as what you tried yourself and a sample list. See also: How to Ask) if you like a more in dept explanation or have problems to implement this.
I am trying to create different variables for the 'costcenter' tag. Each tag key "costcenter" has a respective value of "AB12" for example. I want to create a new variable in kusto to show that for every costcenter with the value of "AB12" I want to have this labeled as "HR Department".
I am unfamiliar with the syntax but need to get the value from costcenter that equals "AB12" and then store that as "Hr Department" so when I create a chart it shows HR department instead of "AB12"
resources
| extend cost = tostring(tags['costcenter'])
you can try using the case() function, or a mapping from actual value to desired value:
resources
| extend costcenter = tostring(tags['costcenter'])
| extend costcenter = case(costcenter == "AB12", "HR Department", costcenter)
| ...
let mapping = dynamic({"AB12":"HR Department", "AB23":"something else"});
resources
| extend costcenter = mapping[tostring(tags['costcenter'])]
| ...
I'm facing a weird behaviour (bug?) using the PowerShell command "Search-AzGraph".
When I use "Azure Resource Graph Explorer", I don't get this issue.
resourcecontainers
| where type =~ 'microsoft.resources/subscriptions/resourcegroups'
| join kind=leftouter (resources | project ResourceId=id, ResourceName=name, ResourceType=type, resourceGroup) on resourceGroup
(You can copy paste this KQL query directly on your Azure Resource Graph Explorer, it's to display all Azure resources with their Azure respective subscriptionId
With "Azure Resource Graph Explorer", after the "project" keyword to display the columns that I look for, I set "ResourceId" to display the Id of the resource. It works fine. I can see on the result that the column name changed.
But when using "Search-AzGraph", "ResourceId" values contain the value id from "resourcecontainers" instead of "resources".
If I remove the "ResourceId=", I got the same (correct) result as "Azure Resource Graph Explorer" but to access to it, I need to use the property "id1".
It's not a big deal but I wish to be able to use my own naming column.
Anyone got this experience or I miss something to get the result result on both cases?
Thanks
I finally found a workaround:
resourcecontainers
| where type =~ 'microsoft.resources/subscriptions/resourcegroups'
| join kind=leftouter (resources | project ResourceId=id, ResourceName=name, ResourceType=type, resourceGroup) on resourceGroup
| project ResourceGroupId=id, ResourceId, ResourceGroupName=name, ResourceName, tags
I added a "project" after my "join" to explicitly give a name to column "id" from "resourcecontainers", so the result don't mess up result (values) between "resourcecontainers" and "resource".
Here my full example of my PowerShell code, in case someone is interested:
$resources = Search-AzGraph -Subscription $subscriptionId -Query "resourcecontainers
| where type =~ 'microsoft.resources/subscriptions/resourcegroups'
| join kind=leftouter (resources | project ResourceId=id, ResourceName=name, ResourceType=type, resourceGroup) on resourceGroup
| project ResourceGroupId=id, ResourceId, ResourceGroupName=name, ResourceName, ResourceType, tags"
foreach ($resource in $resources) {
Write-Host "Resource Group Id: " $resource.ResourceGroupId
Write-Host " Resource Group Name: " $resource.ResourceGroupName
Write-Host " Resource Id: " $resource.ResourceId
Write-Host " Resource Name: " $resource.ResourceName
Write-Host " Resource Type: " $resource.ResourceType
}
I need to use powershell to resolve IP addresses via whois. My company filters port 43 and WHOIS queries so the workaround I have to use here is to ask powershell to use a website such as https://who.is, read the http stream and look for the Organisation Name matching the IP address.
So far I have managed to get the webpage read into powershell (example here with a WHOIS on yahoo.com) which is https://who.is/whois-ip/ip-address/206.190.36.45
So here is my snippet:
$url=Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
now if I do :
$url.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False HtmlWebResponseObject Microsoft.PowerShell.Commands.WebResponseObject
I see this object has several properties:
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
AllElements Property Microsoft.PowerShell.Commands.WebCmdletElementCollection AllElements {get;}
BaseResponse Property System.Net.WebResponse BaseResponse {get;set;}
Content Property string Content {get;}
Forms Property Microsoft.PowerShell.Commands.FormObjectCollection Forms {get;}
Headers Property System.Collections.Generic.Dictionary[string,string] Headers {get;}
Images Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Images {get;}
InputFields Property Microsoft.PowerShell.Commands.WebCmdletElementCollection InputFields {get;}
Links Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Links {get;}
ParsedHtml Property mshtml.IHTMLDocument2 ParsedHtml {get;}
RawContent Property string RawContent {get;}
RawContentLength Property long RawContentLength {get;}
RawContentStream Property System.IO.MemoryStream RawContentStream {get;}
Scripts Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Scripts {get;}
StatusCode Property int StatusCode {get;}
StatusDescription Property string StatusDescription {get;}
but every time I try commands like
$url.ToString() | select-string "OrgName"
Powershell returns the whole HTML code because it interprets the text string as a whole. I found a workaround dumping the output into a file and then read the file through an object (so every line is an element of an array) but I have hundreds of IPs to check so that's not very optimal to create a file all the time.
I would like to know how I could read the content of the web page https://who.is/whois-ip/ip-address/206.190.36.45 and get the line that says :
OrgName: Yahoo! Broadcast Services, Inc.
and just that line only.
Thanks very much for your help! :)
There are most likely better ways to parse this but you were on the right track with you current logic.
$web = Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
$web.tostring() -split "[`r`n]" | select-string "OrgName"
Select-String was returning the match as it, previously, was one long string. Using -split we can break it up to just get the return you expected.
OrgName: Yahoo! Broadcast Services, Inc.
Some string manipulation after that will get a cleaner answer. Again, many ways to approach this as well
(($web.tostring() -split "[`r`n]" | select-string "OrgName" | Select -First 1) -split ":")[1].Trim()
I used Select -First 1 as select-string could return more than one object. It would just ensure we are working with 1 when we manipulate the string. The string is just split on a colon and trimmed to remove the spaces that are left behind.
Since you are pulling HTML data we could also walk through those properties to get more specific results. The intention of this was to get 1RedOne answer
$web = Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
$data = $web.AllElements | Where{$_.TagName -eq "Pre"} | Select-Object -Expand InnerText
$whois = ($data -split "`r`n`r`n" | select -index 1) -replace ":\s","=" | ConvertFrom-StringData
$whois.OrgName
All that data is stored in the text of the PRE tag in this example. What I do is split up the data into its sections (Sections are defined with blank lines separating them. I look for consecutive newlines). The second group of data contains the org name. Store that in a variable and pull the OrgName as a property: $whois.OrgName. Here is what $whois looks like
Name Value
---- -----
Updated 2013-04-02
City Sunnyvale
Address 701 First Ave
OrgName Yahoo! Broadcast Services, Inc.
StateProv CA
Country US
Ref http://whois.arin.net/rest/org/YAHO
PostalCode 94089
RegDate 1999-11-17
OrgId YAHO
You can also make that hashtable into a custom object if you prefer dealing with those.
[pscustomobject]$whois
Updated : 2017-01-28
City : Sunnyvale
Address : 701 First Ave
OrgName : Yahoo! Broadcast Services, Inc.
StateProv : CA
Country : US
Ref : https://whois.arin.net/rest/org/YAHO
PostalCode : 94089
RegDate : 1999-11-17
OrgId : YAHO
it it very simple to use whois app this is for microsoft put app in System32 or windir and in powershell use whois command then get-string get "orgname" like this
PS C:\> whois.exe -v 206.190.36.45 | Select-String "Registrant Organization"
Registrant Organization: Yahoo! Inc.
I advise you this app because has more information for your work
Here you go, the way to do this is in fact to do an Invoke-WebRequest. If we take a look at some of the properties of the object we get from Invoke-WebRequest, we can see that PowerShell has already parsed some of the HTML and text for us.
All that we have to do is pick out some of the values we'd like to work with. For instance, taking a peek at the ParsedText field, we see these results.
These fields begin on about line 30 or so. In my approach to solving this problem we know that we'll find good data like this mid-way down the page, so if we could scrape the values from these lines, we'd be on our way to working with the data. The code to accomplish this first part is this:
$url = "https://who.is/whois-ip/ip-address/$ipaddress"
$Results = Invoke-WebRequest $url
$ParsedResults = $Results.ParsedHtml.body.outerText.Split("`n")[30..50]
Now, PowerShell has a number of very powerful commands to import and convert data into various formats. For instance, if we could only replace the ':' colon character with an equals sign '=', we could send the whole mess over to ConverFrom-StringData and have rich PowerShell objects to work with. It turns out that we can easily do that using the universal -Replace operator, like this
$Results.ParsedHtml.body.outerText.Split("`n")[30..50] -replace ":","="
I figured you might want to do this again in the future, so I took the entire thing and made it into a simple five line function for you. Throw this into your $Profile and enjoy.
So the finished result looks like this:
Function Get-WhoIsData {
param($ipaddress='206.190.36.45')
$url = "https://who.is/whois-ip/ip-address/$ipaddress"
$Results = Invoke-WebRequest $url
$ParsedResults = $Results.ParsedHtml.body.outerText.Split("`n")[30..50] -replace ":","=" | ConvertFrom-StringData
$ParsedResults }
and using it works this way:
PS C:\windows\system32> Get-WhoIsData -ipaddress 206.190.36.45
Name Value
---- -----
NetRange 206.190.32.0 - 206.190.63.255
CIDR 206.190.32.0/19
NetName NETBLK1-YAHOOBS
NetHandle NET-206-190-32-0-1
Parent NET206 (NET-206-0-0-0-0)
NetType Direct Allocation
OriginAS
Organization Yahoo! Broadcast Services, Inc. (YAHO)
RegDate 1995-12-15
Updated 2012-03-02
Ref http=//whois.arin.net/rest/net/NET-206-190-32-0-1
OrgName Yahoo! Broadcast Services, Inc.
OrgId YAHO
Address 701 First Ave
City Sunnyvale
StateProv CA
PostalCode 94089
You can then select any of the properties you'd like using normal Select-Object or Where-Object commands. For example, to pull out just the orgName property, you'd use this command:
(Get-WhoIsData).OrgName
>Yahoo! Broadcast Services, Inc.