Find specific sentence in a web page using powershell - string

Question

I need to use powershell to resolve IP addresses via whois. My company filters port 43 and WHOIS queries so the workaround I have to use here is to ask powershell to use a website such as https://who.is, read the http stream and look for the Organisation Name matching the IP address.
So far I have managed to get the webpage read into powershell (example here with a WHOIS on yahoo.com) which is https://who.is/whois-ip/ip-address/206.190.36.45
So here is my snippet:
$url=Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
now if I do :
$url.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False HtmlWebResponseObject Microsoft.PowerShell.Commands.WebResponseObject
I see this object has several properties:
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
AllElements Property Microsoft.PowerShell.Commands.WebCmdletElementCollection AllElements {get;}
BaseResponse Property System.Net.WebResponse BaseResponse {get;set;}
Content Property string Content {get;}
Forms Property Microsoft.PowerShell.Commands.FormObjectCollection Forms {get;}
Headers Property System.Collections.Generic.Dictionary[string,string] Headers {get;}
Images Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Images {get;}
InputFields Property Microsoft.PowerShell.Commands.WebCmdletElementCollection InputFields {get;}
Links Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Links {get;}
ParsedHtml Property mshtml.IHTMLDocument2 ParsedHtml {get;}
RawContent Property string RawContent {get;}
RawContentLength Property long RawContentLength {get;}
RawContentStream Property System.IO.MemoryStream RawContentStream {get;}
Scripts Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Scripts {get;}
StatusCode Property int StatusCode {get;}
StatusDescription Property string StatusDescription {get;}
but every time I try commands like
$url.ToString() | select-string "OrgName"
Powershell returns the whole HTML code because it interprets the text string as a whole. I found a workaround dumping the output into a file and then read the file through an object (so every line is an element of an array) but I have hundreds of IPs to check so that's not very optimal to create a file all the time.
I would like to know how I could read the content of the web page https://who.is/whois-ip/ip-address/206.190.36.45 and get the line that says :
OrgName: Yahoo! Broadcast Services, Inc.
and just that line only.
Thanks very much for your help! :)

Answer 1

There are most likely better ways to parse this but you were on the right track with you current logic.
$web = Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
$web.tostring() -split "[`r`n]" | select-string "OrgName"
Select-String was returning the match as it, previously, was one long string. Using -split we can break it up to just get the return you expected.
OrgName: Yahoo! Broadcast Services, Inc.
Some string manipulation after that will get a cleaner answer. Again, many ways to approach this as well
(($web.tostring() -split "[`r`n]" | select-string "OrgName" | Select -First 1) -split ":")[1].Trim()
I used Select -First 1 as select-string could return more than one object. It would just ensure we are working with 1 when we manipulate the string. The string is just split on a colon and trimmed to remove the spaces that are left behind.
Since you are pulling HTML data we could also walk through those properties to get more specific results. The intention of this was to get 1RedOne answer
$web = Invoke-WebRequest https://who.is/whois-ip/ip-address/206.190.36.45
$data = $web.AllElements | Where{$_.TagName -eq "Pre"} | Select-Object -Expand InnerText
$whois = ($data -split "`r`n`r`n" | select -index 1) -replace ":\s","=" | ConvertFrom-StringData
$whois.OrgName
All that data is stored in the text of the PRE tag in this example. What I do is split up the data into its sections (Sections are defined with blank lines separating them. I look for consecutive newlines). The second group of data contains the org name. Store that in a variable and pull the OrgName as a property: $whois.OrgName. Here is what $whois looks like
Name Value
---- -----
Updated 2013-04-02
City Sunnyvale
Address 701 First Ave
OrgName Yahoo! Broadcast Services, Inc.
StateProv CA
Country US
Ref http://whois.arin.net/rest/org/YAHO
PostalCode 94089
RegDate 1999-11-17
OrgId YAHO
You can also make that hashtable into a custom object if you prefer dealing with those.
[pscustomobject]$whois
Updated : 2017-01-28
City : Sunnyvale
Address : 701 First Ave
OrgName : Yahoo! Broadcast Services, Inc.
StateProv : CA
Country : US
Ref : https://whois.arin.net/rest/org/YAHO
PostalCode : 94089
RegDate : 1999-11-17
OrgId : YAHO

Answer 2

it it very simple to use whois app this is for microsoft put app in System32 or windir and in powershell use whois command then get-string get "orgname" like this
PS C:\> whois.exe -v 206.190.36.45 | Select-String "Registrant Organization"
Registrant Organization: Yahoo! Inc.
I advise you this app because has more information for your work

Answer 3

Here you go, the way to do this is in fact to do an Invoke-WebRequest. If we take a look at some of the properties of the object we get from Invoke-WebRequest, we can see that PowerShell has already parsed some of the HTML and text for us.
All that we have to do is pick out some of the values we'd like to work with. For instance, taking a peek at the ParsedText field, we see these results.
These fields begin on about line 30 or so. In my approach to solving this problem we know that we'll find good data like this mid-way down the page, so if we could scrape the values from these lines, we'd be on our way to working with the data. The code to accomplish this first part is this:
$url = "https://who.is/whois-ip/ip-address/$ipaddress"
$Results = Invoke-WebRequest $url
$ParsedResults = $Results.ParsedHtml.body.outerText.Split("`n")[30..50]
Now, PowerShell has a number of very powerful commands to import and convert data into various formats. For instance, if we could only replace the ':' colon character with an equals sign '=', we could send the whole mess over to ConverFrom-StringData and have rich PowerShell objects to work with. It turns out that we can easily do that using the universal -Replace operator, like this
$Results.ParsedHtml.body.outerText.Split("`n")[30..50] -replace ":","="
I figured you might want to do this again in the future, so I took the entire thing and made it into a simple five line function for you. Throw this into your $Profile and enjoy.
So the finished result looks like this:
Function Get-WhoIsData {
param($ipaddress='206.190.36.45')
$url = "https://who.is/whois-ip/ip-address/$ipaddress"
$Results = Invoke-WebRequest $url
$ParsedResults = $Results.ParsedHtml.body.outerText.Split("`n")[30..50] -replace ":","=" | ConvertFrom-StringData
$ParsedResults }
and using it works this way:
PS C:\windows\system32> Get-WhoIsData -ipaddress 206.190.36.45
Name Value
---- -----
NetRange 206.190.32.0 - 206.190.63.255
CIDR 206.190.32.0/19
NetName NETBLK1-YAHOOBS
NetHandle NET-206-190-32-0-1
Parent NET206 (NET-206-0-0-0-0)
NetType Direct Allocation
OriginAS
Organization Yahoo! Broadcast Services, Inc. (YAHO)
RegDate 1995-12-15
Updated 2012-03-02
Ref http=//whois.arin.net/rest/net/NET-206-190-32-0-1
OrgName Yahoo! Broadcast Services, Inc.
OrgId YAHO
Address 701 First Ave
City Sunnyvale
StateProv CA
PostalCode 94089
You can then select any of the properties you'd like using normal Select-Object or Where-Object commands. For example, to pull out just the orgName property, you'd use this command:
(Get-WhoIsData).OrgName
>Yahoo! Broadcast Services, Inc.