I have a very big Excel spreadsheet that includes addresses. I want to write a code to automatically open https://www.greatschools.org/, one by one paste addresses in the search box, get the name of best schools for this address, and paste them into the Excel file.
I use this piece of code to paste one of the addresses in the web page but when it pastes and push the search button it doesn't show me any result.
Sub SearchBot()
'dimension (declare or set aside memory for) our variables
Dim objIE As InternetExplorer
'special object variable representing the IE browser
Dim aEle As HTMLLinkElement
'special object variable for an <a> (link) element
Dim y As Integer
'integer variable we'll use as a counter
Dim result As String
'string variable that will hold our result link
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate "https://www.greatschools.org/"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.document.getElementsByClassName("form-control")(0).Focus
objIE.document.getElementsByClassName("form-control")(0).Value = _
Sheets("Sheet1").Range("A1").Value
'click the 'go' button
objIE.document.getElementsByClassName("input-group-btn")(0).Click
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
I even added the above line code to give it more time to find result but this doesn't help me at all.
At first, it pastes the address in the search box and after a while the address disappears from the search box and there is no result for me there.
Can anybody help me solving this problem?
Thank you.
These two lines between the Do While loops worked for me:
objIE.Document.getElementsByClassName("form-control")(1).Value = Sheets("Sheet1").Range("A1").Value
'click the 'go' button
objIE.Document.getElementsByClassName("btn search-btn")(0).Click
Notice the use of (1) index on form-control entry.
Related
I am new to web scraping and VBA but am eager to learn further.
I have a small project in mind and am taking it slowly one step at a time.
I have a small piece of code in an Excel module which is almost working. I just cannot get it to submit my search box text.
I basically have the example horse name in Cell A2 in my Excel worksheet. The code opens the website and also enters the text into the search box.
I just can't get it to click the Go button.
I'd appreciate any help and an explanation of what I'm doing wrong so I can avoid it again!
When inspecting the Go button on the site, the HTML is thus:
My code:
Sub HorseSearch()
'define objIE
Dim objIE As InternetExplorer
'create an instance objIE
Set objIE = New InternetExplorer
'make web page visible
objIE.Visible = True
'navigate objIE to this web page
objIE.navigate "https://www.britishhorseracing.com/racing/horses/racehorse-search-results/"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'in the search box put cell "A2" value which holds a horse name Tiger Roll
objIE.document.getElementById("text-search").Value = Sheets("Sheet1").Range("A2").Value
'place focus on the Go button
objIE.document.getElementById("Submit").Focus
'click the 'go' button
objIE.document.getElementById("Submit").Click
End Sub
Improving your code
Your problem is that when you fire the click on the submit button it is not enabled. You have to enable it. If you observe it the submit button got enabled when you manually insert a text into the input box, with a length > 3 characters.
Your code insert the text changing the value attribute of the input box but it doesn't fire the 'onchange' event.
Using web developer tools in Chrome I saw that the form behaviour (input box vs submit button) is built in jQuery.
One of the possible solutions can be to use
objIE.Document.parentWindow.ExecScript "jQuery(""#text-search"").trigger(""change"")"
after inserting the value and before clicking on the submit button.
Here the complete code:
Sub HorseSearch()
'Dim objIE As Object
'Set objIE = CreateObject("InternetExplorer.Application")
Dim obJe As InternetExplorer
Set objIE = New InternetExplorer
'make web page visible
objIE.Visible = True
'navigate objIE to this web page
objIE.navigate "https://www.britishhorseracing.com/racing/horses/racehorse-search-results/"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'in the search box put cell "A2" value which holds a horse name Tiger Roll
objIE.Document.getElementById("text-search").Value = Sheets("Sheet1").Range("A2").Value
On Error Resume Next
objIE.Document.parentWindow.ExecScript "jQuery(""#text-search"").trigger(""change"")"
On Error GoTo 0
'click the 'go' button
objIE.Document.getElementById("Submit").Click
End Sub
Another solution
Why to use the form when we can directly go to the result page?
Result page is something like
https://www.britishhorseracing.com/racing/horses/racehorse-search-results/#!?pagenum=1&q=tiger%20roll&rated=false
where the q parameter value is the text you put in the input box. We could navigate directly to that page (using the value from cell A22 for q parameter in the URL).
Sub HorseSearch()
'Dim objIE As Object
'Set objIE = CreateObject("InternetExplorer.Application")
Dim obJe As InternetExplorer
Set objIE = New InternetExplorer
'make web page visible
objIE.Visible = True
'navigate objIE to this web page
objIE.navigate "https://www.britishhorseracing.com/racing/horses/racehorse-search-results?pagenum=1&q=" & Sheets("Sheet1").Range("A2").Value & "&rated=false"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'You have loaded the result page :)
End Sub
I have some ISIN-codes from bonds in an excel sheet (column A). I would like to find the corresponding rating from the site https://www.moodys.com.
As this should be done on a regular basis, I would like to automate this process through Excel VBA.
The website does not use any ID's so I can't use getElementById.
I tried using through getElementsByClassName, but this does not work. Also I am not sure if the "search-widget" is the correct class name.
The Error message is:
Object doesn't support this property or method (Error 438)
Sub SearchBot()
Dim objIE As InternetExplorer 'special object variable representing the IE browser
Nbr = ThisWorkbook.Sheets("Sheet1").Cells(1, 1).End(xlDown).Row - 1
With Worksheets("Sheet1").Range(Cells(2, 2), Cells(100000, 6))
.ClearContents
End With
For i = 1 To 1 'Nbr
'initiating a new instance of Internet Explorer and asigning it to objIE
Set objIE = New InternetExplorer
'make IE browser visible (False would allow IE to run in the background)
objIE.Visible = True
'navigate IE to this web page (a pretty neat search engine really)
objIE.navigate "https://www.moodys.com"
'wait here a few seconds while the browser is busy
Application.Wait (Now + TimeValue("00:00:04"))
'Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Do
DoEvents
Loop Until objIE.readyState = 4
objIE.document.getElementsByClassName("search-widget").Value = Sheets("Sheet1").Range("A" & i + 1).Value
...
The getElementsByClassName method returns an array of all elements with that classname, even if that is only one element. So if you know it is the first and only element with that classname, you can use getElementsByClassName("search-widget")(0).Value
Furthermore, you can find the right classname by inspecting the pages html. In chrome, you can do that by right clicking the element and press "inspect". The only searchbar I saw on the site has "typeahead search-box-input left" as classname by the way.
I'm a complete newbie. Can you guys help me with the following?
1. I have a spreadsheet with productcodes.
2. I have an active Internet Explorer session (logged in) on an index page filled with links that among others contain the said productcodes.
3. For each code, I need it to find the code, open the link. Then find a line containing the string "udsalg" and copy the complete line i.e. "Udsalg 10994"
4. Insert the string next to the productcode.
5. Enter IE and "go back" to index page.
5. Rinse and repeat until all productcodes have been searched.
I have no skill regarding this and hoping for help. I understand basic programming (php etc). I hope somebody can help or point in the right direction. Best regards.
You can try to refer code example below will give you most of the answers for your above questions. Note that this is not the exact code.
'start a new subroutine called SearchBot
Sub SearchBot()
'dimension (declare or set aside memory for) our variables
Dim objIE As InternetExplorer 'special object variable representing the IE browser
Dim aEle As HTMLLinkElement 'special object variable for an <a> (link) element
Dim y As Integer 'integer variable we'll use as a counter
Dim result as String 'string variable that will hold our result link
'initiating a new instance of Internet Explorer and asigning it to objIE
Set objIE = New InternetExplorer
'make IE browser visible (False would allow IE to run in the background)
objIE.Visible = True
'navigate IE to this web page (a pretty neat search engine really)
objIE.navigate "https://duckduckgo.com"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'in the search box put cell "A2" value, the word "in" and cell "C1" value
objIE.document.getElementById("search_form_input_homepage").Value = _
Sheets("Sheet1").Range("A2").Value & " in " & Sheets("Sheet1").Range("C1").Value
'click the 'go' button
objIE.document.getElementById("search_button_homepage").Click
'wait again for the browser
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'the first search result will go in row 2
y = 2
'for each <a> element in the collection of objects with class of 'result__a'...
For Each aEle In objIE.document.getElementsByClassName("result__a")
'...get the href link and print it to the sheet in col C, row y
result = aEle
Sheets("Sheet1").Range("C" & y).Value = result
'...get the text within the element and print it to the sheet in col D
Sheets("Sheet1").Range("D" & y).Value = aEle.innerText
Debug.Print aEle.innerText
'is it a yellowpages link?
If InStr(result, "yellowpages.com") > 0 Or InStr(result, "yp.com") > 0 Then
'make the result red
Sheets("Sheet1").Range("C" & y).Interior.ColorIndex = 3
'place a 1 to the left
Sheets("Sheet1").Range("B" & y).Value = 1
End If
'increment our row counter, so the next result goes below
y = y + 1
'repeat times the # of ele's we have in the collection
Next
'add up the yellowpages listings
Sheets("Sheet1").Range("B1").Value = _
Application.WorksheetFunction.Sum(Sheets("Sheet1").Range("B2:B100"))
'close the browser
objIE.Quit
'exit our SearchBot subroutine
End Sub
You can try to refer the code and try to understand it. Then you can try to modify it based on your requirement. If you have any further questions than you can try to post your sample code. We will try to check it and try to provide you suggestions for it.
Helpful References to learn about IE VBA Automation.
(1) Excel + VBA + IE = web automation
(2) IE (Internet Explorer) Automation using Excel VBA
(3) Automate Internet Explorer (IE) Using VBA
I have a webpage on which i want to autologin and click on submit button using vba the code is below
'start a new subroutine called SearchBot
Sub SearchBot()
'dimension (declare or set aside memory for) our variables
Dim objIE As InternetExplorer 'special object variable representing the IE browser
Dim aEle As HTMLLinkElement 'special object variable for an <a> (link) element
Dim y As Integer 'integer variable we'll use as a counter
Dim result As String 'string variable that will hold our result link
'initiating a new instance of Internet Explorer and asigning it to objIE
Set objIE = New InternetExplorer
'make IE browser visible (False would allow IE to run in the background)
objIE.Visible = True
'navigate IE to this web page (a pretty neat search engine really)
objIE.navigate "mywebpage"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'in the search box put cell "A2" value, the word "in" and cell "C1" value
objIE.document.getElementById("loginID").Value = ("userid")
objIE.document.getElementById("password").Value = ("password")
after this code i want to click on submit button using vba
HTML inspect element code is below
<INPUT class=btn2 type=submit value=Submit name=submit1>
Try finding element by name and click it.
objIE.document.getElementsByName("submit1")(0).click
I am trying to loop through a list of names in an Excel spreadsheet using VBA and I would like to paste a surname into a text box on a webpage and then extract the returned phone number into a cell after the page has been submitted.
I can only do this one name at a time so I would like to automate this process as there are several hundreds of them. I can do the Excel looping but I don't know how to interact with the webpage.
Can anybody help, or point me in the right direction please?
Regards, Ian.
this should help.
As an example, I've used one of the millions of random name generators that exist on the 'Net. This code opens an IE instance, navigates to "http://www.behindthename.com/random/", inputs the surname to the form, submits the form, and returns the resulting HTML body.
I haven't parsed the result page HTML to return the actual name, as this will be different for your site, but this should help you get started.
You'll need a reference to "Microsoft Internet Controls" to access the SHDocVw.InternetExplorer object.
Private Function GetFromSubmittedForm(strSurname As String) As String
Dim strReturnBody As String
'Requires reference to "Microsoft Internet Controls"'
Dim IE As SHDocVw.InternetExplorer
Set IE = CreateObject("InternetExplorer.Application")
'Navigate to the URL'
IE.Navigate "http://www.behindthename.com/random/"
'No need to show the window'
IE.Visible = False
'Wait for IE to load the page'
While IE.Busy: DoEvents: Wend
Do While IE.ReadyState <> 4: DoEvents: Loop
'NOTE: I have refered to each element by name, but indexes _
can also be used: Forms("random") could be Forms(0)'
'Select the form by name:'
With IE.Document.Forms("random")
'Reference each Input by name'
.Surname.Value = strSurname
'Submit the form'
.Submit
'Sometimes you may need to submit using _
the name of the submit button like this:'
'.SubmitButtonName.Click'
End With
'Wait for IE to load the new page'
While IE.Busy: DoEvents: Wend
Do While IE.ReadyState <> 4: DoEvents: Loop
'Set the body of the new page to a string'
strReturnBody = IE.Document.Body.InnerHTML
'/CODE THAT ANALYSES THE BODY HTML GOES HERE'
GetFromSubmittedForm = strReturnBody
'\CODE THAT ANALYSES THE BODY HTML GOES HERE'
'Close the IE doc'
IE.Document.Close
IE.Quit
End Function
Hope this helps.