Excel VBA: Check all check boxes on Web Page - excel

I'm trying to check all the checkboxes available on a webpage via VBA since the name convention doesnt appear to be one in which I can be selective. However I cannot seem to get anything to work. I can login to the website and navigate to the section of the website I want but cannot cross this hurdle. Any help would be greatly appreciate. Below is the source code from the webpage.
<li data-product-family="30yr"
data-product-amortizationTerm="30"
data-product-type="Conventional"
data-product-amortizationType="Fixed"
>
<label>
<input type="checkbox"
value="154232"
class="product-Conventional product-item"
data-authorized-remittance-types="ActualActual "
/>30-Year Fixed Rate - 110k Max Loan Amount</label>
</li>
VBA I attempted to write (edited)... code I'm using presently:
Public Sub TestIE()
Dim IE As Object
Dim aNodeList As Object, i As Long
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
' You can uncoment Next line To see form results
IE.Visible = False
' Send the form data To URL As POST binary request
IE.Navigate "https://"
' Statusbar
Application.StatusBar = "Page is loading. Please wait..."
' Wait while IE loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
IE.Visible = True
Set aNodeList = IE.document.querySelectorAll("input[type=checkbox]")
If aNodeList Is Nothing Then Exit Sub
For i = 0 To aNodeList.Length
aNodeList.Item(i).Checked = True
Next i
End Sub

You can try to get a nodeList of the checkboxes with:
IE.document.querySelectorAll("input[type=checkbox]")
You can traverse the nodeList along its .Length property.
E.g.
Dim aNodeList As Object, i As Long
Set aNodeList = IE.document.querySelectorAll("input[type=checkbox]")
If aNodeList Is Nothing Then Exit Sub
For i = 0 To aNodeList.Length -1
On Error Resume Next
aNodeList.item(i).Checked = True
On Error GoTo 0
Next i

Related

Run-time error '9': Subscript out of range -Error

I am running VBA in excel in order to do some web-scraping, but am coming up with this error when attempting to reach Google Chrome. How would one avoid this? I will click debug then a single line becomes highlighted...
This is where the error pops up
This is the highlighted line of code that must be giving the issue
Full Code in this picture
FULL CODE BELOW:
Private Sub time_sheet_filling()
Dim I As Long
Dim IE As Object
Dim doc As Object
Dim objElement As Object
Dim objCollection As Object
' Create InternetExplorer Object
Set IE = CreateObject("ChromeTab.ChromeFrame")
IE.Visible = True
' Send the form data To URL As POST binary request
IE.navigate "https://wistar.dispondo.net/#/login"
' Wait while IE loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
'Load the logon page
Set objCollection = IE.Document.getElementsByTagName("input")
I = 0
While I < objCollection.Length
If objCollection(I).Name = "username" Then
' Set text to enter
objCollection(I).Value = "MyUsername"
End If
If objCollection(I).Name = "password" Then
' Set text for password
objCollection(I).Value = "MyPasword"
End If
If objCollection(I).Type = "submit" And objCollection(I).Name = "btnSubmit" Then ' submit button clicking
Set objElement = objCollection(I)
End If
I = I + 1
Wend
objElement.Click ' click button to load the form
' Wait while IE re-loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
' Show IE
IE.Visible = True
Dim links, link
Dim n, j
Set links = IE.Document.getElementById("dgTime").getElementsByTagName("a")
n = links.Length
For j = 0 To n - 1 Step 2
links(j).Click
'I have some operations to be done will post another question for this
Next
End Sub
That error would occur if there is not a worksheet named "Website Data" in the active workbook at the time of the code running. Either the worksheet name does not match, or a different workbook is active when the code is running.
Make sure the worksheet name is correct and also explicitly reference the workbook it is located in (so that it doesn't matter which workbook is active when the code is running). If the "Website Data" sheet is in the workbook where the code is running, then refer to the value in the cell using:
ThisWorkbook.Sheets("Website Data").Cells(1,1).Value
You can't use Chrome instead of IE in this way. Chrome can only use via Selenium for web scraping via VBA. The IE is the only browser with a COM Interface which is needed to control an application via VBA:
For more information:
About the Component Object Model (COM)
About Selenium browser automation

Get a single value from a table with no ID with VBA

I am developing a web bot that scrapes the importation taxes from different countries customs website, and I have a problem retrieving the value I want from the following site : http://www.aduanet.gob.pe/itarancel/arancelS01Alias , using the test value 3303000000 next to CODIGO. The value I want to retrieve is the 6% next to "Ad / Valorem", but the table it is in has no ID properties nor class or something relevant to get directly to it or at least near to it. I have been trying to use .parent and .child methods, but without success. My code so far is as follows:
Function Peru(partida As String) As String
'Open IE
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate "http://www.aduanet.gob.pe/itarancel/arancelS01Alias"
'Load sub
Cargar
'Navigate further into the website (Im using partida = 3303000000)
For Each box In objIE.document.getElementsByTagName("input")
If box.Name = "cod_partida" Then
box.Value = partida
Exit For
End If
Next
For Each boton In objIE.document.getElementsByTagName("input")
If boton.Value = "Consultar" Then
boton.Click
Exit For
End If
Next
'Get the 6% value (This part is the one I cant figure out)
End Function
This is how you can get the data from that page. It was needed to switch two iframes from that page to reach the required content.
Sub Aduanet_Info()
Dim IE As New InternetExplorer, html As HTMLDocument
Dim elem As Object, frm As Object, frm1 As Object
With IE
.Visible = False
.navigate "http://www.aduanet.gob.pe/itarancel/arancelS01Alias"
Do While .readyState <> READYSTATE_COMPLETE: Loop
Set html = .document
End With
html.getElementsByTagName("input")(0).Value = "3303000000"
html.getElementsByTagName("input")(3).Click
Application.Wait Now + TimeValue("00:00:05")
Set frm = html.getElementsByClassName("autoHeight")(0).contentWindow.document
Set frm1 = frm.getElementsByClassName("autoHeight")(1).contentWindow.document
For Each elem In frm1.getElementsByTagName("td")
If InStr(elem.innerText, "Valorem") > 0 Then MsgBox elem.NextSibling.NextSibling.innerText: Exit For
Next elem
IE.Quit
End Sub
Output:
6%

VBA Set Dropdown value

I have a macro that I have written in excel and I have navigated to a webpage using "ActiveWorkbook.FollowHyperlink", which works just as I need it!
However, I now need to update a dropdown menu on that webpage.
I have an ID for the dropdown and each selection obviously has a value. I want to set the selected option using the value, which I have in the excel sheet.
I am struggling because I don't know how to access elements on the page, once opened using .FollowHyperlink.
After .FollowHyperlink is the webpage then active, is there something like ActiveWebPage.getElementById?
Appreciate any help.
Mike
what you want to do is use com automation to call an instance of internet explorer and navigate to the page in question, this will give you the document model, from there you can do most anything, see IE (Internet Explorer) Automation using Excel VBA
Sample VBA follows
Private Sub IE_Autiomation()
Dim i As Long
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.Navigate "http://www.excely.com/"
' Statusbar
Application.StatusBar = "www.excely.com is loading. Please wait..."
' Wait while IE loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
' Find 2 input tags:
' 1. Text field
' <input type="text" class="textfield" name="s" size="24" value="" />
'
' 2. Button
' <input type="submit" class="button" value="" />
Application.StatusBar = "Search form submission. Please wait..."
Set objCollection = IE.document.getElementsByTagName("input")
i = 0
While i < objCollection.Length
If objCollection(i).Name = "s" Then
' Set text for search
objCollection(i).Value = "excel vba"
Else
If objCollection(i).Type = "submit" And _
objCollection(i).Name = "" Then
' "Search" button is found
Set objElement = objCollection(i)
End If
End If
i = i + 1
Wend
objElement.Click ' click button to search
' Wait while IE re-loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
' Show IE
IE.Visible = True
' Clean up
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
Application.StatusBar = ""
End Sub

Web-scraping on intranet

I wrote a VBA code to scrape data from my company's intranet.
Issues:
The below error occurs:
Run-time error '91':
object variable or with block variable not set
It happens on:
myPoints = Trim(Doc.getElementsByName("price")(0).getAttribute("value"))
When I debug it and run line by line, it can retrieve all the values.
Input and Output:
I input multiple product ID on column B and retrieve data on column C:
Column B = product ID
Column C = price
HTML:
<td id="myPower_val_9" style="visibility: visible;">
<input type="text" disabled="disabled" value="300" name="price"></input>
</td>
VBA:
Sub Button1_Click()
Dim ie As Object
Dim r As Integer
Dim myPoints As String
Dim Doc As HTMLDocument
Set ie = New InternetExplorerMedium
For r = 2 To Range("B65535").End(xlUp).Row
With ie
.Visible = 0
.navigate "www.example.com/product/" & Cells(r, "B").Value
Do Until .readyState = 4
DoEvents
Loop
End With
Set Doc = ie.document
myPoints = Trim(Doc.getElementsByName("price")(0).getAttribute("value"))
Cells(r, "C").Value = myPoints
Next r
End Sub
Have I missed an error handler?
You need to wait for the document to be fully rendered and the DOM available before accessing any elements. ie.ReadyState changes to READYSTATE_COMPLETE once the page connects and starts loading. The reason that your code works when debugging is that in the couple of seconds it takes for you to start working with the debugger, the page finishes loading.
With ie
.Visible = True
.Navigate "www.example.com/product/" & Cells(r, "B").Value
Do Until .ReadyState = READYSTATE_COMPLETE
DoEvents
Loop
Do Until .Document.ReadyState = "complete"
DoEvents
Loop
End With
I would also recommend that you make the ie Window visible, at least while you're developing. Once you've got your functionality complete and debugging, you can make the window invisible. Keep in mind if you forget to close your invisible IE windows when your code finishes, your users will end up with runaway iexplore.exe processes.
If you only want to ignore the error and continue with the next iteration, use this modified code:
Sub Button1_Click()
Dim ie As Object
Dim r As Integer
Dim myPoints As String
Dim Doc As HTMLDocument
Set ie = New InternetExplorerMedium
For r = 2 To Range("B65535").End(xlUp).Row
With ie
.Visible = 0
.navigate "www.example.com/product/" & Cells(r, "B").Value
Do Until .readyState = 4
DoEvents
Loop
End With
Set Doc = ie.document
'Edit:
myPoints = ""
On Error Resume Next
myPoints = Trim(Doc.getElementsByName("price")(0).getAttribute("value"))
On Error Goto 0
Cells(r, "C").Value = myPoints
Next r
End Sub
You could also loop until element is set (add a timeout clause as well)
Dim a As Object
Do
DoEvents
On Error Resume Next
Set a = Doc.getElementsByName("price")
On Error GoTo 0
Loop While a Is Nothing

Trying to extract ONE value from a webpage with VBA in Excel

I've been trying to find the information now for a couple of days, but all the examples I've found just has a small piece of the code, I need it all =)
What I want to do is to extract one value from a homepage and put it into a cell in Excel
(and then take another value from another page on the same site and put in the next cell etc etc.)
The page is a swedish stock-exchange page, and the page I've used as a test-page is the stock for "Investor B" (https://www.avanza.se/aktier/om-aktien.html/5247/investor-b)
And the value I'm interested in is the one called "Senaste" (this is the page-information surrounding it)
<li>
<span class="XSText">Senast<br/></span>
<span class="lastPrice SText bold"><span class="pushBox roundCorners3" title="Senast uppdaterad: 17:29:59">248,60</span></span>
</li>
And it's the value 248,60 I'm after!
I got some coding experience, but not for VBA-scripting, after reading some forum-posts (mostly here), I've been trying out a few example by myself, but couldn't get any to work.
Since I'm quite basic with VBA, I might have got the structure wrong, so please be basic and patient with me, this was my test, but I got "Runtime error 429"
ActiveX component can't create object
I might be totally on the wrong track
Private Sub CommandButton1_Click()
Dim ie As Variant
Set ie = CreateObject("InternetExplorer")
ie.navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
ie.Visible = True
Do
DoEvents
Loop Until ie.readyState = READYSTATE_COMPLETE
Application.Wait (Now() + TimeValue("00:00:016")) ' For internal page refresh or loading
Dim doc As Variant 'variable for document or data which need to be extracted out of webpage
Set doc = CreateObject("HTMLDocument")
Set doc = ie.document
Dim dd As Variant
dd = doc.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
End Sub
EDIT: 2014-05-12 Current code beeing tested 17:05
under the button command
Private Sub CommandButton1_Click()
Dim IE As Object
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
' You can uncoment Next line To see form results
IE.Visible = False
' Send the form data To URL As POST binary request
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
' Statusbar
Application.StatusBar = "Loading, Please wait..."
' Wait while IE loading...
'Do While IE.Busy
' Application.Wait DateAdd("s", 1, Now)
'Loop
'this should go from ready-busy-ready
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
' Dim Document As HTMLDocument
' Set Document = IE.Document
Dim dd As Variant
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
' Show IE
IE.Visible = True
' Clean up
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
Application.StatusBar = ""
End Sub
And in module1
Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Public Function IEWait(p_ieExp As InternetExplorer)
'this should go from ready-busy-ready
Dim initialReadyState As Integer
initialReadyState = p_ieExp.ReadyState
'wait 250 ms until it's done
Do While p_ieExp.Busy Or p_ieExp.ReadyState <> READYSTATE_COMPLETE
Sleep 250
Loop
End Function
As said earlier, I do not know if I got the structure right with this latest add-in, not to expired in this kind of coding I'm afraid.
Best Regards
Stop editing 2014-05-12 17:08
You are close but have a couple small errors.
Here is how I would set it up (Tested):
Private Sub CommandButton1_Click()
Dim IE As Object
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
' You can uncoment Next line To see form results
IE.Visible = False
' URL to get data from
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
' Statusbar
Application.StatusBar = "Loading, Please wait..."
' Wait while IE loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
Application.StatusBar = "Searching for value. Please wait..."
Dim dd As String
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
' Show IE
IE.Visible = True
' Clean up
Set IE = Nothing
Application.StatusBar = ""
End Sub
Results:
Tested in Excel 2010 with the following references:
Edit - Option B
To get rid of a possible "Run-Time Error '91'" try changing a few lines like this:
Dim dd As Variant
Set dd = IE.Document.getElementsByClassName("lastPrice SText bold")
MsgBox dd(0).textContent
Edit - Option C
Yet another way to get elements:
Dim tag
Dim tags As Object
Set tags = IE.Document.getElementsByTagName("*")
For Each tag In tags
If tag.className = "lastPrice SText bold" Then
MsgBox tag.innerText
Exit For
End If
Next tag
(All three methods have been tested on Excel 2010 and IE10)
I just wanted to add the code I'm currently running which works perfectly fine at the moment, if people run into the same problem. This is to get two values into dedicated cells.
Private Sub CommandButton10_Click()
Dim IE As Object
Dim dd As Variant
' Create InternetExplorer Object
Set IE = GetObject("new:{D5E8041D-920F-45e9-B8FB-B1DEB82C6E5E}")
IE.Visible = False
' Send the form data To URL As POST binary request
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/52476/alk-abell-b"
Application.StatusBar = "Loading, Please wait..."
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
Range("Y2").Value = dd
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/52380/alm--brand"
Application.StatusBar = "Loading, Please wait..."
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
Range("Y3").Value = dd
' Clean up
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
Application.StatusBar = ""
End Sub
If one wants more data, it is just to copy the part starting with
IE.Navigate "https://www.pagewhereyourdatayouwanttoextractis.com"
and stops with
Range("Y2").Value = dd
It is ofcourse based if the page you want to extract data from has a similiar structure to the one above.
Hope this can help some people out there.
Best Regards

Resources