Want to select button on website using excel vba - excel

I want to use excel to navigate a webpage. yet the website doesn't use ID's like a normal site does (amazon,google,ect). the website is http://www.scoopmae.com/. how would i select the "book a demo" button. I would normally use getelementbyID, but i don't know the id. I've also tried tag and class but no luck.
Sub scoop()
Set objie = CreateObject("InternetExplorer.Application")
objie.Top = 0
objie.Left = 0
objie.Width = 1600
objie.Height = 900
objie.Visible = True 'We can see IE
On Error Resume Next
objie.navigate ("http://scoopmae.com")
Do
DoEvents
Loop Until objue.readystate = 4
Application.Wait (Now + TimeValue("0:00:02"))
'MsgBox ("wait")
'click button
'objie.document.getElementsById("scoop-sort").Click
x = objie.document.getElementsByClassName("a")
Cells(1, 2) = x
End Sub

I can access the button by doing this.
Sub test()
Dim oHtml As HTMLDocument
Dim oElement As Object
Set oHtml = New HTMLDocument
With CreateObject("WINHTTP.WinHTTPRequest.5.1")
.Open "GET", "http://www.scoopmae.com/", False
.send
oHtml.body.innerHTML = .responseText
End With
Set mybtn = oHtml.getElementsByClassName("sf-button large orange default dropshadow")(0).getElementsByTagName("span")
i = 0
For Each oElement In mybtn
Debug.Print mybtn(i).innerText
i = i + 1
Next oElement
End Sub
Make certain that you go to Tools -> References and add a reference to the Microsoft HTML Object Library [MSHTML.TLB]
Thanks
Miguel

You are basically right there!! Make sure you add 2 references:
1 Microsoft Internet Controls
2. Microsoft HTML Object Library
Read more at http://vbadud.blogspot.com/2009/08/how-to-login-to-website-using-v
See this link.
http://vbadud.blogspot.com/2009/08/how-to-login-to-website-using-vba.html
Here is the code:
Sub test()
Dim oHtml As HTMLDocument
Dim oElement As Object
Set oHtml = New HTMLDocument
With CreateObject("WINHTTP.WinHTTPRequest.5.1")
.Open "GET", "http://www.scoopmae.com/", False
.send
oHtml.body.innerHTML = .responseText
End With
Set mybtn = oHtml.getElementsByClassName("sf-button large orange default dropshadow")(0).getElementsByTagName("span")
i = 0
For Each oElement In mybtn
Debug.Print mybtn(i).innerText
oElement.Click
i = i + 1
Next oElement
End Sub

Related

Excel VBA - Get all href links from a website

Hi there, hoping that someone can help me.
On this example link: https://www.academiadasapostas.com/stats/competition/brasil/26
I want to get all the href links which are the target of the "VS".
I'm trying examples like this one:
Sub ScrapeScores()
Dim IE As New SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument
Dim HTMLTables As MSHTML.IHTMLElementCollection
Dim HTMLTable As MSHTML.IHTMLElement
Dim HTMLDiv As MSHTML.IHTMLElement
Dim TableSection As MSHTML.IHTMLElement
Dim TableRow As MSHTML.IHTMLElement
Dim TableCell As MSHTML.IHTMLElement
Dim RowText As String
IE.Visible = True
IE.navigate "https://www.academiadasapostas.com/stats/competition/brasil/26"
Do While IE.readyState <> READYSTATE_COMPLETE Or IE.Busy
Loop
Set HTMLDoc = IE.document
Set HTMLDiv = HTMLDoc.getElementById("competition-round-group-0")
Set HTMLTables = HTMLDiv.getElementsByTagName("a")
For Each HTMLTable In HTMLTables
Debug.Print HTMLTable.ID, "&", HTMLTable.className
For Each TableSection In HTMLTable.Children
Debug.Print , TableSection.tagName
Next TableSection
Next HTMLTable
End Sub
But with no success. I think I could use CSS with a SelectorAll, right? Since IE is going to get extinted, would be nice to use the CSS instead.
Thank you in advance for any answer.
You can use the following css pattern with querySelectorAll .competition-rounds td:nth-child(4) > a. Loop the returned nodeList and extract the href from each node. This selects for the 4th column within that table, then the child a tag, from which the href attribute is extracted during the loop.
Required references:
Microsoft Internet Controls
Microsoft HTML Object Library
Option Explicit
Public Sub PrintLinks()
Dim ie As SHDocVw.InternetExplorer, nodeList As MSHTML.IHTMLDOMChildrenCollection
Set ie = New SHDocVw.InternetExplorer
With ie
.Visible = True
.Navigate2 "https://www.academiadasapostas.com/stats/competition/brasil/26"
While .Busy Or .readyState <> READYSTATE_COMPLETE: DoEvents: Wend
Set nodeList = ie.Document.querySelectorAll(".competition-rounds td:nth-child(4) > a")
Dim i As Long
For i = 0 To nodeList.length - 1
Debug.Print nodeList.Item(i).href
Next
Stop
.Quit
End With
End Sub
Reading:
:nth-child()
Child combinator
If you wish to get rid of IE, the following should do it:
Sub GetTragetLinks()
Const Url = "https://www.academiadasapostas.com/stats/competition/brasil/26"
Dim Html As HTMLDocument, I&
Set Html = New HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", Url, False
.send
Html.body.innerHTML = .responseText
End With
With Html.querySelectorAll("tr[id*='gsm_id_'] td[title] > a")
For I = 0 To .Length - 1
Debug.Print .item(I).href
Next I
End With
End Sub
Reference to add:
Microsoft HTML Object Library

Unable to click on webpage based on attribute results

I am trying to navigate in "https://www.ford.co.uk/shop/price-and-locate/build-and-price-gf3#/catalogID/WAEGB-CE1-2015-B479FiestaCarGBR201950/?code=". I have to select Titanium in the model list. But i am not able to select it. Everytime the list of models will vary based on link.
Below is the code which i have tried.
Sub testcode()
Dim ie As New SHDocVw.InternetExplorer
Dim htmldoc As MSHTML.HTMLDocument
Dim HTMLAs As MSHTML.IHTMLElementCollection
Dim HTMLA As MSHTML.IHTMLElement
ie.Visible = True
ie.navigate "https://www.ford.co.uk/shop/price-and-locate/build-and-price-gf3#/catalogID/WAEGB-CE1-2015-B479FiestaCarGBR201950/?code="
Do While ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
Application.Wait (Now + TimeValue("0:00:03"))
Set htmldoc = ie.document
Set HTMLAs = htmldoc.getElementsByClassName("features-list-item feature-box")
For Each HTMLA In HTMLAs
Debug.Print HTMLA.getAttribute("id"), HTMLA.getAttribute("tabindex")
If HTMLA.getAttribute("id") = "feature-1" And HTMLA.getAttribute("tabindex") = 0 Then
HTMLA.Click
End If
Next HTMLA
Set HTMLAs = Nothing
Set HTMLA = Nothing
End Sub
The result should be webpage loaded and selected the titanium series. Kindly assist me achieve this.
HTMLA itself doesn't seem to have a click handler, but you can click on one of its child elements.
Try this:
If HTMLA.getAttribute("id") = "feature-1" And HTMLA.getAttribute("tabindex") = 0 Then
HTMLA.getElementsByTagName("img")(0).Click
Exit For
End If

Web scraping in Investing.com with Excel vba

I have no knowledge of vba. Only the macro recorder is used.
I need to download the data from a web page to an Excel spreadsheet and with my knowledge of vba I am not capable.
In particular, what I want to do a macro to download to Excel a data table of the page: https://www.investing.com/equities/cellnex-telecom-historical-data
This download would have to be configured in terms of time, date range and ordering.
The steps would be the following:
1.- The objective is to copy the data from the "CLNX historical data" table to an Excel spreadsheet.
2.- That download should be done by previously selecting "Monthly" in the drop-down menu by calling "Term".
3.- That the download is made by previously selecting the range of dates for the last 2 years.
4.- Finally, order the table in descending order by the column "Maximum".
5.- Once the term, the date range and the order are selected, copy the data from the "CLNX historical data" table to an Excel spreadsheet.
I have tried with the macro recorder but I am not able to configure the term, the date range or the ordering.
Could someone help me?
Thanks for your help.
The code:
Sub DataInvesting()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.navigate "https://www.investing.com/equities/cellnex-telecom-historical-data"
Do Until IE.readyState = 4
DoEvents
Loop
IE.Document.getElementsByClassName("newInput selectBox float_lang_base_1")(0).Value = "Monthly"
IE.Visible = True
Set IE = Nothing
Set appIE = Nothing
End Sub
I have just tested the following code and it works, instead of creating an instance of internet explorer every time we need to run this macro, we will use xmlhttp requests. Just copy the entire code and paste it into a module in vba. Don't forget to add references (Tools/References) to Microsoft HTML Object Library and Microsoft XML v6.0.
Option Explicit
Sub Export_Table()
'Html Objects---------------------------------------'
Dim htmlDoc As MSHTML.HTMLDocument
Dim htmlBody As MSHTML.htmlBody
Dim ieTable As MSHTML.HTMLTable
Dim Element As MSHTML.HTMLElementCollection
'Workbooks, Worksheets, Ranges, LastRow, Incrementers ----------------'
Dim wb As Workbook
Dim Table As Worksheet
Dim i As Long
Set wb = ThisWorkbook
Set Table = wb.Worksheets("Sheet1")
'-------------------------------------------'
Dim xmlHttpRequest As New MSXML2.XMLHTTP60 '
'-------------------------------------------'
i = 2
'Web Request --------------------------------------------------------------------------'
With xmlHttpRequest
.Open "POST", "https://www.investing.com/instruments/HistoricalDataAjax", False
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.setRequestHeader "X-Requested-With", "XMLHttpRequest"
.send "curr_id=951681&smlID=1695217&header=CLNX+Historical+Data&st_date=01%2F01%2F2017&end_date=03%2F01%2F2019&interval_sec=Monthly&sort_col=date&sort_ord=DESC&action=historical_data"
If .Status = 200 Then
Set htmlDoc = CreateHTMLDoc
Set htmlBody = htmlDoc.body
htmlBody.innerHTML = xmlHttpRequest.responseText
Set ieTable = htmlDoc.getElementById("curr_table")
For Each Element In ieTable.getElementsByTagName("tr")
Table.Cells(i, 1) = Element.Children(0).innerText
Table.Cells(i, 2) = Element.Children(1).innerText
Table.Cells(i, 3) = Element.Children(2).innerText
Table.Cells(i, 4) = Element.Children(3).innerText
Table.Cells(i, 5) = Element.Children(4).innerText
Table.Cells(i, 6) = Element.Children(5).innerText
Table.Cells(i, 7) = Element.Children(6).innerText
i = i + 1
DoEvents: Next Element
End If
End With
Set xmlHttpRequest = Nothing
Set htmlDoc = Nothing
Set htmlBody = Nothing
Set ieTable = Nothing
Set Element = Nothing
End Sub
Public Function CreateHTMLDoc() As MSHTML.HTMLDocument
Set CreateHTMLDoc = CreateObject("htmlfile")
End Function
I can't test this as despite setting up a free account it keeps saying the password is wrong. Fed up with 5 password resets and same problem and suspect it want my social media details.
The following broadly outlines steps I would consider though some timed waits are most likely needed.
Option Explicit
'VBE > Tools > References:
' Microsoft Internet Controls
Public Sub Info()
Dim ie As New InternetExplorer
Const URL As String = ""https://www.investing.com/equities/cellnex-telecom-historical-data""
With ie
.Visible = True
.Navigate2 URL
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector(".login").Click
While .Busy Or .readyState < 4: DoEvents: Wend
.Navigate2 URL
While .Busy Or .readyState < 4: DoEvents: Wend
With .document.querySelector("#loginFormUser_email")
.Focus
.Value = "Bob#gmail.com"
End With
With .document.querySelector("#loginForm_password")
.Focus
.Value = "systemSucksDoesn'tAcceptMyPassword"
End With
Application.Wait Now + TimeSerial(0, 0, 2)
.document.querySelector("[onclick*=submitLogin]").Click
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector("#data_interval").Click
.document.querySelector("[value=Monthly]").Click
With .document.querySelector("#picker")
.Focus
.Value = "03/08/2017 - 03/08/2019"
.FireEvent "onchange"
End With
'TODO Sorting column when clarified which column
.document.querySelector("[title='Download Data']").Click
Application.Wait Now + TimeSerial(0, 0, 10)
Stop
.Quit
End With
End Sub
Try this.
Sub Web_Table_Option()
Dim HTMLDoc As New HTMLDocument
Dim objTable As Object
Dim lRow As Long
Dim lngTable As Long
Dim lngRow As Long
Dim lngCol As Long
Dim ActRw As Long
Dim objIE As InternetExplorer
Set objIE = New InternetExplorer
objIE.Navigate "https://www.investing.com/equities/cellnex-telecom-historical-data"
Do Until objIE.ReadyState = 4 And Not objIE.Busy
DoEvents
Loop
Application.Wait (Now + TimeValue("0:00:03")) 'wait for java script to load
HTMLDoc.body.innerHTML = objIE.Document.body.innerHTML
With HTMLDoc.body
Set objTable = .getElementsByTagName("curr_table")
For lngTable = 0 To objTable.Length - 1
For lngRow = 0 To objTable(lngTable).Rows.Length - 1
For lngCol = 0 To objTable(lngTable).Rows(lngRow).Cells.Length - 1
ThisWorkbook.Sheets("Sheet1").Cells(ActRw + lngRow + 1, lngCol + 1) = objTable(lngTable).Rows(lngRow).Cells(lngCol).innerText
Next lngCol
Next lngRow
ActRw = ActRw + objTable(lngTable).Rows.Length + 1
Next lngTable
End With
objIE.Quit
End Sub

VBA: copy data from website into excel

I have a VBA code that selects info from drop-down menus on a government website and then submits the query. The requested data then opens up in another IE page. I am trying to copy this data into excel; however, I am unable to do so.
My code currently copies the text on the first IE page that contains the drop-down menus. The government website is: http://www.osfi-bsif.gc.ca/Eng/wt-ow/Pages/FINDAT.aspx
I have look all over the internet for a solution but nothing seems to work...
Here is my code:
Sub GetOsfiFinancialData()
Dim UrlAddress As String
UrlAddress = "http://ws1.osfi-bsif.gc.ca/WebApps/FINDAT/DTIBanks.aspx?T=0&LANG=E"
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
With ie
.Silent = True
.Visible = False
.navigate UrlAddress
End With
Do Until Not ie.Busy And ie.readyState = 4
DoEvents
Loop
Application.Wait (Now() + TimeValue("00:00:05"))
'Select Bank
ie.document.getElementById("DTIWebPartManager_gwpDTIBankControl1_DTIBankControl1_institutionTypeCriteria_institutionsDropDownList").Value = Z005
'open window with financial data
Dim objButton
Set objButton = ie.document.getElementById("DTIWebPartManager_gwpDTIBankControl1_DTIBankControl1_submitButton")
objButton.Focus
objButton.Click
'select new pop-up window
marker = 0
Set objshell = CreateObject("Shell.Application")
IE_count = objshell.Windows.Count
For x = 0 To (IE_count - 1)
On Error Resume Next ' sometimes more web pages are counted than are open
my_title = objshell.Windows(x).document.Title
If my_title Like "Consolidated Monthly Balance Sheet" & "*" Then 'compare to find if the desired web page is already open
Set ie = objshell.Windows(x)
marker = 1
Exit For
Else
End If
Next
Do Until Not ie.Busy And ie.readyState = 4
DoEvents
Loop
Application.Wait (Now() + TimeValue("00:00:05"))
Dim doc As MSHTML.HTMLDocument
Dim tables As MSHTML.IHTMLElementCollection
Dim table As MSHTML.HTMLTable
Dim clipboard As MSForms.DataObject
Set doc = ie.document
Set tables = doc.getElementsByTagName("body")
Set table = tables(0)
Set clipboard = New MSForms.DataObject
'paste in sheets
Dim test
Set test = ActiveWorkbook.Sheets("Test")
clipboard.SetText table.outerHTML
clipboard.PutInClipboard
test.Range("A1").PasteSpecial xlPasteAll
clipboard.Clear
MsgBox ("Task Completed")
End Sub
Your help is greatly appreciated!
You were using the current test with document.Title. I found that For Each of all windows looking for the full title worked in combination with copy pasting the pop-up window outerHTML. No additional wait time was required.
Inside the For Each Loop, after you reset the IE instance to the new window, you can obtain the new URL with ie.document.url. As you already have the data loaded you might as well just copy paste it straight away in my opinion.
Code:
Option Explicit
Public Sub GetOsfiFinancialData()
Dim UrlAddress As String, objButton, ie As Object
UrlAddress = "http://ws1.osfi-bsif.gc.ca/WebApps/FINDAT/DTIBanks.aspx?T=0&LANG=E"
Set ie = CreateObject("internetexplorer.application")
With ie
.Silent = True
.Visible = False
.navigate UrlAddress
While .Busy Or .readyState < 4: DoEvents: Wend
.document.getElementById("DTIWebPartManager_gwpDTIBankControl1_DTIBankControl1_institutionTypeCriteria_institutionsDropDownList").Value = "Z005"
Set objButton = .document.getElementById("DTIWebPartManager_gwpDTIBankControl1_DTIBankControl1_submitButton")
objButton.Focus
objButton.Click
Dim objShellWindows As New SHDocVw.ShellWindows, currentWindow As IWebBrowser2
For Each currentWindow In objShellWindows
If currentWindow.document.Title = "Consolidated Monthly Balance Sheet - Banks, Trust and Loan" Then
Set ie = currentWindow
Exit For
End If
Next
Dim clipboard As Object
Set clipboard = GetObject("New:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
clipboard.SetText ie.document.body.outerHTML
clipboard.PutInClipboard
ThisWorkbook.Worksheets("Sheet1").Cells(1, 1).PasteSpecial
.Quit
End With
End Sub
References (VBE > Tools > References):
Microsoft Internet Controls
I don't have time to get into all the stuff about controlling one browser from another, but I think you can figure that part out, especially since you made some great progress on this already. Get URL#2 from URL#1, like you are doing, but with some better data controls around it, and then do this...
Option Explicit
Sub Web_Table_Option_One()
Dim xml As Object
Dim html As Object
Dim objTable As Object
Dim result As String
Dim lRow As Long
Dim lngTable As Long
Dim lngRow As Long
Dim lngCol As Long
Dim ActRw As Long
Set xml = CreateObject("MSXML2.XMLHTTP.6.0")
With xml
.Open "GET", "http://ws1.osfi-bsif.gc.ca/WebApps/Temp/2f40b7ef-d024-4eca-a8a3-fb82153efafaFinancialData.aspx", False
.send
End With
result = xml.responseText
Set html = CreateObject("htmlfile")
html.body.innerHTML = result
Set objTable = html.getElementsByTagName("Table")
For lngTable = 0 To objTable.Length - 1
For lngRow = 0 To objTable(lngTable).Rows.Length - 1
For lngCol = 0 To objTable(lngTable).Rows(lngRow).Cells.Length - 1
ThisWorkbook.Sheets("Sheet1").Cells(ActRw + lngRow + 1, lngCol + 1) = objTable(lngTable).Rows(lngRow).Cells(lngCol).innerText
Next lngCol
Next lngRow
ActRw = ActRw + objTable(lngTable).Rows.Length + 1
Next lngTable
End Sub

Where to find VBA functions for working with internet explorer

I am working on a VBA function to go to a webpage, find a single HTML element, and display its contents. Here is what I have so far.
Function WebTableToSheet(webpage As String, tag As String, count As Integer) As String
'Tested using IE7, Excel 2000 SP1, and Windows XP
Dim objIE As Object
Dim varTables, varTable, document
Dim allTags, tags
Dim varRows, varRow
Dim varCells, varCell
Dim lngRow As Long, lngColumn As Long
Dim strBuffer As String
Set objIE = CreateObject("InternetExplorer.Application")
'Setup
With objIE
.AddressBar = False
.StatusBar = False
.MenuBar = False
.Toolbar = 0
.Visible = True
.Navigate webpage
End With
'Load webpage
While objIE.Busy
Wend
While objIE.document.ReadyState <> "complete"
Wend
varTable = objIE.document.All
allTags = objIE.document.All.tags(tag)
WebTableToSheet = allTags.Item(count)
Debug.Print "Testing function"
Cleanup:
Set varCell = Nothing: Set varCells = Nothing
Set varRow = Nothing: Set varRows = Nothing
Set varTable = Nothing: Set varTables = Nothing
objIE.Quit
Set objIE = Nothing
End Function
I am able to open InternetExplorer, got to the webpage specified in the function, but when I try searching for a specific tag, it seems to fail. I have had trouble searching for Microsoft VBA Internet Explorer documentation and knew what variables and methods are available for this task?
I'd recommend setting a reference to shdocvw.dll and the Microsoft HTML Object Library. In XP shdocvw.dll is in your system32 folder, the HTML Object Library should be in your list of references. I only have access to XP at the moment, so you'll have to do a search for any other ver of windows you're using. Then in your code you can do the following:
Dim IE as SHDocVw.InternetExplorer
Dim doc as HTMLDocument
Dim el as IHTMLElement
Set IE = new SHDocVw.InternetExplorer
With IE
.navigate "some.webpage.com"
.visible = true
End With
Do
'Nothing
Loop Until IE.readyState = READYSTATE_COMPLETE ' = 4
Set doc = IE.Document
Set el = doc.getElementById("someElementId")
If Not el Is Nothing Then
MsgBox el.innerText
End If
Team,
IE webpage full loading check use below mentioned code
Application.Wait (Now + TimeValue("0:00:1"))
Do Until IE.Busy = False
DoEvents
Loop
ieApp.Navigate "Https://duckduckgo.com/"
'Wait for Internet Explorer to finish loading
Do While ieApp.Busy: DoEvents: Loop
Do While ieApp.Busy And Not ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
ieApp.Document.getelementbyid("search_form_input_homepage").Value = "Gajendra Santosh"
ieApp.Document.getelementbyid("search_button_homepage").Click

Resources