VBA to insert data in Search box IE - excel

When i insert Few words in searchbox, Its fetching related data.
I need to select first option from it.
There is one website "https://indiarailinfo.com/"
When i search "ADI" in from station box, system fetching related station having "ADI" in their name?. First option always showing very close match to it.
How can i select First Option from it using vba code
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
ie.navigate "https://indiarailinfo.com/"
While ie.readyState <> 4: DoEvents: Wend
ie.Visible = True
ie.document.querySelector("[placeholder='from station']").Value = "ADI"
HTML Codes can be available from that site
It's Bring First Answer in Dropdown like "ADI/Ahmedabad Junction"
How can i get this answer in selected"
Kindly Suggest

Automation purists won't like using javascript to execute but I will use here for IE to trigger dropdown. If I was going pure route I would use selenium.
Option Explicit
Public Sub MakeSelection()
Dim ie As InternetExplorer, t As Date, dropdown1 As Object
Set ie = New InternetExplorer
Const MAX_WAIT_SEC As Long = 5
With ie
.Visible = True
.Navigate2 "https://indiarailinfo.com/"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document.querySelector("[placeholder='from station']")
.Focus
.Value = "ADI"
ie.document.parentWindow.execScript "document.querySelector('[placeholder^=from]').click();"
End With
t = Timer
Do
DoEvents
On Error Resume Next
Set dropdown1 = .document.querySelectorAll(".icol span")
On Error GoTo 0
If Timer - t > MAX_WAIT_SEC Then Exit Do
Loop While dropdown1.Length = 0
If dropdown1.Length > 0 Then
dropdown1.item(0).Click
End If
Stop
.Quit
End With
End Sub
For automation purists using selenium basic
Option Explicit
Public Sub MakeSelection()
Dim d As WebDriver
Set d = New ChromeDriver
Const Url = "https://indiarailinfo.com/"
With d
.Start "Chrome"
.get Url
.FindElementByCss("[placeholder='from station']").SendKeys "ADI"
.FindElementByCss(".icol span").Click
Stop
.Quit
End With
End Sub

Related

VBA code to copy table data from webpage into Excel

I wish to copy data from a table on a webpage into Excel using VBA code but didn't get anything on the Excel sheet :(.
I have tried to put together some VBA code from different sources. Here is my code:
Sub CopyWebData()
Dim IE As Object
On Error Resume Next
Application.DisplayAlerts = False
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.navigate "https://eresearch.fidelity.com/eresearch/evaluate/fundamentals/earnings.jhtml?tab=details&symbols=GOOG"
Do Until .readyState = 4: DoEvents: Loop
End With
Dim idoc As MSHTML.HTMLDocument
Dim elem As MSHTML.IHTMLElement
Set idoc = IE.document
Set elem = idoc.getElementsByClassName("layout-outer-table-width")(0).innerText
Sheets("Sheet1").Activate
Range("A1:A1000") = "" ' erase previous data
Range("A1").Select
Range("A1").Value = elem
End Sub
This is a password-protected webpage and I have logged in so I can see the webpage has been successfully pulled out by the VBA code. However, the data in the table on this webpage failed to be copied into excel - I saw nothing on the destination worksheet.
As you can see, I used code .getElementsByClassName("layout-outer-table-width") since I used Chrome's "Inspect" function to check the webpage and found that when the mouse was hovering over the statements:
...<table cellspacing="0" cellpadding="0" border="0" class="layout-outer-table-width"> == $0
<tbody>...</tbody>
</table>
part of the webpage covering the table I need was shaded. I then coded in the class name "layout-outer-table-width". However, as I said, I didn't see anything appearing on the Excel sheet.
Any instruction would be much appreciated!
If after the earnings detail table you need a different selector. I am showing a css selector for that table. Your current selector (class), and index 0, is matching on a breadcrumb (nav tree). That class is also not correct for selecting the table on the page.
.earningsHistoryTable-Cont table
I cannot test this but you may also want a timed loop for table to be present
Option Explicit
'VBE > Tools > References:
' Microsoft Internet Controls
Public Sub RetrieveInfo()
Dim IE As InternetExplorer, hTable As Object, clipboard As Object, t As Date
Const MAX_WAIT_SEC As Long = 5
Set clipboard = GetObject("New:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
Set IE = New InternetExplorer
With IE
.Visible = True
.Navigate2 "https://eresearch.fidelity.com/eresearch/evaluate/fundamentals/earnings.jhtml?tab=details&symbols=GOOG"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document
.querySelector("#userId").Value = "xyz"
.querySelector("#password").Value = "123456"
.querySelector("form").submit
End With
While .Busy Or .readyState < 4: DoEvents: Wend
t = Timer 'timed loop for details table to be present
Do
On Error Resume Next
Set hTable = IE.document.querySelector(".earningsHistoryTable-Cont table")
On Error GoTo 0
If Timer - t > MAX_WAIT_SEC Then Exit Do
Loop While hTable Is Nothing
If Not hTable Is Nothing Then 'use clipboard to copy paste
clipboard.SetText hTable.outerHTML
clipboard.PutInClipboard
ThisWorkbook.Worksheets("Sheet1").Range("A1").PasteSpecial
End If
End With
End Sub

Get list of stocks from a URL with excel VBA

I want to download a file to my local worksheet from the site: https://www.bseindia.com/corporates/List_Scrips.aspx#
below is my code which I tried after doing some research.
challenge is, how to avoid the page resubmit (retry, cancel warning window)
secondly, I need to download the 5mb file and extract to the current localworksheet.
Sub bsecode()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
ie.Visible = True
'To open the website
.navigate "https://www.bseindia.com/corporates/List_Scrips.aspx#"
Do While ie.readyState <> 4
Sleep 1000
Loop
ie.document.getElementsByName("ctl00$ContentPlaceHolder1$btnSubmit")(0).Click
Sleep 1000
Do While ie.readyState <> 4
Sleep 1000
Loop
'To download the file
ie.navigate "javascript:__doPostBack('ctl00$ContentPlaceHolder1$lnkDownload','')"
'Do While .Busy: DoEvents: Loop
End With
End Sub
I'm not a fan of sendkeys but the following works
Option Explicit
'VBE > Tools > References:
' Microsoft Internet Controls
Public Sub GetData()
Dim ie As New InternetExplorer
With ie
.Visible = True
.Navigate2 "https://www.bseindia.com/corporates/List_Scrips.aspx#"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document
'status
.querySelector("[value='Active']").Selected = True 'Suspended,Delisted,Select
'group
.querySelector("[value='Select']").Selected = True ' "B ", "C " etc
'industry
.querySelector("[value='Advertising & Media']").Selected = True 'Agrochemicals etc
'segment
.querySelector("#ContentPlaceHolder1_ddSegment [value='Equity']").Selected = True
'Submit
.querySelector("#ContentPlaceHolder1_btnSubmit").Click
Const MAX_WAIT_SEC As Long = 5
Dim t As Date
While ie.Busy Or ie.readyState < 4: DoEvents: Wend
Dim download As Object
t = Timer
Do
On Error Resume Next
Set download = .querySelector("#ContentPlaceHolder1_lnkDownload")
On Error GoTo 0
If Timer - t > MAX_WAIT_SEC Then Exit Do
Loop While download Is Nothing
If Not download Is Nothing Then
download.Click
End If
Application.Wait Now + TimeSerial(0, 0, 10)
Application.SendKeys "%N", True
Application.SendKeys "%S", True
Application.Wait Now + TimeSerial(0, 0, 10)
Application.SendKeys "%O", True
End With
Stop
.Quit
End With
End Sub
It is a whole lot easier with selenium though you need to change the extension of the downloaded file from .tmp to .csv. After installing selenium basic be sure to go VBE > Tools > References and add a reference to Microsoft Scripting Runtime.
Option Explicit
Public Sub MakeSelections()
Dim d As WebDriver
Set d = New ChromeDriver
Const URL = "https://www.bseindia.com/corporates/List_Scrips.aspx#"
With d
.Start "Chrome"
.get URL
'status
.FindElementByCss("#ContentPlaceHolder1_ddlStatus").AsSelect.SelectByText "Suspended"
'group
.FindElementByCss("#ContentPlaceHolder1_ddlGroup").AsSelect.SelectByText "Select" ' "B ", "C " etc
'industry
.FindElementByCss("#ContentPlaceHolder1_ddlIndustry").AsSelect.SelectByText "Agrochemicals" 'Agrochemicals etc
'segment
.FindElementByCss("#ContentPlaceHolder1_ddSegment").AsSelect.SelectByText "Equity"
.FindElementByCss("#ContentPlaceHolder1_btnSubmit").Click
.FindElementByCss("#ContentPlaceHolder1_lnkDownload").Click
.Quit
End With
End Sub

Get a single value from a table with no ID with VBA

I am developing a web bot that scrapes the importation taxes from different countries customs website, and I have a problem retrieving the value I want from the following site : http://www.aduanet.gob.pe/itarancel/arancelS01Alias , using the test value 3303000000 next to CODIGO. The value I want to retrieve is the 6% next to "Ad / Valorem", but the table it is in has no ID properties nor class or something relevant to get directly to it or at least near to it. I have been trying to use .parent and .child methods, but without success. My code so far is as follows:
Function Peru(partida As String) As String
'Open IE
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate "http://www.aduanet.gob.pe/itarancel/arancelS01Alias"
'Load sub
Cargar
'Navigate further into the website (Im using partida = 3303000000)
For Each box In objIE.document.getElementsByTagName("input")
If box.Name = "cod_partida" Then
box.Value = partida
Exit For
End If
Next
For Each boton In objIE.document.getElementsByTagName("input")
If boton.Value = "Consultar" Then
boton.Click
Exit For
End If
Next
'Get the 6% value (This part is the one I cant figure out)
End Function
This is how you can get the data from that page. It was needed to switch two iframes from that page to reach the required content.
Sub Aduanet_Info()
Dim IE As New InternetExplorer, html As HTMLDocument
Dim elem As Object, frm As Object, frm1 As Object
With IE
.Visible = False
.navigate "http://www.aduanet.gob.pe/itarancel/arancelS01Alias"
Do While .readyState <> READYSTATE_COMPLETE: Loop
Set html = .document
End With
html.getElementsByTagName("input")(0).Value = "3303000000"
html.getElementsByTagName("input")(3).Click
Application.Wait Now + TimeValue("00:00:05")
Set frm = html.getElementsByClassName("autoHeight")(0).contentWindow.document
Set frm1 = frm.getElementsByClassName("autoHeight")(1).contentWindow.document
For Each elem In frm1.getElementsByTagName("td")
If InStr(elem.innerText, "Valorem") > 0 Then MsgBox elem.NextSibling.NextSibling.innerText: Exit For
Next elem
IE.Quit
End Sub
Output:
6%

Selecting a dropdown list when inserting data from web (VBA)

I want to download some data from a webpage (http://www.debentures.com.br/exploreosnd/consultaadados/sndemumclique/) into an Excel spreadsheet.
After loading this page I have to manually choose an option from the "Código do Ativo" dropdown list, and then choose "Agenda".
Is there a way I can do it automatically via VBA?
For example: selecting "RDVT11" from the "Código do Ativo" dropdown list, selecting "Agenda" and then downloading the data from the table that will appear in the bottom part of the page?
My macro so far:
Private Sub Agenda()
Sheets("Dados").Select
Dim ProductionAddress As String
ProductionAddress = "http://www.debentures.com.br/exploreosnd/consultaadados/sndemumclique/x_pu_historico_r.aspx?"
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Silent = True
.Visible = True
.Navigate ProductionAddress
End With
While ie.ReadyState <> 4 Or ie.Busy: DoEvents: Wend
ie.document.getElementByid("ctl00_ddlAti").Value = "RDVT11|11001110111100001"
While ie.ReadyState <> 4 Or ie.Busy: DoEvents: Wend
Set objButton = ie.document.getElementByid("ctl00_x_agenda_r")
objButton.Focus
objButton.Click
While ie.ReadyState <> 4 Or ie.Busy: DoEvents: Wend
ie.Quit
Set ie = Nothing
End Sub
You need to capture the request that a browser sends when the dropdown is activated. Open up Chrome dev tools and watch the network tab. You will see a POST request to sndemumclique/. This will have some headers and form data. Your code will need to basically replicate this request. Likely, not all of the header and form fields are required but there is not way to know without trying.
Here are all 3 parts. The making two selections and the writing of the table to the sheet.
Notes:
① Making first selection:
To make the RDVT11 selection, I first use the Id of the dropdown to capture the element in a variable with:
Set a = .document.getElementById("ctl00_ddlAti")
Next, I loop the drop down options, using a.getElementsByTagName("Option") to generate the collection which I loop over. When the target selection text is found, I set that option to Selected and exit the loop.
For Each currentOption In a.getElementsByTagName("Option")
If InStr(currentOption.innerText, optionText) > 0 Then
currentOption.Selected = True
Exit For
End If
Next currentOption
② Making Agenda selection:
I then target the agenda option of Sobre e emissão by its id and click it and wait for a refresh of the page:
.document.getElementById("ctl00_x_agenda_r").Click
While .Busy Or .readyState < 4: DoEvents: Wend
③ Getting the table and writing to sheet:
I then target the table that is loaded by its id. This is done within a loop to ensure the table is present:
Do: On Error Resume Next: Set nTable = .document.getElementById("aGENDA"): On Error GoTo 0: DoEvents: Loop While nTable Is Nothing
I finally, loop the rows and columns in the table, writing out to the Activesheet.
Code:
Option Explicit
Public Sub MakeSelectiongGetData()
Dim IE As New InternetExplorer
Const URL = "http://www.debentures.com.br/exploreosnd/consultaadados/sndemumclique/"
Const optionText As String = "RDVT11"
Application.ScreenUpdating = False
With IE
.Visible = True
.navigate URL
While .Busy Or .readyState < 4: DoEvents: Wend
Dim a As Object
Set a = .document.getElementById("ctl00_ddlAti")
Dim currentOption As Object
For Each currentOption In a.getElementsByTagName("Option")
If InStr(currentOption.innerText, optionText) > 0 Then
currentOption.Selected = True
Exit For
End If
Next currentOption
.document.getElementById("ctl00_x_agenda_r").Click
While .Busy Or .readyState < 4: DoEvents: Wend
Dim nTable As HTMLTable
Do: On Error Resume Next: Set nTable = .document.getElementById("aGENDA"): On Error GoTo 0: DoEvents: Loop While nTable Is Nothing
Dim nRow As Object, nCell As Object, r As Long, c As Long
With ActiveSheet
Dim nBody As Object
Set nBody = nTable.getElementsByTagName("tbody")(0).getElementsByTagName("tr")
.Cells(1, 1) = nBody(0).innerText
For r = 2 To nBody.Length - 1
Set nRow = nBody(r)
For Each nCell In nRow.Cells
c = c + 1: .Cells(r + 1, c) = nCell.innerText
Next nCell
c = 0
Next r
End With
.Quit
End With
Application.ScreenUpdating = True
End Sub
Data on page (sample)
Code output (sample):

VBA error when navigating with Internet Explorer

I am trying to download a table of proprietary investments/positions/pricing from Nationwide. The code seems to do what I want, EXCEPT for producing an "object required" error when I attempt to select a particular account (click)
I thought I had the proper code to tell my macro to wait until IE was ready to go on, but clearly I am missing something.
In the code, the relevant line is highlighted. If I enter a STOP above the error line, I can wait until I "see" the link appear, then "continue" the code and it runs as expected.
Because this goes to my financial accounts, I cannot provide the user name and password to allow someone to replicate the exact problem, but here is the code, and the error message and highlight. Suggestions appreciated.
Option Explicit
'set Reference to Microsoft Internet Controls
Sub DownLoadFunds()
Dim IE As InternetExplorer
Dim sHTML
Const sURL As String = "https://www.nationwide.com/access/web/login.htm"
Const sURL2 As String = "https://isc.nwservicecenter.com/iApp/isc/app/ia/balanceDetail.do?basho.menuNodeId=12245"
Dim wsTemp As Worksheet
Set wsTemp = Worksheets("Scratch")
Set IE = New InternetExplorer
With IE
.Navigate sURL
.Visible = True 'for debugging
Do While .ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
Do While .Busy = True
DoEvents
Loop
'Login: User Name and Password "remembered" by IE
.Document.all("submitButton").Click
Do While .ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
Do While .Busy = True
DoEvents
Loop
'Select this account to show
.Document.all("RothIRA_#########").Click '<--Error at this line
Do While .ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
Do While .Busy = True
DoEvents
Loop
.Navigate sURL2
Do While .ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
Do While .Busy = True
DoEvents
Loop
Set sHTML = .Document.GetElementByID("fundByFundOnly")
With wsTemp
.Cells.Clear
.Range("a2") = sHTML.innertext
End With
.Quit
End With
Set IE = Nothing
End Sub
This is the error message:
This shows the highlighted line:
EDIT:
At Tim Williams suggestion, I added a loop to test for the presence of the desired element. This seems to work:
...
On Error Resume Next
Do
Err.Clear
DoEvents
Application.Wait (Time + TimeSerial(0, 0, 1))
.Document.getelementbyid("RothIRA_#########").Click
Loop Until Err.Number = 0
On Error GoTo 0
....
IE.Document.all("#RothIRA_....") is returning Nothing (null in more refined languages), so calling the Click method is causing the error.
Your code is the same as doing this:
Dim rothElement As Whatever
rothElement = IE.Document.all("#RothIRA_....")
rothElement.Click
...when you should do this:
Dim rothElement As Whatever
rothElement = IE.Document.all("#RothIRA_....")
If rothElement <> Nothing Then
rothElement.Click
End If
I suggest using the modern document.GetElementById method instead of the deprecated (if not obsolete) document.All API.
It's possible/likely that the page is using script to dynamically load some content or generate some layout after your "wait" loop has finished. That loop only waits until all linked content/resources have been loaded - it does not wait for scripts on the loaded page to finish, etc.
One approach is to loop your code waiting for the desired element to be rendered:
Const MAX_WAIT_SEC as Long = 5 'limit on how long to wait...
Dim t
t = Timer
Do While .Document.all("RothIRA_#########") Is Nothing
DoEvents
'or you can Sleep here
If Timer - t > MAX_WAIT_SEC Then Exit Do
Loop
'carry on...

Resources