vba code to fetch data from website - excel

I am a newbie in this website and in VBA programming as well. I am stuck into a problem where I have to fetch the data from this page. I need to have the hyperlink url of Check Rates 10 button. Can anyone help me with this problem.
I have done the following code:
Sub GetData()
Dim IE As New InternetExplorer
IE.navigate "http://www.kieskeurig.nl/zoeken/index.html?q=4960999543345"
IE.Visible = False
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Application.Wait (Now() + TimeValue("00:00:016")) ' For internal page refresh or loading
Dim doc As HTMLDocument 'variable for document or data which need to be extracted out of webpage
Set doc = IE.document
Dim dd As Variant
dd = doc.getElementsByClassName("lgn")(0).outerHtml
'Range("a1").Value = dd
MsgBox dd
End Sub
In which I am getting text of the button but I want to have the value of the class. I think I am very close to the result but somehow cant reach to the goal...can anyone please help me...
Regards,

I think this is what you're looking for:
(Code modified slightly from Kyle's answer here)
Sub Test()
'Must have the Microsoft HTML Object Library reference enabled
Dim oHtml As HTMLDocument
Dim oElement As Object
Dim link As String
Set oHtml = New HTMLDocument
With CreateObject("WINHTTP.WinHTTPRequest.5.1")
.Open "GET", "http://www.kieskeurig.nl/zoeken/index.html?q=4960999543345", False
.Send
oHtml.Body.innerHTML = .responseText
End With
If InStr(1, oHtml.getElementsByClassName("lgn")(0).innerText, "Bekijk 10 prijzen") > 0 Then
link = Mid(oHtml.getElementsByClassName("lgn")(0).href, 7)
Debug.Print "http://www.kieskeurig.nl" & link
End If
End Sub
This code prints the URL to the immediate window. Hope that helps!

This works for me...
Sub GetData()
Set IE = CreateObject("InternetExplorer.Application")
my_url = "http://www.kieskeurig.nl/zoeken/index.html?q=4960999543345"
With IE
.Visible = True
.navigate my_url
.Top = 50
.Left = 530
.Height = 400
.Width = 400
Do Until Not IE.Busy And IE.readyState = 4
DoEvents
Loop
End With
Application.Wait (Now() + TimeValue("00:00:016")) ' For internal page refresh or loading
Set Results = IE.document.getElementsByTagName("a")
For Each itm In Results
If itm.classname = "lgn" Then
dd = itm.getAttribute("href")
Exit For
End If
Next
' if you wnat to click the link
itm.Click
' otherwise
'Range("a1").Value = dd
MsgBox dd
End Sub

Related

How to close a web message from IE VBA?

I,ve been dealing with a issue during my web scraping.
My problem is that when I click to "Submit" a form, it pops-up a web message that I've not been able to close.
I´ve already try the sendKeys Method but no success.
here is the link of the web page:
https://www3.bcb.gov.br/CALCIDADAO/publico/corrigirPorIndice.do?method=corrigirPorIndice
Here is a print of the info to put in the webpage
Here is a print of the info to put in the webpage
The message appers when you click the button on the left "Corrigir Valor"
enter image description here
the messege
enter image description here
obs. pressing enter or esc with the keyboard close it, but I`m not been able to do it in the code
here is the code if may be helpful
Sub atualizacao_valores()
Dim data_base As String
Dim data_atualizacao As String
Dim valor_face As String
Dim drp As HTMLFormElement
Dim html As HTMLDocument
Set ie = CreateObject("internetexplorer.application")
ie.navigate "https://www3.bcb.gov.br/CALCIDADAO/publico/exibirFormCorrecaoValores.do?method=exibirFormCorrecaoValores&aba=1"
ie.Visible = True
Do While ie.busy And ie.readyState <> "READYSTATE_COMPLETE"
DoEvents
Loop
data_base_ipcae = "01/2022"
data_atualizacao_ipcae = "12/2022"
valor_face = "100000"
Application.Wait (Now + TimeValue("00:00:02"))
Set html = ie.document
Set drp = html.getElementById("selIndice")
drp.selectedIndex = 4
ie.document.getelementsbytagname("input")(1).Value = data_base_ipcae
ie.document.getelementsbytagname("input")(2).Value = data_atualizacao_ipcae
ie.document.getelementsbytagname("input")(3).Value = valor_face
ie.document.getelementsbyclassname("botao")(0).Click
`here happens the pop up that I cant close
...
end sub
I agree with the suggestion given by Tim Williams.
Adding code below just above the line when you click the button could suppress the Alert().
With html.parentWindow
.execScript "window.alert = function(){return true;};", "JScript"
End With
Your modified code:
Sub atualizacao_valores()
Dim data_base As String
Dim data_atualizacao As String
Dim valor_face As String
Dim drp As HTMLFormElement
Dim html As HTMLDocument
Set ie = CreateObject("internetexplorer.application")
ie.navigate "https://www3.bcb.gov.br/CALCIDADAO/publico/exibirFormCorrecaoValores.do?method=exibirFormCorrecaoValores&aba=1"
ie.Visible = True
Do While ie.busy And ie.readyState <> "READYSTATE_COMPLETE"
DoEvents
Loop
data_base_ipcae = "01/2022"
data_atualizacao_ipcae = "12/2022"
valor_face = "100000"
Application.Wait (Now + TimeValue("00:00:02"))
Set html = ie.document
Set drp = html.getElementById("selIndice")
drp.selectedIndex = 4
ie.document.getelementsbytagname("input")(1).Value = data_base_ipcae
ie.document.getelementsbytagname("input")(2).Value = data_atualizacao_ipcae
ie.document.getelementsbytagname("input")(3).Value = valor_face
With html.parentWindow
.execScript "window.alert = function(){return true;};", "JScript"
End With
ie.document.getelementsbyclassname("botao")(0).Click
End Sub
I have tested this code on my end and it is suppressing the Alert message.
You could test it and let us know your test results.

VBA Excel open multitude websites in the one IE window

I have managed with opening the Internet Explorer using VBA Excel.
My code looks as follows:
Sub IE()
Dim ie As object
Dim location
'Dim button
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate ("https://fulcrum/NewBuild/Record.aspx?ID=" & Range("B1").Value)
.Navigate ("http://gcommswebmapgb/portal/")
.Top = 5
.Left = 5
.Height = 1300
.Width = 1900
While ie.ReadyState <>4
Do Events
Wend
Set location = .document.getElementById("__VIEWSTATE")
'Set button = .document.getElementById("btnContainer").Children(0)
'button.Click
While ie.ReadyState <> 4
DoEvents
Wend
End With
Set ie = Nothing
End Sub
In this event, the 2nd link is opened and 1st one completely omitted.
On top of that, I would like to open them in the same window (as a different tab).
I found some solutions here:
Excel VBA control IE
VBA Excel input data into already opened ie window
where it looks that some functions are required.
Is anyone able to help?
I suggest you store both URLs in a variables like below.
Dim url1, url2 As String
url1 = "https://fulcrum/NewBuild/Record.aspx?ID=" & Range("B1").Value
url2 = "http://gcommswebmapgb/portal/"
Then try to pass the URL variables to .Navigate without round brackets like below.
With ie
.Visible = True
.Navigate url1
.Navigate url2, CLng(2048)
It will fix your syntax error and urls will be opened in 2 tabs.
Modified code:
Sub ie()
Dim ie As Object
Dim location
'Dim button
Dim url1, url2 As String
url1 = "https://fulcrum/NewBuild/Record.aspx?ID=" & Range("B1").Value
url2 = "http://gcommswebmapgb/portal/"
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate url1
.Navigate url2, CLng(2048)
' .Top = 5
' .Left = 5
' .Height = 1300
' .Width = 1900
While ie.ReadyState <> 4
'Do Events
Wend
'Set location = .document.getElementById("__VIEWSTATE")
'Set button = .document.getElementById("btnContainer").Children(0)
'button.Click
While ie.ReadyState <> 4
DoEvents
Wend
End With
Set ie = Nothing
End Sub
Output:
Further, you need to check that your code properly references the correct tab to execute further code. Note that you cannot automate both pages at the same time. You need to switch the tabs using your VBA code to execute code on specific page.

VBA HTML elements to Excel

I am working on a code that uses VBA-Excel to navigate to a website and copy some values to Excel.
I can open the website and navigate, but I can't save the "Precipitation" values in excel sheet
Sub accuweather()
Dim ie As InternetExplorer
Dim pagePiece As Object
Dim webpage As HTMLDocument
Set ie = New InternetExplorer
ie.Visible = True
ie.navigate ("http://www.accuweather.com/en/pt/abadia/869773/daily-weather-forecast/869773?day=2")
Do While ie.readyState = 4: DoEvents: Loop
Do Until ie.readyState = 4: DoEvents: Loop
While ie.Busy
DoEvents
Wend
Set webpage = ie.document
Set mtbl = webpage.getElementsByTagName("details-card card panel details allow-wrap")
Set table_data = mtbl.getElementsByTagName("div")(1)
For itemNum = 1 To 240
For childNum = 0 To 5
Cells(itemNum, childNum + 1) = table_data.Item(itemNum).Children(childNum).innerText
Next childNum
Next itemNum
ie.Quit
Set ie = Nothing
End Sub
The method you are using is getElementsByTagName but the reference is for a multi-valued class. So the correct method would be getElementsByClassName.
However, you don't need the browser as that content is static and you can just use faster xmlhttp request and a single (more robust and faster) class to target.
This
html.querySelectorAll(".list")
is retrieving the two parent nodes which have the various p tag children. The first child in both cases
.Item(i).FirstChild
is the precipitation info.
Option Explicit
Public Sub GetPrecipitationValues()
Dim html As MSHTML.HTMLDocument, i As Long
Set html = New MSHTML.HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://www.accuweather.com/en/pt/abadia/869773/daily-weather-forecast/869773?day=2", False
.setRequestHeader "User-Agent", "Mozilla/5.0"
.send
html.body.innerHTML = .responseText
End With
With html.querySelectorAll(".list")
For i = 0 To .Length - 1
Debug.Print .Item(i).FirstChild.innerText
Next
End With
End Sub

Excel VB Macro to scrape webpage. Can't code to click html button

I have a short excel macro that is designed to:
1) Open Internet Explorer and navigate to "http://www.puco.ohio.gov/pucogis/address/search.cfm"
2) Fill out a form on that site with data from the excel workbook
3) Click a button to submit the form
4) Scrape some innertext from the website and place it in a cell in the workbook
5) Close Internet Explorer
I can not get step 3 to work. That is, I can not get the click/submit function to work with this website. When the button is clicked the website populates with information specific to the information entered in the form. Everything else in the code is working. I have searched for an answer and tried the submit verses click approach with no luck.
Thanks for you help.
Code below:
Private Sub SiteData()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
Set HTMLButton = ie.Document.getElementsByTagName("input")(1)
HTMLButton.Click
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D50") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
Try this solution, which I found from this answer to a similar question. That answer was not accepted, but I have tested this with your code and seems to be working.
Private Sub SiteData()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
Call ie.Document.parentWindow.execScript("codeAddress()")
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D50") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
If you don't know or can't reasonably anticipate the function call codeAddress(), then you can try something like this to derive it from the button's onclick property:
Dim fn$
fn = HTMLButton.onclick
fn = Mid(fn, InStr(fn, "{"))
fn = Trim(Replace(Replace(Replace(fn, "{", vbNullString), "}", vbNullString), vbLf, vbNullString))
Call ie.Document.parentWindow.execScript(fn)
You can call the JavaScript directly. try this it will work
Instead of:
Set HTMLButton = ie.Document.getElementsByTagName("input")(2)
HTMLButton.Click
use
ie.Document.parentWindow.execScript code:="codeAddress()"
note that IE may prompt you to confirm every run so you may need to
stop showing this message for smooth operation
Private Sub CommandButton1_Click()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
ie.Document.parentWindow.execScript code:="codeAddress()"
'Set HTMLButton = ie.Document.getElementsByTagName("input")(2)
'HTMLButton.Click
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D16") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
thanks also to this article helped me to solve your problem
How to find and call javascript method from vba

Trying to extract ONE value from a webpage with VBA in Excel

I've been trying to find the information now for a couple of days, but all the examples I've found just has a small piece of the code, I need it all =)
What I want to do is to extract one value from a homepage and put it into a cell in Excel
(and then take another value from another page on the same site and put in the next cell etc etc.)
The page is a swedish stock-exchange page, and the page I've used as a test-page is the stock for "Investor B" (https://www.avanza.se/aktier/om-aktien.html/5247/investor-b)
And the value I'm interested in is the one called "Senaste" (this is the page-information surrounding it)
<li>
<span class="XSText">Senast<br/></span>
<span class="lastPrice SText bold"><span class="pushBox roundCorners3" title="Senast uppdaterad: 17:29:59">248,60</span></span>
</li>
And it's the value 248,60 I'm after!
I got some coding experience, but not for VBA-scripting, after reading some forum-posts (mostly here), I've been trying out a few example by myself, but couldn't get any to work.
Since I'm quite basic with VBA, I might have got the structure wrong, so please be basic and patient with me, this was my test, but I got "Runtime error 429"
ActiveX component can't create object
I might be totally on the wrong track
Private Sub CommandButton1_Click()
Dim ie As Variant
Set ie = CreateObject("InternetExplorer")
ie.navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
ie.Visible = True
Do
DoEvents
Loop Until ie.readyState = READYSTATE_COMPLETE
Application.Wait (Now() + TimeValue("00:00:016")) ' For internal page refresh or loading
Dim doc As Variant 'variable for document or data which need to be extracted out of webpage
Set doc = CreateObject("HTMLDocument")
Set doc = ie.document
Dim dd As Variant
dd = doc.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
End Sub
EDIT: 2014-05-12 Current code beeing tested 17:05
under the button command
Private Sub CommandButton1_Click()
Dim IE As Object
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
' You can uncoment Next line To see form results
IE.Visible = False
' Send the form data To URL As POST binary request
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
' Statusbar
Application.StatusBar = "Loading, Please wait..."
' Wait while IE loading...
'Do While IE.Busy
' Application.Wait DateAdd("s", 1, Now)
'Loop
'this should go from ready-busy-ready
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
' Dim Document As HTMLDocument
' Set Document = IE.Document
Dim dd As Variant
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
' Show IE
IE.Visible = True
' Clean up
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
Application.StatusBar = ""
End Sub
And in module1
Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Public Function IEWait(p_ieExp As InternetExplorer)
'this should go from ready-busy-ready
Dim initialReadyState As Integer
initialReadyState = p_ieExp.ReadyState
'wait 250 ms until it's done
Do While p_ieExp.Busy Or p_ieExp.ReadyState <> READYSTATE_COMPLETE
Sleep 250
Loop
End Function
As said earlier, I do not know if I got the structure right with this latest add-in, not to expired in this kind of coding I'm afraid.
Best Regards
Stop editing 2014-05-12 17:08
You are close but have a couple small errors.
Here is how I would set it up (Tested):
Private Sub CommandButton1_Click()
Dim IE As Object
' Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
' You can uncoment Next line To see form results
IE.Visible = False
' URL to get data from
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/5247/investor-b"
' Statusbar
Application.StatusBar = "Loading, Please wait..."
' Wait while IE loading...
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
Application.StatusBar = "Searching for value. Please wait..."
Dim dd As String
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
MsgBox dd
' Show IE
IE.Visible = True
' Clean up
Set IE = Nothing
Application.StatusBar = ""
End Sub
Results:
Tested in Excel 2010 with the following references:
Edit - Option B
To get rid of a possible "Run-Time Error '91'" try changing a few lines like this:
Dim dd As Variant
Set dd = IE.Document.getElementsByClassName("lastPrice SText bold")
MsgBox dd(0).textContent
Edit - Option C
Yet another way to get elements:
Dim tag
Dim tags As Object
Set tags = IE.Document.getElementsByTagName("*")
For Each tag In tags
If tag.className = "lastPrice SText bold" Then
MsgBox tag.innerText
Exit For
End If
Next tag
(All three methods have been tested on Excel 2010 and IE10)
I just wanted to add the code I'm currently running which works perfectly fine at the moment, if people run into the same problem. This is to get two values into dedicated cells.
Private Sub CommandButton10_Click()
Dim IE As Object
Dim dd As Variant
' Create InternetExplorer Object
Set IE = GetObject("new:{D5E8041D-920F-45e9-B8FB-B1DEB82C6E5E}")
IE.Visible = False
' Send the form data To URL As POST binary request
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/52476/alk-abell-b"
Application.StatusBar = "Loading, Please wait..."
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
Range("Y2").Value = dd
IE.Navigate "https://www.avanza.se/aktier/om-aktien.html/52380/alm--brand"
Application.StatusBar = "Loading, Please wait..."
IEWait IE
Application.StatusBar = "Searching for value. Please wait..."
dd = IE.Document.getElementsByClassName("lastPrice SText bold")(0).innerText
Range("Y3").Value = dd
' Clean up
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
Application.StatusBar = ""
End Sub
If one wants more data, it is just to copy the part starting with
IE.Navigate "https://www.pagewhereyourdatayouwanttoextractis.com"
and stops with
Range("Y2").Value = dd
It is ofcourse based if the page you want to extract data from has a similiar structure to the one above.
Hope this can help some people out there.
Best Regards

Resources