VBA extract text value from webpage? - excel

I have a webpage with some text in a HTML Span like so:
<span id="ctl00_ContentPlaceHolder1_FormView1_GridView1_ctl02_lb_ExpiryDate">Expiry Date : 16/02/2018</span>
I am trying to get this value display it as a message in excel using the below code:
Sub PullExpiry()
Dim appIE As Object
Set appIE = CreateObject("internetexplorer.application")
With appIE
.Navigate "https://www.brcdirectory.com/InternalSite//Site.aspx?BrcSiteCode=" & Range("J6").Value
.Visible = True
End With
Do While appIE.Busy Or appIE.ReadyState <> 4
DoEvents
Loop
Set getPrice = appIE.Document.getElementById("ctl00_ContentPlaceHolder1_FormView1_GridView1_ctl02_lb_ExpiryDate")
Dim myValue As String
myValue = getPrice.innerText
appIE.Quit
Set appIE = Nothing
MsgBox myValue
End Sub
This was working on my laptop (operating windows) but it does not work on my computer (also operating windows). Both windows are the same version with the same version of IE. I cannot explain it.
I have Microsoft Office and Excel Object libraries turned on in both references.
I get an error about an active x component not being able to create something
Please can someone show me where i am going wrong?

You need to add Microsoft Internet Controls to your references to be able to create the object. You may have to register the shdocvw.dll library on your computer. It may be registered on your laptop already which is why it might be bombing on your computer.
How to register a .dll
MSDN Documentation: look at the last sentence before the C# example.
Similar Question
After playing around a little bit and ensuring everything was registered, this ran fine on my PC:
Public Sub ie()
Dim ieApp As SHDocVw.InternetExplorerMedium
Dim html As HTMLDocument
Set ieApp = New SHDocVw.InternetExplorerMedium
ieApp.Visible = True
ieApp.Navigate "http://internalAddress/"
Do While ieApp.Busy Or ieApp.ReadyState <> 4
DoEvents
Loop
Set html = ieApp.Document.getElementById("myID")
End Sub

Related

Fetching text from website using VBA for Mac

I am sorry if this is a very basic question, but I'm really desperate. I want to fetch the output of the website https://cactus.nci.nih.gov/chemical/structure/78-70-6/smiles, and input it into an Excel cell using VBA. Does anyone know of a simple and straightforward way of doing this?
In a way, I would like something like "curl", in bash, but that could be used in VBA for Excel.
I'm using Excel for Mac, Version 16.16.14, with VBA 7.1.
I've tried many of the approaches suggested online, and nothing seemed to work on my version of Excel.
Thanks a lot.
This should do the trick;
Dim InternetExplorer As Object 'dim your internet explorer
Dim WebsiteURL As String: WebsiteURL = "https://cactus.nci.nih.gov/chemical/structure/78-70-6/smiles" 'dim and set your website url
Dim WebsiteText As String 'dim a variable to store the website text
Set InternetExplorer = CreateObject("InternetExplorer.Application") 'set your internet explorer
With InternetExplorer
.Visible = 1 'set to show (if you put 0 the browser would be hidden - just remember to set to nothing at end)
.navigate WebsiteURL 'navigates to websiteurl
While .Busy Or .readyState <> 4 'waits for page to load
DoEvents
Wend
End With
Dim WebPage As HTMLDocument 'dim your webpage as htmldocument
Set WebPage = InternetExplorer.document 'set your webpage to the internet explorer
WebsiteText = WebPage.body.innerText: Range("A1").Value = WebsiteText 'puts website text into variable : write to whatever cell you want
EDIT:
Just noticed you said for MAC - My bad as this uses internet explorer

getElementById won't work in VBA, error 438

I am relatively new at VBA (I know the basics, but not much else) and I am trying to get some code set up that will fill out online forms for me, but when I run my code I get a 438 error:
object doesnt support this property or method
when it gets to
ie.document.getElementById ("q")
I have added the HTML object library and Microsoft internet controls to my references. I have looked at tons of online forums. I have even copied and pasted an entire script directly into VBA. Nothing will make it use getElementById(). Here is my code:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate ("https://www.google.com/")
Set searchbx = ie.document.getElementById("q")
searchbx.Value = "Howdy!"
End Sub
What should happen is that it should open InternetExplorer, go to Google, and populate the search bar with "Howdy!".
Instead, It only opens google, then I get the error message and it stops running.
Because "q" is not an ID, it's a Name.
You have to use GetElementsByName and select the 1st Element
try this:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.Navigate ("https://www.google.com/")
Do Until ie.ReadyState >= 4
DoEvents
Loop
Set searchbx = ie.document.getElementsByName("q")(0)
searchbx.Value = "Howdy!"
End Sub
I have also added a waiting Event, in case it takes time to load Google.
Result:
A more concise and efficient way is to use querySelector to return first match for the name attribute
Option Explicit
Public Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate2 "https://www.google.com/"
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector("[name=q]").Value = "Howdy"
Stop '<delete me later
.Quit
End With
End Sub

Scraping source code of website does not work on VDI

I have a problem with extracting data from the website using VBA on Citrix Virtual Desktop.
I have wrote my code on my local desktop first and it works good - HTML source has been extracted to the cell in Excel.
On VDI IE opens the website without any problems.
Code:
Sub GetBody()
Dim Body As String
the_start:
Set ObjIE = CreateObject("InternetExplorer.Application")
ObjIE.Visible = False
ObjIE.navigate ("https://pl.wikipedia.org/wiki/Wikipedia:Strona_g%C5%82%C3%B3wna")
Do
DoEvents
If Err.Number <> 0 Then
ObjIE.Quit
Set ObjIE = Nothing
GoTo the_start:
End If
Loop Until ObjIE.readyState = 4
Body = ObjIE.document.Body.innerHTML
Cells(1, 1).Value = Body
End Sub
When I try to run this code on VDI I am getting following error:
Run-time error '-2147467259(80004005)': Method 'Document' of object 'IWebBrowser2' failed.
Any ideas where this error comes from and what I should add to run it successfully on VDI?
I have done some changes mentioned in the comments (like changing the endless loop etc.) and also have another errors ( Automation error The object invoked has disconnected from its clients).. Previously I have declared IE as a object in this line below:
Set ObjIE = CreateObject("InternetExplorer.Application")
Soultion for all my problems:
Dim IE as SHDocVw.InternetExplorer
Set IE = New InternetExplorerMedium
Thank you all for participating in this thread and THANK YOU SO MUCH for your help!

VBA IE.Document empty error

I've been running a query for a while now getting data from a webpage. After numerous runs it has decided to stop working, and I've traced the issue back to the ie.document object - it never returns anything.
When compiling my project I see that the "Document" element of ie returns an error of "Applicaiton-defined or Object-defined error" - even before I navigate to a webpage. Also some other elements return this error also - namely "Status Text" and "Type"
The link contains a screenshot of my error:
https://www.dropbox.com/s/wcxxep8my10nu8h/vba%20ie%20document.jpg?dl=0
In case that doesn't work here a scaled back version of the code I'm running
Sub getCard()
Dim ie As InternetExplorer
Dim url1 As String
url1 = "google.com"
Set ie = New InternetExplorer
ie.Visible = True
ie.Navigate url1
WaitBrowserQuiet ie
End Sub
Sub WaitBrowserQuiet(objIE As InternetExplorer)
Do While objIE.Busy Or objIE.ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
End Sub
As soon as I get to the "Set ie = New InternetExplorer" part of the code is when the ie object is created and I see the errors. If I do happen to navigate to webpage, then the ie.document object is empty.
I've searched around and tried a few things to stop this happening - restarted my computer, run "ie.quit" and "Set ie = Nothing", reset my Internet Explorer, etc... Nothing seems to work.
It seems like it may be a deeper issue given I'm getting an error message even before navigating to a webpage. Hope someone knows how to stop the error.
Your URL is URL1, try changing that, or just putting the URL in there.
In your code you have the object "ie" locally defined in the sub getCard and when this sub finishes,so goes the binding. Also changing from private to public internet zones can remove the binding to that object. What I rather do is use a global object appIE and then when it runs into such an error I catch the error (if TypeName(appIE) = "Object" Then FindIE) and find the object again with this sub:
Sub FindIE() 'Needs reference to "Microsoft Shell Controls And Automation" from VBA->Tools->References
Dim sh
Dim eachIE
Dim SearchUntil
SearchUntil = Now() + 20 / 24 / 60 / 60 'Allow to search for 20 seconds otherwise it interrupts search
Do
Set sh = New Shell32.Shell
For Each eachIE In sh.Windows
If InStr(1, eachIE.LocationURL, ServerAddress) Then
Set appIE = eachIE
'IE.Visible = False 'This is here because in some environments, the new process defaults to Visible.
'Exit Do
End If
Next eachIE
If TypeName(appIE) <> "Object" Then
If InStr(1, appIE.LocationURL, ServerAddress) > 0 Or SearchUntil < Now() Then Exit Do
End If
Loop
Set eachIE = Nothing
Set sh = Nothing
End Sub
This code contains parts of other people here from stackoverflow, but I forgot who to credit for some essential parts of the code. Sorry.

Using Excel VBA, open a web page and then save it

I'm trying to write a macro that will open a given URL and perform a File >> SaveAs... The URL returns an XML output on the screen and I want to save it. Then I can have Excel open the file and work on it (this part I can do).
Here's basically what I want to do and it works except for the SaveAs bit:
Sub Main()
REM Reference to "Microsoft Internet Controls" is being used.
Dim objIE As InternetExplorer
Set objIE = New InternetExplorer
With objIE
.Visible = True
.navigate "http://exampleURL.comany.com/ServerOutput/XmlFile.xml"
Do Until .readyState = 4
DoEvents
Loop
REM The next line fails
.SaveAs "C:\temp\test.xml"
.Application.Quit
End With
Set objIE = Nothing
End Sub
A plain "Save" is fine if that's all that will work. Just looking for a way to save the output on the screen. Thanks!
Maybe something like this can help?
Public Sub GetXML()
Dim oXML As New MSXML2.XMLHTTP60
Dim xURL As String
'Retrieve XML.
xURL = "http://exampleURL.comany.com/ServerOutput/XmlFile.xml"
oXML.Open "GET", xURL, False
oXML.send
'Put into sheet.
ThisWorkbook.Sheets("Sheet1").Range("A1").Value = oXML.responseXML.XML
End Sub
EDIT: Make sure the reference to Microsoft XML, v3.0 is enabled for this.

Resources