Web scraping - 'IWebBrowser2' failed - excel

Currently, I have provided a dummy website below, and I was using the code below to navigate to the website and enter the value in the specific cell. After running this code, it navigates to the website, but no values are filled in and an error message appears.
Sub Test1()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
'IE.navigate "https://www.facebook.com/"
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
IE.document.getElementById("WD0282").Value = "1234"
End Sub
Error: Run-time error '2147467259 (80004005) Method Document of object
'IWebBrowser2' failed

Try to replace the line below.
Set IE = CreateObject("InternetExplorer.Application")
with the line below.
Set IE = GetObject("new:{D5E8041D-920F-45e9-B8FB-B1DEB82C6E5E}")
It could help you fixed the said error.
Helpful reference: Method Document of object 'IWebBrowser2' failed on a dedicated company server

Related

Login to web page and scraping data using VBA

I'm coding a macro that will load a web page, login in it and get the datas from a table.
Thanks to several topics here I've done that:
Sub sail2()
Dim ie As Object
Set ie = New(InternetExplorer.Application")
ie.Visible = True
ie.Navigate "http://pfrwtekref/websail/"
With ie
Dim oLogin As Object, oPassword As Object
Set oLogin = .document.getElementById("txtPwd")(0)
Set oPassword = .document.getElementById("txtPwd")(0)
oLogin.Value = test
oPassword.Value = test
.document.forms(0).submit
End With
End Sub
But I have two type of errors "automation error" if I launch the macro and "error 462 - server not available"
Can anyone help me ?
Thanks for reading and have a good day !
The first issue seems to be that you have a mismatched comma on the third line; however, seeing as you're are complaining about an automation error I think that may be just a typo on this site.
I can't see your internal website so I'm just guessing but I suspect the next issue is that there is no element with ID "txtPwd". You can check to confirm by pressing Ctrl-Shift-C and then selecting the username and password entry boxes.
Finally, depending on how the site is set up your .document.forms(0).submit may not work. You may need to find the ID for the submit button class submit. Below is a function I created a while back for such a task:
Function logIn(userName As String, password As String) As Boolean
'This routine logs into the grade book using given credentials
Dim ie As New InternetExplorer
Dim doc As HTMLDocument
On Error GoTo loginFail
ie.Navigate "[website here]"
'ie.Visible = True
Do While ie.ReadyState <> READYSTATE_COMPLETE Or ie.Busy: DoEvents: Loop 'Wait server to respond
Set doc = ie.Document
doc.getElementsByName("u_name").Item(0).Value = userName 'These may be different for you
doc.getElementsByName("u_pass").Item(0).Value = password
doc.getElementsByClassName("btn").Item(0).Click
Do While ie.ReadyState <> READYSTATE_COMPLETE Or ie.Busy: DoEvents: Loop 'Wait server to respond
Set doc = ie.Document
'Add a check to confirm you aren't on the same page
ie.Quit
Set ie = Nothing
LogIn = True
Exit Function
loginFail:
MsgBox "There was an issue logging in. Please try again."
logIntoGradeBook = False
End Function
Note that the site I was dealing with was set up poorly and so I needed to switch to GetElementsByName and GetElementsByClassName to get access to what I needed. You may be fine with IDs.
Why don't you try to use the Excel PowerQuery to get your data from the tables you need? When your desired columns are done, you just click in close and load, select data model e now you can use a macro to perform a PivotTable whenever you load the document or just use a PivotTable and the data will be refreshed when you use the refresh data in data ribbon.

Object Not Set error when Excel VBA to pull HTML data from web

I have been using code from this youtube tutorial (https://www.youtube.com/watch?v=7sZRcaaAVbg) to have VBA in Excel to pull web data into my sheet. What I don't understand is that it worked perfectly at first for a a few weeks, but, without touching any part of the code, about a month or two ago I ran into an error that reads:
"Run-time error '91':
Object variable or With block variable not set"
Here is the part of my code that the trips the error, where 'address' is the web address I want to pull data from:
ActiveSheet.Cells(1, 1) = "Opening Web Page"
Dim IE As New InternetExplorer
IE.Visible = false
Dim address As String
address = ActiveSheet.Cells(4, 3)
IE.navigate address
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim askingPrice As Variant
ActiveSheet.Cells(1, 1) = "Getting Info From Web Page"
askingPrice = Trim(Doc.getElementsByClassName("ds-value")(0).innerText)
Can anyone lead me to correctly setting the variable 'Doc' so that I can use 'getElementsByClassName'? Or show me if there is something else I need to do?
Thank you

Engage with widget on IE by VBA

I haven't really tried working on this before, so I have no idea what I am doing at the moment. I have limited knowledge of html so not sure whether I am doing right. Basically what I aim to do is opening the Internet explorer by macro, changing some elements based id and click submit button on the website to show the data. Then I need keep working on the next step.
As you can see from the code I was trying to engage with the widget on IE by id number from html codes.
Sub Automate_IE_Enter_Data()
'This will load a webpage in IE
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objbutton As Object
'Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
'Set IE.Visible = True to make IE visible, or False for IE to run in the background
IE.Visible = True
'Define URL
URL = "http://cfpsg1/plant/Reports/ScrapReport.aspx"
'Navigate to URL
IE.Navigate URL
' Statusbar let's user know website is loading
Application.StatusBar = URL & " is loading. Please wait..."
' Wait while IE loading...
'IE ReadyState = 4 signifies the webpage has loaded (the first loop is set to avoid inadvertantly skipping over the second loop)
Do While IE.ReadyState = 4: DoEvents: Loop
'Webpage Loaded
Application.StatusBar = URL & " Loaded"
IE.Document.getelementbyid("1stGroupBy").Value = "3"
'Find & Fill Out Input Box
IE.Document.getelementbyid("PageContent_uxStartDate").Value = "06/21/2019"
IE.Document.getelementbyid("PageContent_uxEndDate").Value = "06/21/2019"
Set objbutton = IE.Document.getelementbyid("PageContent_btnQuery")
objbutton.Focus
objbutton.Click
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
End Sub
First thing first the webpage popped up but nothing changed of widgets besides an
error message "method 'Document' of object 'IWebBrowser 2' failed"
show on IE.Document.getelementbyid("1stGroupBy").Value = "3" row.
You are trying to interact with a dropdown so you want syntax such as
IE.Document.querySelector("[value='3']").Selected = True
You could also use
IE.Document.querySelector("#1stGroupBy").SelectedIndex = 2 'change to appropriate index
Error "method 'Document' of object 'IWebBrowser 2' failed" may be due to Integrity level
You can try below mentioned code by changing Integrity level as Medium
Dim IE As InternetExplorer
Set IE = New InternetExplorerMedium
IE.Visible = True
URL = "http://cfpsg1/plant/Reports/ScrapReport.aspx"
IE.Navigate URL
Do While IE.ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
and try other line of codes to avoid this issue also please refer Here
Please add ref Microsoft Internet Controls and Microsoft HTML Object Library based on your req

getElementById won't work in VBA, error 438

I am relatively new at VBA (I know the basics, but not much else) and I am trying to get some code set up that will fill out online forms for me, but when I run my code I get a 438 error:
object doesnt support this property or method
when it gets to
ie.document.getElementById ("q")
I have added the HTML object library and Microsoft internet controls to my references. I have looked at tons of online forums. I have even copied and pasted an entire script directly into VBA. Nothing will make it use getElementById(). Here is my code:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate ("https://www.google.com/")
Set searchbx = ie.document.getElementById("q")
searchbx.Value = "Howdy!"
End Sub
What should happen is that it should open InternetExplorer, go to Google, and populate the search bar with "Howdy!".
Instead, It only opens google, then I get the error message and it stops running.
Because "q" is not an ID, it's a Name.
You have to use GetElementsByName and select the 1st Element
try this:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.Navigate ("https://www.google.com/")
Do Until ie.ReadyState >= 4
DoEvents
Loop
Set searchbx = ie.document.getElementsByName("q")(0)
searchbx.Value = "Howdy!"
End Sub
I have also added a waiting Event, in case it takes time to load Google.
Result:
A more concise and efficient way is to use querySelector to return first match for the name attribute
Option Explicit
Public Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate2 "https://www.google.com/"
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector("[name=q]").Value = "Howdy"
Stop '<delete me later
.Quit
End With
End Sub

Scraping source code of website does not work on VDI

I have a problem with extracting data from the website using VBA on Citrix Virtual Desktop.
I have wrote my code on my local desktop first and it works good - HTML source has been extracted to the cell in Excel.
On VDI IE opens the website without any problems.
Code:
Sub GetBody()
Dim Body As String
the_start:
Set ObjIE = CreateObject("InternetExplorer.Application")
ObjIE.Visible = False
ObjIE.navigate ("https://pl.wikipedia.org/wiki/Wikipedia:Strona_g%C5%82%C3%B3wna")
Do
DoEvents
If Err.Number <> 0 Then
ObjIE.Quit
Set ObjIE = Nothing
GoTo the_start:
End If
Loop Until ObjIE.readyState = 4
Body = ObjIE.document.Body.innerHTML
Cells(1, 1).Value = Body
End Sub
When I try to run this code on VDI I am getting following error:
Run-time error '-2147467259(80004005)': Method 'Document' of object 'IWebBrowser2' failed.
Any ideas where this error comes from and what I should add to run it successfully on VDI?
I have done some changes mentioned in the comments (like changing the endless loop etc.) and also have another errors ( Automation error The object invoked has disconnected from its clients).. Previously I have declared IE as a object in this line below:
Set ObjIE = CreateObject("InternetExplorer.Application")
Soultion for all my problems:
Dim IE as SHDocVw.InternetExplorer
Set IE = New InternetExplorerMedium
Thank you all for participating in this thread and THANK YOU SO MUCH for your help!

Resources