I am trying to learn web scraping with VBA and i am running into an issue in the most basic first step.
The problem (I think) is that after IEObject.Navigate is performed, all the variables of IEObject dissapear. So I get the error
"Run-time error '426': The remote server machine does not exist or is
unavailable"
on line 8. Any help is appreciated.
Code:
Sub VBAWeb()
Dim IEObject As InternetExplorer
Set IEObject = New InternetExplorer
IEObject.Visible = True
IEObject.Navigate URL:="https://google.com"
Do While IEObject.Busy = True Or IEObject.ReadyState <> READYSTATE_COMPLETE
Application.Wait Now + TimeValue("00:00:01")
Loop
Debug.Print IEObject.LocationURL
End Sub
I suggest you try to run code below on your side may help to fix the issue.
Sub demo()
Dim URL As String
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "https://www.microsoft.com"
IE.Navigate URL
Do While IE.ReadyState = 4: DoEvents: Loop
Do Until IE.ReadyState = 4: DoEvents: Loop
Debug.Print IE.LocationURL
Set IE = Nothing
End Sub
You can see that the IE object was created differently. Also, the loop is a little bit different.
Related
Having some issues with my code, I've removed the URL since it requires specific access. I have confirmed the ID is correct and copy/pasted rather than manual typing. My code does open up IE, opens the correct URL, but does not paste "test" into the description box. What have I done wrong? I've bolded the line that returns the error.
Sub Test1()
Dim IE As Object
Dim doc As HTMLDocument
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
Url = "website"
With IE
.navigate Url
.Visible = True
'Waiting till page loads
Do While .readyState <> READYSTATE_COMPLETE
DoEvents
Debug.Print "Waiting on IE" & Time
Loop
End With
Application.Wait (Now + TimeValue("00:00:08"))
**IE.document.getElementById("new_call.description").Value = "test"**
End Sub
I'm quite new to VBA so please bear with me. I've been trying to create an automation to fill in username and password and login to a site (to start with) but I've been having trouble trying to click on the submit button. Scoured the internet and learnt a whole bunch of things but I didnt find anything that seems to work. The page loads and fills in the details and nothing happens when I run the code below.
Would greatly appreciate some help with this. Thanks in advance as always!
Sub worldcheck()
Dim lastrow As Long
Dim IE As Object
Dim cel As Range
Dim post As Object
Dim ws As Worksheet
Dim element As Object
Set ws = Sheets("sheet1")
Set IE = CreateObject("internetexplorer.application")
lastrow = ws.Range("B" & ws.Rows.Count).End(xlUp).Row
IE.Visible = True
IE.Navigate "https://www.world-check.com/frontend/login/"
Do While IE.busy
DoEvents
Loop
Application.Wait (Now + TimeValue("0:00:2"))
IE.document.getElementbyID("username").Value = ws.Range("D2")
IE.document.getElementbyID("password").Value = ws.Range("D3")
IE.document.getElementbyClass("button").click
Do While IE.busy
DoEvents
Loop
End Sub
Nothing else happens? You should be getting an error message at the very least as you are trying to use a non existent method (VBA Run-time error 438 Object doesn't support this property or method) . The method is getElementsByClassName - note the s indicating it returns a collection and the ending is ClassName. You would then need to index into that collection before attempting to access the Click method
As there is only a single element with classname button you can use a faster css class selector (this is also faster than using a type selector of form; likewise, you can use the faster css equivalent of getElementById for the other two DOM elements). document.querySelector stops at the first match so is also more efficient.
Finally, rather than hard coded waits use proper page load waits as shown below:
Option Explicit
Public Sub WorldCheck()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate2 "https://www.world-check.com/frontend/login/"
While .busy Or .readystate <> 4: DoEvents: Wend
With .document
.querySelector("#username").Value = "ABCDEF" ' equivalent to .getElementbyID("username").Value = "ABCDEF"
.querySelector("#password").Value = "GHIJKL" '.getElementbyID("password").Value = "GHIJKL"
.querySelector(".button").Click
End With
While .busy Or .readystate <> 4: DoEvents: Wend
Stop '<== delete me later
.Quit
End With
End Sub
I am relatively new at VBA (I know the basics, but not much else) and I am trying to get some code set up that will fill out online forms for me, but when I run my code I get a 438 error:
object doesnt support this property or method
when it gets to
ie.document.getElementById ("q")
I have added the HTML object library and Microsoft internet controls to my references. I have looked at tons of online forums. I have even copied and pasted an entire script directly into VBA. Nothing will make it use getElementById(). Here is my code:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate ("https://www.google.com/")
Set searchbx = ie.document.getElementById("q")
searchbx.Value = "Howdy!"
End Sub
What should happen is that it should open InternetExplorer, go to Google, and populate the search bar with "Howdy!".
Instead, It only opens google, then I get the error message and it stops running.
Because "q" is not an ID, it's a Name.
You have to use GetElementsByName and select the 1st Element
try this:
Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.Navigate ("https://www.google.com/")
Do Until ie.ReadyState >= 4
DoEvents
Loop
Set searchbx = ie.document.getElementsByName("q")(0)
searchbx.Value = "Howdy!"
End Sub
I have also added a waiting Event, in case it takes time to load Google.
Result:
A more concise and efficient way is to use querySelector to return first match for the name attribute
Option Explicit
Public Sub internetstuff()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate2 "https://www.google.com/"
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector("[name=q]").Value = "Howdy"
Stop '<delete me later
.Quit
End With
End Sub
I have a short excel macro that is designed to:
1) Open Internet Explorer and navigate to "http://www.puco.ohio.gov/pucogis/address/search.cfm"
2) Fill out a form on that site with data from the excel workbook
3) Click a button to submit the form
4) Scrape some innertext from the website and place it in a cell in the workbook
5) Close Internet Explorer
I can not get step 3 to work. That is, I can not get the click/submit function to work with this website. When the button is clicked the website populates with information specific to the information entered in the form. Everything else in the code is working. I have searched for an answer and tried the submit verses click approach with no luck.
Thanks for you help.
Code below:
Private Sub SiteData()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
Set HTMLButton = ie.Document.getElementsByTagName("input")(1)
HTMLButton.Click
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D50") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
Try this solution, which I found from this answer to a similar question. That answer was not accepted, but I have tested this with your code and seems to be working.
Private Sub SiteData()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
Call ie.Document.parentWindow.execScript("codeAddress()")
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D50") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
If you don't know or can't reasonably anticipate the function call codeAddress(), then you can try something like this to derive it from the button's onclick property:
Dim fn$
fn = HTMLButton.onclick
fn = Mid(fn, InStr(fn, "{"))
fn = Trim(Replace(Replace(Replace(fn, "{", vbNullString), "}", vbNullString), vbLf, vbNullString))
Call ie.Document.parentWindow.execScript(fn)
You can call the JavaScript directly. try this it will work
Instead of:
Set HTMLButton = ie.Document.getElementsByTagName("input")(2)
HTMLButton.Click
use
ie.Document.parentWindow.execScript code:="codeAddress()"
note that IE may prompt you to confirm every run so you may need to
stop showing this message for smooth operation
Private Sub CommandButton1_Click()
Dim ie As Object
Dim utility As Variant
Dim HTMLButton
Set ie = CreateObject("InternetExplorer.Application")
ie.Navigate "http://www.puco.ohio.gov/pucogis/address/search.cfm"
ie.Visible = True
While ie.Busy
DoEvents
Wend
ie.Document.all("address").Value = ThisWorkbook.Sheets("Site Info").Range("D14")
While ie.Busy
DoEvents
Wend
ie.Document.parentWindow.execScript code:="codeAddress()"
'Set HTMLButton = ie.Document.getElementsByTagName("input")(2)
'HTMLButton.Click
While ie.Busy
DoEvents
Wend
Set utility = ie.Document.getElementById("supName")
ThisWorkbook.Sheets("Site Info").Range("D16") = utility.innerText
ie.Quit
Set ie = Nothing
End Sub
thanks also to this article helped me to solve your problem
How to find and call javascript method from vba
I am currently using the following sub to close my IE after automating:
Public Sub CloseIE()
Dim Shell As Object
Dim IE As Object
Set Shell = CreateObject("Shell.Application")
For Each IE In Shell.Windows
If TypeName(IE.Document) = "HTMLDocument" Then
IE.Quit
End If
Next
End Sub
This works great but the problem occurs when I try to run the IE code again, I get the following:
Run-time error '-2147023706 (800704a6)':
Automation error
A system shutdown has already been scheduled.
After 20 secs, I can re-run the code. Is there any way of "force closing" IE so that I can run the code again directly after without the error?
EDIT:
Heres the code that initiates IE:
Sub testSub()
Dim IE As Object, Doc As Object, strCode As String
Set IE = CreateObject("internetexplorer.application")
IE.Visible = True
IE.Navigate "website name"
Do While IE.ReadyState <> 4: DoEvents: Loop
Set Doc = CreateObject("htmlfile")
Set Doc = IE.Document
CODE HERE
CloseIE
End Sub
I used a simple procedure to create an InternetExplorer object, navigate, then close it using IE.Quit and Set IE = Nothing.
Apparently after closing, the internet was still running in the background for about another minute (I noticed it using task manager, background processes). I went into Internet Explorer options and unclicked “delete browsing history on exit”.
That fixed my issue. I am not sure why IE takes that long to clear the history and I’m not sure if there is a workaround but it is the only viable option for me thus far.
Modify your main sub to Quit the IE application, and then set the object variable to Nothing:
Sub testSub()
Dim IE As Object, Doc As Object, strCode As String
Set IE = CreateObject("internetexplorer.application")
IE.Visible = True
IE.Navigate "website name"
Do While IE.ReadyState <> 4: DoEvents: Loop
Set Doc = CreateObject("htmlfile")
Set Doc = IE.Document
'### CODE HERE
IE.Quit
Set IE = Nothing
End Sub
You will not need the CloseIE procedure anymore.