I have been using code from this youtube tutorial (https://www.youtube.com/watch?v=7sZRcaaAVbg) to have VBA in Excel to pull web data into my sheet. What I don't understand is that it worked perfectly at first for a a few weeks, but, without touching any part of the code, about a month or two ago I ran into an error that reads:
"Run-time error '91':
Object variable or With block variable not set"
Here is the part of my code that the trips the error, where 'address' is the web address I want to pull data from:
ActiveSheet.Cells(1, 1) = "Opening Web Page"
Dim IE As New InternetExplorer
IE.Visible = false
Dim address As String
address = ActiveSheet.Cells(4, 3)
IE.navigate address
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim askingPrice As Variant
ActiveSheet.Cells(1, 1) = "Getting Info From Web Page"
askingPrice = Trim(Doc.getElementsByClassName("ds-value")(0).innerText)
Can anyone lead me to correctly setting the variable 'Doc' so that I can use 'getElementsByClassName'? Or show me if there is something else I need to do?
Thank you
Related
I'm coding a macro that will load a web page, login in it and get the datas from a table.
Thanks to several topics here I've done that:
Sub sail2()
Dim ie As Object
Set ie = New(InternetExplorer.Application")
ie.Visible = True
ie.Navigate "http://pfrwtekref/websail/"
With ie
Dim oLogin As Object, oPassword As Object
Set oLogin = .document.getElementById("txtPwd")(0)
Set oPassword = .document.getElementById("txtPwd")(0)
oLogin.Value = test
oPassword.Value = test
.document.forms(0).submit
End With
End Sub
But I have two type of errors "automation error" if I launch the macro and "error 462 - server not available"
Can anyone help me ?
Thanks for reading and have a good day !
The first issue seems to be that you have a mismatched comma on the third line; however, seeing as you're are complaining about an automation error I think that may be just a typo on this site.
I can't see your internal website so I'm just guessing but I suspect the next issue is that there is no element with ID "txtPwd". You can check to confirm by pressing Ctrl-Shift-C and then selecting the username and password entry boxes.
Finally, depending on how the site is set up your .document.forms(0).submit may not work. You may need to find the ID for the submit button class submit. Below is a function I created a while back for such a task:
Function logIn(userName As String, password As String) As Boolean
'This routine logs into the grade book using given credentials
Dim ie As New InternetExplorer
Dim doc As HTMLDocument
On Error GoTo loginFail
ie.Navigate "[website here]"
'ie.Visible = True
Do While ie.ReadyState <> READYSTATE_COMPLETE Or ie.Busy: DoEvents: Loop 'Wait server to respond
Set doc = ie.Document
doc.getElementsByName("u_name").Item(0).Value = userName 'These may be different for you
doc.getElementsByName("u_pass").Item(0).Value = password
doc.getElementsByClassName("btn").Item(0).Click
Do While ie.ReadyState <> READYSTATE_COMPLETE Or ie.Busy: DoEvents: Loop 'Wait server to respond
Set doc = ie.Document
'Add a check to confirm you aren't on the same page
ie.Quit
Set ie = Nothing
LogIn = True
Exit Function
loginFail:
MsgBox "There was an issue logging in. Please try again."
logIntoGradeBook = False
End Function
Note that the site I was dealing with was set up poorly and so I needed to switch to GetElementsByName and GetElementsByClassName to get access to what I needed. You may be fine with IDs.
Why don't you try to use the Excel PowerQuery to get your data from the tables you need? When your desired columns are done, you just click in close and load, select data model e now you can use a macro to perform a PivotTable whenever you load the document or just use a PivotTable and the data will be refreshed when you use the refresh data in data ribbon.
I am trying to enter info into an Internet Explorer 11 website form. The website/form was designed a long time ago (around 15-20 years). The website can only be accessed through Internet Explorer.
I cannot share the website/source code as it is internal to my company.
I have browsed online for a solution, but none worked (I tried many different versions).
I am looking to login, then go through a few pages of entering information, while clicking next/submit at each stage. I am failing after I login.
I have the following references on Excel:
Microsoft Internet Controls, Microsoft HTML Object Library, Microsoft XML, v6.0
I am following the wise owl tutorial https://www.youtube.com/watch?v=dShR33CdlY8. Skip to about 17 mins in to see where I got the code.
I got an error message at the line htmlinput.Value = "excel".
The error message was
object variable or with block variable not set - Run Time error '91'
Sub navigate_website()
Dim ie As New SHDocVw.InternetExplorer
Dim htmldoc As MSHTML.HTMLDocument
Dim htmlinput As MSHTML.IHTMLElement
ie.Visible = True
ie.navigate Sheet1.Range("C2").Text
Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
'enter in userid
ie.document.forms("formsamplename").elements("usedid").Value = ThisWorkbook.Sheets("sheet1").Range("B6")
'enter in password
ie.document.forms("formsamplename").elements("userpassword").Value = ThisWorkbook.Sheets("sheet1").Range("B7")
'click the login button
ie.document.forms("formsamplename").elements("cmdSubmit").Click
Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
' ----- I tried the below code as an alternative but it didn't work -----
'ie.document.forms("formsamplename").elements("usernumber").Value = ThisWorkbook.Sheets("sheet1").Range("B6")
Set htmldoc = ie.document
Set htmlinput = htmldoc.getElementById("usernumber")
htmlinput.Value = "excel" **'error occurs here**
' ----- I also tried the below code, but it didn't work -----
'htmldoc.forms("formsamplename").elements("usernumber").Value = "test"
Set ie = Nothing
Set htmldoc = Nothing
Set htmlinput = Nothing
End Sub
I haven't really tried working on this before, so I have no idea what I am doing at the moment. I have limited knowledge of html so not sure whether I am doing right. Basically what I aim to do is opening the Internet explorer by macro, changing some elements based id and click submit button on the website to show the data. Then I need keep working on the next step.
As you can see from the code I was trying to engage with the widget on IE by id number from html codes.
Sub Automate_IE_Enter_Data()
'This will load a webpage in IE
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objbutton As Object
'Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
'Set IE.Visible = True to make IE visible, or False for IE to run in the background
IE.Visible = True
'Define URL
URL = "http://cfpsg1/plant/Reports/ScrapReport.aspx"
'Navigate to URL
IE.Navigate URL
' Statusbar let's user know website is loading
Application.StatusBar = URL & " is loading. Please wait..."
' Wait while IE loading...
'IE ReadyState = 4 signifies the webpage has loaded (the first loop is set to avoid inadvertantly skipping over the second loop)
Do While IE.ReadyState = 4: DoEvents: Loop
'Webpage Loaded
Application.StatusBar = URL & " Loaded"
IE.Document.getelementbyid("1stGroupBy").Value = "3"
'Find & Fill Out Input Box
IE.Document.getelementbyid("PageContent_uxStartDate").Value = "06/21/2019"
IE.Document.getelementbyid("PageContent_uxEndDate").Value = "06/21/2019"
Set objbutton = IE.Document.getelementbyid("PageContent_btnQuery")
objbutton.Focus
objbutton.Click
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
End Sub
First thing first the webpage popped up but nothing changed of widgets besides an
error message "method 'Document' of object 'IWebBrowser 2' failed"
show on IE.Document.getelementbyid("1stGroupBy").Value = "3" row.
You are trying to interact with a dropdown so you want syntax such as
IE.Document.querySelector("[value='3']").Selected = True
You could also use
IE.Document.querySelector("#1stGroupBy").SelectedIndex = 2 'change to appropriate index
Error "method 'Document' of object 'IWebBrowser 2' failed" may be due to Integrity level
You can try below mentioned code by changing Integrity level as Medium
Dim IE As InternetExplorer
Set IE = New InternetExplorerMedium
IE.Visible = True
URL = "http://cfpsg1/plant/Reports/ScrapReport.aspx"
IE.Navigate URL
Do While IE.ReadyState <> READYSTATE_COMPLETE
DoEvents
Loop
and try other line of codes to avoid this issue also please refer Here
Please add ref Microsoft Internet Controls and Microsoft HTML Object Library based on your req
I get the error mentioned in the title every time I try to run my VBA code. Error is appearing on this line:
Set Button_Top_Result = IE.document.getElementById_
("javascript:gx.evt.execEvt('EUSRACNAM.CLICK.0001',this);").href
Also did some research online and find this alternative code and tried it as well.
For Each ele In IE.document.getElementsByTagName("a")
If InStr(ele.href, "javascript:gx.evt.execEvt('EUSRACNAM.CLICK.0001',this);") > 0 Then IE.navigate_
ele.href: Exit For
Next
This line worked once, and now no longer works.
VBA Novice here, so this might be a simple solution but I am stumped - working on navigating through the company website and wrote line to click on hyperlink to navigate me to this next page. It worked the first time, and then won't work and spits out the above error. Really stumped here, already did some digging online and can't figure out where my error is.
Here is the html part of the web page
https://i.stack.imgur.com/kgJOo.png
'''Option Explicit
Sub GetMasterDetailKeyedDataTest2()
'The goal of this macro is to quickly and conveniently update BNA Depreciation by automating the steps necessary to complete the process.
'Dimensions identfy things I will define later to use.
Dim name As String
Dim IE As InternetExplorerMedium
Dim store As String
Dim client As String
Dim URL As String
Dim username
Dim Password
Dim Button_Next
Dim Button_Login
Dim Button_OnDemandReporting
Dim Search_Bar
Dim Button_Mag_Glass
Dim Button_Top_Result As Action
'Line below bypasses login if user is already logged in
'On Error Resume Next
store = Workbooks("Learning").Sheets("Sheet1").Range("A2")
client = Workbooks("Learning").Sheets("Sheet1").Range("B2")
'Abbreviates Internet Explorer. Note I need the correct references enabled in tools in order to run a web query using this name.
Set IE = New InternetExplorerMedium
'Define URL
URL = "Company website"
'make IE browser visible (False would allow IE to run in the background)
'Once program is working I will want to turn this off so that the user doesn't see the webbrowser.
IE.Visible = True
'Navigate to Login page
IE.navigate URL
'This loop prevents Excel from continuing the code
Do While IE.Busy Or IE.readyState <> 4
DoEvents
Loop
'These next four steps navigate through the login
Set username = IE.document.getElementById("username") 'id of the username control (HTML Control)
username.Value = "username"
Set Button_Next = IE.document.getElementById("next") 'id of the button control (HTML Control)
Button_Next.Click
Set Password = IE.document.getElementById("password") 'id of the password control (HTML Control)
Password.Value = "password"
Set Button_Login = IE.document.getElementById("submit") 'id of the button control (HTML Control)
Button_Login.Click
Do While IE.Busy Or IE.readyState <> 4
DoEvents
Loop
'Connects to OnDemand Reporting
Set Button_OnDemandReporting = IE.document.getElementById("IMAGE1_0004")
Button_OnDemandReporting.Click
Do While IE.Busy Or IE.readyState <> 4
DoEvents
Loop
Set Search_Bar = IE.document.getElementById("vNAME")
Search_Bar.Value = Workbooks("Learning").Sheets("Sheet1").Range("B2")
Set Button_Mag_Glass = IE.document.getElementById("IMAGE1")
Button_Mag_Glass.Click
Set Button_Top_Result = IE.document.getElementById("javascript:gx.evt.execEvt('EUSRACNAM.CLICK.0001',this);").href
Button_Top_Result.Click
End Sub
'''
I expect the last step to navigate me to the next part of the web page, but instead I keep getting the above error.
I am want to scrap from amazon some fields.
Atm I am using a link and my vba script returns me name and price.
For example:
I put the link into column A and get the other fields in the respective columns, f.ex.: http://www.amazon.com/GMC-Denali-Black-22-5-Inch-Medium/dp/B00FNVBS5C/ref=sr_1_1?s=outdoor-recreation&ie=UTF8&qid=1436768082&sr=1-1&keywords=bicycle
However, I would also like to have the product description.
Here is my current code:
Sub ScrapeAmz()
Dim Ie As New InternetExplorer
Dim WebURL
Dim Docx As HTMLDocument
Dim productDesc
Dim productTitle
Dim price
Dim RcdNum
Ie.Visible = False
For RcdNum = 2 To ThisWorkbook.Worksheets(1).Range("A65536").End(xlUp).Row
WebURL = ThisWorkbook.Worksheets(1).Range("A" & RcdNum)
Ie.Navigate2 WebURL
Do Until Ie.ReadyState = READYSTATE_COMPLETE
DoEvents
Loop
Set Docx = Ie.Document
productTitle = Docx.getElementById("productTitle").innerText
'productDesc = Docx.getElementsByClassName("productDescriptionWrapper")(0).innerText
price = Docx.getElementById("priceblock_ourprice").innerText
ThisWorkbook.Worksheets(1).Range("B" & RcdNum) = productTitle
'ThisWorkbook.Worksheets(1).Range("C" & RcdNum) = productDesc
ThisWorkbook.Worksheets(1).Range("D" & RcdNum) = price
Next
End Sub
I am trying to get the product description by using productDesc = Docx.getElementsByClassName("productDescriptionWrapper")(0).innerText.
However, I get an error.
Object variable or with block variable not set.
Any suggestion why my statement does not work?
I appreciate your replies!
I'm pretty sure your problem is being caused by attempting to access the document before it's completely loaded. You're just checking ie.ReadyState.
This is my understanding of the timeline for loading a page with an IE control.
Browser connects to page: ie.ReadyState = READYSTATE_COMPLETE. At this point, you can access ie.document without causing an error, but the document has only started loading.
Document fully loaded: ie.document.readyState = "complete"
(note that frames may still be loading and AJAX processing may still be occurring.)
So you really need to check for two events.
Do
If ie.ReadyState = READYSTATE_COMPLETE Then
If ie.document.readyState = "complete" Then Exit Do
End If
Application.Wait DateAdd("s", 1, Now)
Loop
edit: after actually looking at the page you're trying to scrape, it looks like the reason it's failing is because the content you're trying to get at is in an iframe. You need to go through the iframe before you can get to the content.
ie.document.window.frames("product-description-iframe").contentWindow.document.getElementsByClassName("productDescriptionWrapper").innerText