As new coding maybe this question may be silly, but when I try to run this code, it keeps showing an Error 91 highlighting this line
'Get the number from the specified element on the page
number = html.getElementsByClassName("arial_14 redFont").Item(0).innerText
As I already identified number at the beginning, I have no idea why it keeps showing the variable wasn't defined.
Sub Get_Housing_Starts_From_Investing()
Dim request As Object
Dim response As String
Dim html As New HTMLDocument
Dim website As String
Dim number As Variant
' Source: https://www.youtube.com/watch?v=IOzHacoP-u4
' Remember to activate References: Microsoft Office 16.0 Object Library, Microsoft HTML Object Library, Microsoft XML, v6.0, Visual Basic For Applications, Microsoft Excel 16.0 Object Library, OLE Automation
' Website to go to
website = "https://www.investing.com/economic-calendar/housing-starts-151"
' Create the object that will make the website request
Set request = CreateObject("Msxml2.ServerXMLHTTP.6.0")
' Where to go and how to go there - don´t need to change it
request.Open "GET", website, False
' Get fresh data
request.setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
' Send the request for the webpage
request.send
' Get the webpage response data into a variable
response = StrConv(request.responseBody, vbUnicode)
' Put the webpage into an html object to make data references easier
html.body.innerHTML = response
'Get the number from the specified element on the page
number = html.getElementsByClassName("arial_14 redFont").Item(0).innerText
' Output the number into a message box
MsgBox number
End Sub
Related
I am using the VBA code to extract information from a website into excel cells, and the numerical information is fine but I have a problem with text strings. I am mostly extracting information from Georgian websites, and the texts with the Georgian language are not properly displayed in excel, so I was wondering if there is any chance (code or something else) I could convert these symbols into proper language.
Sub GetData()
Dim request As Object
Dim response As String
Dim html As New HTMLDocument
Dim website As String
Dim price As Variant
Dim address As Variant
Dim x As Integer
Dim y As Range
x = 1
Do Until x = 9
Set y = Worksheets(1).Range("A21:A200"). _
Find(x, LookIn:=xlValues, lookat:=xlWhole)
website = "https://www.myhome.ge/ka/pr/11247371/iyideba-Zveli-ashenebuli-bina-veraze-T.-WoveliZis-qucha"
' Create the object that will make the webpage request.
Set request = CreateObject("MSXML2.XMLHTTP")
' Where to go and how to go there.
request.Open "GET", website, False
' Get fresh data.
request.setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
' Send the request for the webpage.
request.send
' Get the webpage response data into a variable.
response = StrConv(request.responseBody, vbUnicode)
' Put the webpage into an html object.
html.body.innerHTML = response
' Get info from the specified element on the page.
address = html.getElementsByClassName("address").Item(0).innerText
price = html.getElementsByClassName("d-block convertable").Item(0).innerText
y.Offset(0, 1).Value = address
y.Offset(0, 5).Value = price
x = x + 1
Loop
End Sub
This is the code that I took from a youtube video (https://www.youtube.com/watch?v=IOzHacoP-u4) and slightly modified, and it works, I just have a problem with how excel displays the characters in text strings.
For your issue in the question
Remove this line response = StrConv(request.responseBody, vbUnicode) as it's not required.
Change html.body.innerHTML = response to html.body.innerHTML = request.responseText.
For your issue in comment
To retrieve the ID of the property, it can be retrieved from the class id-container, you will need to perform some string processing though to remove the extract :
propertyID = Trim$(Replace(html.getElementsByClassName("id-container")(0).innerText, ":", vbNullString))
Note: You should try to avoid declaring variable as Variant. innerText property returns a String datatype so you should declare address and price as String.
I have been using the following Excel VBA macro to bring back data from a website. It worked fine until a few days ago when the website stopped supporting IE. Of course the macro just fails now as there is no data on the webpage to bring back to Excel, Is there a way to have the "Get method" (MSXML2.XMLHTTP)
here is my Code
Public Sub GGGG()
Dim MSX As Object
Dim HTML As HTMLDocument
Dim URL As String
Dim UrlResponse As String
Dim N As Long
Dim sht1, sht2 As Worksheet
' On Error Resume Next
Set MSX = CreateObject("MSXML2.XMLHTTP")
Set HTML = New HTMLDocument
URL = "https://www.justdial.com/Agra/Yogi-General-Store-Opp-Eclave-Satiudum-Sadar-Bazaar/0562P5612-5612-120207212812-H5I2_BZDET"
With MSX
.Open "GET", URL, False
.setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
.send
UrlResponse = StrConv(.responseBody, vbUnicode)
End With
ActiveCell.Offset(0, 1) = UrlResponse
End Sub
I get response like
Error
An error occurred while processing your request.
Reference #97.ec8a2c31.1621136928.281f3ca8
Please anyone can support me how to get data when IE dose not support
I am not an expert in coding
Okay, try this to get the title and votes from that site using vba in combination with selenium.
Sub FetchInfo()
Dim driver As Object, oTitle As Object
Dim oVotes As Object
Set driver = CreateObject("Selenium.ChromeDriver")
driver.get "https://www.justdial.com/Agra/Yogi-General-Store-Opp-Eclave-Satiudum-Sadar-Bazaar/0562P5612-5612-120207212812-H5I2_BZDET"
Set oTitle = driver.FindElementByCss("span.item > span", Raise:=False, timeout:=10000)
Set oVotes = driver.FindElementByCss("span.rtngsval > span.votes", Raise:=False, timeout:=10000)
Debug.Print oTitle.Text, oVotes.Text
End Sub
When the webpage no longer support IE in future, you can try out web scrape using Google Chrome with new add-in installed, please see following link for the add-in installation adn how to write in VBA. However, it is in my opinion the most simple way to perform your work is to use Uipath free community version, it work for all type of web-browser.
VBA guideline:
https://www.wiseowl.co.uk/vba-macros/videos/vba-scrape-websites/web-scraping-selenium-chrome/
VBA library installation for Selenium:
https://code.google.com/archive/p/selenium-vba/downloads
You probably need to set the Feature Browser Emulation to zero as detailed by Daniel here:
Everything You Never Wanted to Know About the Access WebBrowser Control
That said, your URL fails even when opened in Edge Chromium, so the site may suffer from a general failure.
I am trying to bring into vba the data displayed under the -Dividend Summary- title of this webpage:
https://seekingalpha.com/symbol/ABBV/dividends/scorecard
By running this line of code in the Google Chrome console I managed to get the info, so I am trying to replicate this in VBA.
document.querySelectorAll("div [data-test-id='dynamic-tooltips-area']")[1].innerText
The VBA code I have written is this:
Public Sub Stackoverflow_Question()
Dim sResponse As String, i As Long, Html As New HTMLDocument
Dim oSelectors As MSHTML.IHTMLDOMChildrenCollection 'Object
'Get response from webpage
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://seekingalpha.com/symbol/ABBV/dividends/scorecard", False
.send
sResponse = .responseText
End With
'Write and read HTML
With Html
.body.innerHTML = sResponse
Set oSelectors = .querySelectorAll("div [data-test-id='dynamic-tooltips-area']")
End With
'Print obtained data -this produces no result-
'I loop only to be sure that I don't request the wrong index
For i = 0 To oSelectors.Length - 1
Debug.Print oSelectors(i).innerText
Next i
'Auxiliary: Create txt with response text
Dim FilePath As String
Dim TextFile As Integer
FilePath = Application.ActiveWorkbook.Path & "\HTML_ResponseText.txt"
TextFile = FreeFile
Open FilePath For Output As TextFile
Print #TextFile, sResponse
Close TextFile
End Sub
Not only do I get no result from the debug.print, but a simple search in the responseText string shows that contents from the webpage such as the header "Div Yield (FWD)" or the values (4.99% as of today) are not there.
Why is the .ResponseText not working as I expect?
Is there and alternative way to retrieve the webpage content as text?
Thank you in advance
As pointed out by Tim Williams, loading a webpage usually takes more than one request.
In Google Chrome, pressing F12 and going to the "Network" tab shows all of them, as well as their respective Request URL's.
Looking for the wanted one looks like a trial and error task, but it all seems to be there.
In my case, the URL I was interested in is:
https://seekingalpha.com/api/v3/symbol_data?fields[]=divYieldFwd&fields[]=divRate&fields[]=payoutRatio&fields[]=divGrowRate5&fields[]=dividendGrowth&fields[]=divDistribution&fields[]=dividends&slugs=ABBV
Any one of three highlighted part are the value i want to print. I am trying below code
Sub JJ()
Dim IE As New SHDocVw.InternetExplorer
Dim hdoc As MSHTML.HTMLDocument
Dim ha As String
IE.Visible = True
IE.navigate "https://www.nseindia.com/get-quotes/equity?symbol=DIVISLAB"
Do While IE.readyState <> READYSTATE_COMPLETE
Loop
Set hdoc = IE.document
ha = hdoc.getElementById("preOpenFp").innerText
Debug.Print ha
End Sub
But the output is nothing pls help.
The website you're trying to scrape offers a very convenient way to do it. All you need to do is send an HTTP request and get the corresponding JSON response which looks like so:
If you take a look at the network traffic in your browser's developer tools, you'll see the requests that are being sent to the server when the page is being loaded. Among these requests you'll find the following one:
To send this request and get the info you need, you have to do the following:
Option Explicit
Sub nse()
Dim req As New MSXML2.XMLHTTP60
Dim url As String
Dim json As Object
url = "https://www.nseindia.com/api/quote-equity?symbol=DIVISLAB"
With req
.Open "GET", url, False
.send
Set json = JsonConverter.ParseJson(.responseText)
End With
Debug.Print json("preOpenMarket")("IEP")
End Sub
This will print the value of IEP to your immediate window (in this case 2390). You can modify the code to best fit your needs.
To parse a JSON string you will need to add this to your project. Follow the installation instructions in the link and you should be set to go.
You will also need to add the following references to your project (VBE>Tools>References):
Microsoft XML version 6.0
Microsoft Scripting Runtime
I have a HTML web page at work that I want to query data from tables into excel 2007. This web page requires I sign on with a password. I sign in with my normal IE7 browser, then I go to DATA -> connections -> my connections and edit the query. This reads the IE7 cookie cache and I re-POST the data to connect to the server's security by clicking "retry" when it says "the web query returned no data". After I do this, the data imports fine.
I can do this just fine and it only needs to be done once a day. Other users of my application find this difficult which leads to my question:
Is there a way to automatically POST this data back with VB? I'm thinking maybe I should use the cookie property of the IE.Document.cookie?
I'm calling the following login script, before I continue with the web query (set reference to XML library). Look around to find some instructions how you can find your POST parameters.
Sub XMLHttpLogin()
Dim i As Integer
Dim sExpr As String
Dim sPar As String, sURL as String
Dim sResp As String
Dim XMLHttp As MSXML2.XMLHTTP60
Set XMLHttp = New MSXML2.XMLHTTP60
sPar = "name=user1&pass=pass1&form_id=form1" 'The parameters to send.
sURL = "http://www.stackoverflow.com"
With XMLHttp
.Open "POST", sURL, True 'Needs asynchronous connection
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.send (sPar)
i = 0 'wait until data has been downloaded
Do While i = 0
If .readyState = 4 Then
If .Status = 200 Then Exit Do
End If
DoEvents
Loop
sResp = .responseText 'contains source code of website
sExpr = "not-logged-in" 'look for this string in source code
If InStr(1, sResp, sExpr, vbTextCompare) Then
MsgBox "Not logged in. Error in XMLHttpLogin"
End If
End With
End Sub