Retrieve the URL from the Internet Explorer - excel

i am struggling on the following: I try to open an URL(Link), become redirected to a new URL and retrieve this new URL into a Cell in my excel worksheet.
I have written the following code but it is not retrieving the new URL and is not quitting the Internet Explorer at the end:
Sub Get_URL()
Dim ISIN As String
Dim Link As String
Dim IE As Object
ISIN = Range("A1").Value
Link = "https://www.finanzen.net/suchergebnis.asp?_search=" & ISIN
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate Link
End With
Range("A2") = IE.LocationURL
Set IE = Nothing
IE.Quit
End Sub
The ISIN in A1 is KYG875721634.
I would be very glad if some of you guys could find the problem. Thank you very much!
Greetings, Robin

This works. Read the comments please:
Sub Get_URL()
Dim ISIN As String
Dim Link As String
Dim IE As Object
'ISIN = Range("A1").Value
ISIN = "KYG875721634"
Link = "https://www.finanzen.net/suchergebnis.asp?_search=" & ISIN
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate Link
End With
'You must wait to load the page
Do While IE.readystate <> 4: DoEvents: Loop
'Range("A2") = IE.LocationURL
MsgBox IE.LocationURL
'First quit the IE
'because this step needs the reference ;-)
IE.Quit
Set IE = Nothing
End Sub

There're two issues in your code:
You need to add a wait to load page.
Do Until IE.readyState = 4
DoEvents
Loop
First quit IE. If you first set IE to Nothing then there will be no reference to IE when quit.
IE.Quit
Set IE = Nothing
The final code is like this which can work well:
Sub Get_URL()
Dim ISIN As String
Dim Link As String
Dim IE As Object
ISIN = Range("A1").Value
Link = "https://www.finanzen.net/suchergebnis.asp?_search=" & ISIN
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate Link
Do Until IE.readyState = 4
DoEvents
Loop
Range("A2") = IE.LocationURL
IE.Quit
Set IE = Nothing
End Sub
Result:

Related

Scrape data from multiple url in excel and save it in different row

getting url form sheet1 range a1:a1000 and after exporting saving data too sheet2
ie.navigate Workbooks("1000").Worksheets("sheet1").Range("A6").Value
Set ht = ie.document
Workbooks("1000").Worksheets("sheet2").Range("C6").Value = ht.getElementById("formNo").Value
Workbooks("1000").Worksheets("sheet2").Range("D6").Value = ht.getElementById("fullName").Value
Workbooks("1000").Worksheets("sheet2").Range("E6").Value = ht.getElementById("idNo").Value
I do not what exactly is that you want. But I am providing a code snippet with which you can achieve your task by simply modifying it.
Sub Scrape()
Dim url As String
Dim IE As New InternetExplorer
Dim doc As HTMLDocument
Set IE = New InternetExplorer
IE.Visible = False
For Each cell In Range("A1:A1000")
url = cell.Value
IE.navigate url
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Set doc = IE.document
Sheet2.Range("A1").Value = doc.getElementsByTagName("span")(0).innerText
Sheet2.Range("A2").Value = doc.getElementsByTagName("span")(1).innerText
Sheet2.Range("A3").Value = doc.getElementsByTagName("span")(2).innerText
Sheet2.Range("A4").Value = doc.getElementsByTagName("span")(3).innerText
Next cell
IE.Quit
End Sub

Retrieving a URL from Internet Explorer with VBA

I have written some VBA code in Excel to retrieve the latitude and longitude from a Google Maps URL and paste it into a cell in my worksheet. My problem is in retrieving the URL from internet explorer. Below I have two examples of my code, one macro returns an about:blank as though the object doesn't have the LocationURL property, and the other example seems like it is saving all of my previous searches, so it cycles through all of my previous searches and pastes the very first searches' URL. Example 2 uses a shell suggestion that I found online to reassign the properties to the oIE object. I can get both to slightly work, but neither will do exactly what I need from the macro.
Cell(8,8) is a hyperlink to google maps where I'm searching an address, and Cell(8,9) is where I want to paste the URL after google maps has redirected and has the latitude and longitude in the URL.
Example 1:
Sub CommandButton1_Click()
Dim ie As Object
Dim Doc As HTMLDocument
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.Navigate "http://www.google.com/maps?q=" & Range("I7").Value
Do
DoEvents
Loop Until ie.ReadyState = 4
Set Doc = ie.Document
Cells(8, 9).Value = ie.LocationName
End Sub
Example 2:
Sub Macro()
Dim oIE, oShell, objShellWindows, strPath, X
strPath = Cells(8, 8)
Set oIE = CreateObject("InternetExplorer.Application")
'This is to resolve oIE.navigate "about:blank" issue
oIE.Top = 0
oIE.Left = 0
oIE.Width = 500
oIE.Height = 500
oIE.Navigate strPath
Do While oIE.Busy And oIE.ReadyState < 2
DoEvents
Loop
'Reassigning oIE.LocationName & vbCrLf & oIE.LocationURL values after redirect in IE
Set oShell = CreateObject("WScript.Shell")
Set objShellWindows = CreateObject("Shell.Application").Windows
For X = objShellWindows.Count - 1 To 0 Step -1
Set oIE = objShellWindows.Item(X)
If Not oIE Is Nothing Then
If StrComp(oIE.LocationURL, strPath, 1) = 0 Then
Do While oIE.Busy And oIE.ReadyState < 2
DoEvents
Loop
oIE.Visible = 2
Exit For
End If
End If
Cells(8, 9).Value = oIE.LocationURL
Set oIE = Nothing
Next
Set objShellWindows = Nothing
Set oIE = Nothing
End Sub
Thanks,
Andrew
Is this as simple as looping until the document.URL changes? In my timed loop I wait for the string safe=vss in the original page load to disappear.
Option Explicit
Public Sub GetNewURL()
Dim IE As New InternetExplorer, newURL As String, t As Date
Const MAX_WAIT_SEC As Long = 5
With IE
.Visible = True
.navigate2 "http://www.google.com/maps?q=" & "glasgow" '<==Range("I7").Value
While .Busy Or .readyState < 4: DoEvents: Wend
t = Timer
Do
DoEvents
newURL = .document.URL
If Timer - t > MAX_WAIT_SEC Then Exit Do
Loop While InStr(newURL, "safe=vss") > 0
Debug.Print newURL
End With
End Sub

how to select a specific dropdown element in webpage by excel vba

Currently, I am working on a webpage to extract some data related to my work. Here I am facing a problem while selecting the drop-down menu in the production list. I want to select the "All" option in the drop-down list. By default, it takes only 25 values in the production table.
Here is my code
Sub Production_table()
Dim objIE As InternetExplorer
Dim ele As Object
Dim ele2 As Object
Dim ele3 As Object
Dim s As String
Dim i As Integer
i = 2
Application.ScreenUpdating = False
Application.DisplayAlerts = False
Do While Range("A" & i).Value <> ""
Set objIE = New InternetExplorer
objIE.Visible = True
source = "https://secure.conservation.ca.gov/WellSearch/Details?api=01120716&District=&County=&Field=&Operator=&Lease=&APINum=&address=&ActiveWell=false&ActiveOp=false&Location=&sec=&twn=&rge=&bm=&PgStart=0&PgLength=10&SortCol=6&SortDir=asc&Command=Search
objIE.navigate source"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Sheets.Add
ActiveSheet.Name = "Data"
On Error Resume Next
Set ele2 = objIE.document.getElementById("productionTable_length"). _
getElementsByTagName("label")
For Each ele3 In ele2
If ele3.Value = -1 Then
ele3.Focus
ele3.Selected = True
ele3.Click
Exit For
End If
Next
webpage link-- https://secure.conservation.ca.gov/WellSearch/Details?api=01120716&District=&County=&Field=&Operator=&Lease=&APINum=&address=&ActiveWell=false&ActiveOp=false&Location=&sec=&twn=&rge=&bm=&PgStart=0&PgLength=10&SortCol=6&SortDir=asc&Command=Search
Please help me on the same.
I always recommend using Chrome Browser for such cases as it let's you inspect the element in a userfriendly way.
Try this
Sub Sample()
Dim objIE As Object, ele As Object, opt As Object
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True
Source = "https://secure.conservation.ca.gov/WellSearch/Details?api=01120716&District=&County=&Field=&Operator=&Lease=&APINum=&address=&ActiveWell=false&ActiveOp=false&Location=&sec=&twn=&rge=&bm=&PgStart=0&PgLength=10&SortCol=6&SortDir=asc&Command=Search"
objIE.navigate Source
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Set ele = objIE.document.getElementsByTagName("select")
For Each opt In ele
If opt.getAttribute("name") = "tblWellRecords_length" Then
opt.Focus
opt.Value = -1
Exit For
End If
Next opt
End Sub
I was looking for the "All" element to click on the production data. it's changing the value of the drop-down if I fix the attribute name as "productionTable_length" but it's not clicking the element. If it is clicked then the total number of shows will be changed from 1-25 to 1-166. Please help me on that.... – Partha 2 hours ago
You did not say about that :) Your question was how to select a specific dropdown element in webpage by excel vba For which I gave you the answer. For what you want, there is a different approach.
Try this
Sub Sample()
Dim objIE As Object, source As String, script As String
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True
source = "https://secure.conservation.ca.gov/WellSearch/Details?api=01120716&District=&County=&Field=&Operator=&Lease=&APINum=&address=&ActiveWell=false&ActiveOp=false&Location=&sec=&twn=&rge=&bm=&PgStart=0&PgLength=10&SortCol=6&SortDir=asc&Command=Search"
objIE.navigate source
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
script = "$('select[name=productionTable_length]').val('-1'); $('select[name=productionTable_length]').trigger('change');"
objIE.document.parentWindow.execScript script, "jscript"
End Sub

Excel VBA code reads wrong innerHTML code

I am absolutely new in webscraping and have some minor previous VBA knowledge.
I am trying to make a scraper which enters a site makes a search and then scrapes the details of the search.
I am very annoyed that my scraper can make the search with the given parameters, but after the search is made and the website is loaded, I make a innerHTML read request within VBA and the results are NOT the source code of the new page. So I cannot extract any information because my VBA code does not see the actual webpage html data. Why is that happening? What is the source code that my VBA extracts?
Thank you very much for your help in advance!
Public Sub my_scraper()
Dim my_data1, my_data2 As String
Dim my_Coll As String
i = 1
my_data1 = ActiveSheet.Cells(1, 1).Value
my_data2 = ActiveSheet.Cells(1, 2).Value
my_Coll = profession_hu_scraper(my_data1, my_data2)
Cells(2, 2).Value = my_Coll
End Sub
Public Function profession_hu_scraper(ByVal my_data1 As String, ByVal my_data2 As String) As String
Dim objIE As InternetExplorer
Dim html As HTMLDocument
Dim Link As Object
Dim ElementCol As Object
Dim erow As Long
'Dim all_inp_el As Object
'Application.ScreenUpdating = False
Set objIE = CreateObject("InternetExplorer.Application")
With objIE
.Visible = True
.Navigate "https://www.profession.hu/"
Do While .ReadyState <> READYSTATE_COMPLETE
Application.StatusBar = "Loading website..."
DoEvents
Loop
Set html = .Document
Range("A16") = html.DocumentElement.innerHTML
.Document.getElementById("header_keyword").Value = my_data1
.Document.getElementById("header_location").Value = my_data2
Set my_classes = .Document.getElementsByClassName("p2_button_inner")
For Each my_class In my_classes
If my_class.getAttribute("value") = "Keresés" Then
Range("c4") = "Clicked"
my_class.Click
i = i + 1
End If
Next my_class
Do While .ReadyState <> READYSTATE_COMPLETE
Application.StatusBar = "Loading website..."
DoEvents
Loop
Set html = .Document
Range("B16") = html.DocumentElement.innerHTML
End With
Set objIE = Nothing
Application.StatusBar = "Finished"
'Application.StatusBar = ""
End Function
After a few days of struggling I finally was able to find out that the code works OK. The problem was that the max character of a cell is 32k so it could not show the whole html code.
If you are a beginner watch out for it!
Updated:
Public Function profession_Hu_Scraper(myData1 As String, my_data2 As String)
Dim ie As New InternetExplorer
Dim doc As HTMLDocument
Dim ws As Worksheet: Set ws = ActiveSheet
ie.navigate "https://www.profession.hu/"
Do While ie.readyState <> READYSTATE_COMPLETE
Loop
Set doc = ie.document
ws.Range("A16") = doc.getElementById(myData1).innerText
ws.Activate("B16") = doc.getElementById(mydata2).innerText
'whatever else you wish to do
End Function

Where to find VBA functions for working with internet explorer

I am working on a VBA function to go to a webpage, find a single HTML element, and display its contents. Here is what I have so far.
Function WebTableToSheet(webpage As String, tag As String, count As Integer) As String
'Tested using IE7, Excel 2000 SP1, and Windows XP
Dim objIE As Object
Dim varTables, varTable, document
Dim allTags, tags
Dim varRows, varRow
Dim varCells, varCell
Dim lngRow As Long, lngColumn As Long
Dim strBuffer As String
Set objIE = CreateObject("InternetExplorer.Application")
'Setup
With objIE
.AddressBar = False
.StatusBar = False
.MenuBar = False
.Toolbar = 0
.Visible = True
.Navigate webpage
End With
'Load webpage
While objIE.Busy
Wend
While objIE.document.ReadyState <> "complete"
Wend
varTable = objIE.document.All
allTags = objIE.document.All.tags(tag)
WebTableToSheet = allTags.Item(count)
Debug.Print "Testing function"
Cleanup:
Set varCell = Nothing: Set varCells = Nothing
Set varRow = Nothing: Set varRows = Nothing
Set varTable = Nothing: Set varTables = Nothing
objIE.Quit
Set objIE = Nothing
End Function
I am able to open InternetExplorer, got to the webpage specified in the function, but when I try searching for a specific tag, it seems to fail. I have had trouble searching for Microsoft VBA Internet Explorer documentation and knew what variables and methods are available for this task?
I'd recommend setting a reference to shdocvw.dll and the Microsoft HTML Object Library. In XP shdocvw.dll is in your system32 folder, the HTML Object Library should be in your list of references. I only have access to XP at the moment, so you'll have to do a search for any other ver of windows you're using. Then in your code you can do the following:
Dim IE as SHDocVw.InternetExplorer
Dim doc as HTMLDocument
Dim el as IHTMLElement
Set IE = new SHDocVw.InternetExplorer
With IE
.navigate "some.webpage.com"
.visible = true
End With
Do
'Nothing
Loop Until IE.readyState = READYSTATE_COMPLETE ' = 4
Set doc = IE.Document
Set el = doc.getElementById("someElementId")
If Not el Is Nothing Then
MsgBox el.innerText
End If
Team,
IE webpage full loading check use below mentioned code
Application.Wait (Now + TimeValue("0:00:1"))
Do Until IE.Busy = False
DoEvents
Loop
ieApp.Navigate "Https://duckduckgo.com/"
'Wait for Internet Explorer to finish loading
Do While ieApp.Busy: DoEvents: Loop
Do While ieApp.Busy And Not ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
ieApp.Document.getelementbyid("search_form_input_homepage").Value = "Gajendra Santosh"
ieApp.Document.getelementbyid("search_button_homepage").Click

Resources