loop through rows of data to submit web form - excel

Sub AutoLoadAccounts()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Navigate "https://www.urlhere.com/admin/accounts/create"
IE.Visible = True
While IE.busy
DoEvents
Wend
IE.Document.All("title").Value = ThisWorkbook.Sheets("sheet1").Range("a1")
IE.Document.All("names").Value = ThisWorkbook.Sheets("sheet1").Range("b1")
IE.Document.All("floor").Value = 30
IE.Document.getElementById("status").selectedindex = 1
IE.Document.getElementById("email_state").selectedindex = 1
IE.Document.All("id").Value = ThisWorkbook.Sheets("sheet1").Range("c1")
IE.Document.All("years").Value = ThisWorkbook.Sheets("sheet1").Range("d1")
IE.Document.All("submit").Click
End Sub
The above code I use to populate a web form and submit it. I have around 150 rows of data ranging from A1:D1. I am trying to find a way to loop through the rows 1 by 1 after submitting the form until it reaches the end.
So essentially it will start on the first row and populate the fields from A1:D1, then once complete go down to the next row and do the same for A2:D2. and so on

The trick here is to organise your source data. Using two columns you can record the field name and the required value:
A B
1 Title Sample Title
2 Names Sample Names
3 Floor Sample Floor
To loop:
Sub AutoLoadAccounts()
Dim IE As Object
Dim cRow As Range ' Current row, used to extract values from Excel.
Set IE = CreateObject("InternetExplorer.Application")
IE.Navigate "https://www.urlhere.com/admin/accounts/create"
IE.Visible = True
While IE.busy
DoEvents
Wend
' Executes once for each row in the source range.
For Each cRow In ThisWorkbook.Sheets("sheet1").Range("A1:A3")
' Read field name and value from current row.
IE.Document.All(cRow.Value).Value = cRow.Offset(0, 1)
Next
IE.Document.All("submit").Click
End Sub
This code could be improved. At the moment the source range is hard coded (Range("A1:A3")). You could improve this, so the code automatically identifies all completed rows in Excel. If you are interested research the worksheets UsedRange object.
EDIT
Added example that reads source data from columns, not rows.
Sub AutoLoadAccounts_Columns()
Dim IE As Object
Dim cRow As Range ' Current row, used to extract values from Excel.
Set IE = CreateObject("InternetExplorer.Application")
IE.Navigate "https://www.urlhere.com/admin/accounts/create"
IE.Visible = True
While IE.busy
DoEvents
Wend
' Executes once for each row in the source range.
For Each cRow In ThisWorkbook.Sheets("sheet1").Range("A1:C1")
' Read field name and value from current row.
IE.Document.All(cRow.Value).Value = cRow.Offset(1, 0).Value
Next
IE.Document.All("submit").Click
End Sub

Related

Web scraping with Excel VBA to get hotel details

I am trying to scrape www.booking.com to get some hotel information based on a search i did on the website. The url is containing my search like address, check-in etc.
After the search, i am trying to scrape the hotel name, score, distance and other information without going further into detail (by clicking on the hotel). But in some way i cant go through the steps to get this information.
If i am right, all the search results are placed in the ID "hotellist_inner", which contains all the seperate hotel results in class "sr_property_block". But i already get an error on the first line:
For Each itemEle In objIE.Document.getElementsById("hotellist_inner"), as it seems that the result of getElementById is empty.
See my example code below.
Sub test()
Dim ws As Worksheet: Set ws = Sheets("scrape")
Dim objIE As Object
Set objIE = CreateObject("InternetExplorer.Application")
Dim url As String
url = ws.Range("A1")
With objIE
.navigate url
.Visible = True
End With
'wait IE
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
ResRw = 2
For Each itemEle In objIE.Document.getElementById("hotellist_inner")
'Every booking goes on a new line, so reset the column to 1
ResCol = 1
For Each d In itemEle.getElementByClassName("sr_property_block ") 'Each result (hotel) has this Class
'For the next result, add 1 to the column
ResCol = ResCol + 1
Next d
'For the next result, add 1 to the row
ResRw = ResRw + 1
Next
objIE.Quit
Set objIE = Nothing
End Sub
Sample url: https://www.booking.com/searchresults.nl.html?label=gen173nr-1DCAEoggI46AdIM1gEaKkBiAEBmAEcuAEXyAEM2AED6AEB-AEDiAIBqAIDuAL-25KJBsACAdICJDQ5MDk4ZmU4LWE3MDYtNGEwNi1hZTY2LWYzYTc5YmQwZWU2M9gCBOACAQ&sid=c7a70093e9732a158e970a642f2af63f&sb=1&sb_lp=1&src=index&src_elem=sb&error_url=https%3A%2F%2Fwww.booking.com%2Findex.nl.html%3Flabel%3Dgen173nr-1DCAEoggI46AdIM1gEaKkBiAEBmAEcuAEXyAEM2AED6AEB-AEDiAIBqAIDuAL-25KJBsACAdICJDQ5MDk4ZmU4LWE3MDYtNGEwNi1hZTY2LWYzYTc5YmQwZWU2M9gCBOACAQ%3Bsid%3Dc7a70093e9732a158e970a642f2af63f%3Bsb_price_type%3Dtotal%26%3B&ss=Amsterdam%2C+Noord-Holland%2C+Nederland&is_ski_area=&checkin_year=2021&checkin_month=8&checkin_monthday=24&checkout_year=2021&checkout_month=8&checkout_monthday=31&group_adults=1&group_children=0&no_rooms=1&b_h4u_keep_filters=&from_sf=1&ss_raw=Amsterdam&ac_position=0&ac_langcode=nl&ac_click_type=b&dest_id=-2140479&dest_type=city&iata=AMS&place_id_lat=52.372898&place_id_lon=4.893&search_pageview_id=694a3bbf21c20007&search_selected=true&search_pageview_id=694a3bbf21c20007&ac_suggestion_list_length=5&ac_suggestion_theme_list_length=0
Does anyone have an idea how to fix this?
Tnx!

Web Scraping with VBA - Almost There

After studying many posts here, i finally patched together code that worked (below)! It extracts the index value from yahoo finance when the index is basically the only data point on the page (https://finance.yahoo.com/quote/%5EHSI?p=%5EHSI&.tsrc=fin-srch).
However, when I want the same index from a website containing several index values (https://www.hkex.com.hk/?sc_lang=en), the web code looks overwhelming for me to correctly reference the element and put into my coding. Can anyone please help with this final hurdle? Thank you.
Sub HSI_Scrape()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.Navigate "https://finance.yahoo.com/quote/%5EHSI?p=%5EHSI&.tsrc=fin-srch"
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
Dim hsi As String
hsi = IE.Document.getElementsByClassName("Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)")(0).innerText
Worksheets("Tickers").Select
Range("z11").value = hsi
End Sub
Each of the tickers on HKEX has class "listitem". So to get each ticker, search for that class using getElementsByClassName("listitem"). Loop through each element, and the first child (class="col_name") of the element will be the ticker name, the second child (class="col_last") will be the price.
Sub HSI_Scrape()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.Navigate "https://www.hkex.com.hk/?sc_lang=en"
Do While IE.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
Dim listitems As Object 'list of tickers
Dim listitem As Object 'each ticker
Set listitems = IE.document.getElementsByClassName("listitem")
Dim row As Integer
row = 1
For Each listitem In listitems
Worksheets("Tickers").Range("A" & row) = listitem.Children(0).innerText 'Ticker Name
Worksheets("Tickers").Range("B" & row) = listitem.Children(1).innerText 'Ticker Price
row = row + 1
Next listitem
End Sub

Copy dropdown table from internet VBA

I am trying to copy a table from this website: http://www.nzfma.org/data/search.aspx#
I need to select the date as yesterday's date, then copy and paste the table to a file.
My code is below:
Sub Test1()
'open IE, navigate to the website of interest and loop until fully loaded
Dim NZFMA As Worksheet
Dim TodayN As Range
Dim elemCollection As Object
Set NZFMA = Sheets("NZFMA")
Set TodayN = NZFMA.Range("B2")
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.navigate "http://www.nzfma.org/data/search.aspx"
Do Until Not ie.Busy And ie.ReadyState = 4
DoEvents
Loop
'Select the dates from the drop-down box
ie.Document.getElementbyid("ctl00_cphBody_rdpDate_dateInput").Value = Format(TodayN, "yyyy-mm-dd")
'Click the submit button
ie.Document.getElementbyid("cphBody_btnSearch").Value = "Search"
'Copy the results
Set elemCollection = ie.Document.getElementbyid("cphBody_upResults")
While ie.ReadyState = 4
DoEvents
Wend
End With
End Sub
For some reason, my macro stops after the first getElementbyID line. Can someone advise as to which part of the code is wrong?
Alright, haven't tested this but something like it should work to change the first dropdown box
Dim elemCollection As HTMLFormElement
set elemCollection = ie.Document.getElementbyid("cphBody_lstMarket")
dim teststring as string
teststring = whatever
Select Case TestString
Case "BKBM"
elemCollection.selectedIndex = 1
Case "BBRS030"
elemCollection.selectedIndex = 2
etc etc
To change the date, you will need something like
ie.Document.getElementbyid("ctl00_cphBody_rdpDate_dateInput")(0).value = "05/07/2016"
Anyway, have a go with that, and please tell me the results

Looping through a row and copying each cell In a specific procedure

What I have to do is use Excel VBA to:
login to Amazon Seller
open a workbook
loop through a column to get an order number
put it in the search box
hit the search button
go to the order page and extract the data
then have the extracted data go back into a specified column in
another Excel workbook
The loop and order number parts are what I'm currently stumped on. I've figured out this much code as of this moment:
Sub MyAmazonSeller()
Dim MyHTML_Element As IHTMLElement
Dim MyURL As String
Dim oSignInLink As HTMLLinkElement
Dim oInputEmail As HTMLInputElement
Dim oInputPassword As HTMLInputElement
Dim oInputSigninButton As HTMLInputButtonElement
'InputSearchOrder will be the destination for order numbers taken from the workbook
Dim InputSearchOrder As HTMLInputElement
Dim InputSearchButton As HTMLInputButtonElement
Dim IE As InternetExplorer
Dim AAOrder As Workbook
Dim AAws As Worksheet
MyURL = "https://sellercentral.amazon.com/gp/homepage.html"
Set IE = New InternetExplorer
' Open the browser and navigate.
With IE
.Silent = True
.Navigate MyURL
.Visible = True
Do
DoEvents
Loop Until .ReadyState = READYSTATE_COMPLETE
End With
' Get the html document.
Set HTMLDoc = IE.Document
' See if you have the sign in link is because you are in the main
' page
Set oSignInLink = HTMLDoc.getElementById("signin-button-container")
If Not oSignInLink Is Nothing Then
oSignInLink.Click
Do
DoEvents
Loop Until IE.ReadyState = READYSTATE_COMPLETE
End If
' Get the email field and the next button
Set oInputEmail = HTMLDoc.getElementById("username")
Set oInputPassword = HTMLDoc.getElementById("password")
' Click the button and wait
oInputEmail.Value = "xxxxxx#xxxxxx.net"
' Get the password field and the sign in button
Set oInputPassword = HTMLDoc.getElementById("password")
Set oInputSigninButton = HTMLDoc.getElementById("sign-in-button")
' Click the button and wait
oInputPassword.Value = "xxxxxxxx"
oInputSigninButton.Click
Do
DoEvents
Loop Until IE.ReadyState = READYSTATE_COMPLETE
Application.Wait (Now + TimeValue("0:00:05"))
Set AAOrder = Application.Workbooks.Open("Z:\Employee Folders\Employee\trackingnumber_sample_spreadsheet.xls")
Set AAws = AAws.Worksheets("PrimeOrdersWithNoFulfillmentRe")
Set InputSearchOrder = HTMLDoc.getElementById("sc-search-field")
'What I'm currently stuck on
InputSearchOrder.Value = "001-7163923-7572632"
Set InputSearchButton = HTMLDoc.getElementsByClassName("sc-search-button")(0)
InputSearchButton.Click
Do
DoEvents
Loop Until IE.ReadyState = READYSTATE_COMPLETE
'Was able to add this snippet, but I'm getting an error 13, most likely with
'my e variable. I'm basically trying to do a loop within a loop, extracting 5
'pieces of data and sticking them back into their respective columns in the
'original Excel sheet. The problem comes when scraping the HTML. I'm basically
'trying to get text in the tables which have a few levels and it's frustrating
'me to no end.
With HTMLDoc
Set elems = HTMLDoc.getElementsByTagName("td")
For Each e In elems
If e.innerText Like "*1Z*" Then
Range("D2").Value = e.innerText
End If
Next e
End With
Err_Clear:
If Err <> 0 Then
Err.Clear
Resume Next
End If
End Sub

excel vba copying fedex tracking information

I want to copy into Excel 3 tracking information tables that website generates when I track a parcel. I want to do it through Excel VBA. I can write a loop and generate this webpage for various tracking numbers. But I am having a hard time copying tables - the top table, travel history and shipments track table. Any solution? In my vba code last 3 lines below are giving an error :( - run time error '438' Object doesn't support this property or error.
Sub final()
Application.ScreenUpdating = False
Set ie = CreateObject("InternetExplorer.Application")
my_url = "https://www.fedex.com/fedextrack/index.html?tracknumbers=713418602663&cntry_code=us"
With ie
.Visible = True
.navigate my_url
Do Until Not ie.Busy And ie.readyState = 4
DoEvents
Loop
End With
ie.document.getElementById("detailsBody").Value
ie.document.getElementById("trackLayout").Value
ie.document.getElementById("detail").Value
End Sub
.Value is not a method available in that context. also, you will want to assign the return value of the method call to a variable. Also, you should declare your variables :)
I made some modifications and include one possible way of getting data from one of the tables. YOu may need to reformat the output using TextToColumns or similar, since it prints each row in a single cell.
I also notice that when I execute this, the tables have sometimes not finished loading and the result will be an error unless you put in a suitable Wait or use some other method to determine when the data has fully loaded on the webpage. I use a simple Application.Wait
Option Explicit
Sub final()
Dim ie As Object
Dim my_url As String
Dim travelHistory As Object
Dim history As Variant
Dim h As Variant
Dim i As Long
Application.ScreenUpdating = False
Set ie = CreateObject("InternetExplorer.Application")
my_url = "https://www.fedex.com/fedextrack/index.html?tracknumbers=713418602663&cntry_code=us"
With ie
.Visible = True
.navigate my_url
'## I modified this logice a little bit:
Do While .Busy And .readyState <> 4
DoEvents
Loop
End With
'## Here is a simple method wait for IE to finish, you may need a more robust solution
' For assistance with that, please ask a NEW question.
Application.Wait Now() + TimeValue("0:00:10")
'## Get one of the tables
Set travelHistory = ie.Document.GetElementByID("travel-history")
'## Split teh table to an array
history = Split(travelHistory.innerText, vbLf)
'## Iterate the array and write each row to the worksheet
For Each h In history
Range("A1").Offset(i).Value = h
i = i + 1
Next
ie.Quit
Set ie = Nothing
End Sub

Resources