I wish to make a macro which can auto login a page and then autofill the search input field and finally web scraping the search result.
I have been struggling for a long time that I can successfully login the page. However, when I click the login button, I cannot further get the input tag for the search field as well as adding the value to that input field. The index of those input tags are absolutely correct.
Below is my vba script, many thanks!
Sub login_and_search()
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate "http://loginpage.html"
Do While .Busy Or _
.readyState <> 4
DoEvents
Loop
Set inputElement = .document.getELementsByTagName("input")
inputElement.Item(0).Value = "abc" 'username
inputElement.Item(1).Value = "xyz" 'password
inputElement.Item(2).Click
Do While ie.Busy Or _
.readyState <> 4
DoEvents
Loop
Set searchField = .document.getELementsByTagName("input")
searchField.Item(0).Value = "searchitems" 'search field
searchField.Item(1).Click
End With
End Sub
Below are the 2 html files, sorry for the too simplified html files as I roughly make them for testing the vba script only. The first one is the login page and the second one is the page after login.
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<title></title>
</head>
<body>
<form class="" action="search.html" method="post">
<input type="text" name="" value="">
<input type="password" name="" value="">
<input type="submit" name="" value="login">
</form>
</body>
</html>
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<title></title>
</head>
<body>
<form class="" action="searchResults.html" method="post">
<input type="text" name="" value="">
<input type="submit" name="" value="Search">
</form>
</body>
</html>
There is no error showing when running the macro, I think it is a logic error instead of syntax error.
Once again, thanks all!
Related
<div id="xe-editor-container-1" class="input_area xpress_xeditor_editing_area_container" style="height: 400px;">
<iframe id="editor_iframe_1" allowtransparency="true" frameborder="0" src="http://my_URL.or.kr/xe/modules/editor/styles/default/editor.html" scrolling="yes" style="width: 100%; height: 400px; display: block;">
<html xmlns="http://www.w3.org/1999/xhtml>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel="stylesheet" type="text/css" href="editor.css">
<title>XpressEngine</title>
</head>
<body class="xe_content editable"></body>
</html>
</iframe>
<textarea id="xpress-editor-1" rows="8" cols="42" style="display: none; width: 100%;"></textarea>
<textarea rows="8" cols="42" class="input_syntax " style="display:none"></textarea>
</div>
I want to copy two tables, ListObjects ("Tbl1") on Sheet1, ListObjects ("Tbl2") on Sheet2, and upload them as a single post on the web.
The range of the table can be changed every time.
Logging in to the web, navigating to the bulletin board, pressing the write button and typing the title succeeded.
But I have failed to upload the post.
Perhaps you can not find the bulletin board object.
Code I created by searching the web.
The HTML above is the board's HTML code.
With ie
.navigate "http://my_URL/offering"
ieBusy ie 'Procedure fetched as search (check the ready status)
.Document.getElementsByClassName("ico_16px write")(0).Click
ieBusy ie
Dim oTitle As Object, Ocontents As Object
Set oTitle = .Document.getElementsByname("title")(0) 'Sometimes fail(sometimes Nothing)
Set oContents = .Document.getElementsByClassName("xe_content editable")(0) 'evry time fail(=Nothing)
oTitle.Value = "my Title"
oContents.Value = ????
.Document.forms(0).submit 'I could not confirm it because it did not work anymore.
End With
Sub ieBusy(ie As Object)
Do While ie.Busy Or ie.readyState < 4
DoEvents
Loop
End Sub
title HTML
board HTML
1) Use an additional timed loop to set oTitle as per https://stackoverflow.com/a/55334183/6241235
2) Your oContents variable is selecting for an element which is inside an iframe I think. I would expect you to instead be targeting a textarea element. There are two that come after the iframe. The first has id xpress-editor-1
I try to automate some manual processes by using VBA automation. One of them is to click on a element that has an empty content and at the moment I am not able to figure out how to deal with it
The HTML code that I am trying to click on:
<div id="searchcombobox-1077-triggerWrap" data-ref="triggerWrap" class="x-form-trigger-wrap x-form-trigger-wrap-toolbar">
<div id="searchcombobox-1077-inputWrap" data-ref="inputWrap" class="x-form-text-wrap x-form-text-wrap-toolbar">
<input id="searchcombobox-1077-inputEl" data-ref="inputEl" type="text" role="combobox" size="1" name="searchcombobox-1077-inputEl" placeholder="Account Number" tabindex="-1" class="x-form-field x-form-text x-form-text-toolbar " autocomplete="off" componentid="searchcombobox-1077">
</div>
<div id="searchcombobox-1077-trigger-picker" class="x-form-trigger x-form-trigger-toolbar x-form-search-trigger x-form-search-trigger-toolbar "></div>
</div>
The VBA code used is:
HTMLDoc.getElementById("searchcombobox-1077-inputEl").Value = '11xx111'
Set click_el = HTMLDoc.querySelector("#searchcombobox-1077-trigger-picker")
With click_el
.Focus
.FireEvent "onclick"
End With
What should be the approach that I need to take into consideration since the div element that I need to click on is empty?
Thanks,
I am trying to punch the radio button question and want to input the value in an open text box as well (using VBA). Below is the script I am using.
I tried multiple things, looked at other websites as well but it is not working.
It would be great if someone can guide me.
Dim IE As Object
Dim Region, VOCSentDate As String
Sheets("Sheet1").Select
Region = ThisWorkbook.Worksheets("Sheet1").Range("A1").Value
VOCSentDate = ThisWorkbook.Worksheets("Sheet1").Range("B1").Value
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate "website link....."
Application.StatusBar = "Submitting"
While IE.Busy
DoEvents
Wend
I'm not sure how to punch the radio button question. I tried the 3 types for Open textbox below but none of them are not working, throwing object required error
IE.Document.getElementById("ctl00_body_7F22FA6E-5D77-426A-AA84-68D833FA05C1_3").Value = VOCSentDate
IE.Document.getElementById("ctl00_body_7F22FA6E-5D77-426A-AA84-68D833FA05C1_3").innerText = VOCSentDate
IE.Document.all("ctl00_body_7F22FA6E-5D77-426A-AA84-68D833FA05C1_3").Value = VOCSentDate
Application.StatusBar = "Form Submitted"
IE.Quit
Set IE = Nothing
-------------- HTML of the Radio button --------------
<div class="SurveyItem RadioButtonList Item1">
<div class="ItemText">
<span class="NoItemNumber"></span><span class="QuestionText">Please select your region:</span>
</div>
<div class="Validators">
<span id="ctl00_body_ctl12" style="color:Red;display:none;">A response to this question is required</span>
</div>
<div id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" class="Response">
<div class="ResponseOption">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_1" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_1" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_1">Americas</label>
</div>
<div class="ResponseOption AlternatingRow">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_2" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_2" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_2">ANZ</label>
</div>
<div class="ResponseOption">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_3" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_3" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_3">APAC</label>
</div>
<div class="ResponseOption AlternatingRow">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_4" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_4" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_4">APME</label>
</div>
<div class="ResponseOption">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_5" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_5" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_5">Europe</label>
</div>
<div class="ResponseOption AlternatingRow">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_6" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_6" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_6">India</label>
</div>
<div class="ResponseOption">
<input id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_7" type="radio" name="ctl00$body$ctl00_body_ctl00_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1" value="01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_7" /><label for="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_7">North America</label>
</div>
------------------- HTML of Open End -------------------------
<div class="SurveyItem Item3">
<div class="ItemText">
<span class="NoItemNumber"></span><span class="QuestionText">EOPR First Sent Date (dd/mm/yyyy)</span>
</div>
<div class="Validators">
<span id="ctl00_body_ctl52" style="color:Red;display:none;">A response to this question is required</span><span id="ctl00_body_ctl53" style="color:Red;display:none;">Incorrect Date Format</span>
</div>
<div class="Response">
<input name="ctl00$body$01EE9560-B1F9-4BA3-A922-9D53A1120FC2_3" type="text" id="ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_3" class="TextBox" />
</div>
</div>
tl;dr;
The following are based on the HTML as shown. If there are parent form/frame/iframe tags involved they will also need to be negotiated before making the selections below.
Radio buttons:
For the radio buttons you can use a CSS selector combination to target the page styling and return a nodeList of all of the radio button elements. You can then select by index the appropriate button
Dim aNodeList As Object
Set aNodeList = ie.document.querySelectorAll(".ResponseOption [type=radio]")
aNodeList.item(0).Click '<==Select first option
I think click may be likely method here as I can't see a checked attribute, else the syntax would be aNodeList.item(0).checked = True
The returned nodeList is as follows by index
Textbox:
There is an ID for the input box so you can use an ID selector, #, to target
ie.document.querySelector("#ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_3").Value = "myText"
To choose a radio button style input, make the checked property true.
IE.Document.getElementById("ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_1_6").checked = true
An input textbox gets its value set.
IE.Document.getElementById("ctl00_body_01EE9560-B1F9-4BA3-A922-9D53A1120FC2_3").value = format(VOCSentDate, "d/m/yy")
there are many article on this site on how to read tags and tables in web sites with Excel VBA, but I am stuck here.
This website gives me business locations after entering a Zip code.
("Where is the closest location relative to my Zip Code")
I managed to navigate to the site, enter the Zip code and click Submit:
Dim Browser As SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument
Set Browser = New SHDocVw.InternetExplorer ' create a browser
Browser.Visible = True ' make it visible
Application.StatusBar = ".... opening page"
Browser.navigate "https://www.thewebsite.com" ' navigate to page
WaitForBrowser Browser, 1 ' wait for completion or timeout
Application.StatusBar = "gaining control over DOM object"
Set HTMLDoc = Browser.document ' load the DOM object
WaitForBrowser Browser, 1
HTMLDoc.getElementById("ZipCode").Value = "28278"
HTMLDoc.getElementById("localTeamZipSubmit").Click
The site opens and the relevant content looks like this:
<div>
<div class="columns">
<div class="column boldText paddingFive" style="padding-left: 20px; width: 70px;">
Location:
</div>
<div class="column paddingTopFive">CHARLOTTE</div>
</div>
<div class="columns">
<div class="column boldText paddingFive" style="padding-left: 20px; width: 120px;">
Location Number:
</div>
<div class="column paddingTopFive">102340</div>
</div>
<div class="columns">
<div class="column boldText paddingTopFive paddingLeftTwenty" style="vertical-align: top;">
Address:
</div>
<div class="column paddingTopFive paddingLeftTwenty">
<div>8848 Main St.</div>
<div>Suite F</div>
<div></div>
<div>Charlotte, NC 27218</div>
</div>
</div>
<div class="columns">
<div class="column boldText paddingFive" style="padding-left: 20px; width: 70px;">
Phone:
</div>
<div class="column paddingTopFive">(704) 911-4440</div>
</div>
<div class="columns">
<div class="column boldText paddingFive" style="padding-left: 20px; width: 70px;">
Fax:
</div>
<div class="column paddingTopFive">(704) 911-4441</div>
</div>
</div>
As you can see, this section has no table, no named tags and classes that are use over and over.
I was not able to read this information yet. I would be happy to get the whole blob into a String and parse it"
"Text = HTMLDoc.getEverything()"
Thanks a lot for your help!!!
In the meantime I found another code snippet that I modified but I am getting stuck at the same point:
Post and submit works but how to get the answer....
{ Private Sub PostalCodes()
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
On Error GoTo errHandler
ie.Visible = 1
With ie
.navigate "https://www.pattersondental.com/ContactUs/MyLocalTeam"
Do While .busy: DoEvents: Loop
Do While .ReadyState <> 4: DoEvents: Loop
With .document.Forms("GetBranchFromZipForm")
.ZipCode.Value = "28273"
.submit
End With
' Do While Not CBool(InStrB(1, .document.URL, _
' "cp_search_response-e.asp"))
' DoEvents
' Loop
Do While .busy: DoEvents: Loop
Do While .ReadyState <> 4: DoEvents: Loop
' MsgBox .document.all.tags("Colums").Item(1).Rows(1).Cells(1).innerText
MsgBox .document.all.tags("Colums").innerText
' MsgBox .document}
I guess I have to search no for "how to dissect a HTML document"...
Add on:
It seems that while ie is a valid item (in the watch window) IE.Document is empty... why can this be, The website is still there with new data.
I even tried another code snippet that looks for open websites in IE, it finds the site (with the correct data) but the document is still empty and getelementBY... does not find anything of course.
I am about to start drinking...
I can't believe it.
After 3 days of poking I found this:
With ActiveSheet.QueryTables.Add(Connection:="URL;
https://www.pattersondental.com/ContactUs/MyLocalTeam",
Destination:=Range("A1"))
.PostText = "ZipCode=70032"
.RefreshStyle = xlOverwriteCells
.SaveData = True
.Refresh
I don't pretend to understand why it works, but is does.
John, I will still check out, what you suggested. Thanks
I have a small DropDownList with 3 options .
Problem is when I select , let's say , 2nd option .
No probs , a new page is opened .
If I click on option 1 or 3 , no probs .
However , if I click on 2nd option again , instead of clicking on 1 or 2 , no new page is opened .
Is there a way to correct this , so that clicking on whatever always opens a new window .
Thanks...Vern
SORRY I COULDN'T GET THE "CODE" OR "HTML" TO WORK .
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>myFavs-HtmlProbs.html</title>
<meta name="Generator" content="PureBasic">
<meta name="Description" content="Your description here...">
<meta name="Description" content="...Created by myFavs % PureBasic...">
<style type="text/css">
</style>
</head>
<body text="#000000" style="background-color:#A69E80; text-align:center;">
<br> <br> <br>
<!--dd <div style=-->
<table width="100%" border="0" style="text-align:center" cellspacing="0" cellpadding="0">
<tr style="text-align:center">
<!-- EXAMPLE: <MenuName="aaMost-Used"> --> <!-- rgb(250,240,255) -->
<select style="width:200px; font:14px Arial Black; color:rgb(0,0,0); background-color:rgb(231,169,126);" name="menu" onchange="window.open(this.value)">
<option selected="0" value="">aaMost-Used</option>
<option value="http://alternativeto.net/">AlternativeToSoftwares</option>
<option value="https://www.biblegateway.com/reading-plans/chronological/today?version=NLT">One Year Chronological Bible NLT</option>
<option value="http://www.portablefreeware.com/">portablefreeware.com/</option>
</select></td></tr></table><br></body></html>
Just add this attribute to your select tag
onfocus="this.selectedIndex=0;"
It will change the option to the default option each time after you select one, in your case
aaMost-Used and hence you can then select your required next option again.
When you click on an option more than once, it does not open the page because the option is already selected, and no change occurs. Therefore, onchange isn't fired.
You can actually open a link twice by opening that link, opening another (deselects first) and then clicking on the first link again.
But here's the fix.
Change onchange="window.open(this.value)" to onclick="window.open(this.value)" (onchange to onclick).
That way, the link will be opened whenever an option is clicked, rather than when it changes.
JSFiddle: https://jsfiddle.net/SanPilot/mqqh5u73/1/