I'm using Selenium to do some webscraping.
I always struggle with finding elements when Name or ID isn't an options.
The page is
https://www.portfolio123.com/holdings.jsp?portid=1637063
And I'm trying to find the log in button.
the element looks like:
I've tried
FindElementByClass("btn-primary btn-sm", 10000)
FindElementsByLinkText("Log In", 10000)
FindElementByXPath("//div[#class='btn btn-primary btn-sm']//[#href='javascript:void(goToLoginPage())")
But without any success (I suspect I need to use XPath, but I can't seem to get it right
Any help please? (and if you could explain how to figure this out, so I don't have the issues in future that would be really appreciated)
To locate the element Log In you can use either of the following Locator Strategies:
Using FindElementByLinkText:
FindElementByLinkText("Log In")
Using FindElementByCss:
FindElementByCss("a[href*='goToLoginPage']")
Using FindElementByXPath:
FindElementByXPath("//a[contains(#href, 'goToLoginPage')]")
I tested. try this code below code. its working for me..
Sub loginCode()
Dim bot As New WebDriver
bot.start "chrome", "https://www.portfolio123.com/holdings.jsp?portid=1637063"
bot.Get "/"
Call WaitForSec(5)
bot.FindElementByXPath("//*[#id=""wrapper""]/div/div/div/a[1]").Click
End Sub
Sub WaitForSec(Sec As Integer)
Dim lngTime As Long
lngTime = Timer
While Timer < lngTime + Sec
DoEvents
Wend
End Sub
Related
Code is required
which type of value format is needed to enter in Excel cell, with one example
these are the coding stuff, also provided you an image where any can visualize how the problem look like. And in this problem we can't just use .SendKeys here it is more typical, because it have the Date-Month-Time, so help me out in this.
I tried, after removing "readonly" word in HTML .. then its working fine, but this is not the way can you edit in this code,
Sub google_search()
Dim row As Integer
row = 2
Dim bot As WebDriver
Set bot = New WebDriver
Dim GenderDD As Selenium.WebElement
bot.Start "chrome"
bot.Get "https://abcd.com/"
bot.FindElementbyName("sample_cdate").SendKeys "Value"
Stop
End Function
Also giving Inspect of Targeted Site, for the reference
<input type="text" class="form-control datetimepicker" name="sample_cdate" id="sample_cdate" placeholder="Date and Time of Sample Collection" **readonly**="">
I tried, after removing "readonly" word in HTML .. then its working
fine, but this is not the way can you edit in this code
You should replace .SendKeys() method:
'bot.FindElementbyName("patient_id").SendKeys Sheet1.Cells(row, 3).Value
bot.ExecuteScript "arguments[0].setAttribute('value', arguments[1])", _
Array(bot.FindElementById("sample_cdate"), _
Format(Sheet1.Cells(row, 16).Value, "yyyy-mm-ddThh:mm:ss"))
As a readonly element, similar as on graphic WebBrowser, you cannot type input using .SendKeys(), but you can use JavaScript to set .Value attribute through programming.
As you show, your input id may be id="sample_rdate", not sample_cdate.
In selenium excel vba I am trying to learn more about how to deal with the CSS selectors
And I am wondering because when inspecting an element with ID and when run the code I got a message that the element not found
Here's the code till now
Private bot As New selenium.ChromeDriver
Sub Test()
Dim win, mainWin As selenium.Window, sCode As String, i As Long
Dim urlImage As String, urlPost As String
Dim sCase As String
sCase = "192160470"
Set bot = New ChromeDriver
With bot
.Start "Chrome"
'First Window (Main Window)
.Get "https://www.kuwaitcourts.gov.kw/searchPages/searchCases.jsp"
'.FindElementById("txtCaseNo").SendKeys sCase
.FindElementByCss("input[type=text][name='txtCaseNo']").SendKeys sCase
'MsgBox "Click OK After Entering Captcha", 64
Stop
.Quit
End With
End Sub
and here's the HTML part for that element
<td><input type="text" name="txtCaseNo" id="txtCaseNo" maxlength="9" class="inputTextBox" onkeypress="return onlyNumbers(event);"></td>
I am stuck at this line
.FindElementByCss("input[type=text][name='txtCaseNo']").SendKeys sCase
Thanks advanced for any help or any ideas
To send a character sequence to the Username field as the the desired element are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
With bot
.Start "Chrome"
.Get "https://www.kuwaitcourts.gov.kw/searchPages/searchCases.jsp"
.SwitchToFrame "searchCaseDiv"
.FindElementByCss("input[type=text][name='txtCaseNo']").SendKeys sCase
You can find a relevant discussion in How to send text with in the username field within an iframe using Selenium VBA
tl; dr
Ways to deal with #document under iframe
The element is inside of an iframe with id searchCaseDiv. You have to switch to that iframe to be able to access the element.
Use .SwitchToFrame to switch frame.
For java, it would be like this,
driver.switchTo().frame("searchCaseDiv");
The below code worked up until a few days ago to go to the url, find the table and import the contents of the table into Excel. I then did some other formatting to get the table into the appropriate rows and columns. But now this code cannot locate the table. I do not fully understand the "Set a = .FindElementsByTag("iframe")(2)" and the ".SwitchToFrame 1". But my general understanding is that this portion of the code switches to a different frame which then extracts the internal url, which then is used to get the data form the table.
I need help identifying what to change in order to get the intended "url2", which is "https://docs.google.com/spreadsheets/d/e/2PACX-1vT__QigQ9cJV03ohUkeK5dgQjfAbJqxrc68bXh9Is1WFST8wjxMxDy7hYUCFHynqRvInsANUI22GdIM/pubhtml?gid=817544912&single=true&chrome=false&widget=false&headers=false" url. *note: I do not use this docs.google url because I do not know if this url will change periodically. I know the rosterresource.com/mlb-roster-grid url will stay consistent.
I have tried changing some of the integers for "Set a = .FindElementsByTag("iframe")(2)" and the ".SwitchToFrame 1", but I am doing that blindly since I am not familiar with this art of the code.
Sub GetRRgrid()
'"Selenium type library" is a reference used
Dim d As WebDriver, a As Object
Set d = New ChromeDriver
Const url = "https://www.rosterresource.com/mlb-roster-grid/"
With d
.Start "Chrome"
.Get url
Set a = .FindElementsByTag("iframe")(2)
.SwitchToFrame 1
url2 = .FindElementByCss("iframe").Attribute("src")
.Get url2
ele = .FindElementByTag("tbody").Attribute("innerText")
d.Close
End With
' other processes t format the data after it is imported
end sub
````
Getting the iframe and switching to it:
You need to pass the iframe element (identifier argument) to SwitchToFrame, you are then within that document and can interact with its contents. No need to .get on that with Selenium. You have to switch to .SwitchToDefaultContent to go back to parent document.
You can identify the iframe in question in a number of ways. Modern browsers are optimized for css selectors so I usually go with those. The css equivalent of
.FindElementByTag("iframe")
is
.FindElementByCss("iframe")
Your iframe is the first (and only) so I wouldn't bother gathering a set of webElements and indexing into it. Also, you want to try for a short selector of a single element where possible to be more efficient.
VBA:
Option Explicit
Public Sub Example()
Dim d As WebDriver
Const URL As String = "https://www.rosterresource.com/mlb-roster-grid/"
Set d = New ChromeDriver
With d
.Start "Chrome"
.get URL
.SwitchToFrame .FindElementByCss("iframe")
Stop
.Quit
End With
End Sub
Writing to Excel (.AsTable.ToExcel) :
Something I only just discovered, haven't seen documented anywhere, and am excited by, is that there is a method to write the table direct to Excel:
Option Explicit
Public Sub Example()
Dim d As WebDriver
Const URL As String = "https://www.rosterresource.com/mlb-roster-grid/"
Set d = New ChromeDriver
With d
.Start "Chrome"
.get URL
.SwitchToFrame .FindElementByTag("iframe")
.FindElementByCss(".waffle").AsTable.ToExcel ThisWorkbook.Worksheets("Sheet1").Range("A1")
Stop
.Quit
End With
End Sub
Here is what I ended up doing for this question. Thanks to QHarr for the guidance.
Public Sub GetRRrostergrid()
Dim d As WebDriver
Const URL As String = "https://www.rosterresource.com/mlb-roster-grid/"
Dim URL2 As String
Set d = New ChromeDriver
Sheet20.Activate
With d
.Start "Chrome"
.Get URL
URL2 = .FindElementByClass("post_content").FindElementByTag("iframe").Attribute("src")
.Get URL2
.FindElementByCss(".waffle").AsTable.ToExcel ThisWorkbook.Worksheets("RRchart").Range("b1")
.Quit
End With
End Sub
i would like to use vba to access a web site and perform other operation once logged in, but i have problem with log in, using this code
Dim obj As New WebDriver
Dim Keys As New Keys
obj.Start "chrome", ""
obj.Get "https://a14onlinetrackers.azurewebsites.net/trackers/trackerView/1?i=1"
obj.FindElementByName("loginfmt").SendKeys Keys.Control & "t"
obj.FindElementByName("loginfmt").SendKeys ("xxxx#yyy.com")
obj.FindElementById("idSIButton9").Click
obj.FindElementByName("passwd").SendKeys ("xxxx")
obj.FindElementById("idSIButton9").Click
after auto-typing the password i have an error on the submit button.
Can you help?
after login in what would be the code to open a new tab on the same browser?
Thank you.
I use the following codes to log in with VBA, and it works.
Sub Test()
Dim obj As New WebDriver
With obj
.Start "Chrome"
.get "https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=13&ct=1594461347&rver=7.0.6738.0&wp=MBI_SSL&wreply=https:%2F%2Faccount.microsoft.com%2Fauth%2Fcomplete-signin%3Fru%3Dhttps%253A%252F%252Faccount.microsoft.com%252F%253Frefd%253Dlogin.live.com%2526ru%253Dhttps%25253A%25252F%25252Faccount.microsoft.com%25252F%25253Frefd%25253Dlogin.live.com&lc=1033&id=292666&lw=1&fl=easi2"
.FindElementByXPath("//input[#class='form-control ltr_override']").SendKeys ("Your Account")
.FindElementByXPath("//input[#id='idSIButton9']").Click
.FindElementByXPath("//input[#class='form-control']").SendKeys ("Your Password")
.FindElementByXPath("//input[#id='idSIButton9']").Click
End With
obj.Wait (60000)
End Sub
#Fabio Giallombardo, please let me know if this works for you or not. :)
I have some experience with VBA but I am very new to web scraping with VBA. However I am very enthusiastic about it and thought of a 1000 ways how could I use it and make my job easier. :)
My problem is that I have a website with two input fields and one button. I can write in the input fields (they have ID so I can easily find them)
My code for the input fields:
.Document.getElementById("header_keyword").Value = my_first
.Document.getElementById("header_location").Value = my_last
But I am really stuck with clicking the button.
Here is the html code for the buttons:
<span class="p2_button_outer p2_button_outer_big"><input class="p2_button_inner" type="submit" value="Keresés" /></span>
<span class="p2_button_outer p2_button_outer_big light hide_floating"><a id="tour_det_search" class="p2_button_inner" href="http://www.profession.hu/kereses">Részletes keresés</a></span>
As you can see there are two different buttons near each other, and they share the same class. I am looking for the first/upper one. My problem is that it has no ID, only class, type and value. But I was not able to find getelementsbytype or getelementsbyvalue method.
Is there any solution to find the button by type or value (or both)?
Sorry if I am asking something stupid but as I said previously I am new in scraping...:)
Thank you in advance and have a nice weekend!
Fortunatelly I have worked out the solution. :)
What I did is the following. I made searched for the relevant classes and then using the getAttribute() method and looping thru the classes I searched for the specific value and clicked on it when found it.
Below is the working code:
Set my_classes = .Document.getElementsByClassName("p2_button_inner")
For Each class In my_classes
If class.getAttribute("value") = "Keresés" Then
Range("c4") = "Clicked"
class.Click
Exit For
End If
Next class
Thank you!
You can use the following function. It looks for a first HTML element with the given caption. You can also limit the searching by HTML tag.
(The code is compatible with IE <9 that doesn't contain getElementsByClassName method).
Public Function FindElementByCaption(dom As Object, Caption As String, _
Optional Tag As String, Optional Nested As Boolean = True) As Object
Dim ControlsSet As Variant
Dim Controls As Variant
Dim Control As Object
'------------------------------------------------------------------------------------
Set ControlsSet = VBA.IIf(Nested, dom.all, dom.childNodes)
If VBA.Len(Tag) Then
Set Controls = ControlsSet.tags(VBA.LCase(Tag))
Else
Set Controls = ControlsSet
End If
For Each Control In Controls
If VBA.StrComp(Control.InnerHtml, Caption, vbTextCompare) = 0 Then
Set FindElementByCaption = Control
Exit For
End If
Next Control
End Function
Here is how to apply it in your code:
Dim button As Object
Set button = FindElementByCaption(.Document, "Keresés", "INPUT", True)
If Not button Is Nothing Then
Call button.Click
Else
Call MsgBox("Button has not been found")
End If
CSS selector:
Use a CSS selector to target the element of:
input[value='Keresés']
This says element with input tag, having attribute value with value 'Keresés'.
CSS query:
VBA:
You apply the selector via the querySelector method of document.
ie.document.querySelector("input[value='Keresés']").Click