I'm trying to build a data scraper with Selenium Python that searches a webpage. The search page contains a dojoComboBox that allows you to select a name from a dropdown list.
Here is the HTML for the dojoComboBox:
<span _="whitespace and CR's between tags adds in FF" class="dojoComboBoxOuter dj_khtml dj_safari dropSel" style="null">
<input style="display:none" tabindex="-1" name="txtCaseName" value="" dojoattachpoint="comboBoxValue">
<input style="display:none" tabindex="-1" name="txtCaseName_selected" value="" dojoattachpoint="comboBoxSelectionValue">
<input type="text" autocomplete="off" class="dojoComboBox dropSel" dojoattachevent="key:_handleKeyEvents; keyUp: onKeyUp; compositionEnd; onResize;" dojoattachpoint="textInputNode" style="null">
<img hspace="0" vspace="0" class="dojoComboBox dropSel" dojoattachpoint="downArrowNode" dojoattachevent="onMouseUp: handleArrowClick; onResize;" src="dojo/src/widget/templates/images/combo_box_arrow.png" style="width: 13px; height: 13px;">
</span>
I can get the DojoComboBox scroll menu to open with the following code:
dropdown = browser.find_element_by_css_selector("img.dojoComboBox")
dropdown.click()
However, I can't seem to find a way to actually select one of the values.
Update:
The HTML I posted above is from the page I pull up when I inspect elements (the HTML under the 'Elements' tab of Chrome DevTools). I've been looking at some other files that I found in the 'Source' tab, and I found some code in a page called main.aspx?e=nauqov2blnhlnh45eseztnao that looks also related to the DojoComboBox:
<select name="txtCaseName" id="txtCaseName" class="dropSel"dojotype="ComboBox"
setSelectedValue="dojo.byId('txtCaseName1').value=arguments[0]"
autocomplete="true" dataurl="CaseCode.aspx?match=%{searchString}&lang=e"
mode="remote" maxlistlength="7">
</select>
However, neither of these seem to encode options that I can select. I thought that perhaps the options would be stored in the url labeled 'dataurl' in the piece of HTML above, but when I copy that link into my address bar, it doesn't take me to a valid webpage. From what I've found in the Dojo documentation, it seems like the options should be stored in a separate file/database, but I can't find any suggestions about how to access that file.
Any suggestions? (Thanks!)
you will have to store every options in a list. Then based on some conditions such as name of option, you can choose it and click on it.
all_options = browser.find_elements_by_css_selector("your locator for every options")
for options in all_options:
if "option_name" in options.text.strip():
options.click()
Related
I am using the Firefox webdriver for Selenium to scrape a webpage that looks to be rendered with React on the client side. The classes in the rendered DOM look dynamically generated, and seem to change with every new request. There are also many button elements on the page, some of which are not in the viewport. So my strategy is to search for a way to click on a button that contains text that I enter using selenium. Several buttons will contain the text, and I want to just find the first such button.
Using selenium/xpath, how would I select the first button that contained the text E9 1QJ?
<button>
<div><svg ...> </div>
<div>
<div>London</div>
<div>E9 1QJ</div>
</div>
</button>
<button>
<div><svg ...> </div>
<div>
<div>London Foo Bar</div>
<div>E9 1QJ</div>
</div>
</button>
Thanks
This should work:
{driver}.find_element_by_xpath("//button[div/div[text()='E9 1QJ']][1]")
But keep in mind that a solution like this it is not very flexible and could break with a minimum change in the html structure.
I have created a libary with about 80 sublime-snippet's.
They work great on my machine which is the machine I created them on. I then passed my folder out to two of my fellow employees so that they could use the new library of snippets I created. The only issue is the snippets won't work on their machines. They are installing it in /Sublime Text 3/Packages/MySnippetFolder. These snippets are meant to be used in .html files. However they are only showing up in .py files for my coworkers. Scope is commented out in the sublime-snippet files so shouldn't they appear in .html files as well? Any advice is appreciated thank you very much for your time in advance.
Edit: Ok So the snippets are working as intended I believe. The thing I am confused about now is when I use it in a python file a dropdown list appears that contains my snippets. When I try this in my html files there is no drop list containing my snippets. Is there a way to have to drop down list appear in my html files as well like it does for the bootstrap 3/4 snippets package?
Here is an example of the code for one of my snippets.
<snippet>
<content><![CDATA[
<div class="form-group v-spacing-extra-large">
<p>Minimum: <strong>1</strong> | Maximum: <strong>10</strong> | Multiple <small>(Step)</small>: <strong>1</strong>
</p>
<div class="input-stepper-group">
<a aria-label="Decrease Quantity by 1" tabindex="0" data-stepper-role="decrease" data-stepper-target="#demo-stepper-three" href="#" class="btn btn-secondary js-stepper-control disabled"><i aria-hidden="true" class="icon-minus"></i></a>
<input aria-describedby="product-qty-three-stepper-aria-describedby" aria-label="quantity" data-toggle="stepper" data-round="up" id="demo-stepper-three" class="form-control stepper" pattern="[0-9.]+" type="text" data-min="1" data-max="10" data-step="1" value="1">
<a aria-label="Increase Quantity by 1" tabindex="0" data-stepper-role="increase" data-stepper-target="#demo-stepper-three" href="#" class="btn btn-secondary js-stepper-control"><i aria-hidden="true" class="icon-plus"></i></a>
</div>
<span id="product-qty-three-stepper-aria-describedby" class="ada screen-reader-only">Initial Quantity is set to 1. Max Quantity is set to 10 and the step multiple is set to 1.</span>
</div>
]]></content>
<!-- Optional: Set a tabTrigger to define how to trigger the snippet -->
<tabTrigger>ff-stepper-min1-max10-stepby1</tabTrigger>
<!-- Optional: Set a scope to limit where the snippet will trigger -->
<!-- <scope>source.python</scope> -->
</snippet>
To have the dropdown list appear in your html files unless you have another package installed to enable this just type < before your snippet code. So if you have a my-snippet.sublime-snippet , then to use that you would type "<"my-snippet wihtout the "'s and you would see the drop down once you began typing.
Just press CTRL + Space anywhere in the code to get the snippets dropdown.
We have a mindbreeze Installation, which defines 3 Datasources. And with the editor I created a searchmask.
The generated/exported code seems to be plain Mustache-html,
<div data-template="filteredfacet" data-name="datasource/category" role="treeitem" tabindex="-1" aria-expanded="true" aria-labelledby="label7841" style="position: relative;">
<script type="text/x-mustache-template" data-tag-name="span" data-attr-title="{{name}}{{^name?}}{{html}}{{/name?}}">
<span class="{{#showAddAnd?}}mb-add-and-available{{/showAddAnd?}}">
{{{html}}}
{{#showAddAnd?}} <a class="action mb-add-and"><i class="icon-plus-sign"></i></a> {{/showAddAnd?}}
</span>
<span class="pull-right" title="{{count}}">
{{#count?}}{{count}}{{/count?}}{{#excluded}}–{{/excluded}}
</span>
</script>
</div>
Where it only seems possible to change the look and feel.
In the Online Documentation an Cheat-Sheet I could find some metadata, but not the desired.
Does someone have any experience with this? Or some pointer how I can limit a search on only on specific data source.
You can add a constraint for the whole search application. This also allows you to restrict to a specific data source. Just add a results display like "List", "Image Gallery" or "Map":
After adding you can change the option "Restrict the Results" to every query you want. In your case e.g. datasource/category:"Web" does the trick.
I have been using display.none, for the hid-whens for example, all the hidden fields were kept in a section and section was hidden from web by using Display.none.
This is working for Internet Explorer till IE 9, but for IE 10 all the hidden fields are shown.
Can anyone help in this matter. Any alternative or approach.
Without seeing the page it sis very difficult to guess.
Try validating the html through one of the many online html validators as something may not be closed or Notes might have given you an unwanted code addition .
Try adding a background color to the css #wrapper to make sure the css is being called.
Take a copy of the form and start removing all other elements one section at a time to see if something else is causing the issue.
Add {meta http-equiv="X-UA-Compatible" content="IE=10;IE=9; IE=8; IE=7; IE=EDGE" /} as the very top meta tag and see if that fixes it. Replace the curly braces obviously.
All the best in finding the issue.
It sounds like just the section element is getting hidden. Without seeing the code I can't tell why that changes between ie 9 and 10 but ie is famous for having varying behavior between versions.
One alternative that comes to mind: You could wrap the section and the fields in a DIV element using pass thru HTML and set that div's style to display:none. That is pretty standard and should work across browsers.
Update: To give you an idea what I'm talking about, check out this jsfiddle.
HTML:
<form>
<div class="wrapper">
<input type="text" name="Field 1" /><br />
<input type="text" name="Field 2" /><br />
<input type="text" name="Field 3" />
</div>
<span>Some text that won't be hidden.</span>
</form>
CSS:
.wrapper {
#display:none;
}
You can remove the # next to the display:none and see the difference, even in IE 10.
You'll need to look closely at the HTML being rendered by Domino and make sure that in fact all the fields you are trying to hide are surrounded by the DIV that is hidden.
I have an read-only text-field in my site. And I need to enter a date using cucumber and watir. But I can't set any value. Even passing the value of date with value = method I can't input any output. There is a JavaScript calendar.
I wrote:
browser.text_field(:name => "deal[start_date]").value = 'test'
it shows the following error:
Watir::Exception::ObjectReadOnlyException: Watir::Exception::ObjectReadOnlyException
from /var/lib/gems/1.8/gems/watir-webdriver-0.1.7/lib/watir-webdriver/elements/element.rb:252:in `assert_writable'
from /var/lib/gems/1.8/gems/watir-webdriver-0.1.7/lib/watir-webdriver/elements/text_field.rb:24:in `value='
from (irb):10
Please help me, I am in a trouble.
This goes along with what xboxer21 noted. I found this code worked for me on a site that had a similar calendar widget.
Lets say you have an HTML form with input text fields that are set to readonly:
<form name="FindRange" method="post" action="FindRange.asp" onsubmit="return false">
...
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr>
<td colspan="3" class="FieldLabel">
<input type="text" name="FromDate" size="16" readonly="true" style="width:85px; background-color:#F8F6E7;">
</td>
<td>
<input type="text" name="ToDate" size="16" readonly="true" style="width:85px; background-color:#F8F6E7;">
</td>
</tr>
...
</form>
Using code like '#ie.text(:name, "FromDate").set("3/23/2011")' in your Ruby Watir script would result in an error in Ruby stating the field was read-only. But using the eval() method may allow you to use Javascript to set the read-only fields behind the scenes.
#Code above these lines instantiate the Watir object in #ie and navigate to the page #containing the HTML form
#named 'FindRange'
#ie.document.parentWindow.eval("document.FindRange.FromDate.value = '3/23/2011'")
#ie.document.parentWindow.eval("document.FindRange.ToDate.value = '3/24/2011'")
Well, the text field is read only, as the error message says. That means it can not be changed. How would you change the value of the text field without Watir? Can you provide link to the page or relevant HTML?
This is what I did to enter a date in a read only text field, The JS calendar script used was http://www.dynarch.com/projects/calendar/
There is a small icon next to the date field which displays the calendar widget upon clicking it.
browser.image(:id,"datewidget-trigger").click # Will display the Calendar
browser.send_keys("{ENTER}") # Will select current date
If you want to select a future date or previous date
browser.send_keys("{LEFT}")
browser.send_keys("{RIGHT}")
This has been tested using IE only.
Try executing the JavaScript itself.
browser.document.parentWindow.execScript("Date_JS_script('date')")