I'm writing a rails app which fetches text from an HTML page using Watir and Chrome Headless. All good so far!
The problem starts when I request a page which has a long load time to completely load all elements despite the fact that I don't need them.
Current code I use:
browser = Watir::Browser.new :chrome, headless: true
browser.goto(url)
The .goto function call, however, blocks until ALL elements have loaded. That's not really what I need - what I need is for goto to just start fetching the page, then continue running code since I really just want to wait until the text I need is present, then fetch it.
Any ideas?
Goto will not leave the control until 60 seconds, If page load time exceeds 60 seconds, then it would throw the error. And also Watir.default_timeout has nothing to do with Goto's page loading. You need to set the timings for page_load which you can do by directly calling selenium driver as I have done below because Watir hasn't offered any systax for that
Write the below code, you could achieve what you want
begin
b.driver.manage.timeouts.page_load=5
b=Watir::Browser.new
b.goto(url)
rescue #I have written the rescue block here because goto will the error for you If page is not loaded within a given time
end
AND THEN you can write your rest of the code here, for an example,
puts b.span(text: 'something').text
What happens here is, goto will be block the execution of the code followed by goto for 5 seconds, and then it would fall into the rescue block, so program would continue to execute next line as you expected.
With the new w3c webdriver specification, you can set the page load strategy to 'none.' https://w3c.github.io/webdriver/webdriver-spec.html#navigation
Only Firefox and IE might have this implemented already.
Related
I'm working on a project which uses nodejs and nighwatch for test automation. The problem here is that the tests are not reliable and give lots of false positives. I did everything to make them stable and still getting the errors. I went through some blogs like https://bocoup.com/blog/a-day-at-the-races and did some code refactoring. Did anyone have some suggestions to solve this issue. At this moment I have two options, either I rewrite the code in Java(removing nodejs and nightwatch from solution as I'm far more comfortable in Java then Javascript. Most of the time, struggle with the non blocking nature of Javascript) or taking snapshots/reviewing app logs/run one test at a time.
Test environment :-
Server -Linux
Display - Framebuffer
Total VM's -9 with selenium nodes running the tests in parallel.
Browser - Chrome
Type of errors which I get is element not found. Most of the time the tests fail as soon the page is loaded. I have already set 80 seconds for timeout so time can't be issue. The tests are running in parallel but on separate VM's so I don't know whether it can be issue or not.
Edit 1: -
Was working on this to know the root cause. I did following things to eliminate random fails: -
a. Added --suiteRetries to retry the failed cases.
b. Went through the error screenshot and DOM source. Everything seems fine.
c. Replaced the browser.pause with explicit waits
Also while debugging I observed one problem, maybe that is the issue which is causing random failures. Here's the code snippet
for (var i = 0; i < apiResponse.data.length; i++) {
var name = apiResponse.data[i];
browser.useXpath().waitForElementVisible(pageObject.getDynamicElement("#topicTextLabel", name.trim()), 5000, false);
browser.useCss().assert.containsText(
pageObject.getDynamicElement("#topicText", i + 1),
name.trim(),
util.format(issueCats.WRONG_DATA)
);
}
I added the xpath check to validate if i'm waiting enough for that text to appear. I observed that visible assertion is getting passed but in next assertion the #topicText is coming as previous value or null.This is an intermittent issue but on test server happens frequently.
There is no magic bullet to brittle UI end to end tests. In the ideal world there would be an option set avoid_random_failures=true that would quickly and easily solve the problem, but for now it's only a dream.
Simple rewriting all tests in Java will not solve the problem, but if you feel better in java, then I would definitely go in that direction.
As you already know from this article Avoiding random failures in Selenium UI tests there are 3 commonly used avoidance techniques for race conditions in UI tests:
using constant sleep
using WebDriver's "implicit wait" parameter
using explicit waits (WebDriverWait + ExpectedConditions + FluentWait)
These techniques are also briefly mentioned on WebDriver: Advanced Usage, you can also read about them here: Tips to Avoid Brittle UI Tests
Methods 1 and 2 are generally not recommended, they have drawbaks, they can work well on simple HTML pages, but they are not 100% realiable on AJAX pages, and they slow down the tests. The best one is #3 - explicit waits.
In order to use technique #3 (explicit waits) You need to familiarize yourself and be comfortable with the following WebDriver tools (I point to theirs java versions, but they have their counterparts in other languages):
WebDriverWait class
ExpectedConditions class
FluentWait - used very rarely, but very usefull in some difficult cases
ExpectedConditions has many predefinied wait states, the most used (in my experience) is ExpectedConditions#elementToBeClickable which waits until an element is visible and enabled such that you can click it.
How to use it - an example: say you open a page with a form which contains several fields to which you want to enter data. Usually it is enought to wait until the first field appears on the page and it will be editable (clickable):
By field1 = By.xpath("//div//input[.......]");
By field2 = By.id("some_id");
By field3 = By.name("some_name");
By buttonOk = By.xpath("//input[ text() = 'OK' ]");
....
....
WebDriwerWait wait = new WebDriverWait( driver, 60 ); // wait max 60 seconds
// wait max 60 seconds until element is visible and enabled such that you can click it
// if you can click it, that means it is editable
wait.until( ExpectedConditions.elementToBeClickable( field1 ) ).sendKeys("some data" );
driver.findElement( field2 ).sendKeys( "other data" );
driver.findElement( field3 ).sendKeys( "name" );
....
wait.until( ExpectedConditions.elementToBeClickable( buttonOK)).click();
The above code waits until field1 becomes editable after the page is loaded and rendered - but no longer, exactly as long as it is neccesarry. If the element will not be visible and editable after 60 seconds, then test will fail with TimeoutException.
Usually it's only necessary to wait for the first field on the page, if it becomes active, then the others also will be.
I have a step like this:
Then(/^I can see the Eligible Bugs list "([^"]*)"$/) do |bugs|
bugs_list = bugs.split(", ")
assert page.has_css?(".eligible-bugs", :visible => true)
within(".eligible-bugs") do
bugs_list.each do |bug|
assert page.has_content?(bug)
end
end
end
But the step fail sometimes at the " within(".eligible-bugs") do" with the error 'Unable to find css ".eligible-bugs"'
I feel it is odd.
for the assertion has been passed. it means the css is visible.
why within cannot find css? How it happen.
But the step fail sometimes at the " within(".eligible-bugs") do" with the error 'Unable to find css ".eligible-bugs"'
I feel it is odd.
for the assertion has been passed. it means the css is visible.
why within cannot find css? How it happen.
BTW, I have set my max wait time to 5.
Capybara.default_max_wait_time = 5
The only way that should happen, is if the page is dynamically changing while you're running the test - sometime during your check for all bugs content on the page it is changing and the '.eligible-bugs' element is going way. The test and the browser run separately, so how/why it is happening depends on what else your page is doing in the browser, it would also depend on what steps have come before this in the test.
Also, note that it's not necessarily disappearing between the has_css? and the within statement first running. If it disappears at any point during the code inside the within running it could throw the same error as it attempts to reload the '.eligible-bugs' element.
From the title of the question I assume the list that you want to check is the result of a search or filtering action. If it is, does that action remove the existing '.eligible-bugs' element and then after some time replace it with a new one returned from an ajax request or something? If that is the case then, since you control the test data, you should be waiting for the correct results count to show, thereby ensuring any element replacements have completed, before checking for the text. How you do that would depend on the exact makeup of the page, but if each eligible bug was a child of '.eligible-bugs' and had a class of '.eligible-bug' I would write your test something like
Then(/^I can see the Eligible Bugs list "([^"]*)"$/) do |bugs|
bugs_list = bugs.split(", ")
assert_css(".eligible-bugs > .eligible_bug", count: bugs_list.size) # wait until the expected number of results have shown up
within(".eligible-bugs") do
bugs_list.each do |bug|
assert_content(bug)
end
end
end
When I close the browser watir has been using (or hit ctrl+C and close it out that way), I get these types of errors...
b = Watir::Browser.new
#ctrl+C and closes browser window
b.url
Errno::ECONNREFUSED: Connection refused - connect(2)
...when I run something like #url or #goto on the watir object.
I tried methods like #closed? and #closed on the object, but they aren't recognized. Have also tried this:
b.methods - Object.methods
and perused the available methods but nothing has worked so far.
Short of responding to raised error messages with begin/rescue blocks, what methods can I use to determine if the browser window is closed?
A bonus would be a method that would allow me to reinitialize the browser window, keeping any preferences picked up along the way. That would be the best possible solution.
Use exists? method to check whether browser is closed or not, browser.exists? would true if browser is open, If not, it would return false.
Going to answer my own question here because I'm not sure if a straightforward answer is possible. Monkey-patching time.
class Watir::Browser
def try(meth)
begin
self.send(meth.to_sym)
true
rescue
false
end
end
def open?
if try(:exists?)
exists?
else
false
end
end
end
Works no manner which way you closed your browser window.
I may submit a pull request
The correct way to close the browser is browser.quit or browser.close
Then you can call browser.exists? and it will return false. Sending ctrl+C exits your session entirely, so there is no longer anything listening at the url/port you are making calls to, and you will always get that error.
Is there any way we can slow down the execution of Watir WebDriver under Cucumber?
I would like to visually track the actions performed by Watir. At the moment, it goes too fast for my eyes.
While Watir itself does not have an API for slowing down the execution, you could use the underlying Selenium-WebDriver's AbstractEventListener to add pauses before/after certain types of actions.
Given you want to see the result of actions, you probably want to pause after changing values and clicking elements. This would be done by creating the following AbstractEventListener and passing it in when creating the browser:
class ActionListener < Selenium::WebDriver::Support::AbstractEventListener
def after_change_value_of(element, driver)
sleep(5)
end
def after_click(element, driver)
sleep(5)
end
end
browser = Watir::Browser.new :firefox, :listener => ActionListener.new
For a full list of events that you can listen for, see the
Selenium::WebDriver::Support::AbstractEventListener documentation.
Not universally. You could Monkey Patch the element_call method to add a sleep after every interaction with a Selenium Element. Import this code after requiring watir-webdriver.
module Watir
class Element
alias_method :watir_element_call, :element_call
def element_call &block
watir_element_call &block
sleep 1
end
end
end
Also note, that Monkey Patching is generally a bad idea, and when I change the implementation (which I plan to), this code will break.
I know that there are several questions related to implementation of waiting and timeouts in Watir, however I have not found an answer to my problem (which must be common). I use Watir-webdriver for testing of a page that due to AJAX implementation loads portion-by-portion for very long time (more than 5 min). I need to be able just to sample this page for a limited time (20-40 sec) and to be able to analyze the information that is loaded during this short time. However, as I know, there is no straightforward direct mechanism to tell Watir::Browser to stop. I can use the Timeout, but although my script gets the control after rescue, it is impossible to interrogate the browser and verify the information that it is able to received during the timeout window. All I can do at this point is to kill the process and restart the browser as discussed here: Make headless browser stop loading page and elsewhere.
The code below illustrates my situation. In this example I have a global timeout (30 sec) and a local timeout (15 sec) used for reading the page. It never gets to b.text call; the script just outputs the first exception after 15 sec and then it keeps waiting for the browser to be released and after the global timeout of 30 sec prints the second exception message.
Time out. Got into exception branch
Dropped to bottom rescue.
The end.
I also tried to send an 'escape' key to the browser, but any communication with it while it is in the goto method is impossible. Any tips and suggestions will be appreciated!
require 'watir-webdriver'
require 'timeout'
client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 30 # Set the global timeout
b = Watir::Browser.new :chrome, :http_client => client
my_url = '...here is my address...'
begin
begin
Timeout::timeout(15) { b.goto my_url } # Access the page with local timeout
b.close # if all is unbelievably good and the page is loaded
rescue Exception => e
puts 'Time out. Got into exception branch'
if b.text.include? 'my_text' # NEVER GETS HERE
puts 'Yes, I see the text!'
else
puts 'I do not see the text.'
end
end
rescue Exception => e
puts 'Dropped to bottom rescue.'
end
puts 'The end.'
Watir relies on Selenium WebDriver to handle calls to the browser. At this time all browsers require that the document.readyState of the current frame return "complete" before returning control to your code.
A recent update to the webdriver specification appears to allow for the possibility of a browser driver implementing a page loading strategy that is not blocking, but it is not a requirement and is not supported at this time.
https://w3c.github.io/webdriver/webdriver-spec.html#the-page-load-strategy