How can I make cucumber run all the steps (not skip them) even if one of them fails? - cucumber

Question

I am using Cucumber with RubyMine, and I have a scenario with steps that verify some special controls from a form (I am using cucumber for automation testing). The controls don't have anything to do with each other, and there is no reason for the steps to be skipped if one in front of them fails.
Does anyone know what configurations or commands should I use to run all the steps in a scenario even if they all fail?

Answer 1

I think the only way to achieve desired behavior (which is quite uncommon) is to define custom steps and catch exceptions in it yourself. According to cucumber wiki step is failed if it raises an error. Almost all default steps raise error if they can't find or interact with an element on the page. If you'll catch this exceptions the step will be marked as passed, but in rescue you can provide custom output. Also I recommend you to carefully define exceptions you want to catch, I think if you're Ok if selenium can't find an element on the page rescue only from ElementNotFound exceptions, don't catch all exceptions.

Answer 2

I've seen a lot of threads on the Web about people wanting to continue steps execution if one failed.
I've discussed with Cucumber developers: they think this is a bad idea: https://groups.google.com/forum/#!topic/cukes/xTqSyR1qvSc
Many times, scenarios can be reworked to avoid this need: scenarios must be split into several smaller and independent scenarios, or several checks can be aggregated into one, providing a more human scenario and a less script-like scenario.
But if you REALLY need this feature, like our project do, we've done a fork of Cucumber-JVM.
This fork let you annotate steps so that when they fail with a determined exception, they will let let next steps execute anyway (and the step itself is marked as failed).
The fork is available here:
https://github.com/slaout/cucumber-jvm/tree/continue-next-steps-for-exceptions-1.2.4
It's published on the OSSRH Maven repository.
See the README.md for usage, explanation screenshot and Maven dependency.
It's only available for the Java language, tough: any help is welcome to adapt the code to Ruby, for instance. I don't think it will be a lot of work.

Answer 3

The question is old, but hopefully this will be helpful. What I'm doing feels kind of "wrong", but it works. In your web steps, if you want to keep going, you have to catch exceptions. I'm doing that primarily to add helpful failure messages. I'm checking a table full of values that are identified in Cucumber with a table having a bunch of rows like:
Then my result should be:
| Row Identifier | Column Identifier | Subcolum Identifier | $1,247.50 |
where the identifiers make sense in the application domain, and name a specific cell in the results table in a human-friendly way. I have helpers that convert the human identifiers to DOM IDs, which are used to first check whether the row I'm looking for exists at all, then look for the specific value in a cell in that row. The default failure message for a missing row is clear enough for me (expected to find css "tr#my_specific_dom_id" but there were no matches). But the failure message for checking specific text in a cell is completely unhelpful. So I made a step that catches the exception and uses the Cucumber step info and some element searching to get a good failure message:
Then /^my application domain results should be:$/ do |table|
table.rows.each do |row|
row_id = dom_id_for(row[0])
cell_id = dom_id_for(row[0], row[1], row[2])
page.should have_css "tr##{row_id}"
begin
page.should have_xpath("//td[#id='#{cell_id}'][text()=\"#{row[3].strip.lstrip}\"]")
rescue Capybara::ExpectationNotMet => exception
# find returns a Capybara::Element, native returns a Selenium::WebDriver::Element
contents = find(:xpath, "//td[#id='#{cell_id}']").native.text
puts "Expected #{ row[3] } for #{ row[0,2].join(' ') } but found #{ contents } instead."
#step_failures_were_rescued = true
end
end
end
Then I define a hook in features/support/hooks.rb like:
After do |scenario|
unless scenario.failed?
raise Capybara::ExpectationNotMet if #step_failures_were_rescued
end
end
This makes the overall scenario fail, but it masks the step failure from Cucumber, so all the step results are green, including the ones that aren't right. You have to see the scenario failure, then look back at the messages to see what failed. This seems kind of "bad" to me, but it works. It's WAY more convenient in my case to get the expected and found values listed in a domain-friendly context for the whole table I'm checking, rather than to get a message like "I looked for "$123.45" but I couldn't find it." There might be a better way to do this using the Capybara "within" method. This is the best I've come up with so far though.