What is the difference between Cucumber and QAF bdd, What are the features which QAF has which is not available in Cucumber and what are the features which are available in Cucumber and not available in QAF Bdd.
Let's first take a look at BDD2 and Gherkin syntax:
Gherkin is syntax that cucumber (and QAF as well) understands. BDD2 is superset of Gherkin syntax. That means BDD2 supports all gherkin keywords and structure and have additional provisions. Below are addition provision in BDD2 syntax:
Metadata support
Provision to have external data file for examples
Parameter support
Below scenario authored in BDD2:
#smoke #TestcaseId:12345
Scenario: A scenario is a collection of executable steps of different type
Given step represents a precondition to an event
When step represents the occurrence of the event
Then step represents the outcome of the event
See more examples of BDD2.
BDD2 looks like Gherkin and any gherkin editor can understands it as well. But it depends on runner to understand and use additional features. You can use BDD2 either with QAF or Cucumber. When using cucumber QAF-Cucumber will enable cucumber runner to get benefit of addition syntax feature.
Now let's take look on features of Cucumber and QAF.
Both are open source frameworks. Cucumber is more popular and well known while QAF is widely used by different enterprises but not as much popular as compared to cucumber.
Cucumber is dedicated for BDD implementation using Gherkin.
Where as QAF supports coded, keyword driven and BDD implementation using TestNG. QAF supports QAF BDD and advance QAF BDD2 syntax in addition to Gherkin.
Till cucumber 4, java implementation of cucumber was not thread-safe
but with cucumber-4 entire architecture of implementation get changed
and it started supporting parallel execution with thread safety.
QAF formally known as ISFW was thread safe from beginning and supports scenario level parallel execution.
Cucumber is Gherkin runner can be used with Junit, TestNG or standalone. Supports only gherkin syntax. Cucumber can run BDD2 QAF-Cucumber
QAF is Built upon TestNG and have native TestNG implementation for BDD. You can use cucumber runner by using QAF-Cucumber
Cucumber doesn't have inbuilt implementation for testing of
web,mobile, web-service. Either you need to write your own implementation or use third party framework that supports cucumber.
using cucumber for different black box testing needs.
QAF has inbuilt Web, Mobile, WebServices support with different
design concepts along with resource and test data management, integration with third party tools (like jira, ALM, QMetry, Rally etc).
Cucumber supports for different programing languages.
QAF is Java only.
Cucumber supports data-driven testing by use of Examples in gherkin
syntax.
QAF supports data-driven testing by use of Examples in Gherkin
or BDD2 syntax. It also supports External data providers and
data-provider interceptor.
Cucumber has hooks for scenario lifecyle.
QAF has support for all TestNG listeners with additional step listener. In addition to that it has support for webdriver listener, webelement listener, result updator and data-provider interceptor.
Till cucumber 4, Step implementation was required to use regular
expressions. With Cucumber 4 new way similar to QAF available to get
rid of regular expression in step definition.
QAF uses simple and easy way to provide description of step.
Cucumber have predefined Parameter Type support as method arguments. It also supports custom types. For any custom type you need to implement and configure type registry. User need to use defined parameter types in step definition.
QAF natively supports all complex types including Map, List, and any custom classes. It also has transformation support to customize parameter transformation behavior. In step description user is free to use meaningful name for parameter placeholder regardless of method argument type.
Cucumber doesn't support step definition in BDD.
QAF supports new step definition in BDD as well
Related
Today I've got some theoretical question. I have a little experience in BDD with Cucumber. Now I started working in API testing and I faced with Serenity BDD framework on my new project. It is used with Cucumber. So I would like to clarify what is the difference between Serenity BDD Framework and Cucumber and why can't we use just Cucumber for our tests, because it seems to me, like these two tools are doing the same job. Could anybody give me some explanation or maybe some link to correspondent documentation. Thanks in advance!
No, those are 2 different things.
Cucumber is the layer to map BDD syntax, which is written in .feature file, with actual code that does the job.
Serenity BDD is the framework supporting 3 different approaches:
Cucumber: same features as stand-alone cucumber, can work with UI or API Automation
Page Object: works with UI automation (selenium)
Screenplay: a design pattern for UI and API automation
Serenity BDD does many things for auto testers that:
Config-oriented: serenity.properties or serenity.conf. For example: take screenshot when FOR_EACH_ACTION, BEFORE_AND_AFTER_EACH_STEP, AFTER_EACH_STEP, FOR_FAILURES, DISABLED. This is really helpful for debugging. https://serenity-bdd.github.io/theserenitybook/latest/serenity-system-properties.html#_serenity_take_screenshots
Living report: much more better comparing to cucumber. https://serenity-bdd.github.io/theserenitybook/latest/living-documentation.html
Wrap other libs in nice and clean APIs: serenity-appium, serenity-browserstack, serenity-cucumber, serenity-rest-assured, serenity-saucelabs, serenity-shutterbug1x ... You see all of them at https://github.com/serenity-bdd/serenity-core
I am able to run the Cucumber/groovy (with maven/pom.xml) test locally. I'm trying to import the test results (results.jon) generated by cucumber to Jira with Xray.
I'm unable to find the steps or procedures required for this. The only source found: https://confluence.xpand-it.com/display/public/XRAY/Import+Execution+Results
but not applicable to my project. There is no CICD yet at the moment. Is there any way to generate results/reports imported to Jira every time I run a test/multiple tests?
For Gherkin-based frameworks, such as Cucumber, you can't simply submit the results. This is because Xray needs to have the Gherkin phrases, which cannot be inferred from the results file.
So you need to choose one of the possible cucumber flows. You need to either choose Xray as the master for the edition of Cucumber scenarios, or you need to use Git/SVN for that and then synch them to Xray.
These steps are detailed in the previous link.
You can see some useful tutorials here.
There are some cucumber specific tutorials, such as this one, but they're not fully detailed. You can see a more technically detailed tutorial for Serenity BDD that make those steps more visible, for the two different flows (you'll need then to adapt it to cucumber specifics, but the principles are the same).
Can anyone please help me in understanding what is difference or advantages of using Cucumber+TestNG over Cucumber+Junit ?
TestNG provides more Annotations Like #BeforeSuite .
TestNG it self provides support for Junit.
With Latest upgrade of cucumber(4.0) Cucumber - TestNg provides Parallel Execution at Scenario Level where as Junit provides at Feature level.
TestNG and Junit are just a unit testing frameworks, when you use Cucumber with Junit you can use Junit Annotation/Assertions in step definition file and similarly when you use Cucumber with TestNG you can use TestNG Annotation/Assertions in step definition file.
Actually in both cases you would be writing the Test Steps using selenium and probably will use the classes inside JUnit or TestNG to run the cucumber Feature file.
As per my knowledge the only place I used such file was the when I'm creating the TestRunner.java class.
It's much easier to do with JUnit since you just have to import org.junit.runner.RunWith But if you are using TestNG you'll need to use dataProviders and go on a bit more complex path to create the TestRunner Class
Other than that there won't be any other differences since you will be using cucumber features for testing.
Due to the above mentioned advantage, I recommend to use JUnit + Cucumber + Selenium for your scripting
I tried both but decided for JUNIT 4.12 (JUNIT 5 does not seem to work with cucumber yet). It has all that you need. But i have to test that parallel thing in Test NG i guess. ;)
I use Cucumber in a Spring Boot Project and therefore JUNIT did not cause any problems for me. I can hand over a browser instance over multiple classes here (#Autowired) and this makes my code so much more modularizable.
You may have more annotation-options with Test NG but somehow i have the feeling that more people work with JUnit, so you have more support and a bigger community here.
At the moment, I'm using two different frameworks for REST APIs integration testing, and load/stress testing. Respectively : geb (or cucumber) and gatling. But most of the time, I'm re-writing some pieces of code in load / performance scenarii that I've been writing for integration testing.
So the question is : is there a framework (running on the JVM) or simply a way, to write integration tests (for a strict REST API use case), preferably programmatically, then assemble load testing scenarios using these integration tests.
I've read cucumber maybe could do that, but I'm lacking a proper example.
The requirements :
write integration tests programmatically
for any integration test, have the ability to "extract" values (the same way gatling can extract json paths for instance)
assemble the integration tests in a load test scenario
If anyone has some experience to share, I'd be happy to read any blog article, GitHub repository, or whatever source dealing with such an approach.
Thanks in advance for your help.
It sounds like you want to extract a library that you use both for your integration tests as well as your load test.
Both tools you are referring to are able to use external jar.
Suppose that you use Maven or Gradle as build tool, create a new module that you refer to from both your integration tests and your load tests. Place all interaction logic in this new module. This should allow you to reuse the code you need.
I'm trying to find possibilities for the BDD approach of testing my web services with SOAP UI & Cucumber. Is there any way to achieve this ? Please advise
Thanks
Exactly, both SOAPUI and Cucumber are different test runners. You have to choose between both. Except that SOAPUI provides you http-clients for calling services and Cucumber can contain any tests that use different libraries/clients to do different type of testing.
Presuming that you are thinking to use SoapUI just as http-client for services testing, its better to use Jersey client and use Cucumber/JBehave as BDD runner.
I Do Agree with #I Am's answer. Using Jersey or other httpclient do the needful. In case of jersy with BDD you can use QAF with Web service plugin. You can use cucumber gherkin or QAF BDD or pure java for test authoring. QAF bdd for web-service may look like below:
SCENARIO: My Fist webservice call
META-DATA: {"description":"Just for learning purpose"}
When user requests '${get.sample.call}'
Then response should have status 'OK'
END
Where, get.sample.call will be in properties file holding request details.