How to implement different data for cucumber scenarios based on environment - cucumber

I have an issue with executing the cucumber-jvm scenarios in different environments. Data that is incorporated in the feature files for scenarios belongs to one environment. To execute the scenarios in different environemnts, I need to update the data in the features files as per the environment to be executed.
for example, in the following scenario, i have the search criteria included in the feature file. search criteria is valid for lets say QA env.
Scenario: search user with valid criteria
Given user navigated to login page
And clicked search link
When searched by providing search criteria
|fname1 |lname1 |address1|address2|city1|state1|58884|
Then verify the results displayed
it works fine in QA env. But to execute the same scenario in other environments (UAT,stage..), i need to modify search criteria in feature files as per the data in those environments.
I'm thinking about maintaing the data for scenarios in properties file for different environments and read it based on the execution environment.
if data is in properties file, scenario will look like below. Instead of the search criteria, I will give propertyName:
Scenario: search user with valid criteria
Given user navigated to login page
And clicked search link
When searched by providing search criteria
|validSearchCriteria|
Then verify the results displayed
Is there any other way I could maintain the data for scenarios for all environments and use it as per the environment the scenario is getting executed? please let me know.
Thanks

I understand the problem, but I don't quite understand the example, so allow me to provide my own example to illustrate how this can be solved.
Let's assume we test a library management software and that in our development environment our test data have 3 books by Leo Tolstoy.
We can have test case like this:
Scenario: Search by Author
When I search for "Leo Tolstoy" books
Then I should get result "3"
Now let's assume we create our QA test environment and in that environment we have 5 books by Leo Tolstoy. The question is how do we modify our test case so it works in both environments?
One way is to use tags. For example:
#dev_env
Scenario: Search by Author
When I search for "Leo Tolstoy" books
Then I should get result "3"
#qa_env
Scenario: Search by Author
When I search for "Leo Tolstoy" books
Then I should get result "5"
The problem here is that we have lots of code duplication. We can solve that by using Scenario Outline, like this:
Scenario Outline: Search by Author
When I search for "Leo Tolstoy"
Then I should see "<number_of_books>" books
#qa_env
Examples:
| number_of_books |
| 5 |
#dev_env
Examples:
| number_of_books |
| 3 |
Now when you execute the tests, you should use #dev_env tag in dev environment and #qa_env in QA environment.
I'll be glad to hear some other ways to solve this problem.

You can do this in two ways
Push the programming up, so that you pass in the search criteria by the way you run cucumber
Push the programming down, so that your step definition uses the environment to decide where to get the valid search criteria from
Both of these involve writing a more abstract feature that does not specify the details of the search criteria. So you should end up with a feature that is like
Scenario: Search with valid criteria
When I search with valid criteria
Then I get valid results
I would implement this using the second method and write the step definitions as follows:
When "I search with valid criteria" do
search_with_valid_criteria
end
module SearchStepHelper
def search_with_valid_criteria
criteria = retrieve_criteria
search_with criteria
end
def retrieve_criteria
# get the environment you are working in
# use the environment to retrieve the search criteria
# return the criteria
end
end
World SearchStepHelper
Notice that all I have done is change the place where you do the work, from the feature, to a helper method in a module.
This means that as you are doing your programming in a proper programming language (rather than in the features) you can do whatever you want to get the correct criteria.

This may have been answered elsewhere but the team I work with currently tends to prefer pushing environmental-specific pre-conditions down into the code behind the step definitions.
One way to do this is by setting the environment name as an environment variable in the process running the test runner class. An example could be $ENV set to 'Dev'. Then #Before each scenario is tested it is possible verify the environment in which the scenario is being executed and load any environment-specific data needed by the scenario:
#Before
public void before(Scenario scenario) throws Throwable {
String scenarioName = scenario.getName();
env = System.getenv("ENV");
if (env == null) {
env = "Dev";
}
envHelper.loadEnvironmentSpecificVariables();
}
Here we set a 'default' value of 'Dev' in case the test runner is run without the environment variable being set. The envHelper points to a test utility class with the method loadEnvironmentSpecificVariables() that could load data from a JSON, csv, XML file with data specific to the environment being tested against.
An advantage of this approach is that it can help to de-clutter Feature files from potentially distracting environmental meta-data which can impact the readability of the feature outside of the development and testing domains.

Related

How to run cucumber scenario's based on Test Case ID that is appended with the Scenario name?

I wanted to run Cucumber Feature file based on the Test case ID that scanerio name contains.
I know we can use #CucumberOptions 'features' tag and specify the line number to execute e.g "src/test/resources/Folder/myfile.feature:7:12"
This will run scenarios at line 7 and 12. But i wanted to run based on the TC ID.
Below is the feature file code
#Run
Feature: Login Functionality
Scenario: First Test Case(TC.No:1)
Given I perform action 1
Scenario: Second Test Case(TC.No:2)
Given I perform action 2
Scenario: Third Test Case(TC.No:3)
Given I perform action 3
Scenario: Fourth Test Case(TC.No:4)
Given I perform action 4
Scenario: Fifth Test Case(TC.No:5)
Given I perform action 5
All the scenario's are in a single feature.
For the feature file code above i wanted some way through which i can execute based on TC Id. E.g I only want to execute TC1,TC2 and TC5( TC id's picked up from scenario names).
There is a property file that contains the TC Id's to be executed. My code should read the file and then execute only those TC id's.
This can help me in reducing the number of automation TC's to be run.
Is it possible?
You can use the name property of #CucumberOptions or use the '-n' option if you are using the cli option. It also supports regular expressions.
To run TC.No:1 and TC.No:4 use something like this
#CucumberOptions(name = { "TC.No:1|TC.No:4" })
or
#CucumberOptions(name = { "TC.No:1","TC.No:4" })
You can get more details at this link.
As you are reading the ids from a file, the second option is the best. Use the cucumber.api.cli.Main class main() method to execute the features. You can create the options dynamically. Refer to this post.
CLI reference docs.
Not familiar with cucumber-jvm.
But, here is the general logic which should work (based on my ruby Cucumber knowledge)
In the hook, you can write the logic to under before method to get the scenario name scenario.name and then extract the TC.No. Compare the TC.No and skip if it's not part of your list.
Here is the link which will give information how to skip the scenario (use this class in the before method)
https://junit.org/junit4/javadoc/4.12/org/junit/AssumptionViolatedException.html
However, the best practice is to use the tags, it would have been easy if you had #TCId-xx tag. Still you can write a simple program that will scan all the feature files and update the scenarios with the tag based on the TC.No in the scenario name.

How to order control m job using REXX? like Control m utility CTMAPI

I have to order few jobs in control m from different scheduling tables. this is manual task so i want to automate it using rexx.
I found below in 'Order or Force under Batch, REXX or CLIST' section of 'CONTROL M USERGUIDE'
EXEC CTMAPI PARM=‘ORDER variable’
I could not find syntax to call CMTAPI using rexx.
ADDRESS 'LINKMVS' is the equivalent of // EXEC PGM=something,PARM='whatever' in REXX. I don't know what the variable is supposed to be, but since this is Control-M, I am going to assume job name. A very simple example:
say 'Enter name of job'
pull jobname
parmvar = 'ORDER' jobname
`ADDRESS 'LINKMVS' 'CTMAPI parmvar'
Please note that for LINKMVS, the variable name goes inside the string passed. The LINKMVS environment substitutes the variable automatically. For example, if I entered MYJOB to the prompt, LINKMVS will build a PARM string of `ORDER MYJOB'. This is the exact equivalent of
// EXEC PGM=CTMAPI,PARM='ORDER MYJOB'
This IBM® Knowledge Center page for the z/OS 2.3 TSO/E REXX Reference manual shows several examples of calling a program in the same manner as // EXEC PGM=,PARM= (item 1). Items 5 through 9 show different ways of using ADDRESS 'LINKMVS'; note how variables are treated in each example.
After suggestions from NicC, zarchasmpgmr and few research, finally i am able to order job with CTMJOB utility. I searched for the loadlib and called TSO using REXX.
/*****REXX*******/
ADDRESS TSO
"CALL 'MY.IN.LOAD(CTMJOB)'
' ORDER DSN=MY.SCHED.LIB TABLE=SCHDTBL,
JOB=JOBNAME,DATE=DATE'"
EXIT
Details found in INCONTROL for ZOS utilities guide. This document was very useful.
http://documents.bmc.com/supportu/952/56/64/195664/195664.pdf

2 Sequential Transactions, setting Detail Number (Revit API / Python)

Currently, I made a tool to rename view numbers (“Detail Number”) on a sheet based on their location on the sheet. Where this is breaking is the transactions. Im trying to do two transactions sequentially in Revit Python Shell. I also did this originally in dynamo, and that had a similar fail , so I know its something to do with transactions.
Transaction #1: Add a suffix (“-x”) to each detail number to ensure the new numbers won’t conflict (1 will be 1-x, 4 will be 4-x, etc)
Transaction #2: Change detail numbers with calculated new number based on viewport location (1-x will be 3, 4-x will be 2, etc)
Better visual explanation here: https://www.docdroid.net/EP1K9Di/161115-viewport-diagram-.pdf.html
Py File here: http://pastebin.com/7PyWA0gV
Attached is the python file, but essentially what im trying to do is:
# <---- Make unique numbers
t = Transaction(doc, 'Rename Detail Numbers')
t.Start()
for i, viewport in enumerate(viewports):
setParam(viewport, "Detail Number",getParam(viewport,"Detail Number")+"x")
t.Commit()
# <---- Do the thang
t2 = Transaction(doc, 'Rename Detail Numbers')
t2.Start()
for i, viewport in enumerate(viewports):
setParam(viewport, "Detail Number",detailViewNumberData[i])
t2.Commit()
Attached is py file
As I explained in my answer to your comment in the Revit API discussion forum, the behaviour you describe may well be caused by a need to regenerate between the transactions. The first modification does something, and the model needs to be regenerated before the modifications take full effect and are reflected in the parameter values that you query in the second transaction. You are accessing stale data. The Building Coder provides all the nitty gritty details and numerous examples on the need to regenerate.
Summary of this entire thread including both problems addressed:
http://thebuildingcoder.typepad.com/blog/2016/12/need-for-regen-and-parameter-display-name-confusion.html
So this issue actually had nothing to do with transactions or doc regeneration. I discovered (with some help :) ), that the problem lied in how I was setting/getting the parameter. "Detail Number", like a lot of parameters, has duplicate versions that share the same descriptive param Name in a viewport element.
Apparently the reason for this might be legacy issues, though im not sure. Thus, when I was trying to get/set detail number, it was somehow grabbing the incorrect read-only parameter occasionally, one that is called "VIEWER_DETAIL_NUMBER" as its builtIn Enumeration. The correct one is called "VIEWPORT_DETAIL_NUMBER". This was happening because I was trying to get the param just by passing the descriptive param name "Detail Number".Revising how i get/set parameters via builtIn enum resolved this issue. See images below.
Please see pdf for visual explanation: https://www.docdroid.net/WbAHBGj/161206-detail-number.pdf.html

Using indexed types for ElasticSearch in Titan

I currently have a VM running Titan over a local Cassandra backend and would like the ability to use ElasticSearch to index strings using CONTAINS matches and regular expressions. Here's what I have so far:
After titan.sh is run, a Groovy script is used to load in the data from separate vertex and edge files. The first stage of this script loads the graph from Titan and sets up the ES properties:
config.setProperty("storage.backend","cassandra")
config.setProperty("storage.hostname","127.0.0.1")
config.setProperty("storage.index.elastic.backend","elasticsearch")
config.setProperty("storage.index.elastic.directory","db/es")
config.setProperty("storage.index.elastic.client-only","false")
config.setProperty("storage.index.elastic.local-mode","true")
The second part of the script sets up the indexed types:
g.makeKey("property").dataType(String.class).indexed("elastic",Edge.class).make();
The third part loads in the data from the CSV files, this has been tested and works fine.
My problem is, I don't seem to be able to use the ElasticSearch functions when I do a Gremlin query. For example:
g.E.has("property",CONTAINS,"test")
returns 0 results, even though I know this field contains the string "test" for that property at least once. Weirder still, when I change CONTAINS to something that isn't recognised by ElasticSearch I get a "no such property" error. I can also perform exact string matches and any numerical comparisons including greater or less than, however I expect the default indexing method is being used over ElasticSearch in these instances.
Due to the lack of errors when I try to run a more advanced ES query, I am at a loss on what is causing the problem here. Is there anything I may have missed?
Thanks,
Adam
I'm not quite sure what's going wrong in your code. From your description everything looks fine. Can you try the follwing script (just paste it into your Gremlin REPL):
config = new BaseConfiguration()
config.setProperty("storage.backend","inmemory")
config.setProperty("storage.index.elastic.backend","elasticsearch")
config.setProperty("storage.index.elastic.directory","/tmp/es-so")
config.setProperty("storage.index.elastic.client-only","false")
config.setProperty("storage.index.elastic.local-mode","true")
g = TitanFactory.open(config)
g.makeKey("name").dataType(String.class).make()
g.makeKey("property").dataType(String.class).indexed("elastic",Edge.class).make()
g.makeLabel("knows").make()
g.commit()
alice = g.addVertex(["name":"alice"])
bob = g.addVertex(["name":"bob"])
alice.addEdge("knows", bob, ["property":"foo test bar"])
g.commit()
// test queries
g.E.has("property",CONTAINS,"test")
g.query().has("property",CONTAINS,"test").edges()
The last 2 lines should return something like e[1t-4-1w][4-knows-8]. If that works and you still can't figure out what's wrong in your code, it would be good if you can share your full code (e.g. in Github or in a Gist).
Cheers,
Daniel

How can I make cucumber run all the steps (not skip them) even if one of them fails?

I am using Cucumber with RubyMine, and I have a scenario with steps that verify some special controls from a form (I am using cucumber for automation testing). The controls don't have anything to do with each other, and there is no reason for the steps to be skipped if one in front of them fails.
Does anyone know what configurations or commands should I use to run all the steps in a scenario even if they all fail?
I think the only way to achieve desired behavior (which is quite uncommon) is to define custom steps and catch exceptions in it yourself. According to cucumber wiki step is failed if it raises an error. Almost all default steps raise error if they can't find or interact with an element on the page. If you'll catch this exceptions the step will be marked as passed, but in rescue you can provide custom output. Also I recommend you to carefully define exceptions you want to catch, I think if you're Ok if selenium can't find an element on the page rescue only from ElementNotFound exceptions, don't catch all exceptions.
I've seen a lot of threads on the Web about people wanting to continue steps execution if one failed.
I've discussed with Cucumber developers: they think this is a bad idea: https://groups.google.com/forum/#!topic/cukes/xTqSyR1qvSc
Many times, scenarios can be reworked to avoid this need: scenarios must be split into several smaller and independent scenarios, or several checks can be aggregated into one, providing a more human scenario and a less script-like scenario.
But if you REALLY need this feature, like our project do, we've done a fork of Cucumber-JVM.
This fork let you annotate steps so that when they fail with a determined exception, they will let let next steps execute anyway (and the step itself is marked as failed).
The fork is available here:
https://github.com/slaout/cucumber-jvm/tree/continue-next-steps-for-exceptions-1.2.4
It's published on the OSSRH Maven repository.
See the README.md for usage, explanation screenshot and Maven dependency.
It's only available for the Java language, tough: any help is welcome to adapt the code to Ruby, for instance. I don't think it will be a lot of work.
The question is old, but hopefully this will be helpful. What I'm doing feels kind of "wrong", but it works. In your web steps, if you want to keep going, you have to catch exceptions. I'm doing that primarily to add helpful failure messages. I'm checking a table full of values that are identified in Cucumber with a table having a bunch of rows like:
Then my result should be:
| Row Identifier | Column Identifier | Subcolum Identifier | $1,247.50 |
where the identifiers make sense in the application domain, and name a specific cell in the results table in a human-friendly way. I have helpers that convert the human identifiers to DOM IDs, which are used to first check whether the row I'm looking for exists at all, then look for the specific value in a cell in that row. The default failure message for a missing row is clear enough for me (expected to find css "tr#my_specific_dom_id" but there were no matches). But the failure message for checking specific text in a cell is completely unhelpful. So I made a step that catches the exception and uses the Cucumber step info and some element searching to get a good failure message:
Then /^my application domain results should be:$/ do |table|
table.rows.each do |row|
row_id = dom_id_for(row[0])
cell_id = dom_id_for(row[0], row[1], row[2])
page.should have_css "tr##{row_id}"
begin
page.should have_xpath("//td[#id='#{cell_id}'][text()=\"#{row[3].strip.lstrip}\"]")
rescue Capybara::ExpectationNotMet => exception
# find returns a Capybara::Element, native returns a Selenium::WebDriver::Element
contents = find(:xpath, "//td[#id='#{cell_id}']").native.text
puts "Expected #{ row[3] } for #{ row[0,2].join(' ') } but found #{ contents } instead."
#step_failures_were_rescued = true
end
end
end
Then I define a hook in features/support/hooks.rb like:
After do |scenario|
unless scenario.failed?
raise Capybara::ExpectationNotMet if #step_failures_were_rescued
end
end
This makes the overall scenario fail, but it masks the step failure from Cucumber, so all the step results are green, including the ones that aren't right. You have to see the scenario failure, then look back at the messages to see what failed. This seems kind of "bad" to me, but it works. It's WAY more convenient in my case to get the expected and found values listed in a domain-friendly context for the whole table I'm checking, rather than to get a message like "I looked for "$123.45" but I couldn't find it." There might be a better way to do this using the Capybara "within" method. This is the best I've come up with so far though.

Resources