Cucumber test cases fail when run together but pass when run individually - cucumber

I am using SpringBoot with Cucumber. One particular test case fails when run together and passes when run individually. Other Scenarios in the same feature file seems to work fine. I tried the following links but they did not work.
link1, link2, link3, link4, link5
I also tried to quarantine the problematic Scenario to a separate feature file and wrote an exclusive step def for the feature, but still no luck.
I tried introducing #After and #Before hooks to reset the value of Instance variable in stepdef file to null.
I even tried switching the position of Sample-payload and id Count Set
Now I ran out of options. Please help.
Feature File:
Scenario Outline: To check if the service returns all the ids when more than 10000 Ids are returned by the query
Given sample request <payload>
When the new end point is invoked
Then the service returns <idCount> Ids
Examples:
| payload | idCount |
| sample-payload-1 | 17575 |
| sample-payload-2 | 4 |
| sample-payload-3 | 23535 |
| sample-payload-4 | 34535 |
Step Def File:
public class MyStepdefs extends AbstractStepsDefs {
#Autowired
MyController myController;
private String requestPayload;
private List<String> ids;
#Given("^sample request (.+)$")
public void sample_request_something(String payload) throws Throwable {
this.requestPayload = payload;
}
#When("^the new end point is invoked$")
public void the_new_end_point_is_invoked() throws Throwable {
String responseJSON = MyUtil.getPostResponseJSON(myController, "/ids", requestPayload);
responseJSON = responseJSON.replace("[", "").replace("]", "").replaceAll("\"", "");
ids = Arrays.asList(responseJSON.split(","));
}
#Then("^service returns list of available (.+)$")
public void service_returns_list_of_available_something(String ids) throws Throwable {
List<String> list = Arrays.asList(ids.split(","));
Assert.assertTrue(this.ids.containsAll(list));
}
#Then("^the service returns (.+) ids$")
public void the_service_returns_ids(String idCount) throws Throwable {
System.out.println("Actual Size:" + this.ids.size());
System.out.println("Expected Size:" + Integer.parseInt(idCount));
Assert.assertEquals(this.ids.size(), Integer.parseInt(idCount));
}
#After
#Before
public void cleanUp() {
ids = null;
}
}
Now I have run out of options. Please help.
UPDATE1:
The second then block in the stepdef class fails. The sample-payload-1 and sample-payload-2 pass, but the other two fails. Even after changing the order of the sample-payloads the tests fail.
The error I get is an Assertion error as the size of the ids list is not same. But when I run the same test Individually I don't get this error as the size matches.

The issue was with the Instance variable I had used as a counter to loop through a logic that fires the query and extracts the result. The first test sample [sample-payload-1] will alone work fine as it has a fresh copy of the instance counter. The subsequent samples will have only the altered counter value.
This explains why the test cases passed when ran individually as there is not another test that would have used the altered counter.
FIX: The fix was to reset the counter back to 0 before I exit the implementation/service class so that the subsequent request will have a fresh counter
Lesson Learnt: Never rely on an instance variable if the scope of the enclosing Class is Singleton. The value will keep changing for every call to the method in the Class

Related

JsfCaptcha : show correct captcha value

I am using JsfCaptcha in an attempt to process offline captcha validation. While there is a method to validate "what the user entered matches what the captcha image has shown", I am having a hard time actually printing out what the server states is the right solution. I anticipated this being fairly easy to complete, but for the life of me, cannot figure it out. Here is how I am using the library:
import botdetect.web.jsf.JsfCaptcha;
[...]
#ManagedBean
#RequestScoped
public class MySampleBean implements Serializable {
private JsfCaptcha captcha;
private String captchaCode;
getters for above two fields
[...]
setters for above two fields
[...]
public boolean checkInputMatches() {
if (!this.captcha.validate(captchaCode)) {
return true;
}
return false;
}
}
The method checkInputMatches() demonstrates how the library validates that the user has entered in the right captcha solution. What I'd want to do now is, for debugging purposes, is to log out what the solution was ( In the event that the user entered in the wrong value ). Potentially, something like this:
final String solution = captcha.getCorrectSolutionToCaptcha();
At first, I've taken a look through all of the public getters, but none of them are blatant in providing me the data I need. After trying all of them, I went down the jdgui route, where I decompiled the libraries and tried to hunt my way around to a solution / method that would give me this data.
Sadly, the JsfCaptcha class goes under 5-6 levels of base class extending, with a multitude of protected / private methods. Obviously, a very tedious and unnecessary hunt for something very simple.
Is it possible to print out the actual JsfCaptcha value that is being validated against?
I finally managed to solve the problem with javassist, by modifying the generated bytecode of the Botdetect library. I did this because I was unable to find any getter method for accessing the actual captcha solution. Obviously, this is not a clean solution, but it is a solution given that you just want to debug your code to determine why the code you entered does not match the code that the backend server has. For now, I'll consider this as a solution until there is a cleaner alternative requiring no bytecode manipulation. Here are the details on the version that I played with and got this to work:
botdetect-4.0.beta3.5jar
botdetect-jsf20-4.0.beta3.5.jar
botdetect-servlet-4.0.beta3.5.jar
When the checkInputMatches() method executes to validate the captcha, this structure is executed on the backend with respect to the mentioned jars:
Step 1: ( botdetect-jsf20-4.0.beta3.5.jar )
com.captcha.botdetect.web.jsf.JsfCaptcha ->
public boolean validate(String paramString)
Step 2: ( botdetect-servlet-4.0.beta3.5.jar )
com.captcha.botdetect.web.servlet.Captcha ->
public boolean validate(String paramString)
Step 3: ( botdetect-jsf20-4.0.beta3.5.jar )
com.captcha.botdetect.internal.core.CaptchaBase ->
public boolean validate(String paramString1, String paramString2, ValidationAttemptOrigin paramValidationAttemptOrigin, boolean paramBoolean)
Step 4: ( botdetect-jsf20-4.0.beta3.5.jar )
com.captcha.botdetect.internal.core.captchacode.CodeCollection ->
public final boolean a(String paramString1, String paramString2, Integer paramInteger, boolean paramBoolean, ValidationAttemptOrigin paramValidationAttemptOrigin)
Step 5: Observe $3 ( third argument from Step 4 ) to show the actual code.
Here is a photo using jdgui, through which I came to this conclusion:
With that in mind, here is how you can go about printing that value out when that code is executed using javassits ( I am using javassist-3.18.1-GA.jar , on Tomcat ) :
#ManagedBean(eager = true)
#ApplicationScoped
public class CustomBean implements Serializable {
private static final long serialVersionUID = 3121378662264771535L;
private static Logger LOG = LogManager.getLogger(CustomBean.class.getName());
#PostConstruct
public void initialize() {
try {
final ClassPool classPool = new ClassPool(ClassPool.getDefault());
classPool.insertClassPath(new ClassClassPath(this.getClass()));
classPool.insertClassPath(new LoaderClassPath(Thread.currentThread().getContextClassLoader()));
final CtClass codeCollectionClass = classPool
.get("com.captcha.botdetect.internal.core.captchacode.CodeCollection");
if (!codeCollectionClass.isFrozen()) {
final CtMethod aMethod = codeCollectionClass.getDeclaredMethod("a",
new CtClass[] { classPool.get("java.lang.String"), classPool.get("java.lang.String"),
classPool.get("java.lang.Integer"), classPool.get("boolean"),
classPool.get("com.captcha.botdetect.internal.core."
+ "captchacode.validation.ValidationAttemptOrigin") });
aMethod.insertAfter("System.out.println(\"Botdetect-DEBUG: entered-captcha: \" + "
+ "$1 + \"; expected-captcha: \" + $3 + \";\" );");
codeCollectionClass.toClass();
} else {
LOG.error("Frozen class : Unable to re-compile BotDetect for debugging.");
}
} catch (final Exception e) {
LOG.error("unable to modify the bot detect java code", e);
}
}
}
Given this input and challenge:
You get a message like this in your logs:
Botdetect-DEBUG: entered-captcha: U33aZ; expected-captcha: U49a6;

In Spock framework, how do I access data table variable from "error" listener?

I have a test with a data table, e.g. like this:
#Unroll
"Basic session start on platform = #platform"() {
when: "user stats a session on a platform"
def response = startSession(platform, id) // it does a REST request
then: "response contains a userId"
assert verifyUserId(response.userId) // method verifies userId with an internal algorithm and returns true\false if it is valid or not
where:
id | platform
1 | "Facebook"
2 | "Google"
3 | "Apple"
}
I have also wrote a listener for errors.
class MyListener extends AbstractListener {
...
public void error(ErrorInfo error) { ... }
...
}
So that in case an assertion error happens during test execution, the code goes into this "error" method.
The question is, how can I get values of variables from the "where" block from inside the "error" method?
I don't do direct assertions for the data table variables, so ErrorInfo.Exception does not contain them.
I also could not find any other suitable members of the "ErrorInfo" object - I can only find variable names in ErrorInfo.getMethod().getFeature(), but not their values that were there when the error happened.
The main issue here is that ErrorInfo doesn't provide current spec instance, that's why you can't get iteration variables. So I would suggest two ways:
First one is to use iteration interceptor:
class ListenForErrorsExtension implements IGlobalExtension {
void visitSpec(SpecInfo specInfo) {
specInfo.addListener(new Listener())
specInfo.allFeatures*.addIterationInterceptor(new IMethodInterceptor() {
#Override
void intercept(IMethodInvocation invocation) {
holder.put(invocation.iteration.dataValues)
invocation.proceed()
}
})
}
The second one was kindly suggested in comments by Opal. You could instantinate not AbstractRunListener, but IRunListener
class ErrorExtension implements IGlobalExtension{
....
#Override
void beforeIteration(IterationInfo iteration) {
holder.put(iteration.dataValues)
}
....
}
It gives you access to each iterration, but the main idea is still following:
You have to create thread safe holder, put iterration data to it BEFORE error happened and then extract.

Calling delegate from (get, set )Properties c#

I need help to use the below piece of code to call Method1. There are no compilation issues but while running the code Method1 is not invoked. I searched the net a lot but didn't find a solution.
public static class Test12
{
public static Test12.ByteDelegate PropertyValue { get; set; }
public delegate byte[] ByteDelegate(byte p1, byte[] p2);
}
The above class has to be used like this:
class Abc
{
internal void Stat()
{
Test12.Propertyvalue = Method1;
}
private byte[] Method1(byte p1, byte[] p2)
{
byte[] abc = ...;
return abc;
}
}
If I am creating an instance of the delegate in stat method and using it like:
Method1(param1,param2);
Then the Method1 in invoked, but if I use it like this:
Test12.PropertyValue = Method1(param1,param2);
compiler is throwing an error saying missing typecast. Can anybody please tell me how to invoke Method1 using Test12.PropertyValue = Method1;?
You receive an error message because when you try to assign the method to the delegate you are using the following line of code:
test12.propertyvalue = method1(param1, param2);
Which firstly calls the method1 and then what it returns it's trying to assign to the delegate, and the error message occurs saying that you can't assign a byte array to a delegate, as #Jon said in the comment. They are different types so you can't assign one to another, at least not without a cast or a conversion method.
In order to invoke the method1 from the delegate, after you do the assignation like this:
test12.propertyvalue = method1;
you can simply call the delegate, as it would be your method:
test12.propertyvalue(param1 , param2);

Find MethodInvocation method bindings in JDT ASTVisitor

I have a java file which uses java.sql.Statement.execute as below.
public class Dummy
{
public void execute(String q) throws SQLException
{
...
Statement stmt = conn.createStatement();
...
stmt.execute(q);
...
}
}
My use case is I want to identify what are all the classes and their method names which use "Statement.execute(String)" using JDT ASTVisitor. Is this possible?
I found below entry using eclipse ASTView plugin.
method binding: Statement.execute(String)
How can I get this method binding value in my ASTVisitor.
I tried this.
#Override
public boolean visit(MethodInvocation node)
{
IMethodBinding iMethod = (IMethodBinding) node.resolveMethodBinding();
if(iMethod != null)
System.out.println("Binding "+iMethod.getName());
return super.visit(node);
}
but node.resolveMethodBinding() always returns null.
... i want to identify what are all the classes and its method names which using "Statement.execute(String)"
This sounds like a job for the org.eclipse.jdt.core.search.SearchEngine, which will produce the results much faster than traversing all your source files using a visitor.
... node.resolveMethodBinding() always returns null
This depends on how you obtained the AST. See, e.g., org.eclipse.jdt.core.dom.ASTParser.setResolveBindings(boolean)

Android Studio - run same test case(s) several times in loop

Before submitting my test cases, I want to make sure they are running stably. Is there any way in Android Studio to run the same test case / class in loop for several times?
Annotate your test with #FlakyTest. See http://developer.android.com/reference/android/test/FlakyTest.html
For instance
#FlakyTest(tolerance = 3)
public void myTest() {
// Test that sometimes fails for no good reason
}
Update: I see you're using Espresso. Then... no, this is not supported by android-test-kit, unfortunately. But here's the feature request: https://code.google.com/p/android-test-kit/issues/detail?id=153
Use parameterized JUnit tests with several instances of empty parameter set:
#RunWith(Parameterized.class)
public class RepeatedTest {
private static final int NUM_REPEATS = 10;
#Parameterized.Parameters()
public static Collection<Object[]> data() {
Collection<Object[]> out = new ArrayList<>();
for (int i = 0; i < NUM_REPEATS; i++) {
out.add(new Object[0]);
}
return out;
}
#Test
public void unstableTest() {
// your test code here
}
}
A parameterized test class runs all its test methods once for every item in the method marked with the #Parameters annotation. It is normally used to run a test with different initial values, but if there are no values to set up, the test is simply repeated as many times as you want.
The test will pass only if all the instances pass.
If you want to run a test couple of times to see if its stable, In Android studio you can repeat a test by mentioning that in Run/Debug configurations for the test
Just use cycle FOR. For example:
#Test // test loop
public void openApp() {
int x;
for(x=1; x < 3; x++) {
PageObject open = new PageObject(driver);
waitUntilElmntToBeClckbl(open.sqlApp);
open.sqlApp.click();
driver.navigate().back;
}

Resources