Before submitting my test cases, I want to make sure they are running stably. Is there any way in Android Studio to run the same test case / class in loop for several times?
Annotate your test with #FlakyTest. See http://developer.android.com/reference/android/test/FlakyTest.html
For instance
#FlakyTest(tolerance = 3)
public void myTest() {
// Test that sometimes fails for no good reason
}
Update: I see you're using Espresso. Then... no, this is not supported by android-test-kit, unfortunately. But here's the feature request: https://code.google.com/p/android-test-kit/issues/detail?id=153
Use parameterized JUnit tests with several instances of empty parameter set:
#RunWith(Parameterized.class)
public class RepeatedTest {
private static final int NUM_REPEATS = 10;
#Parameterized.Parameters()
public static Collection<Object[]> data() {
Collection<Object[]> out = new ArrayList<>();
for (int i = 0; i < NUM_REPEATS; i++) {
out.add(new Object[0]);
}
return out;
}
#Test
public void unstableTest() {
// your test code here
}
}
A parameterized test class runs all its test methods once for every item in the method marked with the #Parameters annotation. It is normally used to run a test with different initial values, but if there are no values to set up, the test is simply repeated as many times as you want.
The test will pass only if all the instances pass.
If you want to run a test couple of times to see if its stable, In Android studio you can repeat a test by mentioning that in Run/Debug configurations for the test
Just use cycle FOR. For example:
#Test // test loop
public void openApp() {
int x;
for(x=1; x < 3; x++) {
PageObject open = new PageObject(driver);
waitUntilElmntToBeClckbl(open.sqlApp);
open.sqlApp.click();
driver.navigate().back;
}
Related
How can I run code in my #RunWith(SpringRunner.class) #SpringBootTest(classes = {...}) JUnit test before Spring starts?
This question has been asked several times (e.g. 1, 2) but was always "solved" by some configuration recommendation or other, never with a universal answer. Kindly don't question what I am about to do in that code but simply suggest a clean way to do it.
Tried so far and failed:
Extend SpringJUnit4ClassRunner to get a class whose constructor can run custom code before initializing Spring. Failed because super(testClass) must be called first thing and already does a whole lot of things that get in the way.
Extend Runner to get a class that delegates to SpringRunner instead of inheriting it. This class could run custom code in its constructor before actually instantiating the SpringRunner. However, this setup fails with obscure error messages like java.lang.NoClassDefFoundError: javax/servlet/SessionCookieConfig. "Obscure" because my test has no web config and thus shouldn't meddle with sessions and cookies.
Adding an ApplicationContextInitializer that is triggered before Spring loads its context. These things are easy to add to the actual #SpringApplication, but hard to add in Junit. They are also quite late in the process, and a lot of Spring has already started.
One way to do it is to leave out SpringRunner and use the equivalent combination of SpringClassRule and SpringMethodRule instead. Then you can wrap the SpringClassRule and do your stuff before it kicks in:
public class SomeSpringTest {
#ClassRule
public static final TestRule TestRule = new TestRule() {
private final SpringClassRule springClassRule =
new SpringClassRule();
#Override
public Statement apply(Statement statement, Description description) {
System.out.println("Before everything Spring does");
return springClassRule.apply(statement, description);
}
};
#Rule
public final SpringMethodRule springMethodRule = new SpringMethodRule();
#Test
public void test() {
// ...
}
}
(Tested with 5.1.4.RELEASE Spring verison)
I don't think you can get more "before" than that. As for other options you could also check out #BootstrapWith and #TestExecutionListeners annotations.
Complementing jannis' comment on the question, the option to create an alternative JUnit runner and let it delegate to the SpringRunner does work:
public class AlternativeSpringRunner extends Runner {
private SpringRunner springRunner;
public AlternativeSpringRunner(Class testClass) {
doSomethingBeforeSpringStarts();
springRunner = new SpringRunner(testClass);
}
private doSomethingBeforeSpringStarts() {
// whatever
}
public Description getDescription() {
return springRunner.getDescription();
}
public void run(RunNotifier notifier) {
springRunner.run(notifier);
}
}
Being based on spring-test 4.3.9.RELEASE, I had to override spring-core and spring-tx, plus javax.servlet's servlet-api with higher versions to make this work.
I am using SpringBoot with Cucumber. One particular test case fails when run together and passes when run individually. Other Scenarios in the same feature file seems to work fine. I tried the following links but they did not work.
link1, link2, link3, link4, link5
I also tried to quarantine the problematic Scenario to a separate feature file and wrote an exclusive step def for the feature, but still no luck.
I tried introducing #After and #Before hooks to reset the value of Instance variable in stepdef file to null.
I even tried switching the position of Sample-payload and id Count Set
Now I ran out of options. Please help.
Feature File:
Scenario Outline: To check if the service returns all the ids when more than 10000 Ids are returned by the query
Given sample request <payload>
When the new end point is invoked
Then the service returns <idCount> Ids
Examples:
| payload | idCount |
| sample-payload-1 | 17575 |
| sample-payload-2 | 4 |
| sample-payload-3 | 23535 |
| sample-payload-4 | 34535 |
Step Def File:
public class MyStepdefs extends AbstractStepsDefs {
#Autowired
MyController myController;
private String requestPayload;
private List<String> ids;
#Given("^sample request (.+)$")
public void sample_request_something(String payload) throws Throwable {
this.requestPayload = payload;
}
#When("^the new end point is invoked$")
public void the_new_end_point_is_invoked() throws Throwable {
String responseJSON = MyUtil.getPostResponseJSON(myController, "/ids", requestPayload);
responseJSON = responseJSON.replace("[", "").replace("]", "").replaceAll("\"", "");
ids = Arrays.asList(responseJSON.split(","));
}
#Then("^service returns list of available (.+)$")
public void service_returns_list_of_available_something(String ids) throws Throwable {
List<String> list = Arrays.asList(ids.split(","));
Assert.assertTrue(this.ids.containsAll(list));
}
#Then("^the service returns (.+) ids$")
public void the_service_returns_ids(String idCount) throws Throwable {
System.out.println("Actual Size:" + this.ids.size());
System.out.println("Expected Size:" + Integer.parseInt(idCount));
Assert.assertEquals(this.ids.size(), Integer.parseInt(idCount));
}
#After
#Before
public void cleanUp() {
ids = null;
}
}
Now I have run out of options. Please help.
UPDATE1:
The second then block in the stepdef class fails. The sample-payload-1 and sample-payload-2 pass, but the other two fails. Even after changing the order of the sample-payloads the tests fail.
The error I get is an Assertion error as the size of the ids list is not same. But when I run the same test Individually I don't get this error as the size matches.
The issue was with the Instance variable I had used as a counter to loop through a logic that fires the query and extracts the result. The first test sample [sample-payload-1] will alone work fine as it has a fresh copy of the instance counter. The subsequent samples will have only the altered counter value.
This explains why the test cases passed when ran individually as there is not another test that would have used the altered counter.
FIX: The fix was to reset the counter back to 0 before I exit the implementation/service class so that the subsequent request will have a fresh counter
Lesson Learnt: Never rely on an instance variable if the scope of the enclosing Class is Singleton. The value will keep changing for every call to the method in the Class
I'm having difficulty using GroovyDocTool.
No matter what paths I pass into it, my rootDoc does not resolve to any packages, classes, or methods. I have to assume I'm just passing in the wrong paths, but for the life of me can't figure out what to pass.
class TestGroovyDoclet extends GroovyDocTool{
public TestGroovyDoclet() {
super([
'~/ ... /src/'
,'/home/ ... /src/ ... /Sample/'
,'/home/ ... /src/ ... /Sample/Results.groovy'
]);
}
public void Execute(){
System.out.println('Begin Processing Javadoc');
System.out.format(' p %d\n', rootDoc.specifiedPackages().length);
System.out.format(' c %d\n', rootDoc.classes().length);
System.out.println('Finished Processing Javadoc');
}
public static void main(String[] args){
(new TestGroovyDoclet()).with{
it.Execute();
}
}
}
What are considered valid parameters to pass into the GroovyRootDocBuilder?
The authors of GroovyDoc appear to have intended for its use from Ant. This means that there is no means to look up the files to be processed, since you should already know what files you are processing. The consumer must specify the individual files to be processed, and call buildPath to actually process the files. (see override of buildPath)
For those only wanting to interact with the Document Tree, GroovyDocTool is really of no value, and users would likely be better extending GroovyRootDocBuilder which actually contains the meat of the processing. If you are doing this, you may as well re-write the constructor to hide away GroovyDocTool altogether. (see the new Constructor)
class TestGroovyDoclet extends GroovyRootDocBuilder{
protected def sourcepaths = [];
public TestGroovyDoclet() {
//HARDCODE: some test values for my testing
this([
'~/ ... /src/'
,'/home/ ... /src/ ... /Sample/'
,'/home/ ... /src/ ... /Sample/Results.groovy'
]);
}
public TestGroovyDoclet(String[] sourcepaths) {
//hide the unused GroovyDocTool
super(new GroovyDocTool(), sourcepaths, new ArrayList<LinkArgument>(), new Properties());
this.sourcepaths = sourcepaths;
}
/**
* Builds the tree by recursively searching for files
* <p>
* It is likely useful to override the original buildTree
* method as well to put some safeties in place. The parsing
* routines do not do a lot of validation. For those of us
* inheritting, it would be a good idea to override build
* tree ot sanitize the input list as much as possible.
* </p>
*/
#Override
public void buildTree(){
def list = [];
// loop through the sourcepaths to recursively find the files
for (String sourcepath : sourcepaths) {
def root = new File(sourcepath);
if(root.exists()){
if(root.isFile()){
list << root.absolutePath;
}
else{
root.eachFileRecurse (FileType.FILES) { file -> list << file.absolutePath; };
}
}
}
buildTree(list);
}
/**
* Method to actually do the processing. Sample only, does not demonstrate anything useful.
*/
public void Execute(){
buildTree();
System.out.println('Begin Processing GroovyDoc');
System.out.format(' p %d\n', rootDoc.specifiedPackages().length);
//System.out.format(' c %d\n', rootDoc.classes().length);
int count = 0;
for(def p : rootDoc.specifiedPackages()){
count += p.allClasses().length;
}
System.out.format(' c %d\n', count);
System.out.println('Finished Processing GroovyDoc');
}
public static void main(String[] args){
(new TestGroovyDoclet()).with{
it.Execute();
}
}
}
The above code is not perfect, and not exactly what I came up with; rather the code is meant to highlight a few of the faulty assumptions contained in the OP, and mechanisms to work around them. It is not gauranteed to compile since it just cut/pastes elements of a larger re-write.
Weirdness:
Interestingly GroovyRootDoc.classes always returns null (unsure why). It was necessary to loop through each package, and inspect the classes that are a subset of each. This was surprising, but the desired usage anyway.
The code above only worked on groovy files, not java. I think GroovyDoc choked on some java1.8 syntax (but I'm not sure).
I am trying to do some dependency injection for my tests using nUnit. I'm new to TDD and nUnit so it's possible I am missing something simple. So basically I've created a SetUp method for my interfaces. I originally was using a constructor but I read it's bad to do this when doing TDD so I now using a method.
When I run my test I construct an object and assign it to the interface and then I call a method using that interface. I want to test if it can parse a string decimal.
When I run my test it says test failed and the message is:Invalid signature for SetUp or TearDown method
See below for the actual code:
public class DonorTests
{
private IDonor _Donor;
private IValidateInput _ValidInput;
//DonorTests(IDonor donor, IValidateInput validInput)
//{
// _Donor = donor;
// _ValidInput = validInput;
//}
[SetUp]
void Setup(IDonor donor, IValidateInput validInput)
{
_Donor = donor;
_ValidInput = validInput;
}
[Test]
public void HandleStringNotDecimal()
{
_ValidInput = new ValidateInput();
Assert.IsTrue(_ValidInput.IsDecimal("3445.3450"));
}
}
My class that uses this interface
public class ValidateInput : IValidateInput
{
public decimal RoundTwoDecimalPlaces(decimal amount)
{
return Math.Round(amount);
}
public bool IsDecimal(string amount)
{
decimal ParsedDecimal;
return Decimal.TryParse(amount, out ParsedDecimal);
}
public decimal ConvertToString(string value)
{
decimal ParsedDecimal;
Decimal.TryParse(value, out ParsedDecimal);
return ParsedDecimal;
}
}
You're injecting dependencies using constructor injection previously, right? I think you will not be able to perform dependency injection using method decorated with SetUpAttribute because such method has to be parameterless. Also Setup method has to be public, see this SO thread.
How are we typically dealing with similar situations in our company is:
[TestFixture]
public class DonorTests
{
private IDonor _Donor;
private IValidateInput _ValidInput;
[SetUp]
public void Setup()
{
_Donor = new Donor();
_ValidInput = new ValidateInput();
}
[Test]
public void HandleStringNotDecimal()
{
Assert.IsTrue(_ValidInput.IsDecimal("3445.3450"));
}
}
Or if construction of ValidInput and Donor is cheap then we simply create new instance for each test, having special method for that purpose so when we decide to test another implementation of IValidateInput then it is enough to change it in one place only:
[TestFixture]
public class DonorTests
{
[Test]
public void HandleStringNotDecimal()
{
var validInput = CreateValidateInput();
Assert.IsTrue(validInput .IsDecimal("3445.3450"));
}
private static IValidateInput CreateValidateInput()
{
return new ValidateInput();
}
}
Besides the cause mentioned in the accepted answer, I have met the same error when leaving method as non-public (private or protected).
NUnit most probably relies on reflection and does not deal with non-public methods, so special methods (i.e. decorated with NUnit specific attributes) must be public.
I am starting to write a unit test (MS Test, with Resharper as the test runner). When I set the LogicalThreadContext (see below), my test cases get 'aborted'. Anybody know why? Is this related to the unit test being on a different thread? How do I resolve this?
[TestClass]
public class ContextInfoTest
{
private ILog _log;
[TestInitialize]
public void TestInitialize()
{
// logging configured in assembly.info
_log = LogManager.GetLogger(this.GetType());
}
[TestMethod]
public void FigureOutWhyAborting()
{
string input = "blah";
LogicalThreadContext.Properties["mypropertyname"] = input;
string output = LogicalThreadContext.Properties["mypropertyname"] as string;
Assert.AreEqual(input, output);
}
[TestMethod]
public void ThisWorks()
{
string input = "blah";
CallContext.LogicalSetData("mypropertyname", input);
string output = CallContext.LogicalGetData("mypropertyname") as string;
Assert.AreEqual(input, output);
}
The weird thing is that if I were to debug and step through the code, the Assert.AreEqual does get called and passes, so something is happening after that line of code... which is why I think it might have something to do with the test thread, etc.
Thanks!
UPDATE:
So I ran this test in MSTest and got this exception (Resharper didn't show it)
Unit Test Adapter threw exception:
Type is not resolved for member 'log4net.Util.PropertiesDictionary,log4net, Version=1.2.13.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a'..
I'm using log4net v1.2.13, on VS2013, .Net 4.5.
This link seems to suggest it is a referenced assemblies problem, but there is no resolution. Any additional ideas would be greatly welcome, GAC'ing log4net is not an option.
https://issues.apache.org/jira/browse/LOG4NET-398
I ended up doing this to get it working:
put this in the TestCleanup() method:
CallContext.FreeNamedDataSlot("log4net.Util.LogicalThreadContextProperties");
So, I can't thank you enough for figuring this out to call FreeNamedDataSlot. This turned me on to my answer that worked for me. Instead of passing in the full namespace of the class, I just had to use the class name:
This was used somewhere deep in my Data Access Layer:
MySession session = (MySession)System.Runtime.Remoting.Messaging.CallContext.LogicalGetData("MySession");
[TestCleanup]
public void Cleanup()
{
CallContext.FreeNamedDataSlot("MySession");
}
This worked perfect for me! Hopefully this helps someone else when using Visual Studio's Test environment.
I know this is a bit old but for the following worked for me.
Although log4net was referenced by my project and my unit tests, it was not configured in the unit test. Updating my ThreadContext helper to the following allowed my tests to succeed. If log4net is configured in your unit tests and you are still getting this issue you could key off a compilation symbol instead.
var is_configured = log4net.LogManager.GetRepository().Configured;
var props = is_configured
? (ContextPropertiesBase)LogicalThreadContext.Properties
: (ContextPropertiesBase)ThreadContext.Properties;
props[key] = value;
Thanks all for the tips, i try with:
[TestCleanup]
public void Cleanup()
{
CallContext.FreeNamedDataSlot("log4net.Util.LogicalThreadContextProperties");
}
And works!!