I'd like an equivalent function to nUnit's Category attribute for test cases.
I have inherited a large number of C++ test cases, some of which are unit tests and some of which are longer-running integration tests, and I need to set up my continuous integration build process to ignore the integration test cases.
I would prefer to simply tag all the integration test cases and instruct cppunit to exclude them during CI builds.
Am I overlooking a feature of cppunit or is there an alternative way to achieve this?
There are no native test category attributes. CppUnit is a bit simpler than that. CppUnit doesn't even come with a command-line test runner for your app. You have to write your own simple main() function that executes a TestRunner.
Here's the canonical example.
#include <cppunit/extensions/TestFactoryRegistry.h>
#include <cppunit/ui/text/TestRunner.h>
int main( int argc, char **argv)
{
CppUnit::TextUi::TestRunner runner;
CppUnit::TestFactoryRegistry ®istry = CppUnit::TestFactoryRegistry::getRegistry();
runner.addTest( registry.makeTest() );
bool wasSuccessful = runner.run( "", false );
return wasSuccessful;
}
A TestSuite is a collection of TestCases. A TestRunner executes a collection of TestSuites. Notice that in this example it gets the TestSuites from the TestFactoryRegistry, which you populate by using a macro call to CPPUNIT_TEST_SUITE_REGISTRATION(MyTestSuite). But the TestCases are still your test classes.
You can certainly implement these attributes yourself, just as you would extend any class with a facade. Derive your new class from TestSuite. Add attributes to your tests that you could select on, then populate your TestRunner executing "just unit tests" or "just integration tests" or whatever you want.
For that matter, a TestRunner can select tests to execute based on name. If you named all your integration tests with a prefix like ITFoo, ITBar, etc., you could select all tests that begin with "IT".
There are dozens of ways to solve your problem, but you'll have to do it yourself. If you can write code worthy of unit testing, it shouldn't be a big deal for you. :-)
Related
I a little bit of confusion about xUnit for test my DAL.
My goal is to verify that my DAL correctly accesses the DB and extract the right data.
I create a xUnit test project and try to do a simpli test with Moq like this
[Fact]
public void Test1()
{
// Arrange
var mockMyClass = new Mock<IMyClassBLL>();
// Setup a mock stat repository to return some fake data within our target method
mockStAverageCost.Setup(ac => ac.GetBy(It.IsAny<MyClassVO>())).Returns(new List<MyClassVO>
{
new MyClassVO { HCO_ID = "1"},
new MyClassVO { HCO_ID = "2"},
new MyClassVO { HCO_ID = "3"},
new MyClassVO { HCO_ID = "4"}
});
// create our MyTest by injecting our mock repository
var MyTest = new MyClassBLL(mockMyClass.Object);
// ACT - call our method under test
var result = MyTest.GetBy();
// ASSERT - we got the result we expected - our fake data has 6 goals we should get this back from the method
Assert.True(result.Count == 4);
}
The method above work fine.
Now I want access directly to the db for get data.
Obviously something escapes me, I did not understand how to perform a data test with .net core 2 simulating dependeny injection and accessing the data.
Can someone clarify my ideas?
Are you looking for a unit test or an integration test? They're fundamentally different things and serve different purposes.
If your goal is to ensure that GetBy (the unit of functionality under test) does what it's supposed to do, then you should not be using live data. A real connection with real data would introduce variables, causing the test to potentially fail when there's actually nothing wrong with GetBy. For a true unit test, you should only use mocks and test data.
If your goal is to ensure that your application can connect to your database and actually draw data out of it, then that's an integration test. You might potentially use GetBy/your repository, in general, in the test, but generally you'd want to avoid that. Again, connecting and querying directly with via something like ADO.NET serves to remove variables, so if the test fails, you'll know it was because there actually was a problem connecting/querying, in general, rather than just some issue with your repository or a particular method thereof.
Long and short, a good test tests just one thing. If that particular thing requires external components (such as a SQL Server database), then it's an integration test, and at that point, you're testing the integration of the component. Something like a repository method should not come into play, as that would be testing two different things in one test. If you need to test GetBy then there should be no external dependencies, such as a SQL Server database.
Additionally:
I did not understand how to perform a data test with .net core 2 simulating dependeny injection and accessing the data.
This would be an example of testing the framework, which is another no-no. You can safely assume that DI works in ASP.NET Core. It has its own test suite covering that. There is no need for you to add tests for that as well.
Im using Spock for my tests and I have multiple classes to test. I want to tell Spock to test each class in specific order. Is there a way to do this? I noticed TestNG has got #AfterTest annotation, so does Spock have something similar?
You can't specify Spock test classes execution order. However Spock allows you to specify methods execution order when you add #Stepwise annotation to your test class - in this case Spock will execute all methods in order from top to bottom and if one method fails it will ignore remaining methods in this test class.
Indicates that a spec's feature methods should be run sequentially in their declared order (even in the presence of a parallel spec runner), always starting from the first method. If a method fails, the remaining methods will be skipped. Feature methods declared in super- and subspecs are not affected.
#Stepwise is useful for specs with (logical) dependencies between methods. In particular, it helps to avoid consecutive errors after a method has failed, which makes it easier to understand what really went wrong.
Reference: http://spockframework.org/spock/javadoc/1.1/spock/lang/Stepwise.html
#Stepwise
class StepwiseExample extends Specification {
def "first test method to run"() {
// ....
}
def "second test method to run"() {
// ....
}
def "if previous method failed this one will be ignored"() {
// ....
}
}
Using org.junit.runners.Suite
Jeff Scott Brown gave a good comment about JUnit's #Suite.SuiteClasses. In this case you create a class where you can aggregate your test classes (specifications) into a single test suite and those classes will be executed in the order they have been defined in the suite. Consider following example:
import org.junit.runner.RunWith
import org.junit.runners.Suite
#RunWith(Suite)
#Suite.SuiteClasses([
Test2Specification,
Test1Specification
])
class TestSuiteSpecification { }
This suite executes two specifications Test2Specification and Test1Specification in the defined order:
How can you get a jenkins groovy script to produce a junit xml results file? I'm doing this purely for the purpose of generating junit results with a specific number of passed/failed and skipped test cases. I need this so that I have a set of test data to test against for another application. This other app goes out to various jenkins jobs and analyzes the junit results from the job's json output. I want to point my functional tests at this jenkins job for testing. (I can't use my real continuous integration jobs because that wouldn't be deterministic).
I've got a basic groovy test case like what's below. It runs but doesn't produce junit output. I didn't expect it to, but I'm also not sure how to get it to generate one.
class BunchOfTests extends GroovyTestCase {
void testOne(){}
void testTwo(){fail()}
}
I also played around with writing code that prints the junit results xml but it's getting lengthy and quite ugly. I've seen the threads on here about what the junit results xsd looks like but I'm thinking there's got to be an easier route to generating some results without needing a pre-made results file. 10 results or so ought to be enough for what I need.
Generally unit testcases in groovy has to be written as below
import groovy.util.GroovyTestCase
class sampleTest extends GroovyTestCase {
assertEquals(true, val);
}
Incase you want only Junit reporting in groovy.
How would I produce JUnit test report for groovy tests, suitable for consumption by Jenkins/Hudson?
I am a beginner in soapui testing. Hopefully you can help me solving this problem.
In my test project I have a test suites which contains several test cases. Multiple test case will start the same test case. To run this test case I need some property values to be transferred to this test case.
I tried to achieve this in two ways. But I failed in both.
I tried to call the test case and set the needed properties in the test case. I start the test case from a Groovy script. But I couldn't find a good example how to set the properties in the called test case.
I tried to get the property values of the calling parent test case inside the called test case. It looks like the parent test case that called the test case isn't available in the context of the running test case.
The test cases, that will call the same test case, will be run in parallel. So, I think it isn't a solution to first set the property values and then start the test case, because they will be overwritten by the other test cases that run at the same time. Also using test suite properties for these values won’t work because of running the test cases in parallel.
My test project looks like this.
MyProject
TestSuite_APLtests
TestCase_user_01
Properties test step
Run_test <groovy script>
Step_01
…..
TestCase_user_02
Properties test step
Run_test <groovy script>
Step_01
…..
TestCase_General
Properties test step
POST sessions
Step_01
…..
The ‘Properties test step’ of each ‘TestCase_user_’ contains a user and password needed in test case ‘TestCase_General’ and will be different for each test case.
In the ‘Run_test’ groovy script of each ‘TestCase_user_’ the test case ‘TestCase_General’ is started by using:
def myTestSuite = testRunner.testCase.testSuite.project.getTestSuiteByName("TestSuite_APLtests")
def myTestCase = myTestSuite.getTestCaseByName("TestCase_General")
myTestCase.run(null, false)
How can I add the properties user and password to the run comment that starts the test case?
If I try to get the property values with a groovy script in test case ‘TestCase_General’ I don’t know how to determine which test case has called ‘TestCase_General’. I found some posts on internet that suggests to use: context.getProperty("#CallingRunTestCaseStep#") to determine the calling test case. But this value is null. And when I try to check if the calling test case is available in the context by using: context.hasProperty("#CallingRunTestCaseStep#") this is false, so this doesn't work to find the calling test case.
Can someone tell me what the solution will be to get this working.
Thanks,
You can set Test Case properties from groovy script with setPropertyValue(name,value) method, however if you run the Test Cases in parallel, this properties as you said will be overwritten for each Test Case calling TestCase_General. So instead of use setPropertyValue you can pass the context properties through the run(StringToObjectMap properties, boolean async) method in the WsdlTestCase.java class. Your groovy code to call TestCase_General could be:
import com.eviware.soapui.support.types.StringToObjectMap
// get test suite
def myTestSuite = testRunner.testCase.testSuite.project.getTestSuiteByName("TestSuite_APLtests")
// get your test case
def myTestCase = myTestSuite.getTestCaseByName("TestCase_General")
// set the user and password properties in the context
context.setProperty("user","userTestCaseN")
context.setProperty("password","passwordTestCaseN")
// run the testCase passing the context
def contextMap = new StringToObjectMap( context )
myTestCase.run(contextMap,false);
To access the context properties in the groovy script of your TestCase_General use this code:
context.getProperty("userPassword")
Or if you prefer to use context.expand:
context.expand('${#user}')
Note that the use of # depends on how you are accessing the properties.
If you also need to use the context properties in the SOAP Test Request of your TestCase_General use this way ${#propetryName} i.e:
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
<Header/>
<Body>
<request>
<user>${#user}</user>
</request>
</Body>
</Envelope>
Hope this helps,
I have heard of this term many times (in the context of programming) but couldn't find any explanation of what it meant. Any good articles or explanations?
I think you're referring to test fixtures:
The purpose of a test fixture is to ensure that there is a well known
and fixed environment in which tests are run so that results are
repeatable. Some people call this the test context.
Examples of fixtures:
Loading a database with a specific, known set of data
Erasing a hard disk and installing a known clean operating system installation
Copying a specific known set of files
Preparation of input data and set-up/creation of fake or mock objects
(source: wikipedia, see link above)
Here are also some practical examples from the documentation of the 'Google Test' framework.
The term fixture varies based on context, programing language or framework.
1. A known state against which a test is running
One of the most time-consuming parts of writing tests is writing the
code to set the world up in a known state and then return it to its
original state when the test is complete. This known state is called
the fixture of the test.
PHP-Unit documentation
A test fixture (also known as a test context) is the set of
preconditions or state needed to run a test. The developer should set
up a known good state before the tests, and return to the original
state after the tests.
Wikipedia (xUnit)
2. A file containing sample data
Fixtures is a fancy word for sample data. Fixtures allow you to
populate your testing database with predefined data before your tests
run. Fixtures are database independent and written in YAML. There is
one file per model.
RubyOnRails.org
3. A process that sets up a required state.
A software test fixture sets up the system for the testing process by
providing it with all the necessary code to initialize it, thereby
satisfying whatever preconditions there may be. An example could be
loading up a database with known parameters from a customer site
before running your test.
Wikipedia
I think PHP-unit tests have very good explaining of this:
One of the most time-consuming parts of writing tests is writing the
code to set the world up in a known state and then return it to its
original state when the test is complete. This known state is called
the fixture of the test.
Also Yii documents described fixtures test in a good shape:
Automated tests need to be executed many times. To ensure the testing
process is repeatable, we would like to run the tests in some known
state called fixture. For example, to test the post creation feature
in a blog application, each time when we run the tests, the tables
storing relevant data about posts (e.g. the Post table, the Comment
table) should be restored to some fixed state.
Here's a simple example of fixtures test:
<?php
use PHPUnit\Framework\TestCase;
class StackTest extends TestCase
{
protected $stack;
protected function setUp()
{
$this->stack = [];
}
protected function tearDown()
{
$this->stack = [];
}
public function testEmpty()
{
$this->assertTrue(empty($this->stack));
}
public function testPush()
{
array_push($this->stack, 'foo');
$this->assertEquals('foo', $this->stack[count($this->stack)-1]);
$this->assertFalse(empty($this->stack));
}
public function testPop()
{
array_push($this->stack, 'foo');
$this->assertEquals('foo', array_pop($this->stack));
$this->assertTrue(empty($this->stack));
}
}
?>
This PHP unit test has functions with names setUp and tearDown so that before running your tests you setup your data and once finished you can restore them to the initial state.
Exactly to that topic, JUnit has a well explained doc. Here is the link!
The related portion of the article is:
Tests need to run against the background of a known set of objects. This set of objects is called a test fixture. When you are writing tests you will often find that you spend more time writing the code to set up the fixture than you do in actually testing values.
To some extent, you can make writing the fixture code easier by paying careful attention to the constructors you write. However, a much bigger savings comes from sharing fixture code. Often, you will be able to use the same fixture for several different tests. Each case will send slightly different messages or parameters to the fixture and will check for different results.
When you have a common fixture, here is what you do:
Add a field for each part of the fixture
Annotate a method with #org.junit.Before and initialize the variables in that method
Annotate a method with #org.junit.After to release any permanent resources you allocated in setUp
For example, to write several test cases that want to work with different combinations of 12 Swiss Francs, 14 Swiss Francs, and 28 US Dollars, first create a fixture:
public class MoneyTest {
private Money f12CHF;
private Money f14CHF;
private Money f28USD;
#Before public void setUp() {
f12CHF= new Money(12, "CHF");
f14CHF= new Money(14, "CHF");
f28USD= new Money(28, "USD");
}
}
In Xamarin.UITest it is explained as following:
Typically, each Xamarin.UITest is written as a method that is referred
to as a test. The class which contains the test is known as a test
fixture. The test fixture contains either a single test or a logical
grouping of tests and is responsible for any setup to make the test
run and any cleanup that needs to be performed when the test finishes.
Each test should follow the Arrange-Act-Assert pattern:
Arrange – The test will setup conditions and initialize things so that the test can be actioned.
Act – The test will interact with the application, enter text, pushing buttons, and so on.
Assert – The test examines the results of the actions performed in the Act step to determine correctness. For example, the
application may verify that a particular error message is
displayed.
Link for original article of the above Excerpt
And within Xamarin.UITest code it looks like following:
using System;
using System.IO;
using System.Linq;
using NUnit.Framework;
using Xamarin.UITest;
using Xamarin.UITest.Queries;
namespace xamarin_stembureau_poc_tests
{
[TestFixture(Platform.Android)]
[TestFixture(Platform.iOS)]
public class TestLaunchScreen
{
IApp app;
Platform platform;
public Tests(Platform platform)
{
this.platform = platform;
}
[SetUp]
public void BeforeEachTest()
{
app = AppInitializer.StartApp(platform);
}
[Test]
public void AppLaunches()
{
app.Screenshot("First screen.");
}
[Test]
public void LaunchScreenAnimationWorks()
{
app.Screenshot("Launch screen animation works.");
}
}
}
Hope this might be helpful to someone who is in search of better understanding about Fixtures in Programming.
I'm writing this answer as quick note for myself on what is "fixture".
same-data-multiple-tests
Test Fixtures: Using the Same Data Configuration for Multiple Tests
If you find yourself writing two or more tests that operate on similar data, you can use a test fixture. This allows you to reuse the same configuration of objects for several different tests.
you can read more at googletest
fixtures can be used for during integration test or during development (lets say ui development where data is comming from development database
fake users for database or testing
myproject/fixtures/my_fake_user.json
[
{
"model": "myapp.person",
"pk": 1,
"fields": {
"first_name": "John",
"last_name": "Lennon"
}
},
{
"model": "myapp.person",
"pk": 2,
"fields": {
"first_name": "Paul",
"last_name": "McCartney"
}
}
]
you can read more from django docs