I have a collection of integration tests running with SpringJUnit4ClassRunner. I'm trying to run these in parallel using maven surefire. However, I have noticed that the the code is blocking before entering the synchronized block in CacheAwareContextLoaderDelegate.loadContext().
Is there a way to bypass this cache? I tried doing this, but it seems like there is more shared state than just the cache itself since my application deadlocked inside Spring code. Or could the synchronization be made more fine-grained by somehow synchronizing on the map key rather than the entire map?
My motivation for parallelising tests is twofold:
In some tests, I replace beans with mocks. Since mocks are inherently stateful, I have to build a fresh ApplicationContext for every test method using #DirtiesContext.
In other tests, I only want to deploy a subset of Jersey resources. To do this, I specify a subset of Spring configuration classes. Since Spring uses the MergedContextConfiguration as a key in the context cache, these tests will be unable to share ApplicationContexts.
It is possible that may get a better turn-around time for your test suit if you disable the parallell test execution. In the testing chapter of Spring's reference docs there is a paragraph about Context caching:
Once the TestContext framework loads an ApplicationContext (or WebApplicationContext) for a test, that context will be cached and reused for all subsequent tests that declare the same unique context configuration within the same test suite.
Why is it implemented like this?
This means that the setup cost for loading an application context is incurred only once (per test suite), and subsequent test execution is much faster.
How does the cache work?
The Spring TestContext framework stores application contexts in a static cache. This means that the context is literally stored in a static variable. In other words, if tests execute in separate processes the static cache will be cleared between each test execution, and this will effectively disable the caching mechanism.
To benefit from the caching mechanism, all tests must run within the same process or test suite. This can be achieved by executing all tests as a group within an IDE. Similarly, when executing tests with a build framework such as Ant, Maven, or Gradle it is important to make sure that the build framework does not fork between tests. For example, if the forkMode for the Maven Surefire plug-in is set to always or pertest, the TestContext framework will not be able to cache application contexts between test classes and the build process will run significantly slower as a result.
One easy thing that I could think of is using #DirtiesContext
Related
I'm currently writing a Node library to execute untrusted code within Docker containers. It basically maintains a pool of containers running, and provides an interface to run code in one of them. Once the execution is complete, the corresponding container is destroyed and replaced by a new one.
The four main classes of the library are:
Sandbox. Exposes a constructor with various options including the pool size, and two public methods: executeCode(code, callback) and cleanup(callback)
Job. A class with two attributes, code and callback (to be called when the execution is complete)
PoolManager, used by the Sandbox class to manage the pool of containers. Provides the public methods initialize(size, callback) and executeJob(job, callback). It has internal methods related to the management of the containers (_startContainer, _stopContainer, _registerContainer, etc.). It uses an instance of the dockerode library, passed in the constructor, to do all the docker related stuff per se.
Container. A class with the attributes tmpDir, dockerodeInstance, IP and a public method executeCode(code, callback) which basically sends a HTTP POST request against ContainerIP:3000/compile along with the code to compile (a minimalist API runs inside each Docker container).
In the end, the final users of the library will only be using the Sandbox class.
Now, my question is: how should I test this?
First, it seems pretty clear to my that I should begin by writing functional tests against my Sandbox class:
it should create X containers, where X is the required pool size
it should correctly execute code (including the security aspects: handling timeouts, fork bombs, etc. which are in the library's requirements)
it should correctly cleanup the resources it uses
But then I'm not sure what else it would make sense to test, how to do it, and if the architecture I'm using is suitable to be correctly tested.
Any idea or suggestion related to this is highly appreciated! :) And feel free to ask for a clarification if anything looks unclear.
Christophe
Try and separate your functional and unit testing as much as you can.
If you make a minor change to your constructor on Sandbox then I think testing will become easier. Sandbox should take a PoolManager directly. Then you can mock the PoolManager and test Sandbox in isolation, which it appears is just the creation of Jobs, calling PoolManager for Containers and cleanup. Ok, now Sandbox is unit tested.
PoolManager may be harder to unit test as the Dockerode client might be hard to mock (API is fairly big). Regardless if you mock it or not you'll want to test:
Growing/shrinking the pool size correctly
Testing sending more requests than available containers in the pool
How stuck containers are handled. Both starting and stopping
Handling of network failures (easier when you mock things)
Retries
Any of failure cases you can think of
The Container can be tested by firing up the API from within the tests (in a container or locally). If it's that minimal recreating it should be straightforward. Once you have that it's really just testing an HTTP client it sounds like.
The source code for the actual API within the container can be tested however you like with standard unit tests. Because you're dealing with untrusted code there are a lot of possibilities:
Doesn't compile
Never completes execution
Never starts
All sorts of bombs
Uses all host's disk space
Is a bot and talks over the network
The code could do basically anything. You'll have to pick the things you care about. Try and restrict everything else.
Functional tests are going to be important too, there are a log of pieces to deal with here and mocking Docker isn't going to be easy.
Code isolation is a difficult problem; I wish Docker was around last time I had to deal with it. Just remember that your customers will always do things you didn't expect! Good luck!
I remember reading somewhere that it isn't advisable to use "exec" within the C code, compiled by NDK.
What is the recommended approach? Do we try and push the EXEC code up to the Java-space; that is, so the JNI (or application) spawns the new process, (and where relevant passes the results back down to the NDK)?
First off, it's not recommended to use either fork or exec. All of your code is generally supposed to live in a single process which is your main Android application process, managed by the Android framework. Any other process is liable to get killed off by the system at any time (though in practice that doesn't happen in present Android versions as far as I have seen).
The rationale as I understand it is simply that the Android frameworks can't properly manage the lifetime and lifecycle of your app, if you go to spawn other processes.
Exec
You have no real alternative here but to avoid launching other executables at all. That means you need to turn your executable code into a library which you link directly into your application and call using normal NDK function calls, triggered by JNI from the Java code.
Fork
Is more difficult. If you really need a multi-process model, and want to fit within the letter of the rules, you need to arrange for the Android framework to fork you from its Zygote process. To do this, you should run all your background code in a different Service which is stated to run in a different process within the AndroidManifest.xml.
To take this to extremes, if you need multiple identical instances of the code running in different processes for memory protection and isolation reasons, you can do what Android Chrome does:
Run all your background/forked code in a subclass of Service
Create multiple subclasses of that
List each of these subclasses as a separate service within your AndroidManifest.xml each with a different process attribute
In your main code, remember exactly which services you've fired up and not, and manage them using startService/stopService.
Of course, if you've turned your native code into a library rather than an executable, you probably don't need fork anyway. The only remaining reason to use fork is to achieve memory protection/isolation.
In practice
In practice quite a lot of apps ignore all this and use fork/exec within their native code directly. At the moment, it works, at least for short-running tasks.
So I have some Java code that takes some time to complete (about 2 minutes). Nothing I can do about that.
But I am wondering how best to approach this in the XPages UI so that the user may still have to wait but has more control/interaction while it is running (not just a spinning wheel).
So from what I can see I can do the following.
Java class called in XPage wrapped in a thread.
Java Agent called from XPage in a thread.
Java Agent called from an XPage, but waits for a document to be updated.
Eclipse plugin (For in the client) is activated. Not sure how it would talk back to XPage though (via document?).
Any other methods?
If you created the thread in the XPage, is that going to cause any problems at the server end? Will I have to avoid using Notes objects in the Java class?
I would suggest using the OSGi Tasklet service, a.k.a. DOTS. This approach allows Java tasks to be scheduled or bound to events, just like agents, but perform significantly more efficiently than agents. Perhaps most pertinent to your need is the additional ability to trigger DOTS tasks via the console, which would allow your XPages code to start the Java code merely by issuing a remote console command via the session object.
In addition, check out the technique used in the XSP Starter Kit to provide a serverScope variable. If your code is running in a DOTS task (or even an agent), it's running in a different Java application, so it can't talk directly to the standard scope variables. The serverScope approach would theoretically allow you to store objects that can be accessed from both the XPage and the triggered task. This could aid in using Mark's technique, as mentioned above by Per, to convey progress to the user while the task is running: you'd just be storing the progress information in serverScope instead of sessionScope.
A solution would be to have an agent react on saving new documents in the database instead of kicking of the agent in your application and use threads ( because threads can be very dangerous and could easily kill your http task )
Another thing you could look into is why the code that you want to execute is taking 2 minutes to complete. What is the code for? Doing things in other databases or connect to other non notes resources?
The Goal: Run our database tests (that use different databases) to run in parallel.
We are using Resharper's Nunit option to integrate our Unit Tests into visual studio. Resharper allows you to set the number of assemblies to run in parallel. The tests never fail when run in serial; however, when we set the number of assemblies to run in parallel to 3 or higher (maybe two, although none have failed yet), some database tests consistently fail.
Our guess, is that data providers are getting switched out from under the tests. We use spring as an IOC and in our tests it also handles the transaction management. Our older tests required seeded data, but our new tests expect an empty database and creates any necessary data for the test. We think that because our bootstrap for the test fixtures is setting the connection string properties of the db provider, when another test runs in parallel, it can change the provider out from under the tests.
Either way, it would be nice to run the database tests in parallel and not lose the transaction management and test cleanup we get with spring.
The bootstrap that runs for each test is setting the connection string (and the db provider connection string).
Any ideas on how to get these tests (with different connection strings) to run in parallel.
I have client and server threads in my applications. When I run these apps as standalone apps, these threads communicate properly.
But when I run client as JUnit and server as standalone, client thread dies within few seconds.
I couldn't get, why such different behavior.
When the JUnit runner terminates, all spawned threads etc. are killed too (as it is most likely run in a separate JVM instance).
Here is a (rather old) article describing the problem you experienced (the GroboUtils library it is recommending seems to have been abandoned long time ago though). And another, recent one, with a more modern solution using the new Java concurrency framework.
The gist of the latter solution is that it runs the threads via an executor, which publishes the results of the runs via Futures. And Future.get is blocking until the thread finishes with the task, automatically keeping the JUnit tests alive. You may be able to adapt this trick to your case.