I have simple Watir tests.
Each test is self-contained, no shared state or dependency of any kind. Each test open and close the browser.
Is it possible to run the test in parallel to reduce the time to run all tests?
Even only 2 or 3 tests in parallel can reduce the time dramatically.
Take a look at parallel_tests Ruby gem. Depending on your setup, running the tests in parallel could be as simple as this:
parallel_cucumber features/
Related
I don't want to run separate tests in parallel, but I want to run steps within one test in parallel, for example if three clusters need to be created, I want them to be created parallelly. I don't fully understand maven-failsafe but I was wondering if it will help me achieve this easily or if i should use normal Java threads?
If we put #parallel=false only at the top of a scenario and not on a feature , then will it stop running the test cases in parallel and still run the features in parellel?
My whole intention is to not run scenarios parallely but run the features parallely.
Features will always run in parallel if you use the parallel runner.
If you add the #parallel=false annotation - it will only affect the individual Scenario.
I have a python web application that uses behave for behavioral testing. I have 5 *.feature files that each take a few minutes to run when I run them, both locally and on our Jenkins build server. I would like to run the five files in parallel rather than sequentially to save time. I can do this locally, but not on my build server. Here are the details:
Locally runs on Windows:
I can run all 5 files in separate command windows using these command:
behave.exe --include "file_01.feature"
behave.exe --include "file_02.feature"
behave.exe --include "file_03.feature"
behave.exe --include "file_04.feature"
behave.exe --include "file_05.feature"
I can also run a python script that spins off 5 separate processes using the same command.
Both of these work, I have no problems
Build server runs on Linux:
When I try to run all five files using a similar command, some of the behave scenarios give me errors. The errors are one of these three:
Message: unknown error: cannot determine loading status from disconnected: Unable to receive message from renderer
Message: chrome not reachable
Message: no such session
The behave scenarios that throw these errors seem to change with every test run.
Oddly, if I rearrange the 5 *.feature files into 3, it works. This is not an ideal solution though. Our application is growing. We'll have more feature files as it grows.
I suspect that there is some shared resource between the chrome drivers in the running behave tests, but I'm not sure. I can't explain why this works for me locally, but not on my build server. Nor can I explain why 3 files work, but not 5.
Has anyone seen errors like this when trying to run multiple behave tests simultaneously? Or do you know what I should be looking for? My project is big enough that it'd be difficult to put together a minimal example of my problem. That's why I haven't posted any code. I'm just wondering what I should be looking for, because I'm at a loss.
This is how I running multiple features in parallel way.
from behave.__main__ import main as behave_main
#step(u'run in parallel "{feature}" "{scenario}"')
def step_impl(context, feature, scenario):
t = threading.Thread(
name='run test parallel',
target=parallel_executor,
args=[context, feature, scenario])
#args=[context, 'parallel_actions.feature', 'Make Cab-Cab communication'])
t.start()
def parallel_executor(context, feature_name, scenario):
os.chdir(testenv.PARALLEACTIONS_PATH)
behave_main('-i "{}" -n "{}" --no-capture --no-skipped'.format(feature_name, scenario))
And feature
Feature: testing parallel
Scenario: parallel run
When run in parallel "parallel_actions-1.feature" "Make Cab-Cab communication"
And run in parallel "parallel_actions-1.feature" "Another Scenario"
And run in parallel "another_parallel.feature" "Another Scenario 2"
I just create new thread and call behave executor directly, you don't need call behave.exe process 5 times separately but once. All features are executed at same time as parallel.
I cant answer your message errors, but you can try another approach (more behave way) to run behave features as parallel.
We're using Jest to power our Node.js tests, these interact with a Postgres database to test CRUD operations. We're currently passing the --runInBand CLI option to ensure our tests operate in serial, this works fine but is obviously slower than we'd like.
Now from reading around (and previous experience) I've found it useful to be able to mark groups of tests as parallelise-able. This is possible with nose in python but I cannot seem to find the syntax in Jest. Is this possible? Or is there another approach to speeding up database (or state constrained to generalise) tests that Jest advocates?
Thanks,
Alex
Put your tests in separate files (in a new sub folder if you want to keep them organized). That way Jest runs the files in parallel.
I'm trying to run cucumber scenarios in parallel from inside my gem. From other answers, I've found I can execute cucumber scenarios with the following:
runtime = Cucumber::Runtime.new
runtime.load_programming_language('rb')
#result = Cucumber::Cli::Main.new(['features\my_feature:20']).execute!(runtime)
The above code works fine when I run one scenario at a time, but when I run them in parallel using something like Celluloid or Peach, I get Ambiguous Step errors. It seems like my step definitions are being loaded for each parallel test and cucumber thinks I have multiple steps definitions of the same kind.
Any ideas how I can run these things in parallel?
Cucumber is not thread safe. Each scenario must be run in a separate thread with it's own cucumber runtime. Celluloid may try to run multiple scenarios on the same actor at the same time.
There is a project called cukeforker that can run scenarios in parallel but it only supports mri on linux and osx. It forks a subprocess per scenario.
I've created a fork of cukeforker called jcukeforker that supports both mri and jruby on linux. Jcukeforker will distribute scenarios to subprocesses. The subprocesses are reused. Subprocesses are used instead of threads to guarantee that each test has it's own global variables. This is important when running the subprocess on a vncserver which requires the DISPLAY variable to be set.