How do I force a Mochajs test to end completely without continuing on to the next tests. A scenario could be prevent any further tests if the environment was accidentally set to production and I need to prevent the tests from continuing.
I've tried throwing Errors but those don't stop the entire test because it's running asynchronously.
The kind of "test" you are talking about --- namely checking whether the environment is properly set for the test suite to run --- should be done in a before hook. (Or perhaps in a beforeEach hook but before seems more appropriate to do what you are describing.)
However, it would be better to use this before hook to set an isolated environment to run your test suite with. It would take the form:
describe("suite", function () {
before(function () {
// Set the environment for testing here.
// e.g. Connect to a test database, etc.
});
it("blah", ...
});
If there is some overriding reason that makes it so that you cannot create a test environment with a hook and you must perform a check instead you could do it like this:
describe("suite", function () {
before(function () {
if (production_environment)
throw new Error("production environment! Aborting!");
});
it("blah", ...
});
A failure in the before hook will prevent the execution of any callbacks given to it. At most, Mocha will perform the after hook (if you specify one) to perform cleanup after the failure.
Note that whether the before hook is asynchronous or not does not matter. (Nor does it matter whether your tests are asynchronous.) If you write it correctly (and call done when you are done, if it is asynchronous), Mocha will detect that an error occurred in the hook and won't execute tests.
And the fact that Mocha continues testing after you have a failure in a test (in a callback to it) is not dependent on whether the tests are asynchronous. Mocha does not interpret a failure of a test as a reason to stop the whole suite. It will continue trying to execute tests even if an earlier test has failed. (As I said above, a failure in a hook is a different matter.)
I generally agree with Glen, but since you have a decent use case, you should be able to trigger node to exit with the process.exit() command. See http://nodejs.org/api/process.html#process_process_exit_code. You can use it like so:
process.exit(1);
As of the mocha documentation, you can add --exit flag when you are executing the tests.
It will stop the execution whenever all the tests have been executed successfully or not.
ex:
mocha **/*.spec.js --exit
Related
I have a Cypress project where I use the Cypress session API to maintain a session throughout features.
Now I try switching from the deprecated Klaveness Cypress Cucumber Preprocessor to the replacement, Badeball's Cypress Cucumber Preprocessor. But I am running into an issue; the beforeEach() step where my authentication takes place gets repeated several times before the tests start. Eventually, Cypress "snaps out of it" and starts running the actual tests - but obviously this is very resource and time intensive, something is going wrong.
My setup:
Dependencies:
"cypress": "^9.6.1",
"#badeball/cypress-cucumber-preprocessor": "^9.1.3",
index.ts:
beforeEach(() => {
let isAuthInitialized = false;
function spyOnAuthInitialized(window: Window) {
window.addEventListener('react:authIsInitialized', () => {
isAuthInitialized = true;
});
}
login();
cy.visit('/', { onBeforeLoad: spyOnAuthInitialized });
cy.waitUntil(() => isAuthInitialized, { timeout: 30000 });
});
login() function:
export function login() {
cy.session('auth', () => {
cy.authenticate();
});
}
As far as I can see, I follow the docs for cy.session almost literally.
My authenticate command has only application specific steps, it does include a cy.visit('/') - after which my application is redirected to a login service (different domain) and then continues.
The problem
cy.session works OK, it creates a session on the first try - then each subsequent time it logs a succesful restore of a valid session. But this happens a number of times, it seems to get stuck in a loop.
Screenshot:
It looks to me like cy.visit() is somehow triggering the beforeEach() again. Perhaps clearing some session data (localstorage?) that causes my authentication redirect to happen again - or somehow makes Cypress think the test starts fresh. But of course beforeEach() should only happen once per feature.
I am looking at a diff of my code changes, and the only difference except the preprocessor change is:
my .cypress-cucumber-preprocessorrc.json (which I set up according to the docs
typing changes, this preprocessor is stricter about typings
plugins/index.ts file, also set up according to the docs
Am I looking at a bug in the preprocessor? Did I make a mistake? Or something else?
There are two aspects of Cypress + Cucumber with preprocessor that make this potentially confusing
Cypress >10 "Run all specs" behaviour
As demonstrated in Gleb Bahmutov PhD's great blog post, if you don't configure Cypress to do otherwise, running all specs runs each hook before each test. His proposed solution is to not use the "run all specs" button, which I find excessive - because there are ways around this; see below for a working solution with the Cucumber preprocessor.
Note: as of Cypress 10, "run all specs" is no longer supported (for reasons related to this unclarity).
Cucumber preprocessor config
The Cypress Cucumber preprocessor recommends to not use the config option nonGlobalStepDefinitions, but instead configure specific paths like (source):
"stepDefinitions": [
"cypress/integration/[filepath]/**/*.{js,ts}",
"cypress/integration/[filepath].{js,ts}",
"cypress/support/step_definitions/**/*.{js,ts}",
]
}
What it doesn't explicitly state though, is that the file which includes your hooks (in my case index.ts) should be excluded from these paths if you don't want them to run for each test! I could see how one might think this is obvious, but it's easy to accidentally include your hooks' file in this filepath config.
TLDR: If I exclude my index.ts file which includes my hooks from my stepDefinitions config, I can use "run all specs" as intended - with beforeEach() running only once before each test.
I wonder if there is any way to prevent tests from running when we have an error.
For example, in the beforeAll() function. I have tried "return" or "throw" an error but after that, Jest runs all of my tests.
So when my code in the beforeAll() function has an error that can affect other test results, I would like to be able to prevent Jest from running all the tests.
Jest tries to run all the tests even though we already know all the tests would fail.
You can try to use bail in config:
bail: 2 // finish after 2 failed tests
or
bail: true // finish after first
https://jestjs.io/docs/cli#--bail
To fail your test use:
fail('something wrong');
How to interrupt all Cypress tests on the first test failure?
We are using semaphore to launch complete e2e tests with Cypress for each PR. But it takes too much time.
I'd like to interrupt all tests on the first test failure.
Getting the complete errors is each developer's business when they develop. I just want to be informed ASAP if there is anything wrong prior to deploy, and don't have to wait for the full tests to complete.
So far the only solution I came up with was interrupting the tests on the current spec file with Cypress.
afterEach(() => {
if (this.currentTest.state === 'failed') {
Cypress.runner.end();
}
});
But this is not enough since it only interrupts the tests located on the spec file, not ALL the other files. I've done some intensive search on this matter today and it doesn't seem like this is a thing on Cypress.
So I'm trying other solutions.
1: with Semaphore
fail_fast:
stop:
when: "true"
It is supposed to interrupt the script on error. But it doesn't work: tests keep running after error. My guess is that Cypress will throw an error only when all tests are complete.
2: maybe with the script launching Cypress, but I'm out of ideas
Right now here are my scripts
"cy:run": "npx cypress run",
"cy:run:dev": "CYPRESS_env=dev npx cypress run",
"cy:test": "start-server-and-test start http-get://localhost:4202 cy:run"
EDIT: It seems like this feature was introduced, but it requires paid version of Cypress (Business Plan). More about it: Docs, comment in the thread
Original answer:
This has been a long-requested feature in Cypress for some reason still has not been introduced. There are some workarounds proposed by the community, however it is not guaranteed they will work. Check this thread on Cypress' Github for more details, maybe you will find a workaround that works for your case.
The solution by #user3504541 is excellent! Thanks a ton. I already started giving up on using Cypress since these issues keep popping up. But in any case, here's my config:
support/index.ts
declare global {
// eslint-disable-next-line
namespace Cypress {
interface Chainable {
interrupt: () => void
}
}
}
function abortEarly() {
if (this.currentTest.state === 'failed') {
return cy.task('shouldSkip', true)
}
cy.task('shouldSkip').then(value => {
if (value) return cy.interrupt()
})
}
commands/index.ts
Cypress.Commands.add('interrupt', () => {
eval("window.top.document.body.querySelector('header button.stop').click()")
})
In my case the Cypress tests were left pending indefinitely on the CI (Github action workflow) but with this fix they interrupt properly.
A little hack that worked for me
Cypress.Commands.add('interrupt', () => {
eval("window.top.document.body.querySelector('header button.stop').click()");
});
This is available as the Auto Cancelation feature, which is part of Smart Orchestration, but is only available to Business Plan. From the Auto Cancelation docs:
Continuous Integration (CI) pipelines are typically costly processes that can demand significant compute time. When a test failure occurs in CI, it often does not make sense to continue running the remainder of a test suite since the process has to start again upon merging of subsequent fixes and other code changes. When Auto Cancellation is enabled, once the number of failed tests goes over a preset threshold, the entire test run is canceled. Note that any in-progress specs will continue to run to completion.
I like letting mocha -w run in a terminal while I work on test so I get immediate feedback, but I can't always tell from a glance if it's changed or not when the status doesn't change - did it run, or did it get stuck (it's happened)?
I'd like to have a way to append a timestamp to the end of each test run, but ideally only when run in 'watch' mode - if I'm running it manually, of course I know if it ran or not.
For now, I'm appending an asynchronous console log to the last test that runs:
it('description', function () {
// real test parts.should.test.things();
// Trick - schedule the time to be printed to the log - so I can see when it was run last
setTimeout(() => console.log(new Date().toDateString() + " # " + new Date().toTimeString()), 5);
});
Obviously this is ugly and bad for several reasons:
It's manually added to the last test - have to know which that is
It is added every time that test is run, but never others - so if I run a different file or test -> no log; if I run only that test manually -> log
It's just kind of an affront to the purpose of the tests - subverting it to serve my will
I have seen some references to mocha adding a global.it object with the command line args, which could be searched for the '-w' flag, but that is even uglier, and still doesn't solve most of the problems.
Is there some other mocha add-in module which provides this? Or perhaps I've overlooked something in the options? Or perhaps I really shouldn't need this and I'm doing it all wrong to begin with?
Mocha supports root level hooks. If you place an after hook (for example) outside any describe block, it should run at the end of all tests. It won't run only in watch mode, of course, but should otherwise be fit for purpose.
I have multiple Mocha test files using one shared base file known as testBase.js. It's responsible for setting up all stubs and spies.
If I run individual file through mocha all test cases pass but when it run tests through mocha *.js, test cases begin to fail and raise error
TypeError: Attempted to wrap send which is already wrapped
Here are my beforeEach and afterEach blocks
beforeEach(function (done) {
context.alexaSpy = sinon.spy(alexa, "send");
}
beforeEach(function (done) {
context.alexaSpy.restore();
}
I actually printed out logs in both blocks and there is a strange thing I noticed. I see logs this way
-- BeforeEach Fired Test1
-- BeforeEach Fired Test1
-- AfterEach Fired Test1
-- AfterEach Fired Test1
I don't know why it's calling twice and its the root cause of the issue. BefireEach must not call twice for one mocha test.
Does importing multiple files call beforeEach twice? Can someone suggest any possible solution to this? I tried sinon.sandbox too but it does not work
We need to see how you require in the base file to be certain.
My guess is simply that you require the file from multiple files, and each time you do this you add the setup and teardown functions. That happens because all the tests share the same outer scope. Requiring the Base file ten times will add the beforeEach ten times too.
The right way to do this would be using sinon.sandbox or sinon-test. Much easier to avoid one test interfering with the next.
But no matter what you do, you would need to export the function and run that in a beforeEach in each file
Typically like this
const base = require('./base')
describe('module one', ()=> {
beforeEach(base.commonStubs);
it('should.... ',..);
})