increase timeout when running against remote grid - serenity-js

when running a test locally it succeeds, but when configuring a remote grid, it fails with
1) Scenario: Login - features/api.feature:10
Step: When he enters his credentials - features/api.feature:13
Step Definition: node_modules/serenity-js/src/serenity-cucumber/webdriver_synchroniser.ts:46
Message:
function timed out after 5000 milliseconds
How can I increase the timeout value?
Thanks & Ciao
Stefan

Hi Stefan and thanks for giving Serenity/JS a try!
You have a couple of options here, depending on what is timing out.
As it's Protractor that's in charge of the timeouts, you'll need to look into your protractor.conf.js file.
Let's assume that your protractor.conf.js file looks more or less like the snippet below. I omit the Serenity/JS and Cucumber.js config for brevity as they're described at serenity-js.org:
exports.config = {
baseUrl: 'http://your.webapp.com',
// Serenity/JS config
framework: ...
specs: [ 'features/**/*.feature' ],
cucumberOpts: {
// ...
},
};
0. Increasing the overall timeout
To start with, you might want to increase the overall timeout of all the tests (for Protractor 5.0.0 the default value is set to 11s).
To do this, add the allScriptsTimeout entry to your config:
exports.config = {
allScriptsTimeout: <appropriate_timeout_in_millis>
// ... rest of the config file
}
1. Loading the page
If the webapp under test is slow to load, you can tweak the getPageTimeout property (default set to 10s):
exports.config = {
getPageTimeout: <appropriate_timeout_in_millis>
// ... rest of the config file
}
2. A specific Cucumber step
If a specific Cucumber step is timing out (which is most likely the case here, as Cucumber.js sets the default value of the cucumber step timeout to 5s), you can increase the timeout by changing the step definition (value in millis):
this.Given(/^When he enters his credentials$/, { timeout: 10 * 1000 }, () => {
return stage.theActorInTheSpotlight().attemptsTo(
Login.withTheirCredentials()
);
});
Please note that in the above answer I'm assuming that you're using Serenity/JS with Cucumber to test an Angular app. If you're using a different web framework (like React), the test might also time out when Protractor is waiting for Angular to load.
If this describes your scenario, you might want to ignoreSynchronization:
exports.config = {
onPrepare: function() {
browser.ignoreSynchronization = false;
}
// ... rest of the config file
}
To find out more, check out the Protractor documentation and the already mentioned Cucumber docs. I'll also add an article on serenity-js.org shortly to describe the different options so everything is in one place :-)
Hope this helps!
Jan

Related

BeforeEach step is repeated with cy.session using cypress-cucumber-preprocessor

I have a Cypress project where I use the Cypress session API to maintain a session throughout features.
Now I try switching from the deprecated Klaveness Cypress Cucumber Preprocessor to the replacement, Badeball's Cypress Cucumber Preprocessor. But I am running into an issue; the beforeEach() step where my authentication takes place gets repeated several times before the tests start. Eventually, Cypress "snaps out of it" and starts running the actual tests - but obviously this is very resource and time intensive, something is going wrong.
My setup:
Dependencies:
"cypress": "^9.6.1",
"#badeball/cypress-cucumber-preprocessor": "^9.1.3",
index.ts:
beforeEach(() => {
let isAuthInitialized = false;
function spyOnAuthInitialized(window: Window) {
window.addEventListener('react:authIsInitialized', () => {
isAuthInitialized = true;
});
}
login();
cy.visit('/', { onBeforeLoad: spyOnAuthInitialized });
cy.waitUntil(() => isAuthInitialized, { timeout: 30000 });
});
login() function:
export function login() {
cy.session('auth', () => {
cy.authenticate();
});
}
As far as I can see, I follow the docs for cy.session almost literally.
My authenticate command has only application specific steps, it does include a cy.visit('/') - after which my application is redirected to a login service (different domain) and then continues.
The problem
cy.session works OK, it creates a session on the first try - then each subsequent time it logs a succesful restore of a valid session. But this happens a number of times, it seems to get stuck in a loop.
Screenshot:
It looks to me like cy.visit() is somehow triggering the beforeEach() again. Perhaps clearing some session data (localstorage?) that causes my authentication redirect to happen again - or somehow makes Cypress think the test starts fresh. But of course beforeEach() should only happen once per feature.
I am looking at a diff of my code changes, and the only difference except the preprocessor change is:
my .cypress-cucumber-preprocessorrc.json (which I set up according to the docs
typing changes, this preprocessor is stricter about typings
plugins/index.ts file, also set up according to the docs
Am I looking at a bug in the preprocessor? Did I make a mistake? Or something else?
There are two aspects of Cypress + Cucumber with preprocessor that make this potentially confusing
Cypress >10 "Run all specs" behaviour
As demonstrated in Gleb Bahmutov PhD's great blog post, if you don't configure Cypress to do otherwise, running all specs runs each hook before each test. His proposed solution is to not use the "run all specs" button, which I find excessive - because there are ways around this; see below for a working solution with the Cucumber preprocessor.
Note: as of Cypress 10, "run all specs" is no longer supported (for reasons related to this unclarity).
Cucumber preprocessor config
The Cypress Cucumber preprocessor recommends to not use the config option nonGlobalStepDefinitions, but instead configure specific paths like (source):
"stepDefinitions": [
"cypress/integration/[filepath]/**/*.{js,ts}",
"cypress/integration/[filepath].{js,ts}",
"cypress/support/step_definitions/**/*.{js,ts}",
]
}
What it doesn't explicitly state though, is that the file which includes your hooks (in my case index.ts) should be excluded from these paths if you don't want them to run for each test! I could see how one might think this is obvious, but it's easy to accidentally include your hooks' file in this filepath config.
TLDR: If I exclude my index.ts file which includes my hooks from my stepDefinitions config, I can use "run all specs" as intended - with beforeEach() running only once before each test.

gulp-eslint not outputting to file - unable to properly configure writableStream

Issue - User cannot get output to print to file for gulp-lint process
Documentation Reference It is observed in the documentation that a writeableStream is a valid configuration, but regrettably it does not denote or provide clarification on how to do this....and I have tried the solution below, along with others to no avail...so am seeking any insight / support that can be provided
Observations - other users had published guidance suggesting a stream similar to this, but when attempting this 2 things are observed....
The IntelliJ IDE notes that the parameter "writable" should be updated to "writableStream"
The build output generates the file, but the file is empty, therefore I am obviously missing something with respect to configuring / establishing the stream properly
Sample Code Block
'use strict';
const {src, task} = require('gulp');
const eslint = require('gulp-eslint');
const fs =require('fs');
task('lint', () => {
return src(['**/*.js', '!**/node_modules/**', '!**/handlebars.runtime-v4.1.2.js', '!**/parsley.js', '!**/slick.js','!*SampleTests.js'])
// Runs eslint
.pipe(eslint())
// Sets the format of the console
.pipe(eslint.format('table',fs.createWriteStream('eslint-result.xml')))
// To have the process exit with an error code (1) on lint error, return the stream and pipe to failAfterError last
.pipe(eslint.failOnError()
.pipe(eslint.results(results => {
// Called once for all ESLint results.
console.log(`Total Results: ${results.length}`);
console.log(`Total Warnings: ${results.warningCount}`);
console.log(`Total Errors: ${results.errorCount}`);
}))
)
});
A new day brings new results I guess....after running a build with this configuration again it worked..much to my surprise.
Note, I am using maven as a build process, and am invoking this using the maven-frontend plugin....and it is important to note that the result file will NOT appear until AFTER the build process has finished

Configure jest timeout once for all tests

According to the docs one can increase the default async timeout from 5000ms using the jest-object
More specifically, by using the jestsettimeouttimeout
The issue I am facing is I am running a series of tests against an API that is very slow, 5-15 second response times, configuring this jest object at the top of each test is painfully annoying.
Is it possible to declare these settings once before all test files are run?
Jest offers a testTimeout configuration option you can add to your package.json:
"jest": {
"testTimeout": 15000,
}
OK, putting bits together:
Option "setupTestFrameworkScriptFile" was replaced by configuration "setupFilesAfterEnv", which supports multiple paths
https://jestjs.io/docs/en/jest-object#jestsettimeouttimeout
https://jestjs.io/docs/en/jest-object#jestdisableautomock
The Jest search box doesn't actually return anything when you search for: setupFilesAfterEnv
And docs talk about: setupTestFrameworkScriptFile (which also doesn't return anything on the search:/ )
Anyway, the docs leave you scratching your head but this works:
jest.config.js:
module.exports = {
setupFilesAfterEnv: ['./setup.js'],
setup.js:
jest.setTimeout(10000); // in milliseconds
The jest folks should make it easier to find this information.
Use testTimeout. In yourjest.config.js (or similar), add the following:
export SECONDS = 1000;
module.exports = {
testTimeout: 60 * SECONDS
}
If you are working with react and initializing you app using create-react-app, then under your src/ directory you should have a file named setupTests.js. Here you can setup a global timeout for all of your tests just by insert this line after the import statement for #testing-libary
jest.setTimeout(15000); // in milliseconds

sails js cannot change timeout load hook, for bootstrap

Hello i have some problem with the bootstrap of sails, i need run a function that take approximately a minute in finished, before sails initialize, so tried do, in the bootstrap hook, but i got this error:
warn: Bootstrap is taking unusually long to execute its callback (2000 milliseconds).
Perhaps you forgot to call it? The callback is the first argument of the function, `cb`.
and searching in internet a the solution of the people was that, i have to create:
config/hookTimeout.js
and put inside :
module.exports.hookTimeout = {
hookTimeout:120000
}
to override the time of load, but still i get the same error, but i figurethat the other hooks had the hookTimeout = 120000, just the bootstrap hook dont.
The field name is bootstrapTimeout instead of hookTimeout (see here for more details).
And your config file (with a name as you like) should be as follows (nothing after module.exports):
module.exports = {
bootstrapTimeout: 60000, // in millis
};
Check if you call callback in the bootstrap file.
The simples version of config/bootstrap.js should be like next:
module.exports.bootstrap = function(cb) {
cb();
};

Intern.io Single Page Application Functional Testing

I have a single page application that uses Dojo to navigate between pages.
I am writing some functional tests using intern and there are some niggly issues I am trying to weed out.
Specifically I am having trouble getting intern to behave with timeouts. None of the timeouts seem to have any effect for me. I am trying to set the initial load timeout using "setPageLoadTimeout(30000)" but this seems to get ignored. I also call "setImplicitWaitTimeout(10000)" but again this seems to have no effect.
The main problem I have is that it may take a couple of seconds in my test environment for the request to be sent and the response parsed and injected into the DOM. The only way I have been able to get around this is by explicitly calling "sleep(3000)" for example but this can be a bit hit & miss and sometimes the DOM elements are not ready by the time I query them. (as mentioned setImplicitWaitTimeout(10000) doesn't seem to have an effect for me)
With the application I fire an event when the DOM has been updated. I use dojo.subscribe to hook into this in the applictaion. Is it possible to use dojo.subscribe within intern to control the execution of my tests?
Heres a sample of my code. I should have also mentioned that I use Dijit so there is also a slight delay when the response comes back and the widgets are being created (via data-dojo-type declarations)...
define([
'intern!object',
'intern/chai!assert',
'require',
'intern/node_modules/dojo/topic'
], function (registerSuite, assert, require, topic) {
registerSuite({
name: 'Flow1',
// login to the application
'Login': function(remote) {
return remote
.setPageLoadTimeout(30000)
.setImplicitWaitTimeout(10000)
.get(require.toUrl('https://localhost:8080/'))
.elementById('username').clickElement().type('user').end()
.elementById('password').clickElement().type('password').end()
.elementByCssSelector('submit_button').clickElement().end();
},
// check the first page
'Page1':function() {
return this.remote
.setPageLoadTimeout(300000) // i've tried these calls in various places...
.setImplicitWaitTimeout(10000) // i've tried these calls in various places...
.title()
.then(function (text) {
assert.strictEqual(text, 'Page Title');})
.end()
.active().type('test').end()
.elementByCssSelector("[title='Click Here for Help']").clickElement().end()
.elementById('next_button').clickElement().end()
.elementByCssSelector("[title='First Name']").clear().type('test').end()
.elementByCssSelector("[title='Gender']").clear().type('Female').end()
.elementByCssSelector("[title='Date Of Birth']").type('1/1/1980').end()
.elementById('next_button').clickElement().end();
},
// check the second page
'Page2':function() {
return this.remote
.setImplicitWaitTimeout(10000)
.sleep(2000) // need to sleep here to wait for request & response injection and DOM parsing etc...
.source().then(function(source){
assert.isTrue(source.indexOf('test') > -1, 'Should contain First Name: "test"');
}).end()
// more tests etc...
}
});
});
I'm importing the relevant Dojo module from the intern dojo node module but I'm unsure of how to use it.
Thanks
Your test is timing out, because Intern tests have an explicit timeout set to 30s that is not accessible through their API. It can be changed by adding 'intern/lib/Test' to your define array, and then overwriting the timeout from the Test's object, e.g. Test.prototype.timeout = 60000;.
For example:
define([
'intern!object',
'intern/chai!assert',
'require',
'intern/node_modules/dojo/topic',
'intern/lib/Test'
], function (registerSuite, assert, require, topic, Test) {
Test.prototype.timeout = 60000;
...
}
This should change the timeout to one minute instead of 30s, to prevent your test timing out.

Resources