I am wondering if it is possible in Jest to skip one or more tests in a suite based on the value of a variable used in the beforeEach function.
My code:
let errLoadingData = false;
beforeEach(async () => {
await page.goto('myurl.com');
try {
await waitForSelector('error-banner');
} catch (error) {
errLoadingData = true;
throw new Error(
'Myurl page displaying error to user.',
);
}
});
test('Visit myurl page', async () => {
// wait for and click elements on myurl page
.
.
.
}
I would like to not run Visit myurl page if errLoadingData = true.
Thanks in advance!
Related
I have this code:
const mocha = require("mocha");
const assert = require("assert");
describe("saving records", function (done) {
it("Saves a record to the database", function (done) {
let mx = false;
setTimeout(() => {
mx = false; //#### note this line
}, 6000);
done();
assert(mx === true);
});
});
I am not sure why, but mocha gives a test pass in the terminal that says:
saving records
✓ Saves a record to the database
1 passing (30ms)
Question no.1 : Why is the test passing at this point
The next weird thing is: It doesn't even wait for 6 seconds seconds before the test passes. Since, I've used done parameter, Question no. 2: shouldn't it wait until the execution is done?
Have I misinterpreted some information?
const mocha = require("mocha");
const assert = require("assert");
describe("saving records", function (done) {
it("Saves a record to the database", function (done) {
let mx = false;
setTimeout(() => {
mx = false; //#### note this line
assert(mx === true);
done();
}, 6000);
});
});
In your variant test passes because assert is after done();
Working with async tests described here
I'm currently setting up a CI environment to automate e2e tests our team runs in a test harness. I am setting this up on Gitlab and currently using Puppeteer. I have an event that fires from our test harness that designates when the test is complete. Now I am trying to "pool" the execution so I don't use up all resources or run out of listeners. I decided to try out "puppeteer-cluster" for this task. I am close to having things working, however I can't seem to get it to wait for the event on page before closing the browser. Prior to using puppeteer-cluster, I was passing in a callback to my function and when the custom event was fired (injected via exposeFunction), I would go about calling it. That callback function is now being passed in data though now and therefore not waiting. I can't seem to find a way to get the execution to wait and was hoping someone might have an idea here. If anyone has any recommendations, I'd love to hear them.
test('Should launch the browser and run e2e tests', async (done) => {
try {
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_CONTEXT,
maxConcurrency: 10,
monitor: false,
timeout: 1200000,
puppeteerOptions: browserConfig
});
// Print errors to console
cluster.on("taskerror", (err, data) => {
console.log(`Error crawling ${data}: ${err.message}`);
});
//Setup our task to be run
await cluster.task( async ({page, data: {testUrl, isLastIndex, cb}, worker}) => {
console.log(`Test starting at url: ${testUrl} - isLastIndex: ${isLastIndex}`);
await page.goto(testUrl);
await page.waitForSelector('#testHarness');
await page.exposeFunction('onCustomEvent', async (e) => {
if (isLastIndex === true){ ;
//Make a call to our callback, finalizing tests are complete
cb();
}
console.log(`Completed test at url: ${testUrl}`);
});
await page.evaluate(() => {
document.addEventListener('TEST_COMPLETE', (e) => {
window.onCustomEvent('TEST_COMPLETE');
console.log("TEST COMPLETE");
});
});
});
//Perform the assignment of all of our xml tests to an array
let arrOfTests = await buildTestArray();
const arrOfTestsLen = arrOfTests.length;
for( let i=0; i < arrOfTestsLen; ++i){
//push our tests on task queue
await cluster.queue( {testUrl: arrOfTests[i], isLastIndex: (i === arrOfTestsLen - 1), cb: done });
};
await cluster.idle();
await cluster.close();
} catch (error) {
console.log('ERROR:',error);
done();
throw error;
}
});
So I got something working, but it really feels hacky to me and I'm not really sure it is the right approach. So should anyone have the proper way of doing this or a more recommended way, don't hesitate to respond. I am posting here shoudl anyone else deal with something similar. I was able to get this working with a bool and setInterval. I have pasted working result below.
await cluster.task( async ({page, data: {testUrl, isLastIndex, cb}, worker}) => {
let complete = false;
console.log(`Test starting at url: ${testUrl} - isLastIndex: ${isLastIndex}`);
await page.goto(testUrl)
await page.waitForSelector('#testHarness');
await page.focus('#testHarness');
await page.exposeFunction('onCustomEvent', async (e) => {
console.log("Custom event fired");
if (isLastIndex === true){ ;
//Make a call to our callback, finalizing tests are complete
cb();
complete = true;
//console.log(`VAL IS ${complete}`);
}
console.log(`Completed test at url: ${testUrl}`);
});
//This will run on the actual page itself. So setup an event listener for
//the TEST_COMPLETE event sent from the test harness itself
await page.evaluate(() => {
document.addEventListener('TEST_COMPLETE', (e) => {
window.onCustomEvent('TEST_COMPLETE');
});
});
await new Promise(resolve => {
try {
let timerId = setInterval(()=>{
if (complete === true){
resolve();
clearInterval(timerId);
}
}, 1000);
} catch (e) {
console.log('ERROR ', e);
}
});
});
Ultimately speaking - is there a practical method (also maybe by inserting some JS constructs into the code) to break or halt the long lasting JS code during the execution? For example: can it be interrupted by some process.* object constructs, or similar? Or the other way? The valid solution may even include the NodeJS process to be killed and/or restarted. Thank you!
EDIT:
I need to execute some particular user code on the server, using Function clause (ala eval, - let alone security concerns). I cannot insert any extra code inside it, only enclose it. What I need is to have a possibility to break user code after 5 minutes, if it is not finished by this time. For example:
usercode = 'Some code from the user';
pre_code = 'some controlling code for breaking the user code';
post_code = 'another controlling code';
fcode = pre_code + usercode + post_code;
<preparations for breaking usercode>
(new Function(fcode))(); // This MUST exit in 5 minutes
Edit:
Answering your edit. I see the intention now. If it is running in nodejs, you can use worker_thread for that https://nodejs.org/api/worker_threads.html#worker_threads_worker_workerdata.
For example:
// main.js
const runCode = (code) => {
const worker = new Worker("./code-executor.js", { workerData: { code: guestCode } });
const promise = new Promise((resolve) => {
setTimeout(() => worker.kill(), 60000 * 5);
worker.on("error", () => {
return reject(new SomeCustomError())
});
worker.on("message", (message) => {
if(message.success) return resolve(message.result);
return reject(new Error(message.error));
});
});
promise.finally(() => { worker.kill() });
return promise;
}
// code-executor.js
const { workerData, parentPort } = require("worker_threads");
const { code } = workerData;
Promise.resolve()
.then(() => (new Function(fcode))())
.then((result) => {
parentPort.postMessage({
success: true,
result: value
})
})
.catch((error) => {
parentPort.postMessage({
success: true,
error: error.message
})
});
If it's in browser https://developer.mozilla.org/en-US/docs/Web/API/Worker
The WebAPI is not exactly the same but the logic should be similar
Original
Killing a process. Also read: https://nodejs.org/api/process.html#process_signal_events
process.kill(pid, "SIGINT")
"Killing" a long running function, you gotta hack a bit. There's no elegant solution. Inject a controller which can be mutated outside of the long running function. To stop it from the outside, set controller.isStopped = true
export const STOP_EXECUTION = Symbol();
function longRunning(controller){
... codes
// add stopping point
if(controller.isStopped) throw STOP_EXECUTION;
... codes
// add stopping point
if(controller.isStopped) throw STOP_EXECUTION;
... codes
}
// catch it by
try{
longRunnning();
}catch(e){
switch(true){
e === STOP_EXECUTION: ...; // the longRunning function is stopped from the outside
default: ...; // the longRunning function is throwing not because of being stopped
}
}
Gist: https://gist.github.com/Kelerchian/3824ca4ce1be390d34c5147db671cc9b
I have a small web scraping application that downloads multiple files from a web application where the URLs require visting the page.
It works fine if I keep the browser instance alive in between runs, but I want to close the instance in between runs. When I call browser.close() my downloads are stopped because the chrome instance is closed before the downloads have finished.
Does puppeteer provide a way to check if downloads are still active, and wait for them to complete? I've tried page.waitForNavigation({ waitUntil: "networkidle0" }) and "networkidle2", but those seem to wait indefinitely.
node.js 8.10
puppeteer 1.10.0
Update:
It's 2022. Use Playwright to get away from this mass. manage downloads
It also has 'smarter' locator, which examine selectors every time before click()
old version for puppeteer:
My solution is to use chrome's own chrome://downloads/ page to managing download files. This solution can be very easily to auto restart a failed download using chrome's own feature
This example is 'single thread' currently, because it's only monitoring the first item appear in the download manager page. But you can easily adapt it to 'infinite threads' by iterating through all download items (#frb0~#frbn) in that page, well, take care of your network:)
dmPage = await browser.newPage()
await dmPage.goto('chrome://downloads/')
await your_download_button.click() // start download
await dmPage.bringToFront() // this is necessary
await dmPage.waitForFunction(
() => {
// monitoring the state of the first download item
// if finish than return true; if fail click
const dm = document.querySelector('downloads-manager').shadowRoot
const firstItem = dm.querySelector('#frb0')
if (firstItem) {
const thatArea = firstItem.shadowRoot.querySelector('.controls')
const atag = thatArea.querySelector('a')
if (atag && atag.textContent === '在文件夹中显示') { // may be 'show in file explorer...'? you can try some ids, classess and do a better job than me lol
return true
}
const btn = thatArea.querySelector('cr-button')
if (btn && btn.textContent === '重试') { // may be 'try again'
btn.click()
}
}
},
{ polling: 'raf', timeout: 0 }, // polling? yes. there is a 'polling: "mutation"' which kind of async
)
console.log('finish')
An alternative if you have the file name or a suggestion for other ways to check.
async function waitFile (filename) {
return new Promise(async (resolve, reject) => {
if (!fs.existsSync(filename)) {
await delay(3000);
await waitFile(filename);
resolve();
}else{
resolve();
}
})
}
function delay(time) {
return new Promise(function(resolve) {
setTimeout(resolve, time)
});
}
Implementation:
var filename = `${yyyy}${mm}_TAC.csv`;
var pathWithFilename = `${config.path}\\${filename}`;
await waitFile(pathWithFilename);
You need check request response.
await page.on('response', (response)=>{ console.log(response, response._url)}
You should check what is coming from response then find status, it comes with status 200
Using puppeteer and chrome I have one more solution which might help you.
If you are downloading the file from chrome it will always have ".crdownload" extension. And when file is completely downloaded that extension will vanish.
So, I am using recurring function and maximum number of times it can iterate, If it doesn't download the file in that time.. I am deleting it. And I am constantly checking a folder for that extention.
async checkFileDownloaded(path, timer) {
return new Promise(async (resolve, reject) => {
let noOfFile;
try {
noOfFile = await fs.readdirSync(path);
} catch (err) {
return resolve("null");
}
for (let i in noOfFile) {
if (noOfFile[i].includes('.crdownload')) {
await this.delay(20000);
if (timer == 0) {
fs.unlink(path + '/' + noOfFile[i], (err) => {
});
return resolve("Success");
} else {
timer = timer - 1;
await this.checkFileDownloaded(path, timer);
}
}
}
return resolve("Success");
});
}
Here is another function, its just wait for the pause button to disappear:
async function waitForDownload(browser: Browser) {
const dmPage = await browser.newPage();
await dmPage.goto("chrome://downloads/");
await dmPage.bringToFront();
await dmPage.waitForFunction(() => {
try {
const donePath = document.querySelector("downloads-manager")!.shadowRoot!
.querySelector(
"#frb0",
)!.shadowRoot!.querySelector("#pauseOrResume")!;
if ((donePath as HTMLButtonElement).innerText != "Pause") {
return true;
}
} catch {
//
}
}, { timeout: 0 });
console.log("Download finished");
}
I didn't like solutions that were checking DOM or file system for the file.
From Chrome DevTools Protocol documentation](https://chromedevtools.github.io/) I found two events,
Page.downloadProgress and Browser.downloadProgress. (Though Page.downloadProgress is marked as deprecated, that's the one that worked for me.)
This event has a property called state which tells you about the state of the download. state could be inProgress, completed and canceled.
You can wrap this event in a Promise to await it till the status changes to completed
async function waitUntilDownload(page, fileName = '') {
return new Promise((resolve, reject) => {
page._client().on('Page.downloadProgress', e => {
if (e.state === 'completed') {
resolve(fileName);
} else if (e.state === 'canceled') {
reject();
}
});
});
}
and await it as follows,
await waitUntilDownload(page, fileName);
Created simple await function that will check for file rapidly or timeout in 10 seconds
import fs from "fs";
awaitFileDownloaded: async (filePath) => {
let timeout = 10000
const delay = 200
return new Promise(async (resolve, reject) => {
while (timeout > 0) {
if (fs.existsSync(filePath)) {
resolve(true);
return
} else {
await HelperUI.delay(delay)
timeout -= delay
}
}
reject("awaitFileDownloaded timed out")
});
},
You can use node-watch to report the updates to the target directory. When the file upload is complete you will receive an update event with the name of the new file that has been downloaded.
Run npm to install node-watch:
npm install node-watch
Sample code:
const puppeteer = require('puppeteer');
const watch = require('node-watch');
const path = require('path');
// Add code to initiate the download ...
const watchDir = '/Users/home/Downloads'
const filepath = path.join(watchDir, "download_file");
(async() => {
watch(watchDir, function(event, name) {
if (event == "update") {
if (name === filepath)) {
browser.close(); // use case specific
process.exit(); // use case specific
}
}
})
Tried doing an await page.waitFor(50000); with a time as long as the download should take.
Or look at watching for file changes on complete file transfer
you could search in the download location for the extension the files have when still downloading 'crdownload' and when the download is completed the file is renamed with the original extension: from this 'video_audio_file.mp4.crdownload' turns into 'video_audio_file.mp4' without the 'crdownload' at the end
const fs = require('fs');
const myPath = path.resolve('/your/file/download/folder');
let siNo = 0;
function stillWorking(myPath) {
siNo = 0;
filenames = fs.readdirSync(myPath);
filenames.forEach(file => {
if (file.includes('crdownload')) {
siNo = 1;
}
});
return siNo;
}
Then you use is in an infinite loop like this and check very a certain period of time, here I check every 3 seconds and when it returns 0 which means there is no pending files to be fully downloaded.
while (true) {
execSync('sleep 3');
if (stillWorking(myPath) == 0) {
await browser.close();
break;
}
}
// Balance.jsx
...
updateToken () {
const parseResponse = (response) => {
if (response.ok) {
return response.json()
} else {
throw new Error('Could not retrieve access token.')
}
}
const update = (data) => {
if (data.token) {
this.data.accessTokenData = data
} else {
throw new Error('Invalid response from token api')
}
}
if (this.props.balanceEndpoint !== null) {
return fetch(this.props.accessTokenEndpoint, {
method: 'get',
credentials: 'include'
})
.then(parseResponse)
.then(update)
.catch((err) => Promise.reject(err))
}
}
componentDidMount () {
this.updateToken()
.then(() => this.updateBalance())
}
}
// Test
it('updates the balance', () => {
subject = mount(<Balance {...props} />)
expect(fetchMock.called('balance.json')).to.be.true
})
I can't figure out how to test the above using Mocha. The code is does work the method updateBalance is called and the fetch api call actually does happen, but the test still fails. If I call updateBalance() synchronously it passes... How do I tell the test to wait for the promise to resolve?
You don't really say what you want to test that the
method does, but if all you want to test is that the method resolves on a network call, then there is no need for Sinon or any of that, as this is all you need:
describe("BalanceComponent", () => {
it("should resolve the promise on a successful network call", () => {
const component = new BalanceComponent({any: 'props', foo: 'bar'});
// assumes you call a network service that returns a
// successful response of course ...
return component.updateToken();
});
});
This will test that the method actually works, but it is slow and is not a true unit test, as it relies on the network being there and that you run the tests in a browser that can supply you with a working implementation of fetch. It will fail as soon as you run it in Node or if the service is down.
If you want to test that the method actually does something specific, then you would need to to that in a function passed to then in your test:
it("should change the token on a successful network call", () => {
const component = new BalanceComponent({any: 'props', foo: 'bar'});
const oldToken = component.data.accessTokenData;
return component.updateToken().then( ()=> {
assert(oldToken !== component.data.accessTokenData);
});
});
If you want to learn how to test code like this without being reliant on there being a functioning link to the networked service you are calling, you can check out the three different techniques described in this answer.