how to handle new page on button click in pupeteer? - node.js

i am using puppeteer in a project to test a web page , in the page i have several buttons that open a new tab in the browser , how can i handle that using puppeteer?
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({ defaultViewport: null });
const page = await browser.newPage();
// go to the URL
await page.goto('https://example.com/', {waitUntil: 'networkidle'});
await page.click('.btnId'); //opens new tab with Page 2
// handle Page 2
// process Page 2
// close Page 2
// go back to Page 1
browser.close();
})();
how can i handle the page 2 ?

await page.waitFor(3 * 1000) // wait for new page to open
const pages = await browser.pages() // get all pages
const page2 = pages[pages.length - 1] // get the new page
// process the new page
await page2.close()

Hope this helps in solving the problem.
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({ defaultViewport: null });
const page = await browser.newPage();
// go to the URL
await page.goto('https://example.com/', {waitUntil: 'networkidle'});
await page.click('.btnId'); //opens new tab with Page 2
// you can make this as dynamic as well depends on the website and use case.
const [tabOne, tabTwo] = (await browser.pages());
// use the tabs Page objects properly
console.log("Tab One Title ",await tabOne.title());
// use the tabs Page objects properly
console.log("Tab Two Title ",await tabTwo.title());
// you can use close property for tab when it's done.
browser.close();
})();

Related

Puppeteer not clicking button with text

I have a simple function that tries to accept the cookies
Here's my code:
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://www.sport1.de/live/darts-sport');
await page.click('button[text=AKZEPTIEREN]');
// await page.screenshot({ path: 'example.png' });
// await browser.close();
})();
The cookie popup is placed in an iframe. You have to switch to iframe by contentFrame to be able to click on the accept button.
Also, if you want to filter by textContent, you need to use XPath. With CSS selector you can't get elements by its textContent.
const cookiePopUpIframeElement=await page.$("iframe[id='sp_message_iframe_373079']");
const cookiePopUpIframe=await cookiePopUpIframeElement.contentFrame();
const acceptElementToClick = await cookiePopUpIframe.$x("//button[text()='AKZEPTIEREN']");
await acceptElementToClick[0].click();

Selecting the radio button with puppeteer

I am trying to fetch data and trigger some automatic buying process with the following website. https://www.klwines.com/
Was using "puppeteer" methods with NodeJS to process the script. According to the following screenshot provided, I got stuck with an issue where I cannot select one of the a radio button from the list since all radio buttons having the same id. What I am trying to do is just trying to select the last radio button from the following list and then trigger he button shown in the image. I was using the following NodeJS code with the help of puppeteer.
await page.waitForNavigation();
await page.waitForSelector('[name="continue"]');
const radio = await page.evaluate("table tr:nth-child(4) > td > input[type=radio]")
radio.click()
Please note that the page variable is defined as the following.
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
If someone can help with this to find a way that would be really great full.
You can try this way;
const puppeteer = require('puppeteer');
exports.yourStatus = async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.klwines.com/');
const data = await page.evaluate(() => {
function cleanData(element) {
const items = element.getElementById('Shepmente_0__shepmentewayCode');
return [...items].map(item => {
console.log(item)
});
}
return data;
};

puppeteer overridePermissions clipboard-read not working on createIncognitoBrowserContext()

The following code is viable for reading the clipboard in headless/headfull:
var context = await client.defaultBrowserContext();
await context.overridePermissions('http://localhost', ['clipboard-read']);
page = await browser.newPage();
await page.goto( 'http://localhost/test/', {waitUntil: 'load', timeout: 35000});
// click button for clipboard..
let clipboard = await page.evaluate(`(async () => await navigator.clipboard.readText())()`);
But when you later start incognito its not working anymore:
const incognito = await client.createIncognitoBrowserContext();
page = await incognito.newPage();
and you get:
DOMException: Read permission denied.
I currently try to figure out to combine both.. Anybody know how to set overridePermissions inside of the new incognito window?
Please notice I do not want to use the incognito chrome arg at the start. I want to manually create new incognito pages inside of my scripts with correct overridePermissions.
I am having the very same issue. Here's a minimal reproducible example.
Nodejs version: v16.13.1
puppeteer version: puppeteer#14.4.1
'use strict';
const puppeteer = require('puppeteer');
const URL = 'https://google.com';
(async () => {
const browser = await puppeteer.launch();
const context = browser.defaultBrowserContext();
context.overridePermissions(URL, ['clipboard-read', 'clipboard-write'])
const page = await browser.newPage();
await page.goto(URL, {
waitUntil: 'networkidle2',
});
await page.evaluate(() => navigator.clipboard.writeText("Injected"));
const value = await page.evaluate(() => navigator.clipboard.readText());
console.log(value);
})();

Cannot reach a dynamic page with puppeteer

I need to read data on https://www.cmegroup.com/tools-information/quikstrike/options-calendar.html
I tried to click on FX tab from page.click in puppeteer, but the page remains on the default.
Any help welcome
const puppeteer = require('puppeteer');
let scrape = async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('https://www.cmegroup.com/tools-information/quikstrike/options-calendar.html');
await page.waitFor(1000);
//div select FX
await page.click('#ctl00_MainContent_ucViewControl_IntegratedCMEOptionExpirationCalendar_ucViewControl_ucProductSelector_lvGroups_ctrl3_lbProductGroup');
//browser.close();
return result;
};
scrape().then((value) => {
console.log(value); // Success!
});
I couldn't find the element you're looking for on that page. However, this might be helpful:
Wait for the selector to appear on the page before clicking on it:
await page.waitForSelector(selector);
If still facing the issue, try using Javascript click method:
await page.$eval(selector, elem => elem.click());

How to get Puppeteer waitForNavigation Working after click

I am trying to get puppeteer to wait for the navigation to finish before moving on to the next statement. Based on the Docs for waitForNavigation() , the code should work below. but it just skips to the next statement and I have to use a workaround to wait for a specific URL in the response.
I have tried all the waituntil options as well
( load, domcontentloaded, networkidle0 and networkidle2 ) .
Any ideas how I could get that working properly is appreciated.
const browser = await puppeteer.launch({
headless: false,
})
const page = await browser.newPage()
const home = page.waitForNavigation()
await page.goto(loginUrl)
await home
const login = page.waitForNavigation()
await page.type('#email', config.get('login'))
await page.type('#password', config.get('password'))
await page.click('#submitButton')
await login // << skips over this
// the following line is my workaround and it works , but ideally I don't want
// to specify the expected "after" page each time I navigate
await page.waitForResponse(request => request.url() === 'http://example.com/expectedurl')
The function page.waitForNavigation() waits for navigation to begin and end.
The navigation has already been initiated with page.click().
Therefore, you can use Promise.all() to avoid the race condition between the mentioned functions:
const browser = await puppeteer.launch({
headless: false,
});
const page = await browser.newPage();
await page.goto(loginUrl);
await page.type('#email', config.get('login'));
await page.type('#password', config.get('password'));
await Promise.all([
page.click('#submitButton'),
page.waitForNavigation({
waitUntil: 'networkidle0',
}),
]);
await browser.close();
I have been going through the same problem, I used pending-xhr-request
it solved many problems when the requests were expected, but when I have late requests I have faced many problems, it took me a while to solve the problem so I built a package Puppeteer-response-waiter to do that
const puppeteer = require('puppeteer');
const {ResponseWaiter} = require('puppeteer-response-waiter');
let browser = await puppeteer.launch({ headless: false });
let page = await browser.newPage();
let responseWaiter = new ResponseWaiter(page);
await page.goto('http://somesampleurl.com');
// start listening
responseWaiter.listen();
// do something here to trigger requests
await responseWaiter.wait();
// all requests are finished and responses are all returned back
// remove listeners
responseWaiter.stopListening();
await browser.close();
hope this will solve your problem.

Resources