page.select() in Puppeteer not working as expeted with kendo dropdown - node.js

page.select() in Puppeteer not working as expected with kendo drop down, its not throwing any error. but the value not getting selected
am using puppeteer 5.6.0
sync function create(page)
{
const engCreationbtn = await page.$('.AddEngBtn');
await engCreationbtn.click();
await page.type('#createEngSidebar input[name="engName"]','puppeteer',{delay:20});
//await page.select('#createEngSidebar select[name="engType"]', 'Audit')
await page.select('select[name="country"]','IN')
await page.type('#createEngSidebar input[name="KPMGOffice"]','Tice',{delay:20});
//await page.select('#createEngSidebar select[name="timezone"]','India Standard Time||Asia/Kolkata',{delay:20})
await page.type('#createEngSidebar input[name="ClientName"]','pup-pepsi',{delay:20});
await page.type('#createEngSidebar .flyoutctrlpart .k-numeric-wrap .k-input','133',{delay:20});
const createbtn = await page.$('#createEngSidebar .flyoutfooter .btnPrimary');
await page.screenshot({path: 'engCreation.png'});
await createbtn.click();
await page.screenshot({path: 'afterengCreation.png'});
}
Html where am trying to set
<div class="flyoutctrlpart">
<span title="" class="k-widget k-dropdown k-header" unselectable="on" role="listbox" aria-haspopup="true" aria-expanded="false" tabindex="0" aria-owns="" aria-disabled="false" aria-busy="false" style="" aria-activedescendant="3f3ec341-a8f4-45bb-b617-0ce20b6b3db2"><span unselectable="on" class="k-dropdown-wrap k-state-default"><span unselectable="on" class="k-input">Select Country/Jurisdiction</span><span unselectable="on" class="k-select"><span unselectable="on" class="k-icon k-i-arrow-s">select</span></span></span><select name="country" kendo-drop-down-list="" k-data-text-field="'name'" k-data-value-field="'code'" k-data-source="engPopCtrl2019V1.countryList" k-option-label="engPopCtrl2019V1.SelectCountry" k-value-primitive="true" k-on-change="engPopCtrl2019V1.CountryChanged()" k-ng-model="engPopCtrl2019V1.country" data-role="dropdownlist" style="display: none;"><option value="" selected="selected">Select Country/Jurisdiction</option><option value="AF">Afghanistan</option><option value="AL">Albania and Kosovo</option><option value="DZ">Algeria</option><option value="AD">Andorra</option><option value="AO">Angola</option><option value="AI">Anguilla</option><option value="AG">Antigua and Barbuda</option><option value="AR">Argentina</option><option value="AM">Armenia</option><option value="AW">Aruba</option><option value="AU">Australia</option><option value="AT">Austria</option><option value="AZ">Azerbaijan</option><option value="SI">Slovenia</option><option value="ZA">South Africa</option><option value="SP">Spain</option><option value="LK">Sri Lanka</option><option value="LC">St. Lucia</option><option value="MF">St. Maarten</option><option value="VC">St. Vincent and the Grenadines</option><option value="SR">Suriname</option><option value="SE">Sweden</option><option value="CH">Switzerland</option><option value="SY">Syria</option><option value="TW">Taiwan</option><option value="TZ">Tanzania</option><option value="TH">Thailand</option><option value="TG">Togo</option><option value="TT">Trinidad and Tobago</option><option value="TN">Tunisia</option><option value="TR">Turkey</option><option value="TC">Turks & Caicos</option><option value="AE">UAE</option><option value="UG">Uganda</option><option value="UA">Ukraine</option><option value="UK">United Kingdom</option><option value="US">United States</option><option value="UY">Uruguay</option><option value="UZ">Uzbekistan</option><option value="VU">Vanuatu</option><option value="VE">Venezuela</option><option value="VN">Vietnam</option><option value="YE">Yemen</option><option value="ZM">Zambia</option><option value="ZW">Zimbabwe</option></select></span>
</div>
i have tried this against the kendo site also same am not able to set the value
const puppeteer = require('puppeteer');
const homepage = 'https://demos.telerik.com/kendo-ui/dropdownlist/index';
async function test() {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.setViewport({width: 1400, height: 1400});
await page.goto(homepage, {waitUntil: 'networkidle2'});
await page.select('#size','XL - 7 5/8"');
}
test()

page.select() will work only on select elements and not custom elements like span, it is behaving as expected.
Kendo does a data binding to the select element hence it does not update when you select/update the select element.
The easiest way is to use the kendoUI itself.
On the demo page, the UI initializes by following,
var size = $("#size").data("kendoDropDownList");
To set the value,
size.value('M - 7 1/4"')
To get the value,
size.value()
Same with the input box,
var color = $("#color").data("kendoDropDownList");
color.value() // "1" => which is Blank
color.value("2") // "2" => Now it's Orange
Working example with puppeteer,
const puppeteer = require('puppeteer');
const homepage = 'https://demos.telerik.com/kendo-ui/dropdownlist/index';
async function test() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(homepage, {waitUntil: 'networkidle2'});
// use the demo pages built in jQuery and Kendo to change the results
const sizeValue = await page.evaluate(()=>{
var size = $("#size").data("kendoDropDownList");
size.value('XL - 7 5/8"')
return size.value()
});
await page.screenshot({path: 'kendoTest.png', fullPage: true});
await browser.close();
return sizeValue;
}
test().then(console.log) // should say XL - 7 5/8"
Result:

Related

Puppeteer: wait for request to finish after dropdown selection FILTER

I'm trying to scrape data off a car sales website. It has several "dropdowns". My problem is that the lists are always filled / complete and when I select the values as make or model the following are filtered, the options can be repeated or different, and I don't know what to wait to know if my list has loaded correctly. If I make my code with timers it works great, but they will be removed. Next I leave a part of my code:
const scraperObject = {
url: 'https://www.example.com/cars',
async scraper(browser){
....
async function getMarcas(page) {
return await getSelectOptions(page,
'select#category__l2 > option');
}
async function getModelos(page) {
return await getSelectOptions(page,
'select#category__l3 > option');
}
async function getAnodesde(page) {
return await getSelectOptions(page,
'select#year-filter-from > option');
}
async function getAnohasta(page) {
return await getSelectOptions(page,
'select#year-filter-to > option');
}
await page.waitForFunction('document.querySelector("#category__l2").length > 0');
let marcas = await getMarcas(page);
for (const [ i, state ] of marcas.entries()) {
await page.select('#category__l2', state.value);
await page.waitForSelector('#category__l3:not(:disabled)');
let modelos = await getModelos(page);
for (const [ j, state1 ] of modelos.entries()) {
await page.select('#category__l3', state1.value);
await page.waitForFunction('document.querySelector("#year-filter-from").length < 29');
await page.waitForFunction('document.querySelector("#year-filter-to").length < 29');
let anodesde = await getAnodesde(page);
let anohasta = await getAnohasta(page);
const desde = await page.evaluate(() => {return document.querySelector('#year-filter-from').length;});
for (var ii=0;ii<5000;ii++){
if (desde > 0){
ii=5001
}else{
desde = await page.evaluate(() => {return document.querySelector('#year-filter-from').length;});
}
}
const hasta = await page.evaluate(() => {return document.querySelector('#year-filter-to').length;});
for (var iii=0;iii<5000;iii++){
if (hasta > 0){
iiii=5001
}else{
hasta = await page.evaluate(() => {return document.querySelector('#year-filter-to').length;});
}
}
for (const [ k, state2 ] of anodesde.entries()) {
await page.select('#year-filter-from', state2.value);
await page.select('#year-filter-to', state2.value);
for(var gg=0,len = anohasta.length;gg<=len;gg++){
if (anohasta[gg].name ===state2.name){
index = gg;
console.log('lo encontro en la pos: '+gg);
var selectedVal2 = await page.$eval("#year-filter-to", selectedValue=> selectedValue.value);
for(let g=0;g<5000;g++){
if(selectedVal2 ==state2.value){
g=5001;
}else{
await console.log(anohasta);
await page.select('#year-filter-to', state2.value);
selectedVal2 = await page.$eval("#year-filter-to", selectedValue=> selectedValue.value);
await page.waitFor(4000);
}
}
break;
}else{
await console.log ('ERROR');
}
}
await page.click('#root-app > section > form > button');
let scrapedData = [];
...
Beyond the result or the error of my code I would like to know what to expect to know when my "dropdowns" are already FILTERED.
Thank you for taking the time to read and sorry for my English
<div class="nav-search-classi-content">
<div class="nav-search-classi__section">
<label for="category">Categorías</label>
<select id="category" name="category" class="category-filter" tabindex="10" style="display: block; min-width: 212px;"><option value="MLA1744">Autos y camionetas</option><option value="MLA1763">Motos</option><option value="MLA58254">Camiones</option><option value="MLA1785">Náutica</option><option value="MLA7312">Maquinaria agrícola</option><option value="MLA1745">Autos de colección</option><option value="MLA405183">Maquinaria vial</option><option value="MLA1784">Planes de ahorro</option><option value="MLA80579">Motorhomes</option><option value="MLA93412">Autos chocados y averiados</option><option value="MLA93430">Semirremolques</option><option value="MLA51547">Colectivos</option><option value="MLA1907">Otros vehículos</option></select>
<select id="category__l2" name="BRAND" class="category-filter category-dynamic" tabindex="11" style="display: block; min-width: 212px;"><option value="">Todas las marcas</option><option value="67695">Alfa Romeo</option><option value="40661">Audi</option><option value="66352">BMW</option><option value="2088960">Baic</option><option value="389167">Changan</option><option value="389168">Chery</option><option value="58955">Chevrolet</option><option value="66395">Chrysler</option><option value="389169">Citroën</option><option value="2103629">DFSK</option><option value="2130485">DS</option><option value="10075">Daihatsu</option><option value="66708">Dodge</option><option value="67781">Fiat</option><option value="66432">Ford</option><option value="396748">Foton</option><option value="389171">Geely</option><option value="60559">Honda</option><option value="1089">Hyundai</option><option value="60648">Isuzu</option><option value="396749">Iveco</option><option value="60395">Jeep</option><option value="374002">Kia</option><option value="380878">Lifan</option><option value="75966">Mercedes-Benz</option><option value="65127">Mini</option><option value="1138">Mitsubishi</option><option value="60505">Nissan</option><option value="60279">Peugeot</option><option value="56870">Porsche</option><option value="2710997">RAM</option><option value="9909">Renault</option><option value="60589">Rover</option><option value="60268">SEAT</option><option value="380886">Shineray</option><option value="38765">Smart</option><option value="60285">Subaru</option><option value="43943">Suzuki</option><option value="60297">Toyota</option><option value="60249">Volkswagen</option><option value="60658">Volvo</option></select>
<select id="category__l3" name="category__l3" class="category-filter category-dynamic" tabindex="12" disabled="" style="display: block; min-width: 212px;"><option value="">Todos los modelos</option></select>
</div>
<div class="nav-search-classi__section">
<div class="nav-search-classi__range-filter">
<select id="price-filter-from" name="price_from" class="range-filter range-filter-from" tabindex="100"><option value="">Precio desde</option><option value="1">Sin precio mínimo</option><option value="650000">$ 650.000</option><option value="1150000">$ 1.150.000</option><option value="1650000">$ 1.650.000</option><option value="2150000">$ 2.150.000</option><option value="2650000">$ 2.650.000</option><option value="3150000">$ 3.150.000</option><option value="3650000">$ 3.650.000</option><option value="4150000">$ 4.150.000</option><option value="4650000">$ 4.650.000</option><option value="5150000">$ 5.150.000</option><option value="5650000">$ 5.650.000</option><option value="6150000">$ 6.150.000</option><option value="6650000">$ 6.650.000</option><option value="7150000">$ 7.150.000</option><option value="7650000">$ 7.650.000</option></select>
<select id="price-filter-to" name="price_to" class="range-filter range-filter-to" tabindex="101"><option value="">Precio hasta</option><option value="650000">$ 650.000</option><option value="1150000">$ 1.150.000</option><option value="1650000">$ 1.650.000</option><option value="2150000">$ 2.150.000</option><option value="2650000">$ 2.650.000</option><option value="3150000">$ 3.150.000</option><option value="3650000">$ 3.650.000</option><option value="4150000">$ 4.150.000</option><option value="4650000">$ 4.650.000</option><option value="5150000">$ 5.150.000</option><option value="5650000">$ 5.650.000</option><option value="6150000">$ 6.150.000</option><option value="6650000">$ 6.650.000</option><option value="7150000">$ 7.150.000</option><option value="7650000">$ 7.650.000</option><option value="0">Sin precio máximo</option></select>
</div>
<div class="nav-search-classi__range-filter">
<select id="year-filter-from" name="years_from" class="range-filter range-filter-from" tabindex="102"><option value="">Año desde</option><option value="2021">2021</option><option value="2020">2020</option><option value="2019">2019</option><option value="2018">2018</option><option value="2017">2017</option><option value="2016">2016</option><option value="2015">2015</option><option value="2014">2014</option><option value="2013">2013</option><option value="2012">2012</option><option value="2011">2011</option><option value="2010">2010</option><option value="2009">2009</option><option value="2008">2008</option><option value="2007">2007</option><option value="2006">2006</option><option value="2005">2005</option><option value="2004">2004</option><option value="2003">2003</option><option value="2002">2002</option><option value="2001">2001</option><option value="2000">2000</option><option value="1999">1999</option><option value="1998">1998</option><option value="1997">1997</option><option value="1996">1996</option><option value="1995">1995</option><option value="1994">1994</option><option value="1993">1993</option><option value="1992">1992</option><option value="1991">1991</option><option value="1989">1989</option><option value="1988">1988</option><option value="1987">1987</option><option value="1984">1984</option><option value="1981">1981</option><option value="1980">1980</option><option value="1978">1978</option><option value="1974">1974</option><option value="1971">1971</option><option value="1970">1970</option><option value="1968">1968</option></select>
<select id="year-filter-to" name="years_to" class="range-filter range-filter-to" tabindex="103"><option value="">Año hasta</option><option value="2021">2021</option><option value="2020">2020</option><option value="2019">2019</option><option value="2018">2018</option><option value="2017">2017</option><option value="2016">2016</option><option value="2015">2015</option><option value="2014">2014</option><option value="2013">2013</option><option value="2012">2012</option><option value="2011">2011</option><option value="2010">2010</option><option value="2009">2009</option><option value="2008">2008</option><option value="2007">2007</option><option value="2006">2006</option><option value="2005">2005</option><option value="2004">2004</option><option value="2003">2003</option><option value="2002">2002</option><option value="2001">2001</option><option value="2000">2000</option><option value="1999">1999</option><option value="1998">1998</option><option value="1997">1997</option><option value="1996">1996</option><option value="1995">1995</option><option value="1994">1994</option><option value="1993">1993</option><option value="1992">1992</option><option value="1991">1991</option><option value="1989">1989</option><option value="1988">1988</option><option value="1987">1987</option><option value="1984">1984</option><option value="1981">1981</option><option value="1980">1980</option><option value="1978">1978</option><option value="1974">1974</option><option value="1971">1971</option><option value="1970">1970</option><option value="1968">1968</option></select>
</div>
</div>
</div>
I add html code requested by #Yves Gurcan
One example you could use is with the page.waitForTimeout() function. This will wait a given time in ms before completing the next function. I believe it should work for your issue, however I'm not sure if this is the best practice.
An example code snippet would be:
// Go to page
await page.goto('https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_select', { waitUntil: 'domcontentloaded' });
// Wait for selector
await page.waitForSelector('#cars');
// Make selection
await page.select('#cars', 'volvo');
// Then you can wait a set amount of time (5s in this example)
await page.waitForTimeout(5000)
// Whatever you want to do after
I'm not sure how your page is being rendered out but sometimes using page.waitForSelector() would work fine and its definitely better practice then page.waitForTimeout().

puppeteer show result in screenshot but cant get value of that result

after click on the search button, the puppeteer take a screenshot but I can't get the element's value
here is my code
await page.$eval('#textInputSelector',(el, licenceInfo) =>
(el.value = licenceInfo),licenceInfo)
const searchBtn = await page.$x('//*[#id="searchBtnXPath"]')
await searchBtn[0].click()
await page.waitFor(4000);
console.log(await page.$eval('#selector1', el => el.innerText));
await makeScreenShot(page, screenPath, { fullPage: true })
and the result is (red box)
result's image
and its HTML code output
<div>
<span id="#selector1" >
Your search returned no results. Please modify your search criteria
and try again.
</span>
</div>
and button html code
<div id="#selector2">
<a id="searchBtnXPath" href="
javascript:
__doPostBack('ctl00$PlaceHolderMain$btnNewSearch','');
var p = new ProcessLoading();p.showLoading(false);">
<span>Search</span>
</a>
</div>
and this is my error
Error: failed to find element matching selector "#selector1"
Use this instead:
console.log(await page.$eval('body #selector1', el => el.innerHTML));

puppeteer press Enter on input type search

I want to press key enter, after fill a input, but nothing happens, it is not a link or button to click
<div class="search-input">
<a class="search machin" style="display:none;z-index: 0;position: absolute;"></a>
<input type="search" id="global-search"> //events jquery keypress and keyup
<a class="close-search" style="z-index: 0;">
<i class="m times"></i>
</a>
</div>
jquery : function(t) {
return "undefined" != typeof w && w.event.triggered !== t.type ? w.event.dispatch.apply(e,
arguments) : void 0
}
I verify with screenshot that the input type="search" was well filled and it is.
await page.focus('#global-search');
await page.type('#global-search',"string to find"); //it works
or await page.$eval('#global-search', (el, value) => el.value = value, myLocalValue);//it works
i try this:
await page.keyboard.press("Enter");
or
await page.keyboard.down('Enter');
await page.keyboard.up('Enter');
the press button Enter have to complete the url https://lesite.fr/recherche?q=string_to_find but it is not the case the url remains desperately https://lesite.fr/
I must find the result of the seach in
<div class="h-container">
<div class="h">
<div class="h-image SearchBanner">
<img src="/-/media/_banner.jpg">
<div class="rot-block"></div>
</div>
<div class="h-details">
<h1>16 RÉSULTATS</h1>
<div class="search-result">HERE THE RESULT<span class="secondary-title">string to find</span></div>
</div>
</div>
</div>
the probleme is the url is not completed
i read a lot of google search (Pressing Enter button in puppeteer.)
Help
await page.focus('#header > div.component.container.float-wrapper > div.search > div > div.search-input');
await page.type('#header > div.component.container.float-wrapper > div.search > div > div.search-input',"PSG");
await page.keyboard.press('ArrowRight');
await page.focus('#header > div.component.container.float-wrapper > div.search > div > div.search-input');
await page.keyboard.press('Enter'); // Enter Key
await page.keyboard.press('NumpadEnter'); // Numeric Keypad Enter Key
await page.keyboard.press('\n'); // Shortcut for Enter Key
await page.keyboard.press('\r'); // Shortcut for Enter Key
it works after 30 hours to search
Thanks a lot for your answer
In this working example, it will search Google from an HTML input search tag.
Try this workaround, and tell me if this solve your problem.
Try also each commented 4 lines above it.
const puppeteer = require ('puppeteer')
;(async () => {
const browser = await puppeteer.launch ({
headless: false,
devtools: true,
slowMo: 50 // delete this if you like
})
const contentHTML = '<form name="search" method="get" action="https://www.google.com/search?"><div class="search-input"><a class="search machin" style="display:none;z-index: 0;position: absolute;"></a><input type="search" id="global-search" name="q"><a class="close-search" style="z-index: 0;"><i class="m times"></i></a></div></form>'
const page = (await browser.pages())[0]
// const google = await page.goto('https://www.google.com', {waitUntil: 'networkidle2'})
const setContent = await page.setContent(contentHTML, {waitUntil: 'domcontentloaded'})
const focusSearch = await page.focus('#global-search')
const typeSearch = await page.type('#global-search',"string_to_find") //it works
const searchSubmitted = await Promise.all([
page.waitForNavigation({waitUntil: 'domcontentloaded'}),
// const pressReturn = page.type('#global-search', '\r'),
// const pressReturn = page.type('#global-search', String.fromCharCode(13)),
// const pressReturn = page.keyboard.type('\n'),
// const pressReturn = page.keyboard.down('NumpadEnter'),
const pressReturn = page.keyboard.press('Enter') // 4 LINES ABOVE ALSO WORKS, TRY THEM ALL
])
// 'Enter': {'keyCode': 13, 'code': 'Enter', 'key': 'Enter', 'text': '\r'}
// <div class="search-result">HERE THE RESULT<span class="secondary-title">string to find</span></div>
// const searchString = await page.$eval('span.secondary-title', el => el.innerText);
})()

With Puppeteer how can I click the parent element of my selector?

The markup i have to work with looks like this:
<label>
<input type="radio" name="myfield" value="Yes" size>
</label>
I want to call page.click(selector) with the radio as the selector, but I can't. I don't think it is visible because of the size attribute.
My javascript looks like this:
const page = await browser.newPage();
const selector = 'input[name="myfield"]';
await page.click(selector);
So I would like to target and click the parent label element.
How do I change the value of my selector constant to target the label?
Sorry, I didn't explain very well. By can't, i mean that the element is not visible and therefore i don't believe it can technically be clicked. Therefore I think i need to target the label which is visible, but i don't know how I target it
not visible or not visible at the moment?
Have you tried to use waitForSelector(selector) ?
const page = await browser.newPage();
const selector = 'input[name="myfield"]';
await page.waitForSelector(selector); // waiting here before click
await page.click(selector);
Or something like:
const page = await browser.newPage();
const selector = 'label';
await page.waitForSelector(selector);
await page.evaluate((_) => {
document.querySelector('label > input[name="myfield"]').parentElement.click()
});

puppeteer howto get element tagName

I would like to get an element's tagName. Should be button in following example.
const puppeteer = require('puppeteer')
async function run () {
const browser = await puppeteer.launch({headless: false})
const page = await browser.newPage()
const html = `
<div>
<button type="button">click me</button>
<span>Some words.</span>
</div>
`
await page.setContent(html)
const elements = await page.$$('button')
const tagName = await elements[0].$eval('*', node => node.tagName)
console.log(tagName) // expect to be 'button'
await browser.close()
}
run()
The error message said Error: failed to find element matching selector "*"
I can tell elements matched one element as elements.length is 1
Where is wrong?
========== Edit ==========
Let's say I already had elements beforehand, how to get the tagName out of it.
Thanks!
Try using page.$eval to select the button, and then get the tagName from the button:
const tagName = await page.$eval('button', button => button.tagName);
If you already have an elementHandle like elements[0], you can get an attribute from that element by passing it through page.evaluate:
const tagName = await page.evaluate(
element => element.tagName,
elements[0]
);
It appears your elements is an array of ElementHandles.
In that case, there may be a slightly more straightforward syntax:
const tag_name = await (await elements[0].getProperty('tagName')).jsonValue()
This does not involve referring to the page object.
Thanks!

Resources