after click on the search button, the puppeteer take a screenshot but I can't get the element's value
here is my code
await page.$eval('#textInputSelector',(el, licenceInfo) =>
(el.value = licenceInfo),licenceInfo)
const searchBtn = await page.$x('//*[#id="searchBtnXPath"]')
await searchBtn[0].click()
await page.waitFor(4000);
console.log(await page.$eval('#selector1', el => el.innerText));
await makeScreenShot(page, screenPath, { fullPage: true })
and the result is (red box)
result's image
and its HTML code output
<div>
<span id="#selector1" >
Your search returned no results. Please modify your search criteria
and try again.
</span>
</div>
and button html code
<div id="#selector2">
<a id="searchBtnXPath" href="
javascript:
__doPostBack('ctl00$PlaceHolderMain$btnNewSearch','');
var p = new ProcessLoading();p.showLoading(false);">
<span>Search</span>
</a>
</div>
and this is my error
Error: failed to find element matching selector "#selector1"
Use this instead:
console.log(await page.$eval('body #selector1', el => el.innerHTML));
Related
I'm using Puppeteer to extract the text of a span by it's class name but I'm getting returned nothing. I don't know if its because the page isn't loading in time or not.
This is my current code:
async function Reload() {
Page.reload()
Price = await Page.evaluate(() => document.getElementsByClassName("text-robux-lg wait-for-i18n-format-render"))
console.log(Price)
}
Reload()
HTML
<div class="icon-text-wrapper clearfix icon-robux-price-container">
<span class="icon-robux-16x16 wait-for-i18n-format-render"></span>
<span class="text-robux-lg wait-for-i18n-format-render">689</span>
</div>
because the function that you passed to Page.evaluate() returns a non-Serializable value.
from the puppeteer official document
If the function passed to the page.evaluate returns a non-Serializable value, then page.evaluate resolves to undefined
so you have to make the function that passed to Page.evaluate() returns the text of span element rather than returns the Element object of span.
like the following code
const puppeteer = require('puppeteer');
const htmlCode = `
<div class="icon-text-wrapper clearfix icon-robux-price-container">
<span class="icon-robux-16x16 wait-for-i18n-format-render"></span>
<span class="text-robux-lg wait-for-i18n-format-render">689</span>
</div>
`;
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setContent(htmlCode);
const price = await page.evaluate(() => {
const elements = document.getElementsByClassName('text-robux-lg wait-for-i18n-format-render');
return Array.from(elements).map(element => element.innerText); // as you see, now this function returns array of texts instead of Array of elements
})
console.log(price); // this will log the text of all elements that have the specific class above
console.log(price[0]); // this will log the first element that have the specific class above
// other actions...
await browser.close();
})();
NOTE: if you want to get the html code from another site by its url use page.goto() instead of page.setContent()
NOTE: because you are using document.getElementsByClassName() the returned value of the function that passed to page.evaluate() in the code above will be array of texts and not text as document.getElementById() do
NOTE: if you want to know what is the difference between Serializable objects and non-serializable objects read the answers of this question on Stackoverflow
Hi Pupeteer community,
I'm struggling with a simple test automation to click an "Accept All" Banner shown on the bottom of a page. The button, unfortunately, is not of type button, but wrapped within an link tag. It is hidden in the following code:
<a _tagc1="" class="accept-all mat-button mat-button-base mat-raised-button" mat-button="" aria-label="Some label" tabindex="0" aria-disabled="false">
<span class="mat-button-wrapper">
<div _tagc1="" class="action-wrapper-notice">
<strong _tagc1="">Some Button Text</strong>
</div>
</span>
<div class="mat-button-ripple mat-ripple" matripple=""></div>
<div class="mat-button-focus-overlay"></div>
</a>
I tried to click it with Puppeteer like this:
async function run () {
const browser = await puppeteer.launch({headless : false});
const page = await browser.newPage();
await page.goto(url, {waitUntil:'networkidle2'});
await page.screenshot({path: 'screenshot1.png'});
await page.click('a.accept-all.mat-button.mat-button-base.mat-raised-button', {waitUntil:'networkidle2'})
await page.screenshot({path: 'screenshot2.png'});
browser.close();
}
.. but cant get the button pressed.
Does anyone know how to best select the button? The only thing I did and seems to work is sending repeatedly Keyboard Tabs to the page and then submitting an Enter key. But I am looking at a more robust and elegant solution.
Thanks for helping a Puppeteer newbie!
I am attempting to perform a web scraping operation and would like to get all the children element in a html tree similar to this:
<div class="main">
<p>Some p</p>
<a>Some a</a>
<br>
<br>
<em>
<p>Another p</p>
<a>Another a</a>
<br>
<br>
<em>
//...
</div>
I scraped the html using Puppeteer like so and managed to get the children but as a string format. Here are my attempts:
const children = await page.evaluate(el => el.children, await page.$('div.main'))
console.log(children)
//prints {"1": {}, "2": {}, "3": {} ...}
I then refer to this post and this post, and attempted this:
const children = await page.evaluate(() => {
var children = [...document.querySelector('div.main').children];
return children.map((e) => e.outerHTML);
})
console.log(children)
//prints all children correctly, but all as strings
Is there a way to get all child elements under a tag but with all the DOM attributes retained so that I can loop over each element, perform some algorithmic operation and extract some attributes.
I want to press key enter, after fill a input, but nothing happens, it is not a link or button to click
<div class="search-input">
<a class="search machin" style="display:none;z-index: 0;position: absolute;"></a>
<input type="search" id="global-search"> //events jquery keypress and keyup
<a class="close-search" style="z-index: 0;">
<i class="m times"></i>
</a>
</div>
jquery : function(t) {
return "undefined" != typeof w && w.event.triggered !== t.type ? w.event.dispatch.apply(e,
arguments) : void 0
}
I verify with screenshot that the input type="search" was well filled and it is.
await page.focus('#global-search');
await page.type('#global-search',"string to find"); //it works
or await page.$eval('#global-search', (el, value) => el.value = value, myLocalValue);//it works
i try this:
await page.keyboard.press("Enter");
or
await page.keyboard.down('Enter');
await page.keyboard.up('Enter');
the press button Enter have to complete the url https://lesite.fr/recherche?q=string_to_find but it is not the case the url remains desperately https://lesite.fr/
I must find the result of the seach in
<div class="h-container">
<div class="h">
<div class="h-image SearchBanner">
<img src="/-/media/_banner.jpg">
<div class="rot-block"></div>
</div>
<div class="h-details">
<h1>16 RÉSULTATS</h1>
<div class="search-result">HERE THE RESULT<span class="secondary-title">string to find</span></div>
</div>
</div>
</div>
the probleme is the url is not completed
i read a lot of google search (Pressing Enter button in puppeteer.)
Help
await page.focus('#header > div.component.container.float-wrapper > div.search > div > div.search-input');
await page.type('#header > div.component.container.float-wrapper > div.search > div > div.search-input',"PSG");
await page.keyboard.press('ArrowRight');
await page.focus('#header > div.component.container.float-wrapper > div.search > div > div.search-input');
await page.keyboard.press('Enter'); // Enter Key
await page.keyboard.press('NumpadEnter'); // Numeric Keypad Enter Key
await page.keyboard.press('\n'); // Shortcut for Enter Key
await page.keyboard.press('\r'); // Shortcut for Enter Key
it works after 30 hours to search
Thanks a lot for your answer
In this working example, it will search Google from an HTML input search tag.
Try this workaround, and tell me if this solve your problem.
Try also each commented 4 lines above it.
const puppeteer = require ('puppeteer')
;(async () => {
const browser = await puppeteer.launch ({
headless: false,
devtools: true,
slowMo: 50 // delete this if you like
})
const contentHTML = '<form name="search" method="get" action="https://www.google.com/search?"><div class="search-input"><a class="search machin" style="display:none;z-index: 0;position: absolute;"></a><input type="search" id="global-search" name="q"><a class="close-search" style="z-index: 0;"><i class="m times"></i></a></div></form>'
const page = (await browser.pages())[0]
// const google = await page.goto('https://www.google.com', {waitUntil: 'networkidle2'})
const setContent = await page.setContent(contentHTML, {waitUntil: 'domcontentloaded'})
const focusSearch = await page.focus('#global-search')
const typeSearch = await page.type('#global-search',"string_to_find") //it works
const searchSubmitted = await Promise.all([
page.waitForNavigation({waitUntil: 'domcontentloaded'}),
// const pressReturn = page.type('#global-search', '\r'),
// const pressReturn = page.type('#global-search', String.fromCharCode(13)),
// const pressReturn = page.keyboard.type('\n'),
// const pressReturn = page.keyboard.down('NumpadEnter'),
const pressReturn = page.keyboard.press('Enter') // 4 LINES ABOVE ALSO WORKS, TRY THEM ALL
])
// 'Enter': {'keyCode': 13, 'code': 'Enter', 'key': 'Enter', 'text': '\r'}
// <div class="search-result">HERE THE RESULT<span class="secondary-title">string to find</span></div>
// const searchString = await page.$eval('span.secondary-title', el => el.innerText);
})()
page.select() in Puppeteer not working as expected with kendo drop down, its not throwing any error. but the value not getting selected
am using puppeteer 5.6.0
sync function create(page)
{
const engCreationbtn = await page.$('.AddEngBtn');
await engCreationbtn.click();
await page.type('#createEngSidebar input[name="engName"]','puppeteer',{delay:20});
//await page.select('#createEngSidebar select[name="engType"]', 'Audit')
await page.select('select[name="country"]','IN')
await page.type('#createEngSidebar input[name="KPMGOffice"]','Tice',{delay:20});
//await page.select('#createEngSidebar select[name="timezone"]','India Standard Time||Asia/Kolkata',{delay:20})
await page.type('#createEngSidebar input[name="ClientName"]','pup-pepsi',{delay:20});
await page.type('#createEngSidebar .flyoutctrlpart .k-numeric-wrap .k-input','133',{delay:20});
const createbtn = await page.$('#createEngSidebar .flyoutfooter .btnPrimary');
await page.screenshot({path: 'engCreation.png'});
await createbtn.click();
await page.screenshot({path: 'afterengCreation.png'});
}
Html where am trying to set
<div class="flyoutctrlpart">
<span title="" class="k-widget k-dropdown k-header" unselectable="on" role="listbox" aria-haspopup="true" aria-expanded="false" tabindex="0" aria-owns="" aria-disabled="false" aria-busy="false" style="" aria-activedescendant="3f3ec341-a8f4-45bb-b617-0ce20b6b3db2"><span unselectable="on" class="k-dropdown-wrap k-state-default"><span unselectable="on" class="k-input">Select Country/Jurisdiction</span><span unselectable="on" class="k-select"><span unselectable="on" class="k-icon k-i-arrow-s">select</span></span></span><select name="country" kendo-drop-down-list="" k-data-text-field="'name'" k-data-value-field="'code'" k-data-source="engPopCtrl2019V1.countryList" k-option-label="engPopCtrl2019V1.SelectCountry" k-value-primitive="true" k-on-change="engPopCtrl2019V1.CountryChanged()" k-ng-model="engPopCtrl2019V1.country" data-role="dropdownlist" style="display: none;"><option value="" selected="selected">Select Country/Jurisdiction</option><option value="AF">Afghanistan</option><option value="AL">Albania and Kosovo</option><option value="DZ">Algeria</option><option value="AD">Andorra</option><option value="AO">Angola</option><option value="AI">Anguilla</option><option value="AG">Antigua and Barbuda</option><option value="AR">Argentina</option><option value="AM">Armenia</option><option value="AW">Aruba</option><option value="AU">Australia</option><option value="AT">Austria</option><option value="AZ">Azerbaijan</option><option value="SI">Slovenia</option><option value="ZA">South Africa</option><option value="SP">Spain</option><option value="LK">Sri Lanka</option><option value="LC">St. Lucia</option><option value="MF">St. Maarten</option><option value="VC">St. Vincent and the Grenadines</option><option value="SR">Suriname</option><option value="SE">Sweden</option><option value="CH">Switzerland</option><option value="SY">Syria</option><option value="TW">Taiwan</option><option value="TZ">Tanzania</option><option value="TH">Thailand</option><option value="TG">Togo</option><option value="TT">Trinidad and Tobago</option><option value="TN">Tunisia</option><option value="TR">Turkey</option><option value="TC">Turks & Caicos</option><option value="AE">UAE</option><option value="UG">Uganda</option><option value="UA">Ukraine</option><option value="UK">United Kingdom</option><option value="US">United States</option><option value="UY">Uruguay</option><option value="UZ">Uzbekistan</option><option value="VU">Vanuatu</option><option value="VE">Venezuela</option><option value="VN">Vietnam</option><option value="YE">Yemen</option><option value="ZM">Zambia</option><option value="ZW">Zimbabwe</option></select></span>
</div>
i have tried this against the kendo site also same am not able to set the value
const puppeteer = require('puppeteer');
const homepage = 'https://demos.telerik.com/kendo-ui/dropdownlist/index';
async function test() {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.setViewport({width: 1400, height: 1400});
await page.goto(homepage, {waitUntil: 'networkidle2'});
await page.select('#size','XL - 7 5/8"');
}
test()
page.select() will work only on select elements and not custom elements like span, it is behaving as expected.
Kendo does a data binding to the select element hence it does not update when you select/update the select element.
The easiest way is to use the kendoUI itself.
On the demo page, the UI initializes by following,
var size = $("#size").data("kendoDropDownList");
To set the value,
size.value('M - 7 1/4"')
To get the value,
size.value()
Same with the input box,
var color = $("#color").data("kendoDropDownList");
color.value() // "1" => which is Blank
color.value("2") // "2" => Now it's Orange
Working example with puppeteer,
const puppeteer = require('puppeteer');
const homepage = 'https://demos.telerik.com/kendo-ui/dropdownlist/index';
async function test() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(homepage, {waitUntil: 'networkidle2'});
// use the demo pages built in jQuery and Kendo to change the results
const sizeValue = await page.evaluate(()=>{
var size = $("#size").data("kendoDropDownList");
size.value('XL - 7 5/8"')
return size.value()
});
await page.screenshot({path: 'kendoTest.png', fullPage: true});
await browser.close();
return sizeValue;
}
test().then(console.log) // should say XL - 7 5/8"
Result: