Web Scraper Pagination - pagination

Question

I have created a web scraper where I am trying to fetch the dynamic data which loads in a div after page is load.
Here it is my code and source website url https://www.medizinerkarriere.de/kliniken-sortiert-nach-name.html
async function pageFunction(context) {
// jQuery is handy for finding DOM elements and extracting data from them.
// To use it, make sure to enable the "Inject jQuery" option.
const $ = context.jQuery;
var result = [];
$('#klinikListBox ul').each(function(){
var item = {
Name: $(this).find('li.klName').text().trim(),
Ort: $(this).find('li.klOrt').text().trim(),
Land: $(this).find('li.klLand').text().trim(),
Url:""
};
result.push(item);
});
// To make this work, make sure the "Use request queue" option is enabled.
await context.enqueueRequest({ url: 'https://www.medizinerkarriere.de/kliniken-sortiert-nach-name.html' });
// Return an object with the data extracted from the page.
// It will be stored to the resulting dataset.
return result;
}
But there are on click pagination and I am not sure how to do it.
I tried all method from this link but it didn't work.
https://docs.apify.com/scraping/web-scraper#bonus-making-your-code-neater
Please help and quick help will be highly appreciated.

Answer 1

In this case the pagination loads dynamically on a single page so enqueuing new pages doesn't make sense. You can get to the next page by simply clicking the page button, it is also a good practice to wait a bit after the click.
$('#PGPAGES span').eq(1).click();
await context.waitFor(1000)
You can scrape all pages with a simple loop
const numberOfPages = 8 // You can scrape this number too
for (let i = 1; i <= numberOfPages; i++) {
// Your scraping code, push data to an array and return them in the end
$('#PGPAGES span').eq(i).click();
await context.waitFor(1000)
}