No matter what I try, the maximum height of a full-page screenshot with headless Chrome 60 (and 59) is 16,348px.
It is not a memory issue. There are no segfaults and no, for example, FATAL:memory_linux.cc(35) Out of memory. messages. None. Capturing screenshots are "successful". Changing the screenshot format - PNG or JPEG - has no affect.
The screenshot width can vary, but the height of the saved screenshot is always limited to 16,348px. Example,
1600x16384, 1200x16384, 2400x16384, etc.
I'm taking upscaled screenshots with this minimal code (full source):
const upscale = 2;
const viewportWidth = 1200;
const viewportHeight = 800;
....
// Set up viewport resolution, etc.
const deviceMetrics = {
width: viewportWidth,
height: viewportHeight,
deviceScaleFactor: 0,
mobile: false,
fitWindow: false,
scale: 1 // Relative to the upscale
};
await Emulation.setDeviceMetricsOverride(deviceMetrics);
await Emulation.setVisibleSize({width: viewportWidth, height: viewportHeight});
await Emulation.setPageScaleFactor({pageScaleFactor: upscale});
// Navigate to target page
await Page.navigate({url});
const {root: {nodeId: documentNodeId}} = await DOM.getDocument();
const {nodeId: bodyNodeId} = await DOM.querySelector({
selector: 'body',
nodeId: documentNodeId,
});
const {model: {height}} = await DOM.getBoxModel({nodeId: bodyNodeId});
// Upscale the dimensions for full page
await Emulation.setVisibleSize({width: Math.round(viewportWidth * upscale), height: Math.round(height * upscale)});
// This forceViewport call ensures that content outside the viewport is
// rendered, otherwise it shows up as grey. Possibly a bug?
await Emulation.forceViewport({x: 0, y: 0, scale: upscale});
const screenshot = await Page.captureScreenshot({format});
const buffer = new Buffer(screenshot.data, 'base64');
file.writeFile(outFile, buffer, 'base64', function (err) {
if (err) {
console.error('Exception while saving screenshot:', err);
} else {
console.log('Screenshot saved');
}
client.close();
});
I was unable to find any useful Chrome switches either. Is this a hardcoded limit of Chrome? 16384 is a suspicious number (2^14 = 16,384). How can this height be increased?
This is a limitation of Chromium as documented in this bug
https://bugs.chromium.org/p/chromium/issues/detail?id=770769&desc=2
Related
I would like to set the window-position within the total available screen, so it takes up half of the screen-size and is positioned at the right end of the screen with its right border. Couldn't find any docs about it, maybe it's not possible?
As the docs state, the args-option of openBrowser() accepts all of the Chromium browser launch options and the nodejs-package "robotjs" helps to get the screen-size. Also "screenres" claims to able to do that, but I haven't tested it.
This opens the browser-window with half of the screen-width, and positions it to be right-aligned within the screen:
const { closeBrowser, openBrowser } = require('taiko')
const screenSize = require('robotjs').getScreenSize()
const screenWidth = screenSize.width / 2
const screenHeight = screenSize.height
const windowSize = '--window-size=' + screenWidth + ',' + screenHeight
const windowPosition = '--window-position=' + screenWidth + ',0'
async () => {
try {
await openBrowser({ args: [ windowPosition, windowSize ] })
} catch (error) {
console.error(error);
} finally {
await closeBrowser();
}
})();
I have the task of stacking several pngs one on top of another and exporting it as one png. The images do not have the same size, however they do not need to be resized, just cropped to the available canvas.
The images come from the file system and are sent as a buffer. There may be somewhere between 8 and 30 images.
My tests have been with 10 images, and I have compared it with node-canvas which takes less than 50% of the time that Sharp does. I think one problem with Sharp is that I have to make and extract operation before compositing.
This is my code running with Sharp, which takes ~600ms with 10 images:
export const renderComponentsSHARP = async ( layers )=>{
const images = [];
for(let i=0; i<layers.length; i++){
const layer = sharp(layers[i]));
const meta = await layer.metadata();
if(meta.height > AVATAR_SIZE.height || meta.width > AVATAR_SIZE.width)
layer.extract({ top:0, left:0, ...AVATAR_SIZE });
images.push({input:await layer.toBuffer()});
}
return sharp({
create: {
...AVATAR_SIZE,
channels: 4,
background: { r: 0, g: 0, b: 0, alpha: 0 }
}})
.composite(images)
.png()
.toBuffer();
};
This is my code running with canvas, which takes ~280ms:
export const renderComponentsCANVAS = async ( layers )=>{
const canvas = createCanvas(AVATAR_SIZE.width, AVATAR_SIZE.height);
const ctx = canvas.getContext('2d');
const images = await Promise.all(layers.map( layer=>loadImage(layer)));
images.forEach(image=>{
ctx.drawImage(image, 0, 0);
});
return canvas.toBuffer();
};
Am I doing something wrong with sharp? Or is it not the tool for this job?
I am using the sharp nodejs library found here: https://github.com/lovell/sharp. I am trying to take several screenshots, and then piece the images together using the sharp library.
Here is my code. I am using puppeteer to take screenshots of the page, saving in memory as a binary file and combining those binary files together using sharp's composite() method.
let pagePath = 'path/to/file.png';
let maxScreenshotHeight = 2000;
// Loop over sections of the screen that are of size maxScreenshotHeight.
for (let ypos = 0; ypos < contentSize.height; ypos += maxScreenshotHeight) {
const height = Math.min(contentSize.height - ypos, maxScreenshotHeight);
let image = await page.screenshot({
encoding: 'binary',
clip: {
x: 0,
y: ypos,
width: contentSize.width,
height
}
});
composites.push({input: image, gravity: 'southeast'});
}
sharp()
.composite(composites)
.toFile(pagePath, function(err) {
if (err) {
console.log('fail');
return;
}
console.log('complete');
});
However, in the toFile callback, nothing ever gets logged. Console logging works, as I've added logs before and after the toFile statement, but it seems that this function call never completes. I want to create a png file that I can later download.
How can I merge these multiple screenshots and store them on the server for a later download? Am I using toFile incorrectly?
I'm using 'puppeteer' for NodeJS to test a specific website. It seems to work fine in most case, but some places it reports:
Error: Node is either not visible or not an HTMLElement
The following code picks a link that in both cases is off the screen.
The first link works fine, while the second link fails.
What is the difference?
Both links are off the screen.
Any help appreciated,
Cheers, :)
Example code
const puppeteer = require('puppeteer');
const initialPage = 'https://website.com/path';
const selectors = [
'div[id$="-bVMpYP"] article a',
'div[id$="-KcazEUq"] article a'
];
(async () => {
let selector, handles, handle;
const width=1024, height=1600;
const browser = await puppeteer.launch({
headless: false,
defaultViewport: { width, height }
});
const page = await browser.newPage();
await page.setViewport({ width, height});
page.setUserAgent('UA-TEST');
// Load first page
let stat = await page.goto(initialPage, { waitUntil: 'domcontentloaded'});
// Click on selector 1 - works ok
selector = selectors[0];
await page.waitForSelector(selector);
handles = await page.$$(selector);
handle = handles[12]
console.log('Clicking on: ', await page.evaluate(el => el.href, handle));
await handle.click(); // OK
// Click that selector 2 - fails
selector = selectors[1];
await page.waitForSelector(selector);
handles = await page.$$(selector);
handle = handles[12]
console.log('Clicking on: ', await page.evaluate(el => el.href, handle));
await handle.click(); // Error: Node is either not visible or not an HTMLElement
})();
I'm trying to emulate the behaviour of a real user clicking around the site, which is why I use .click(), and not .goto(), since the a tags have onclick events.
Instead of
await button.click();
do this:
await button.evaluate(b => b.click());
The difference is that button.evaluate(b => b.click()) runs the JavaScript HTMLElement.click() method on the given element in the browser context, which will fire a click event on that element even if it's hidden, off-screen or covered by a different element, whereas button.click() clicks using Puppeteer's ElementHandle.click() which
scrolls the page until the element is in view
gets the bounding box of the element (this step is where the error happens) and finds the screen x and y pixel coordinates of the middle of that box
moves the virtual mouse to those coordinates and sets the mouse to "down" then back to "up", which triggers a click event on the element under the mouse
First and foremost, your defaultViewport object that you pass to puppeteer.launch() has no keys, only values.
You need to change this to:
'defaultViewport' : { 'width' : width, 'height' : height }
The same goes for the object you pass to page.setViewport().
You need to change this line of code to:
await page.setViewport( { 'width' : width, 'height' : height } );
Third, the function page.setUserAgent() returns a promise, so you need to await this function:
await page.setUserAgent( 'UA-TEST' );
Furthermore, you forgot to add a semicolon after handle = handles[12].
You should change this to:
handle = handles[12];
Additionally, you are not waiting for the navigation to finish (page.waitForNavigation()) after clicking the first link.
After clicking the first link, you should add:
await page.waitForNavigation();
I've noticed that the second page sometimes hangs on navigation, so you might find it useful to increase the default navigation timeout (page.setDefaultNavigationTimeout()):
page.setDefaultNavigationTimeout( 90000 );
Once again, you forgot to add a semicolon after handle = handles[12], so this needs to be changed to:
handle = handles[12];
It's important to note that you are using the wrong selector for your second link that you are clicking.
Your original selector was attempting to select elements that were only visible to xs extra small screens (mobile phones).
You need to gather an array of links that are visible to your viewport that you specified.
Therefore, you need to change the second selector to:
div[id$="-KcazEUq"] article .dfo-widget-sm a
You should wait for the navigation to finish after clicking your second link as well:
await page.waitForNavigation();
Finally, you might also want to close the browser (browser.close()) after you are done with your program:
await browser.close();
Note: You might also want to look into handling unhandledRejection errors.
Here is the final solution:
'use strict';
const puppeteer = require( 'puppeteer' );
const initialPage = 'https://statsregnskapet.dfo.no/departementer';
const selectors = [
'div[id$="-bVMpYP"] article a',
'div[id$="-KcazEUq"] article .dfo-widget-sm a'
];
( async () =>
{
let selector;
let handles;
let handle;
const width = 1024;
const height = 1600;
const browser = await puppeteer.launch(
{
'defaultViewport' : { 'width' : width, 'height' : height }
});
const page = await browser.newPage();
page.setDefaultNavigationTimeout( 90000 );
await page.setViewport( { 'width' : width, 'height' : height } );
await page.setUserAgent( 'UA-TEST' );
// Load first page
let stat = await page.goto( initialPage, { 'waitUntil' : 'domcontentloaded' } );
// Click on selector 1 - works ok
selector = selectors[0];
await page.waitForSelector( selector );
handles = await page.$$( selector );
handle = handles[12];
console.log( 'Clicking on: ', await page.evaluate( el => el.href, handle ) );
await handle.click(); // OK
await page.waitForNavigation();
// Click that selector 2 - fails
selector = selectors[1];
await page.waitForSelector( selector );
handles = await page.$$( selector );
handle = handles[12];
console.log( 'Clicking on: ', await page.evaluate( el => el.href, handle ) );
await handle.click();
await page.waitForNavigation();
await browser.close();
})();
For anyone still having trouble this worked for me:
await page.evaluate(()=>document.querySelector('#sign-in-btn').click())
Basically just get the element in a different way, then click it.
The reason I had to do this was because I was trying to click a button in a notification window which sits outside the rest of the app (and Chrome seemed to think it was invisible even if it was not).
I know I’m late to the party but I discovered an edge case that gave me a lot of grief, and this thread, so figured I’d post my findings.
The culprit:
CSS
scroll-behavior: smooth
If you have this you will have a bad time.
The solution:
await page.addStyleTag({ content: "{scroll-behavior: auto !important;}" });
Hope this helps some of you.
My way
async function getVisibleHandle(selector, page) {
const elements = await page.$$(selector);
let hasVisibleElement = false,
visibleElement = '';
if (!elements.length) {
return [hasVisibleElement, visibleElement];
}
let i = 0;
for (let element of elements) {
const isVisibleHandle = await page.evaluateHandle((e) => {
const style = window.getComputedStyle(e);
return (style && style.display !== 'none' &&
style.visibility !== 'hidden' && style.opacity !== '0');
}, element);
var visible = await isVisibleHandle.jsonValue();
const box = await element.boxModel();
if (visible && box) {
hasVisibleElement = true;
visibleElement = elements[i];
break;
}
i++;
}
return [hasVisibleElement, visibleElement];
}
Usage
let selector = "a[href='https://example.com/']";
let visibleHandle = await getVisibleHandle(selector, page);
if (visibleHandle[1]) {
await Promise.all([
visibleHandle[1].click(),
page.waitForNavigation()
]);
}
I'm making a fun open sourced sample for doing Edge Compute Computer Vision using the Raspberry Pi as my hardware.
The current SDK I have to access hardware is nodejs based (I'll release a second with python when it is available). Warning: I am a node novice.
The issue I am facing is that I want to take pictures using the stock camera in a loop without saving a file. I just want to get access to the buffer, extract the pixels, pass to my second edge module.
Taking pictures with no file save in a while(true) loop appears to never execute.
Here is my sample:
'use strict';
var sleep = require('sleep');
const Raspistill = require('node-raspistill').Raspistill;
var pixel_getter = require('pixel-getter');
while(true) {
const camera = new Raspistill({ verticalFlip: true,
horizontalFlip: true,
width: 500,
height: 500,
encoding: 'jpg',
noFileSave: true,
time: 1 });
camera.takePhoto().then((photo) => {
console.log('got photo');
pixel_getter.get(photo,
function(err, pixels) {
console.log('got pixels');
console.log(String(pixels));
});
});
sleep.sleep(5);
}
console.log('picture taken');
In the above code, none of the console.log functions actually ever log; leading me to believe that photos are never taken and therefor pixels can not be extracted.
Any assistance would be greatly appreciated.
UPDATE:
It looks like the looping mechanic might be funny. I guess I don't really care if it takes pictures in a loop as long as it takes a picture, I pass it off, I take a picture and I pass it off, indefinately.
I decided to approach the problem with a recursive loop instead which worked brilliantly.
'use strict';
const sleep = require('sleep');
const Raspistill = require('node-raspistill').Raspistill;
const pixel_getter = require('pixel-getter')
const camera = new Raspistill({ verticalFlip: true,
horizontalFlip: true,
width: 500,
height: 500,
encoding: 'jpg',
noFileSave: true,
time: 5 });
function TakePictureLoop() {
console.log('taking picture');
camera.takePhoto().then((photo) => {
console.log('got photo');
pixel_getter.get(photo, function(err, pixels) {
console.log('got pixels');
TakePictureLoop();
});
});
}
TakePictureLoop();