Package puppeteer don't work properly (centos 7) - node.js

I used the package puppeteer to generate pdf in nodejs in local developement i use lubuntu 18.04 and every thing works great but in production after deploying the code in centos 7 it works sometimes but the majority of time it doesn't work.
i installed necessary packages of chromium using this command:
sudo yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver.x86_64 libXrandr.x86_64 GConf2.x86_64 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc
I added the argument {args:['--no-sandbox']} but the same problem.
This is my code :
const generator = (data, fileName, prepa) =>
new Promise(async (resolve, reject) => {
try {
const browser = await puppeteer.launch({ args: ["--no-sandbox"] });
const page = await browser.newPage();
let content;
if (prepa === true) {
content = templateCarnetPrepa(data);
} else {
content = templateCarnet(data);
}
await page.setContent(content);
await page.addStyleTag({ path: "./bootstrap.min.css" });
await page.emulateMediaType("screen");
await page.pdf({
path: `./${ENV.MEDIA_STORAGE}/carnet/${fileName}.pdf`,
format: "A4",
margin: { top: 20, left: 20, right: 20, bottom: 20 },
printBackground: true,
});
await browser.close();
//process.exit()
resolve(0);
} catch (e) {
console.log(e);
reject(e);
}
});
Does any one have a solution to this problem ?

Related

Puppeteer performance issue

I'm testing Puppeteer for large scale PDF generation, I'm testing 5 requests per second for 1 min, the problem is that the puppeteer is causing the VM to consume all resources and crash.
code
async generatePdf({
url,
displayHeaderFooter,
headerContent,
footerContent,
width,
height,
marginTop,
marginRight,
marginBottom,
marginLeft}: IPDFRepositoryDTO): Promise<Buffer> {
if (!global.browser) {
await launchPuppeteer();
}
const page = await global.browser.newPage();
await page.goto(url, {
waitUntil: 'networkidle0',
});
const isDisplayHeaderFooter =
displayHeaderFooter != undefined ? displayHeaderFooter : true;
const pageOptions = {
printBackground: true,
format: 'Letter',
displayHeaderFooter: isDisplayHeaderFooter,
headerTemplate: headerContent,
footerTemplate: footerContent,
width,
height,
margin: {
top: marginTop || '80px',
bottom: marginBottom || '80px',
left: marginLeft || '20px',
right: marginRight || '20px',
},
} as PDFOptions;
const pdf = await page.pdf(pageOptions);
await page.close();
return pdf;}
function launch puppeteer:
export const launchPuppeteer = async () => {
global.browser = await launch({
args: [
// '--no-sandbox',
// '--disable-setuid-sandbox',
// '--disable-dev-shm-usage',
// '--disable-accelerated-2d-canvas',
// '--no-first-run',
// '--no-zygote',
// '--single-process',
// '--disable-gpu',
'--no-sandbox',
'--disable-accelerated-2d-canvas',
'--no-zygote',
'--single-process',
'--disable-gpu',
'--disable-canvas-aa', // Disable antialiasing on 2d canvas
'--disable-2d-canvas-clip-aa', // Disable antialiasing on 2d canvas clips
'--disable-gl-drawing-for-tests', // BEST OPTION EVER! Disables GL drawing operations which produce pixel output. With this the GL output will not be correct but tests will run faster.
'--disable-dev-shm-usage',
'--use-gl=swiftshader', // better cpu usage with --use-gl=desktop rather than --use-gl=swiftshader, still needs more testing.
'--enable-webgl',
'--hide-scrollbars',
'--mute-audio',
'--no-first-run',
'--disable-infobars',
'--disable-breakpad',
//'--ignore-gpu-blacklist',
// '--window-size=1280,1024', // see defaultViewport
// '--user-data-dir=./chromeData', // created in index.js, guess cache folder ends up inside too.
'--disable-setuid-sandbox'
],
});
console.log('browser ready');
};
For the tests I'm using a google cloud VM that has 4 cores and 4GB of ram, e2-highcpu-4 , running Ubuntu
enter image description here

Puppeteer in headless false mode on ubuntu error

I have this web-scraping function
async () => {
try {
const browser = await puppeteer.launch({
headless: false,
ignoreHTTPSErrors: true,
args: ['--no-sandbox', "--disabled-setupid-sandbox"]
})
const page = await browser.newPage();
await page.goto('https://finviz.com/map.ashx');
await page.waitForTimeout(3000);
await page.click('.content #root > div > div:nth-child(3) > button:nth-child(1)'); // Fullscreen
await page.click('.content #root > div > div:nth-child(3) > button:nth-child(2)'); // Share map
await page.waitForTimeout(3000);
const imageUrl = await page.$eval('img.w-full', el => el.src);
console.log(imageUrl);
await browser.close();
} catch (err) {
console.log(err);
}
};
When I try to run it on ubuntu I get an error
Missing X server or $DISPLAY The platform failed to initialize.
Exiting. NaCl helper process running without a sandbox!
If I try to run it in headless mode, I get an error
Error: No node found for selector: .content #root > div > div:nth-child(3) > button:nth-child(1)
On my local machine the script runs fine in mode headless : true
How can you get out of this situation?

Puppeteer nodejs Error: input.on is not a function\n at new Interface

M generating a pdf from html using the npm module puppeteer.
When the running the following code m getting an error.
It is working properly on windows , but when the same is executed on linux red hat server , it is giving an error
let poptions = {
path: pdfPath, scale: 0.8, printBackground: true, format: "letter"
,"margin": {
"bottom": 70,
"left": 25,
"right": 35,
"top": 70,
},
landscape:true
}
console.log(htmlPath);
const browser = await puppeteer.launch({ args: [
'--no-sandbox'
],"dumpio": true})
const page = await browser.newPage();
page.on('console', (msg) => console.log('PAGE LOG:', msg.text()));
await page.goto(htmlPath);
// await page.emulateMedia('print');
poption=Object.assign(poptions,pageoptions)
if(pageStyle)await page.addStyleTag(pageStyle);
const pdf = await page.pdf(poptions);
await browser.close();
Error: input.on is not a function
at new Interface (readline.js:207:11)
at Object.createInterface (readline.js:75:10)
at Promise (/microservice/node_modules/puppeteer/lib/Launcher.js:329:25)
at new Promise ()
at waitForWSEndpoint (/microservice/node_modules/puppeteer/lib/Launcher.js:326:10)
at Launcher.launch (/microservice/node_modules/puppeteer/lib/Launcher.js:170:41)
Used the following parameters while launching the chrome.--disable-setuid-sandbox resolved the issue
const browser = await puppeteer.launch({ args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--headless',
'--disable-dev-shm-usage',
'--disable-gpu',
'--disable-features=NetworkService',
'--window-size=1920x1080',
'--disable-features=VizDisplayCompositor',
'--log-file=/home/ec2-user/credence/microservices/reporting-server/log/server.log',
'--log-level=0'
],"dumpio": true})

Using the default chrome profile with puppeteer that my chrome app uses

I'm having issues getting puppeteer to use the default profile that my chrome browser uses. I've tried setting path to the user profile, but when I go to a site with puppeteer that I know is saved with chrome app's userDataDir, there's nothing saved there. What am I doing wrong? I appreciate any help!
const browser = await puppeteer.launch({
headless: false,
userDataDir: 'C:\\Users\\Bob\\AppData\\Local\\Google\\Chrome\\User Data',
}).then(async browser => {
I've also tried userDataDir: 'C:/Users/Phil/AppData/Local/Google/Chrome/User Data',, but still nothing.
UPDATED:
const username = os.userInfo().username;
(async () => {
try {
const browser = await puppeteer.launch({
headless: false, args: [
`--user-data-dir=C:/Users/${username}/AppData/Local/Google/Chrome/User Data`]
}).then(async browser => {
I had same exact issue before. However connecting my script to a real chrome instance helped to solve a lot of problems specially the profile one.
You can see the steps here:
https://medium.com/#jaredpotter1/connecting-puppeteer-to-existing-chrome-window-8a10828149e0
//MACOS
/*
Open this instance first:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --no-first-run --no-default-browser-check --user-data-dir=$(mktemp -d -t 'chrome-remote_data_dir')
// Windows:
- Add this to Target of launching chrome --remote-debugging-port=9222
- Navigate to http://127.0.0.1:9222/json/version
- copy webSocketDebuggerUrl
More Info: https://medium.com/#jaredpotter1/connecting-puppeteer-to-existing-chrome-window-8a10828149e0
*/
// Puppeteer Part
// Always update this socket after running the instance in terminal (look up ^)
and this is abstracted controller written in Typescript, that I always use in any project:
import * as puppeteer from 'puppeteer';
import { Browser } from 'puppeteer/lib/cjs/puppeteer/common/Browser';
import { Page } from 'puppeteer/lib/cjs/puppeteer/common/Page';
import { PuppeteerNode } from 'puppeteer/lib/cjs/puppeteer/node/Puppeteer';
import { getPuppeteerWSUrl } from './config/config';
export default class Puppeteer {
public browser: Browser;
public page: Page;
getBrowser = () => {
return this.browser;
};
getPage = () => {
return this.page;
};
init = async () => {
const webSocketUrl = await getPuppeteerWSUrl();
try {
this.browser = await ((puppeteer as unknown) as PuppeteerNode).connect({
browserWSEndpoint: webSocketUrl,
defaultViewport: {
width: 1920,
height: 1080,
},
});
console.log('BROWSER CONNECTED OK');
} catch (e) {
console.error('BROWSER CONNECTION FAILED', e);
}
this.page = await this.browser.newPage();
this.page.on('console', (log: any) => console.log(log._text));
};
}
Abstracted webosocket fecther:
import axios from "axios";
import { exit } from "process";
export const getPuppeteerWSUrl = async () => {
try {
const response = await axios.get("http://127.0.0.1:9222/json/version");
return response.data.webSocketDebuggerUrl;
} catch (error) {
console.error("Can not get puppeteer ws url. error %j", error);
console.info(
"Make sure you run this command (/Applications/Google Chrome.app/Contents/MacOS/Google Chrome --remote-debugging-port=9222 --no-first-run --no-default-browser-check --user-data-dir=$(mktemp -d -t 'chrome-remote_data_dir')) first on a different shell"
);
exit(1);
}
};
Feel free to adjust the template to suit whatever you enviroment/tools currrently look like.

While using capture-website, puppeteer with webpack throwing error:browser not downloaded, at ChromeLauncher.launch (webpack-internal)

I am trying to take screenshot by providing html file on node js.
I have used capture-website package.
Here is the code:
try{
await captureWebsite.file('file.html', 'file.png', {overwrite: true}, function (error) {
if (error) {
console.log('error',error);
}
});
}catch(e){
console.log('error in capture image:', e)
}
Version:
node:12.16.1
capture-website: "0.8.1"
Angular : 7
You can do something like:
async function screenshotFromHtml ({ html, timeout = 2000 }: ScreenshotOptions) {
const browser = await puppeteer.launch({
headless: !process.env.DEBUG_HEADFULL,
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
]
})
const page = await browser.newPage()
// Set viewport to something big
// Prevents Carbon from cutting off lines
await page.setViewport({
width: 2560,
height: 1080,
deviceScaleFactor: 2
})
page.setContent(html)
const base64 = await page.screenshot({ encoding: "base64" }) as string;
// Wait some more as `waitUntil: 'load'` or `waitUntil: 'networkidle0'
await page.waitFor(timeout)
// Close browser
await browser.close()
return base64
}
This code is in typescript, but you can use the function body in your JS project
In this github file you can see a html render code too

Resources