I'm trying to install spatie/browsershot which uses puppeteer and for Debian systems, the list of dependencies are:
gconf-service
libasound2
libatk1.0-0
libc6
libcairo2
libcups2
libdbus-1-3
libexpat1
libfontconfig1
libgcc1
libgconf-2-4
libgdk-pixbuf2.0-0
libglib2.0-0
libgtk-3-0
libnspr4
libpango-1.0-0
libpangocairo-1.0-0
libstdc++6
libx11-6
libx11-xcb1
libxcb1
libxcomposite1
libxcursor1
libxdamage1
libxext6
libxfixes3
libxi6
libxrandr2
libxrender1
libxss1
libxtst6
ca-certificates
fonts-liberation
libappindicator1
libnss3
lsb-release
xdg-utils
wget
Inside the workspace, I've tried running apt-get install [package], but every one gives a message saying Unable to locate package [package]
I was able to figure this out. It may not be the nicest way, but all I had to do was place the following in the php-fpm Dockerfile:
RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils ca-certificates fonts-liberation libappindicator3-1 libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libfontconfig1 libgbm1 libgcc1 lib lib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release wget xdg-utils wget
Now everything is working.
There is a very large useful thread here: https://github.com/puppeteer/puppeteer/issues/290
I had to combine a few answers that listed dependencies
Related
/usr/share/code/bin/../code: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory
getting this error when install visual studio code in debian amd64
I add the same error trying using codium inside docker container. The solution given by Steve Mak (https://medium.com/#ssmak/how-to-fix-puppetteer-error-while-loading-shared-libraries-libx11-xcb-so-1-c1918b75acc3) work fine.
You need to install missing packages this list : gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
This is the error I get. This occurs when the test code is run in the Gitlab CI.
However, it works fine locally. Someone please help?
2036 $ npm test
2037 > 10_auth#1.1.3 test /builds/brhanuh/domestic-repair
2038 > jest --coverage 'unitTests/'
2039 Error: Failed to launch chrome!
2040
/builds/brhanuh/domestic-repair/node_modules/puppeteer/.local-chromium/linux-686378/chrome-linux/chrome:
error while loading shared libraries: libX11-xcb.so.1: cannot open
shared object file: No such file or directory
Install the missing packages using
sudo apt install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
Credits to #coldner on the puppetteer issue tracker for assembling the required pkgs.
If you encounter E: Unable to locate package errors, run sudo apt-get update.
I just wanted to add to #mohammad's answer, that now Puppetteer now has an official troublshooting guide that helps you determine what to install.
Also, they recommend running the following command to tell which particular packages need installing on *nix systems:
ldd chrome | grep not
Or, if you have Chrome installed via Puppetteer via NPM like me:
ldd ./node_modules/puppeteer/.local-chromium/linux-756035/chrome-linux/chrome | grep not
Then, I was able to see what libraries were still missing, and install accordingly.
you need to install required dependencies
here
https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md
under Chrome headless doesn't launch on UNIX you can find
Debian (e.g. Ubuntu) Dependencies
CentOS Dependencies
For those that are still having this problem, you can try this.
sudo apt update && sudo apt install -y ca-certificates fonts-liberation libappindicator3-1 libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release wget xdg-utils
I'm using Browsershot (which uses Puppeteer) with Laravel. I have it running perfectly in my local dev environment (Mac, Laravel Valet), but when running it on a Forge provisioned server, I get the following error in my log file (I've replaced sensitive URLs with placeholders):
local.ERROR: The command "PATH=$PATH:/usr/local/bin NODE_PATH=`/usr/bin/node /usr/bin/npm root -g` /usr/bin/node '/home/forge/my-app.co.uk/vendor/spatie/browsershot/src/../bin/browser.js' '{"url":"https:\/\/my-test-site.com\/about.html","action":"screenshot","options":{"type":"png","path":"\/home\/forge\/my-app.com\/storage\/screenshots\/13\/full\/21.jpg","args":[],"viewport":{"width":1400,"height":1080},"fullPage":true,"waitUntil":"networkidle0"}}'" failed.
Exit Code: 1(General error)
I've followed the instructions from the readme for installing the latest version of stable Chrome:
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
sudo npm install --global --unsafe-perm puppeteer
sudo chmod -R o+rx /usr/lib/node_modules/puppeteer/.local-chromium
My code looks like this:
I've also tried specifying custom node and npm binaries. I've also tried a custom include path. When doing so, the error I get may vary slightly, but remains essentially the same.
I'm actually installing Puppeteer as part of my build process in Forge. I set that up before I noticed the instructions for installing Chrome in the Browsershot readme.
I'm just not sure how to interpret that Node error. I've tried everything I can think of. So I'm hoping someone can point me in the right direction.
Thanks!
I'm trying to run node apps that use the npm package Puppeteer to scrape websites, using VPSes that use root as the default user I login as.
It never works. I always have to spend half an hour googling to find the solution... well no more.
Do this:
0. Run sudo apt update
1. Install libx stuff
sudo apt-get install libpangocairo-1.0-0 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libgconf2-4 libasound2 libatk1.0-0 libgtk-3-0
2. Set --no-sandbox argument in the launch() part of your Puppeteer app (only required if you are running as root/root's cron)
const browser = await puppeteer.launch({headless: true, args: ['--no-sandbox']})
E: as Niko pointed out, --no-sandbox opens your server to a malicious attack from a website the scraper visits. It looks like there are some SO questions elsewhere that post alternatives that aren't awful to implement. I like David's answer here which seems to suggest that even in 2021 --no-sandbox is still a necessary evil in some cases.
E2: See answer below. It looks like in 2021 we also have to install libgbm-dev
Connect to server by SSH
Open your terminal
Type: ssh root#[server ip]
If it is the first time you connect to the server you will get a security prompt. Type 'yes'
Fill in your password: [password]
You should now be connected to your server
Install app
sudo apt-get update
sudo apt-get install -y libgbm-dev
sudo apt install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
sudo apt install curl
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt install nodejs
git config --global user.name "your name"
git config --global user.email "your email"
git clone "your project git url"
cd "your project folder"
npm i
npm start
If you need to run multiple apps on one VPS server - google tool "Screen"
After I installed libgbm, everything works fine for me!
hope that help you out
sudo apt-get update
sudo apt-get install -y libgbm-dev
Hello I am new to gitlab, and so far i succeded in doing everything (runners, pipelines, etc...)
But I now want to set automatic tests build only once per day, triggered by a CRON but NOT TRIGGERED by each push made on the git repo.
I am using a 8.14.2 gitlab without the new SCHEDULE feature
So I used this gitlab.ci.yml:
stages:
- test
test-karma:
stage: test
image: node:8.9.4
before_script:
- apt-get update
- apt-get --yes install npm
- apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
- npm install #angular/cli#1.4.5
- npm install
environment: dev
script:
- ./node_modules/.bin/ng test --single-run=true --watch=false
only:
- triggers
except:
- pushes
But despite the only / except part, the job is triggered :
() For each git push (which i don't want)
() By the CRON which I want
So I found another way
1- Set a variable SHOULD_EXECUTE to false on gitlab
2- Protect every script line by an if statement
3- Use the following CRON:
00 4 * * * curl -X POST -F token=MY_TOKEN -F ref=master -F "variables[SOULD_EXECUTE]=true" http://MY_GITLAB_URL/api/v3/projects/30/trigger/builds
4- And changed the gitlab.ci.yml to
stages:
- test
test-karma:
stage: test
image: node:8.9.4
before_script:
- if [ ${SOULD_EXECUTE} == "true" ]; then apt-get update;fi
- if [ ${SOULD_EXECUTE} == "true" ]; then apt-get --yes install npm ;fi
- if [ ${SOULD_EXECUTE} == "true" ]; then apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget;fi
- if [ ${SOULD_EXECUTE} == "true" ]; then npm install #angular/cli#1.4.5;fi
- if [ ${SOULD_EXECUTE} == "true" ]; then npm install; else echo "TEST JOB SKIPPED";fi
- echo 'SOULD_EXECUTE => ' + ${SOULD_EXECUTE}
environment: dev
script:
- if [ ${SOULD_EXECUTE} == "true" ]; then ./node_modules/.bin/ng test --single-run=true --watch=false;fi
only:
- triggers
It's Ugly... Could some one help me to set this up ? Thank you so much.
regards.
The below also works without using except
only:
- schedules
I fixed my issue without knowing how or why. I had to move the repo to another group in gitlab. And now it's working perfectly, the only : triggers (without except : pushes) enable me to NOT run a pipeline for each pushes, and I can run this with the CRON post call.