Cloud Functions Puppeteer cannot open browser - node.js

My setup in GCF:
install npm install --save puppeteer from project cloud shell
edit package.json like so:
{ "dependencies": { "puppeteer": "^19.2.2" } }
paste code from medium.com into index.js:
https://gist.githubusercontent.com/Alezco/b9b7ce4ec7ee7f208818e395225fcbbe/raw/8554acc8b311a10e272f5d1b98dce3400945bb00/index.js
deploy with 2 GB RAM, 0-3 instances, max 500s timeout
I get these errors after building or opening the URL:
Internal Server Error
Could not find Chromium (rev. 1056772). This can occur if either 1. you did not perform an installation before running the script (e.g. npm install) or 2. your cache path is incorrectly configured (which is: /workspace/.cache/puppeteer). For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.
When I run npm list both webdriver and puppeteer are installed. I suspect there is an issue this Path but I cannot figure out where it should lead.
I could then provide puppeteer.launch() with argument executablePath which might solve the problem.
I tried reinstalling puppeteer and changing configuration. No luck.

In addition to adding a .puppeteerrc.cjs per Kristofer's answer, I added a postinstall script in my package.json:
"scripts": {
...
"postinstall": "node node_modules/puppeteer/install.js"
},
This fixed the problem and I was able to deploy my Google Cloud Function. This is a temporary fix until issue #9128 is fixed.

I had the exact same issue and it seems to be related to this https://github.com/puppeteer/puppeteer/issues/9128
I'm using Firebase and don't have complete control over the build process when I deploy my functions but I still have access to the build logs. From the issue above, I realized I needed to handle the cache directory and the NPM version for this to work.
As far as I can tell the problem is that the build step installs the Chrome browser needed for Puppeteer in a cache directory outside of the final image that is used for the actuall function. In that context the error message makes more sence, it can't find the browser therefor it doesn't work.
I was using Node 14 in my cloud functions which used NPM 6.14.17 in the build steps. According to the issue you need to use NPM > 7 so I upgraded my function to use Node 16.
Then I added the .puppeteerrc.cjs from https://pptr.dev/guides/configuration/#examples when testing that locally it will add a .cache directory where the Chrome installation is. This has to be ignored when deploying the cloud function or the deplot will fail due to size.
In the firebase.json add:
"functions": {
"ignore": [
".cache"
]
},
The last step is pretty specific for Firebase and I'm not sure how this applies to your build steps etc. But this solved my issue that had the exact same error messages as you had. So double check the following:
NPM version in the build step, needs to beat least v.7 - Node 16 should do this.
Cache directory specified in the .puppeteerrc.cjs
It also looks like your using an old Puppeteer version, I used 19.3

I got this same (very missleading) error in my Puppeteer project which I run in Google Cloud Function. The issue was that Function was finishing (exiting) before the async Puppeteer script was finished.
Resolved this issue by changing the "await browser.close();" to a Promise and creating the response message in promise.then().
... only to hit next problem. My script is not downloading the csv file as expected. Works locally though...

Related

ChromeNotInstalledError while using chrome-launcher npm package

I am using chrome-launcher for running lighthouse programmatically. It works fine locally but when I run it on azure I am getting an error.
On this statement const chrome = await chromeLauncher.launch({chromeFlags: ['--headless']}); I am getting the following error:
ChromeNotInstalledErrorat new LauncherError (C:\home\site\wwwroot\node_modules\chrome-launcher\dist\utils.js:37:22)at new ChromeNotInstalledError (C:\home\site\wwwroot\node_modules\chrome-launcher\dist\utils.js:68:9){message: 'No Chrome installations found.',code: 'ERR_LAUNCHER_NOT_INSTALLED'}
How can I solve this?
You need to install Chrome on the Azure Function app somehow.
One way to do this is by using an npm dependency that installs Chrome as part of its install process. Examples of this are puppeteer and playwright. Although then you end up with some unnecessary dependencies.
You could also have a startup script or something that installs Chrome before running chrome-launcher/lighthouse. You'll need to tell crome-launcher where Chrome is installed if it's not a standard place using chromePath options or CHROME_PATH environment variable link.
You also have to ensure you do a Remote Build for the Function app.
You will also run into this error, which has a possible workaround: https://github.com/GoogleChrome/chrome-launcher/issues/188
Overall it's not easy. I actually ended up moving my workflow to GitHub Actions instead as Chrome is already installed on their runner images.
See:
https://anthonychu.ca/post/azure-functions-headless-chromium-puppeteer-playwright/

Failed to launch the browser process on Heroku

I built an app using as core node, express and sulla (import puppeteer).
Basically I scrape some data and use sulla to send them via whatsapp.
It works fine on local but when I deploy it on heroku I'm faced with this issue :
Failed to launch the browser
process!\n[0601/222716.792459:FATAL:zygote_host_impl_linux.cc(116)] No
usable sandbox! Update your kernel or see
https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md
for more information on developing with the SUID sandbox. If you want
to live dangerously and need an immediate workaround, you can try
using --no-sandbox ...... Core file will not be generated.
TROUBLESHOOTING:
https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md
I've already added the following buildpacks to my heroku app :
https://github.com/jontewks/puppeteer-heroku-buildpack.git
heroku/nodejs
https://github.com/heroku/heroku-buildpack-chromedriver
I've seen solutions like https://stackoverflow.com/a/52228855, but I can't apply it since I'm not directly using puppeteer. Or clear heroku caches without success.
Until you are using the current npm package of sulla unfortunately it won't work for you on Heroku. As the linked question says, you need to launch puppeteer with --no-sandbox (the --disable-setuid-sandbox arg is not mandatory for Heroku):
await puppeteer.launch({ args: ['--no-sandbox'] })
Sulla lacks this arg in the npm package (launch) (current launch config) (not used launch config that would work with Heroku).
It is very good that you've already added the buildpacks, those are needed if puppeteer is running as a dependency in the background.
I.) You could try a fork of sulla, called sulla-hotfix: https://www.npmjs.com/package/#jprustv/sulla-hotfix if it suits your needs. This one still uses the previous sulla puppeteer config, which apparently contains --no-sandbox launch arg.
It is true for the original project sulla was forked from: #open-wa/wa-automate. It may works on Heroku with the buildpacks.
II.) Or you could publish a modified version of sulla under MIT license, containing the right launch parameter.

Node package dependencies on IBM Cloud Foundry - require/module is not defined (Package not loading)

I am working on an application via the toolchain tool on IBM Cloud and editing the code via the Eclipse Orion IDE. As I am not accessing this through my local cli, my understanding is that in order to so call npm install {package}, I would just need to include the package in the package.json file under dependencies and require it in my app. However, when I load the application, I get the require is not defined indicating that the package has not been installed. Moreover, the require() is being used in the app.js file with the application being launched but not from files in my public directory.
After playing around further, it seems it might have to do with the way the directory tree is being traced as the error is only thrown in subdirectories. For example, require('express') works in app.js which is in the main directory ./ but fails when it is called in test.js in ./subdirectory/test.js. I feel like I'm missing something painfully simple like configuration of endpoint or something.
I've been searching around but I can't seem to find how to get the packages loaded, preferably without using the cli. Appreciate any pointers. Thanks!
Update: After playing around further, I am also getting module is not defined error when trying to require from another file in the same directory. For example module.exports = 'str' returns this error. While trying to require('./file') returns the require is not defined. It might have to do with how node is wrapping the functions?
Update 2: Tried "start": "npm install && node app.js" in package.json but no luck. Adding a build stage which calls npm install before deployment also does not work
Update 3: After adding npm install build stage, I am able to see that the dependencies have been successfully built via the logs. However, the require is not defined error still persists.
Update 4: Trying npm install from my CLI doesn't work as well even though all packages and dependencies are present
Update 5: Running cf restage or configuring cache via cacheDirectories does not work as well
Opened a related question regarding deployment here
Found out my confusion was caused due to me not realizing that require() cannot be used on the client side unless via tools such as Browserify.

Issues running React app on Windows environement

I have a working react app that I had hosted on a linux environment, but the client has switched their hosting provider and now I have to run the app on a Windows environment. I personally am unfamiliar with Windows and having some serious issues getting my app to work now.
After loads of debugging and testing I now can get my app to run, but when I start it, it will throw some errors which will results in a broken app in the browser.
So when I run npm start it will start up the app and spit out this error:
Screenshot of the terminal error 1/2
Screenshot of the terminal error 2/2
It seems to spit out some complaints when using npm run watchJS which runs this command babel src --watch --out-dir lib.
I am completely lost at this point and have no idea where to go from here.
I can provide as much detail as anyone needs after looking at the question, I just don't know what else to add to this issue and don't want to fill up this question with irrelevant info.
Thanks in advance for any help
It is likely a bug with the version of npm you are using.
See this thread:
https://github.com/npm/npm/issues/18380
Try rolling back your version of npm to 5.3
The issue appears to be exclusive to windows with babel packages.

Azure and node js __dirname

Probably it is not specifically related to webpack/memory-fs, but I am getting the RangeError: Maximum call stack size exceeded error (see below for a call stack).
I have found out, that __dirname on Azure (webapp) returns \\100.78.172.13\volume-7-default\8f5ecde749dace2bb57a\4e07195f015b45ce8e9ba255dc901988\site\repository\Source\Website\Content\app\node_modules\webpack\node_modules\memory-fs\lib\normalize.js in my situation, while process.cwd() returns D:\home\site\repository\Source\Website\Content\app.
Is anything can be done from my side to configure node js to return D:\... instead of \\.. ?
Gist
How to reproduce:
Clone the https://github.com/intellismiths/webapp1 repository.
Create new Azure Web App (default settings).
Configure deployment source to use GitHub.
Click Sync. It will take 10+ minutes to complete and it will show that the deployment was successful.
Go to Application settings in Azure and change WEBSITE_NODE_DEFAULT_VERSION to 6.2.2
Go to kudu page and open powershell console.
Execute npm cache clean
Check node version by executing node -v. It should be v6.2.2
On Azure, navigate to D:\home\site\respository\src\WebApp1
Execute npm run build
In console, you should see a lot of errors which indicates that modules can not be resolved.
OPTIONAL. Test npm run build on your local machine - it should produce wwwroot/app.js without errors.
Update webpack.config.js to include context: __dirname to fix previous errors.
Execute npm run build
In console, you should see the "RangeError: Maximum call stack size exceeded" error.
Update 1
I only tried to set 6.2.2 runtime after adding the second package.json, so the project structure is not the simplest possible. Maybe just setting node to 6.2.2 breaks the build.
I could reproduce your issue following your steps. I found the key point was setting the WEBSITE_NODE_DEFAULT_VERSION to 6.2.2. And I found the webpack task worked fine if the WEBSITE_NODE_DEFAULT_VERSION was under 6.
Please downgrade the setting WEBSITE_NODE_DEFAULT_VERSION to the version under 6 e.g. 5.9.0 if your node.js modules do not need such high version.
And according the package.json of angular2 athttps://github.com/angular/angular/blob/master/package.json, it seems that the angular2 repository requires the node.js version between 5.4 and 6.
Additionally, the web application's root directory on Azure Web Apps is D:\home\site\wwwroot. So if you want to build your frontend project on Azure Web Apps, you need to locate to D:\home\site\wwwroot\wwwroot\mobile-web-app then run npm run build.
It's been fixed in master and it's proposed to be included in v6.4.0.
See: https://github.com/nodejs/node/issues/7175#issuecomment-239824532 and https://github.com/nodejs/node/pull/8070
After a long day of research, trial-and-error and various experimentation, I've found an acceptable workaround if you're not willing to downgrade to Node 5.*:
Downgrade to Node 6.1.0
Make sure to install webpack globally (with npm install -g webpack).
Just using 6.1.0 gets around the "maximum call stack size exceeded" error, but instead gave me a lot of resolve failures when running webpack from node_modules (using ./node_modules/.bin/webpack). Installing webpack globally finally got me past that.
If I understand it correctly, this whole issue with __dirname in Node >= 6.2 resolving to the UNC folder path instead of the mounted path is going to be fixed, there's an active discussion here.
I had the same issue.
Fixed it with UPGRADING npm not DOWNGRADING.
Bug is fixed in the npm versions newer than 6.5.
https://github.com/aumanjoa/chronas-community/blob/master/package.json#L48
I believe that your __dirname shows your persistant drive where the data is stored, while .cwd gives current directory from where node ran. This is because Azure runs from the Drive but files are stored at the persistent drive.
In your Gruntfile.js add
module.exports = function (grunt) {
grunt.file.setBase(__dirname);
// Code omitted
}
Refer: link

Resources