I am using puppeteer-extra for my firebase cloud function. However, when the function was triggered, I got the error saying that puppeteer is missing and asking me to install puppeteer, while I already had had puppeteer installed. I tried installing it again but still got the same error. What goes wrong?
package.json of my functions folder
{
"name": "functions",
"description": "Cloud Functions for Firebase",
"scripts": {
......
},
"engines": {
"node": "8"
},
"dependencies": {
"firebase-admin": "^8.10.0",
"firebase-functions": "^3.6.1",
"puppeteer": "^5.4.1",
"puppeteer-extra": "^3.1.15",
"puppeteer-extra-plugin-stealth": "^2.6.3"
},
"devDependencies": {
"firebase-functions-test": "^0.2.0"
},
"private": true
}
the beginning of the cloud function
const functions = require('firebase-functions');
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
exports.myFunction = functions.https.onCall((data, context) => {
async function XXX() {
try {
const browser = await puppeteer.launch({
headless: false
});
......
}
catch(e) {
......
}
}
}
full error message
Puppeteer is missing. :-)
Note: puppeteer is a peer dependency of puppeteer-extra,
which means you can install your own preferred version.
- To get the latest stable version run: 'yarn add puppeteer' or 'npm i puppeteer'
Alternatively:
- To get puppeteer without the bundled Chromium browser install 'puppeteer-core'
- To use puppeteer-firefox install 'puppeteer-firefox' and use the 'addExtra' export
update
I replaced
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
with
const puppeteer = require('puppeteer');
Then I got an error deploying the function. Error message was
Function failed on loading user code. Error message: Code in file index.js can't be loaded.
Is there a syntax error in your code?
I checked that if I switched back to puppeteer-extra without modifying any other part of the code the deployment worked fine.
I ran into a very similar issue this days and took a lot of time to realise where the error was coming from and how to fix it.
I was working on a node.js project and installed puppeteer with yarn for scraping a website, and all this setup was into a docker container (will explain later in my answer why i am mentioning this).
So the problem was exactly like yours, i was getting an error that i was missing puppeteer even though it was already installed. Tried to delete node_modules and install all over again, recreate the whole environment etc etc nothing seemed to work.
The solution that unstuck me was to add in my package.json dependencies:
"agent-base": "^6.0.2",
"https-proxy-agent": "^5.0.1",
The reason was that puppeteer was failing with that error message, which was a misleading message, but it was throwing an error from inside puppetter/node_modules/https_agent/src/agent.ts file or a similar named file i am not sure 100% about the file name.
The important thing was that the https_agent/src was not a valid path, and did not even exist but https_agent was trying to access that file and of course failed.
So by adding the agent-base base dependency fixed the issue and i was able to run my yarn command to a point where i got to the next error.
Also code wise i added some extra arguments in the launch function of Puppetter, so launch looked like:
await Puppeteer.launch({ headless: true, defaultViewport: null, args: ['--no-sandbox', '--disable-setuid-sandbox'] })
Also my imports are like the initial ones you had before updating your answer:
import Puppeteer from 'puppeteer-extra'
import Stealth from 'puppeteer-extra-plugin-stealth'
Moving from there and unstucking from my first misleading error that the whole package was missing, i had an issue into opening the headless browser upon puppeteer execution. The solution to this was in my Dockerfile to add those 2 lines:
RUN apk add git netcat-openbsd chromium
ENV PUPPETEER_EXECUTABLE_PATH="/usr/bin/chromium-browser"
After that my command ran normally, added to a crontab and everything was smooth.
Dropping here my solution and hope to help others who are stack in the same issue like i was. The biggest thing that costed most of my time was the misleading error that puppeteer is missing so i hope can i be the one to save some time from another person who is stuck in the same loop as i was.
Related
I'm trying to make a Vue project and use an npm package for connecting to the retroachievements.org api to fetch some data, but I'm getting an error. Here's my process from start to finish to create the project and implement the package.
Navigate to my projects folder and use the vue cli to create the project: vue create test. For options, I usually chose not to include the linter, vue version 2, and put everything in package.json.
cd into the /test folder: cd test and install the retroachievements npm package: npm install --save raapijs
Modify App.vue to the following (apologies for code formatting, not sure why the post isn't formatting/coloring it all properly...):
const RaApi = require('raapijs');
export default {
name: 'App',
data: () => ({
api:null,
user: '<USER_NAME>',
apiKey: '<API_KEY>',
}),
created() {
this.api = new RaApi(this.user, this.apiKey);
},
}
run `npm run serve' and get the error:
ERROR in ./node_modules/raapijs/index.js 2:14-30
Module not found: Error: Can't resolve 'https' in 'C:\Projects\Web\test\node_modules\raapijs'
I'm on Windows 10, Node 16.17.0, npm 8.15.0, vue 2.6.14, vue CLI 5.0.8, raapijs 0.1.2.
The first solution below says he can run it without error but it looks like the exact same code as I'm trying. Can anyone see a difference and a reason for this error?
EDIT: I reworded this post to be more clear about my process and provide more info, like the versions.
This solution works for me. I installed raapijs with npm install --save raapijs command. Then in my Vue version 2 component I used your code as follow:
const RaApi = require('raapijs');
export default {
data: () => ({
api: null,
user: '<USER_NAME>',
apiKey: '<API_KEY>',
}),
created() {
this.api = new RaApi(this.user, this.apiKey);
},
};
It seems the raapijs package was designed to be used in a Node environment, rather than in Vue's browser based environment, so that's the reason I was getting an error. The package itself was looking for the built in https package in Node, but since it wasn't running in Node, it wasn't finding it.
So I solved my problem by looking at the package's github repo and extractingt he actual php API endpoints that were being used and using those in my app directly, rather than using the package wrapper. Not quite as clean and tidy as I was hoping but still a decent solution.
I am a relative noob with Firebase my first time using it and following along with a tutorial. The tutorial is a bit outdated and I have been fixing bugs as I have been going, but this one has got me completely stuck. I try to run a different functions that trigger when a document is created
in certain collections. However, i get the following error:
Error
! functions[createNotificationOnlike(us-central1)]: Deployment error.
Function failed on loading user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs
There are 3 more identical errors that correspond to the other exports in the following index.js file.
Index.js
exports.createNotificationOnlike = functions.firestore.document('likes/{id}').onCreate(async (snapshot) => {
try {
const doc = await db.doc(`posts/${snapshot.data().postId}`).get(); // we have access to user handle via the likes route
if(doc.exists){
await db.doc(`notifications/${snapshot.id}`).set({ // the id of the like is the same as the id of the notification that pertains to the like
createdAt: new Date().toISOString,
recipient: doc.data.userHandle,
sender: snapshot.data().userHandle,
type: 'like',
read: false,
postId: doc.id
});
return;
}
} catch (err) {
console.error(err);
return;
}
});
exports.removeNotificationOnUnlikePost = functions.firestore.document('likes/{id}').onDelete( async (snapshot) => {
try {
await db.doc(`notifications/${snapshot.id}`).delete();
return;
} catch (err) {
console.error(err);
return;
}
})
exports.createNotificationForComments = functions.firestore.document('comments/{id}').onCreate(async (snapshot) => {
try {
const doc = await db.doc(`posts/${snapshot.data().postId}`).get();
if(doc.exists){
db.doc(`notifications/${snapshot.id}`).set({
createdAt: new Date().toISOString,
recipient: doc.data.userHandle,
sender: snapshot.data().userHandle,
type: 'comment',
read: false,
postId: doc.id
})
return;
}
} catch (err) {
console.error(err);
return; // we dont need return messages since this isnt an endpoint it is a db trigger event
}
})
// auto turn the app into base route url/api
exports.api = functions.https.onRequest(app);
I have checked the logs as suggested by the error and i get the following messages which i think are useless, there are three other identical errors for the other functions
Error Logs
removeNotificationOnUnlikePost
{"#type":"type.googleapis.com/google.cloud.audit.AuditLog","status":{"code":3,"message":"Function failed on loading user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs"}
Here is my package.json file
Package.json
{
"name": "functions",
"description": "Cloud Functions for Firebase",
"scripts": {
"serve": "firebase emulators:start --only functions",
"shell": "firebase functions:shell",
"start": "npm run shell",
"deploy": "firebase deploy --only functions",
"logs": "firebase functions:log"
},
"engines": {
"node": "10"
},
"dependencies": {
"busboy": "^0.3.1",
"express": "^4.17.1",
"firebase": "^7.16.0",
"firebase-admin": "^8.10.0",
"firebase-functions": "^3.6.1"
},
"devDependencies": {
"firebase-functions-test": "^0.2.0"
},
"private": true
}
Lastly, here is my config stuff that was used to initialize everything:
admin.js
const admin = require('firebase-admin');
var serviceAccount = require('../../service-acct/socialapp-e5130-firebase-adminsdk-uo6p6-5495e18b97.json');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
storageBucket: "socialapp-e5130.appspot.com",
databaseURL: "https://socialapp-e5130.firebaseio.com"
});
const db = admin.firestore();
module.exports = { db, admin }
Firebase init
const firebase = require('firebase');
const config = require('../util/config.js');
firebase.initializeApp(config);
P.S. It is worth mentioning that the http.onRequest trigger (api) was actually working and I have been developing without deploying using firebase serve. Now that I am ready to deploy these triggers something is going heinously wrong. Any help is greatly appreciated
Enter this command for getting log:
firebase functions:log
I had the exact same problem, trying to deploy the exact same code as you and I was just able to get it to deploy properly.
If yours is the same issue as mine, this is where the problem is:
var serviceAccount = require('../../service-acct/socialapp-e5130-firebase-adminsdk-uo6p6-5495e18b97.json');
It looks like when deploying to firebase you cant have code trying to reach outside your project folder, so the solution would be to have the key inside which isn't recommended or to set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of your service account key json file as explained in https://firebase.google.com/docs/admin/setup#windows and then just having
admin.initializeApp();
I had the same issue while setting up SendGrid.
I got this error because I accidentally installed SendGrid to the root folder instead of the functions folder.
Make sure to install your service to your functions folder using the cmd prompt. It should look something like this:
D:\firebaseProject\functions> npm i #sendgrid/mail
I had this error once and I fixed it by checking my code again, check maybe you require or import a dependency or module that you have not installed and try to check your package.json file make sure you installed all the necessary module and dependencies.
But mostly the problem is due to importing a module you did not install or there is a bug in your code
in my case i had a file starting with capital case letter, but require(...) with a lower case letter, it worked on Mac because it ignores it, as i understand, but deploying these functions results in this error.
Renaming file to lower case fixed this error for me.
Mine was because I tried to import a module that dependents that is not standard-alone
const axioms = require("axioms");
I solved it by adding the module name version to dependencies list in the package.json file.
"dependencies": {
....
"axios": "^0.27.2"
}
I got the module version with this command
npm view axios version
With the suggestion of the recommended answer I moved all my cloud functions code to the single index.js file (rather than 'require' statements) and it seemed to work for me. Note that I omitted any API keys and such and it still worked. I am assuming that since this deployment is handled by the firebase CLI it already knows this is a trusted environment.
I had this same problem. check if your package.json file has all the dependencies that you used to write firebase functions.
e.g. I had written a firebase function in which I called nodemailer dependancy but did not installed in package.json file of 'functions' directory
Note: Remember to install required packages in 'functions' directory only
Another way to view deployment-time logs is to go to the Logs Explorer in the Google Cloud Console.
If you're not familiar with Google Cloud console, maybe because you're only familiar with the Firebase console that's a simplified version of the Google Cloud console, then check out these docs for how to access the Logs Explorer on Google Cloud Console.
In my case, I had to access the logs from the Logs Explorer because when I ran firebase functions:log as suggested by Chinmay Atrawalkar, I got the error Error: Failed to list log entries Failed to retrieve log entries from Google Cloud..
Through the Logs Explorer, I could see the following error: Detailed stack trace: Error: Cannot find module 'file-saver'. From there I was able to figure out I just needed to add file-saver to my functions/package.json dependencies by running yarn add file-saver from my functions directory.
I'm using Firebase functions with a React application. I have some non-trivial code that I don't want to duplicate, so I want to share it between the deployed functions and my React client. I've got this working locally in my React client (though I haven't tried deploying) - but I can't deploy my functions.
The first thing I tried was npm link. This worked locally, but the functions won't deploy (which makes sense, since this leaves no dependency in your package.json). Then I tried npm install ../shared/ - this looked promising because it did leave a dependency in package.json with a file: prefix - but Firebase still won't deploy with this (error below).
My project directory structure looks like this:
/ProjectDir
firebase.json
package.json (for the react app)
/src
* (react source files)
/functions
package.json (for firebase functions)
index.js
/shared
package.json (for the shared module)
index.js
My shared module package.json (extraneous details omitted):
{
"name": "myshared",
"scripts": {
},
"dependencies": {
},
"devDependencies": {
},
"engines": {
"node": "8"
},
"private": true,
"version": "0.0.1"
}
My firebase functions package.json (extraneous details omitted):
{
"name": "functions",
"scripts": {
},
"dependencies": {
"myshared": "file:../shared",
},
"devDependencies": {
},
"engines": {
"node": "8"
},
"private": true
}
When I try to deploy with:
firebase deploy --only functions
It's telling me it can't load the module:
Function failed on loading user code. Error message: Code in file index.js can't be loaded.
Did you list all required modules in the package.json dependencies?
And I don't think the issue is how I export/imported my code- but just in case:
The export:
exports.myFunc = () => { some code };
The import (functions/index.js)
const myFunc = require('myshared');
And in my react code:
import { myFunc } from 'myshared';
So far the searching I've done hasn't yielded anything that works. Someone did mention entering the shared module path in firebase.json, but I couldn't find any details (including in the firebase docs) that show what that would look like. Thanks for any tips to get this going.
I found a solution. I'm not sure if it's the only or even the best solution, but it seems to work for this scenario, and is easy. As Doug noted above, Firebase doesn't want to upload anything not in the functions directory. The solution was to simply make my shared module a subdirectory under functions (ie ./functions/shared/index.js). I can then import into my functions like a normal js file. However, my shared folder also has a package.json, for use as a dependency to the react app. I install it using:
npm install ./functions/shared
This creates a dependency in my react app, which seems to resolve correctly. I've created a production build without errors. I haven't deployed the react app yet, but I don't think this would be an issue.
Another solution is to create a symlink. In terminal, under /ProjectDir, execute:
ln -s shared functions/shared
cd functions
npm i ./shared
I'm running Node version 7.8.0
I have installed the Firebase and Firebase-admin modules into a Server side Node.js app.js file. I want to use these 2 methods:
var myCustomToken = '12345'
firebaseAdmin.auth().createCustomToken(myCustomToken) //firebaseAdmin.auth()
firebase.auth().authenticateWithCustomToken(myCustomToken) // firebase.auth()
The problem is dot .auth() doesn't show up for either module so I can't get to use the 2 methods. There are other methods that are tied to both modules that appear (in pics below) but .auth() isn't one of them.
For e.g.
firebaseAdmin.initializeApp(... //works
firebaseAdmin.credential.cert(... //works
firebase.initializeApp(...) //works
These are the modules I installed in the folder that was initialized with npm:
npm install firebase-admin --save
npm install algoliasearch --save
npm install firebase --save
These are the dependencies inside my package.json file:
"engines": {
"node": "7.8.0"
},
"dependencies": {
"algoliasearch": "^3.22.1",
"firebase": "^3.7.4",
"firebase-admin": "^4.1.4"
}
How can I get .auth() to appear on each module so I can access the 2 methods I need?
Firebase-admin module autocomplete:
Firebase module autocomplete:
Autocomplete results for both modules .auth() doesn't exist:
Have a look at https://firebase.google.com/docs/admin/setup
and https://firebase.google.com/docs/auth/admin/create-custom-tokens#create_custom_tokens_using_the_firebase_admin_sdk
for createCustomToken()
As for the client part, did you try signInWithCustomToken() ?
I filed an issue report at the Firebase-admin github repo and they got back to me and said that even though the .aut() object isn’t showing up in Sublime’s autocomplete it does exist. Since I knew the methods I was looking they said I could just manually put in the code and it would work which in fact it did.
One thing they suggested was that next time I can go into the node_modules folder and I would see that it does exist there:
node_modules/firebase-admin/lib/firebase-namespace.js
this is what is there:
Object.defineProperty(FirebaseNamespace.prototype, "auth", {
/**
* Gets the `Auth` service namespace. The returned namespace can be used to get the
* `Auth` service for the default app or an explicitly specified app.
*/
get: function () {
var _this = this;
var fn = function (app) {
return _this.ensureApp(app).auth();
};
return Object.assign(fn, { Auth: auth_1.Auth });
},
enumerable: true,
configurable: true
});
Basically if your using Sublime and autocomplete isn’t showing what your looking for just manually type it in and look through the node_modules folder to verify it’s there.
Another alternative is to use VSCode or Vim instead of Sublime and the autocomplete should work there.
Trying to bundle the following file with Webpack fails with
ERROR in ./~/pg/lib/native/index.js Module not found: Error: Cannot
resolve module 'pg-native' in
.../node_modules/pg/lib/native
# ./~/pg/lib/native/index.js 9:13-33
I tried several ignore statements in the .babelrc but didnt get it running...
The test-file i want to bundle: handler.js
const Client = require('pg').Client;
console.log("done");
webpack.config.js
module.exports = {
entry: './handler.js',
target: 'node',
module: {
loaders: [{
test: /\.js$/,
loaders: ['babel'],
include: __dirname,
exclude: /node_modules/,
}]
}
};
.babelrc
{
"plugins": ["transform-runtime"],
"presets": ["es2015", "stage-1"]
}
package.json
"dependencies": {
"postgraphql": "^2.4.0",
"babel-runtime": "6.11.6"
},
"devDependencies": {
"babel-core": "^6.13.2",
"babel-loader": "^6.2.4",
"babel-plugin-transform-runtime": "^6.12.0",
"babel-preset-es2015": "^6.13.2",
"babel-preset-stage-0": "^6.5.0",
"babel-polyfill": "6.13.0",
"serverless-webpack": "^1.0.0-rc.3",
"webpack": "^1.13.1"
}
Somewhat related github-issues:
https://github.com/brianc/node-postgres/issues/1187
https://github.com/serverless/serverless-runtime-babel/issues/8
This is indeed an old thread, but one that helped me nonetheless.
The solution provided by Steve Schafer 1 is good, but not the simplest.
Instead, the one provided by Marco Lüthy 2 in the linked issue is probably the easiest to set up because it is pure configuration, without even the need for a dummy file to be created.
It consists of modifying your Webpack config plugins array as follows:
const webpack = require('webpack');
const webpackConfig = {
...
resolve: { ... },
plugins: [
new webpack.IgnorePlugin(/^pg-native$/)
// Or, for WebPack 4+:
new webpack.IgnorePlugin({ resourceRegExp: /^pg-native$/ })
],
output: { ... },
...
}
Updated to include a change suggested in the comments.
This is an old thread but the problem still exists, so for anyone experiencing it, there is a workaround. The problem is an interaction between the way that node-postgres is written and how babel rewrites the code, which forces pg-native to be loaded even when you don't explicitly import/require it.
The simplest workaround is to add a couple of aliases to your webpack.config.js to cause it to link in a dummy do-nothing file instead:
{
...
resolve: {
alias: {
...
'pg-native': path-to-dummy-js-file,
'dns': path-to-dummy-js-file
}
}
...
}
where the dummy file contains a single line:
export default null
See https://github.com/brianc/node-postgres/issues/838 for further discussion and alternative workarounds.
I know that this is an old topic but I'm compelled to share how I solved it. It was maddening to deal with.
So, here is the readers digest version as based on the recollection from the last brain cell that I have.
Error:
Webpack Compilation Error ./node_modules/pg/lib/native/client.js Module not found: Error: Can't resolve 'pg-native'
The error above was thrown when attempting to run a Cypress test that required the npm package 'pg'.
Attempting to install the pg-native package was not successful and resulted in another error; namely ->
Call to 'pg_config --libdir' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
I found that executing pg_config --libdir in the VSCode cmd prompt resulted in that command failing.
However, I knew that it should work since running that command from the system command prompt resulted in this -> C:/PROGRA~1/POSTGR~1/9.3/lib
That is the path that contains a required dll.
So, instead of running npm install from the VSCode command prompt, I ran it from the command prompt as launched from windows.
The result...success!!! pg-native was installed successfully.
After, the Cypress test was able to run as well.
Errors in now way helped me to arrive at this solution. It was more just checking that things were installed that were required, etc.
You may have pg-native globally installed locally. Hence the packet manager does not include the pg-native in the lock file. That was a issue i experienced where it did run fine locally but every time i build in the cloud webpack complained about pg-native missing. I solved it by removing the lockfile in the files pushed to the cloud (In this case seed.run).