Module not found: Can't resolve 'fs' on #google-cloud/storage - node.js

Getting the Module not found: Can't resolve 'fs' error when trying to list out buckets from GCP Storage.
import { Storage } from '#google-cloud/storage';
const googleCloud = new Storage({
keyFilename: '../../my-project-c1a44bf80be3.json',
projectId: 'my-project',
});
googleCloud.getBuckets().then((x: any) => console.log(x));
my-project-c1a44bf80be3.json (Download from GCP) exists and is the project level
Error:
event - compiled successfully
event - build page: /
wait - compiling...
error - ./node_modules/#google-cloud/storage/build/src/file.js:24:0
Module not found: Can't resolve 'fs'
null
Could not find files for / in .next/build-manifest.json
event - compiled successfully
The same error appears when using googleapis.
However, instead of ./node_modules/#google-cloud/storage/build/src/file.js:24:0 it's in the google-auth-library module that comes with the googleapis.
Added using yarn.

This usually happens when you import or reference firebase-admin in the client side, check your code for wrong imports where you might be calling
import * as firebase from "firebase-admin";
instead of
import firebase from "firebase";

I faced the same problem but this guide was the solution in my case.
As mentioned before, the problem is that the firebase-admin package is designed to run in a NodeJs environment and not on the client. Extending webpack.config.js or next.config.js with certain entries did not solve the problem for me.
I have noticed your are using NextJS. So the solution is to move the firebase-admin calls to the NextJS API which you can create within the /pages/api folder.
Here is a simple example to fetch user data from the client.
You need:
/src/lib/firebase.ts: firebase initialization
/src/pages/api/user.ts: NextJS API handler
/src/components/xyz.tsx: component on client, which calls the API
// /src/lib/firebase.ts
import admin from "firebase-admin";
import { cert } from "firebase-admin/app";
import certConfig from "./firebase-adminsdk-config.json"
admin.initializeApp({
credential: cert(certConfig)
});
console.log("Firebase initialized.");
export default admin;
// /src/pages/api/user.ts
import admin from 'lib/firebase'
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const firebase = admin.firestore()
const collection = firebase.collection('users')
// get any data from requests body
const { user } = req.body
// perform your firebase collection call
const data = await collection.get(user.id)
// return data to client
res.status(200)
res.json({
success: true,
result: data
});
return res
}
On the client side then you just have to perform a GET to your own API
// /src/components/xyz.tsx
...
const onLogin = async (user) => {
const response = await fetch(`/api/users`, {
method: "GET",
body: JSON.stringify({ user }),
});
// data => { success: boolean, result: any }
const data = await response.json();
return data;
}
...

Posting the solution provided in the comments for visibility.
In your webpack configuration, you can indicate that certain modules
like fs should be stubbed out; at which point you should be able to
run this library client-side. For example:
node: {
// Some libraries import Node modules but don't use them in the browser.
// Tell Webpack to provide empty mocks for them so importing them works.
...config.node,
fs: 'empty',
child_process : 'empty',
net : 'empty',
tls: 'empty',
}

The scenario you are describing is now being handled by Google. Although there is no ETA, the issue is being prioritized.

Related

How to properly Organize Firebase Functions in Groups?

I am following This Guide to create a Functions project with multiple function files in NodeJS. while this is a good base, it does not show how to properly initiate a firebase app by using const app = admin.initializeApp() in a way that could be reused across the different files. my initial thought was to add the following in index.ts and import app as needed, but this raises a Maximum call stack size exceeded error.
import admin = require('firebase-admin');
export const app = admin.initializeApp();
While searching online I could only find some older posts that don't apply to current versions. can anyone provide a guideline for this?
This looks like a circular dependency issue. You need to initialize Firebase Admin only once at the beginning of index.ts and import other services as needed in other files as shown below:
// index.ts
// Don't export anything from this file.
import { initializeApp } from "firebase-admin/app";
initializeApp({
databaseURL: "<DB_URL>",
storageBucket: "<BUCKET>.appspot.com",
});
export * from "./metric";
// metric.ts
import { https } from "firebase-functions/v1";
import { getFirestore } from "firebase-admin/firestore";
const db = getFirestore();
export const test = https.onRequest(async (req, res) => {
const metricData = await db.collection("metrics").get();
res.json(metricData.docs.map((doc) => doc.data()));
});

How to configure firebase-admin-sdk for `verifyIdToken` to pass?

I try to use Firebase in my application. The frontend logs the user in using the Web SDK, without any backend. Later, I would like to call some backend APIs. For this reason, I pass the idToken to the backend and try to validate the user as described in the Firebase docs.
When I do the above flow locally using the Firebase Emulator everything works as expected.
When I switch off the Emulator the idToken validation fails with
{
errorInfo: {
code: 'auth/argument-error',
message: 'Firebase ID token has invalid signature. See https://firebase.google.com/docs/auth/admin/verify-id-tokens for details on how to retrieve an ID token.'
},
codePrefix: 'auth'
}
I created a Google hosted Firebase function to check if I can get the idToken validated there. The above setup works when the validation happens within the Google infrastructure.
Based on the above, I think the issue is in my FirebaseApp setup in the API. What that issue might be?
This is my setup.
I define 3 environment variables:
FIREBASE_DB_URL=https://<project-id>.firebaseio.com
FIREBASE_PROJECT_ID=<project-id>
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
I checked and cat $GOOGLE_APPLICATION_CREDENTIALS prints the correct file.
I initialize Firebase in the API with
import admin from "firebase-admin";
if(admin.apps.length == 0) {
admin.initializeApp({
credential: admin.credential.applicationDefault(),
databaseURL: process.env.FIREBASE_DB_URL,
projectId: process.env.FIREBASE_PROJECT_ID,
});
console.log('Firebase initialized')
} else {
console.warn('Firebase already initialized')
}
and this is the validating code
import { DecodedIdToken } from 'firebase-admin/lib/auth/token-verifier';
import { getAuth } from 'firebase-admin/auth';
import './initializeFirebase';
export default async function needsLoggedInUser(idToken: string): Promise<DecodedIdToken|false> {
try {
return await getAuth().verifyIdToken(idToken)
} catch(err) {
console.error(err)
return false
}
}
I use the above in a NextJS API code as
import { NextApiRequest, NextApiResponse } from 'next'
import { getDatabase } from 'firebase-admin/database';
import 'services/backend/initializeFirebase';
import needsLoggedInUser from 'services/backend/needsLoggedInUser';
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
// As an admin, the app has access to read and write all data, regardless of Security Rules
const decodedToken = await needsLoggedInUser(req.body.user)
if(!decodedToken) {
return res.status(403).send("403 Forbidden")
}
/// ... rest of the API
}

Error trying to set auto back up Firestore, cloud function

I am following this tutorial here: Tutorial
everything seems ok and it allows me to do everything in the tutorial, but when I run the function I get this error.
textPayload: "TypeError: Cannot read property 'charCodeAt' of undefined
at peg$parsetemplate (/workspace/node_modules/google-gax/build/src/pathTemplateParser.js:304:17)
at Object.peg$parse [as parse] (/workspace/node_modules/google-gax/build/src/pathTemplateParser.js:633:18)
at new PathTemplate (/workspace/node_modules/google-gax/build/src/pathTemplate.js:55:54)
at segments.forEach.segment (/workspace/node_modules/google-gax/build/src/pathTemplate.js:120:29)
at Array.forEach (<anonymous>)
at PathTemplate.render (/workspace/node_modules/google-gax/build/src/pathTemplate.js:114:23)
at FirestoreAdminClient.databasePath (/workspace/node_modules/#google-cloud/firestore/build/src/v1/firestore_admin_client.js:904:57)
at exports.scheduledFirestoreExport (/workspace/index.js:13:31)
at Promise.resolve.then (/layers/google.nodejs.functions-framework/functions-framework/node_modules/#google-cloud/functions-framework/build/src/invoker.js:330:28)
at process._tickCallback (internal/process/next_tick.js:68:7)
insertId: "000000-8410c5c7-8304-42b6-b2b6-dd55a54e8cab"
resource: {2}
timestamp: "2020-07-11T18:14:35.981Z"
severity: "ERROR"
labels: {1}
logName: "projects/b-b-b-app/logs/cloudfunctions.googleapis.com%2Fcloud-functions"
trace: "projects/b-b-b-app/traces/d7c07a715d0106225d9963ce2a046489"
receiveTimestamp: "2020-07-11T18:14:44.813410062Z"
}
I can't see what the problem may be.
I changed the buckets and the app ids like asked in the tutorial.
I am on a Blaze plan and can export the database to the bucket manually by using shell command and using
gcloud firestore export gs://bbbdata-backup
I am using the GCP console on the firebase site and using this code.
const firestore = require('#google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient();
const bucket = 'gs://bbbdata-backup'
exports.scheduledFirestoreExport = (event, context) => {
const databaseName = client.databasePath(
process.env.GCLOUD_PROJECT,
'(default)'
);
return client
.exportDocuments({
name: databaseName,
outputUriPrefix: bucket,
// Leave collectionIds empty to export all collections
// or define a list of collection IDs:
// collectionIds: ['users', 'posts']
collectionIds: [],
})
.then(responses => {
const response = responses[0];
console.log(`Operation Name: ${response['name']}`);
return response;
})
.catch(err => {
console.error(err);
});
};
Following the tutorial referred by the OP I run into precisely the same error. Runtime used: Node.js 14.
Root cause of the issue: value of process.env.GCLOUD_PROJECT is undefined.
Workaround: Go to GCP console -> Home. Note your Project ID. Replace process.env.GCLOUD_PROJECT with the 'Project ID' string. The Cloud Function will then work as expected
Note: it appears to be a known issue that GCLOUD_PROJECT environment variable was missing in the Node.js 10 runtime. This bug report contains a lot of additional pointers: https://github.com/firebase/firebase-functions/issues/437
I had a similar issue last year, probably you are missing some permission, I would do it this way, hope this works for you:
import * as functions from 'firebase-functions'
import { auth } from 'google-auth-library'
export const generateBackup = async () => {
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/datastore',
'https://www.googleapis.com/auth/cloud-platform'
]
})
const path = `YOUR_FOLDER_NAME_FOR_THE_BACKUP`
const BUCKET_NAME = `YOUR_BUCKET_NAME_HERE`
const projectId = await auth.getProjectId()
const url = `https://firestore.googleapis.com/v1beta1/projects/${projectId}/databases/(default):exportDocuments`
const backup_route = `gs://${BUCKET_NAME}/${path}`
return client.request({
url,
method: 'POST',
data: {
outputUriPrefix: backup_route,
// collectionsIds: [] // if you want to specify which collections to export, none means all
}
})
.catch(async (e) => {
return Promise.reject({ message: e.message })
})
}
You can then decide that is your trigger for this function and execute it accordingly.
Note: Go to the IAM section of your project and find the App Engine service account, you will need to add the role Cloud Datastore Import Export Admin, otherwise, It will fail.
You can read more about it here It's very detailed.
Cheers.

Node v 13 : Am I getting the 'type error: jwt.sign is not a function" because I'm using ES Modules?

Console Error: TypeError: jwt.sign is not a function
Trying out the latest Node version 13 using "type": "module" in my package.json. All is going well so far until I tried to add token authentication with 'jsonwebtoken'. Not sure if it's my code or maybe a compatibility issue? Using their ES Module has some differences from those I use in React.
create new token helper function
import * as jwt from 'jsonwebtoken'
export const newToken = user => {
return jwt.sign({id: user.id}, JWT_SECRET, {
expiresIn: '1d'
})
}
signup function
export const signup = async (req, res, next) => {
// ...
try {
//.. create user code ..
const user = new User({email, password})
user.save()
const token = newToken(user)
return res.json({token, user})
} catch (error) {
console.error(error)
res.status(500).send('Server Error')
}
and everytime I hit the signup route, I get the 500 Error, and a user still gets registered to my database. Hitting a wall a bit..
Thanks and happy holidays guys!
Edit: I just changed my import/export statements to common modules, and I was able to get tokens. Still don't know how to fix this to work with ESModules, or even what the issue is
jsonwebtoken uses default exports to expose its functions (https://github.com/auth0/node-jsonwebtoken/blob/master/index.js).
Therefore, you can load the module using import like this:
import jsonwebtoken from 'jsonwebtoken';
const token = jsonwebtoken.sign({ foo: 'bar' }, 'shhhhh');

Cloud Functions for Firebase: how to issue a request to my Cloud Endpoint

I'm trying to issue a request to my cloud endpoint project when a certain value is written in the firebase database. I can't find any example of how perform a request to Endpoints in Node.js. Here's what I come up with so far:
"use strict";
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const gapi = require('googleapis');
admin.initializeApp(functions.config().firebase);
exports.doCalc = functions.database.ref('/users/{uid}/calc').onWrite(event => {
return gapi.client.init({
'apiKey': 'AIzxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
'clientId': '1234567890-xxx.apps.googleusercontent.com',
'scope': 'donno what to put here'
}).then(function() {
return gapi.client.request({
'path': 'https://myproj.appspot.com/_ah/api/myApi/v1',
'params': {'query': 'startCalc', uid: event.params.uid }
})
}).then(function(response) {
console.log(response.result);
}, function(reason) {
console.log('Error: ' + reason.result.error.message);
});
});
When triggered, Functions' log spouts: TypeError: Cannot read property 'init' of undefined. i.e. doesn't even recognize gapi.client.
First, what is the right package to use for this request? googleapis? request-promise?
Second, am I setting up the correct path and parameters for a call to an endpoint? Assume the endpoint function is startCalc(int uid).
Update
It seems that Cloud Functions for Firebase blocks requests to their App Engine service - at least on the Spark plan (even though they're both owned by Google - so you'd assume "on the same network"). The request below, works on a local machine running Node.js, but fails on the Functions server, with a getaddrinfo EAI_AGAIN error, as described here. Evidently, it is not considered accessing Google API when you perform a request to your server running on Google's App Engine.
Can't explain why Firebase advocates here steer clear of this question like from fire.
Original Answer
Figured it out - switched to 'request-promise' library:
"use strict";
const functions = require('firebase-functions');
const request = require('request-promise');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.doCalc = functions.database.ref('/users/{uid}/calc').onWrite(event => {
return request({
url: `https://myproj.appspot.com/_ah/api/myApi/v1/startCalc/${event.params.uid}`,
method: 'POST'
}).then(function(resp) {
console.log(resp);
}).catch(function(error) {
console.log(error.message);
});
});

Resources