How to use Google Cloud FireStore emulator with NodeJS project? - node.js

I am trying to use this tool in order to test my NodeJS application :
https://cloud.google.com/sdk/gcloud/reference/beta/emulators/firestore
when I try to add something to the firestore database, I get this error :
Error: Unable to detect a Project Id in the current environment.
It is normal I think because locally there is no GCP Project.
I guess that I must configure firestore :
const Firestore = require('#google-cloud/firestore');
const db = new Firestore({
projectId: 'YOUR_PROJECT_ID',
keyFilename: '/path/to/keyfile.json',
});
When using the Google Cloud Firestore emulator, how may I create a "virtual" project Id ?
Also, is the keyfile mandatory ?
The final goal is to use a local emulator for firestore if a "FIRESTORE_EMULATOR_HOST" env var exist.

You'll need to setup and start the emulator in a separate terminal
gcloud beta emulators firestore start
Then for the project ID you can follow the documentation, and the github instructions:
firebase serve --only firestore
and then
require "google/cloud/firestore"
# Make Firestore use the emulator
ENV["FIRESTORE_EMULATOR_HOST"] = "127.0.0.1:8080"
firestore = Google::Cloud::Firestore.new project_id: "emulator-project-id"
# Get a document reference
nyc_ref = firestore.doc "cities/NYC"
nyc_ref.set({ name: "New York City" }) # Document created
You can also check this example here

You need to use express.js framework with node js to make requests to #google-cloud/firestore, and best practices will be to start from https://cloud.google.com/nodejs/getting-started?hl=fa

Related

How can I force Firebase Realtime Database URL without "default-rtdb"

My realtime database URL is like https://myprojectid.firebaseio.com
When I start my node.js server I have error message :
#firebase/database: FIREBASE WARNING: Firebase error. Please ensure
that you have the URL of your Firebase Realtime Database instance
configured correctly.
(https://myprojectid-default-rtdb.firebaseio.com/)
My Firebase config is :
# Firebase
apiKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
authDomain=myprojectid.firebaseapp.com
databaseURL=https://myprojectid.firebaseio.com
projectId=myprojectid
storageBucket=myprojectid.appspot.com
messagingSenderId=00000000000000
appId=1:0000000000000:web:000000000000000
When I'm trying to debug npm library, I can see that namespace is indeed "myprojectid-default-rtdb" but can't find where to change it.
I tried to set databaseURL with query string ?ns=myprojectid but not better.
By searching on Google I could read :
For recently created Firebase projects the default database URI
usually has the format https://
-default-rtdb.firebaseio.com. Databases in projects
created before September 2020 had the default database URI
https://.firebaseio.com. For backward compatibility
reasons, if you don’t specify a database URI, the SDK will use the
project ID defined in the Service Account JSON file to automatically
generate it
Could you help me to find a solution please?
There's no way to change the name of the default database instance that is created in your Firebase project. If your project is on the paid plan, you can add additional database instances where you fully control the name, but not for the default one.
You will have to update your project configuration to use the correct database URL. If the database in the US data center, that'd be:
databaseURL=https://myprojectid-default-rtdb.firebaseio.com

Google Cloud Platform / Firebase Function not triggering with onWrite

My application makes use of Firestore Function Triggers to perform background actions based on changes in the Firestore, e.g. user profile changes. For example, if they change their mobile number, a verification code is sent.
I have a trigger that should run when an onWrite() event happens on a specific collection. onWrite() runs when any of the following actions occur in Firebase on a specific collection:
onCreate()
onUpdate()
onDelete()
In my usecase, I need it to run for onCreate() and onUpdate(), thus I use onWrite()
For Firebase Triggers to work, a specific format is expected in addition to a document id/wildcard representing a document that was created/changed/deleted.
Constants:
const collections = {
...
conversations: "conversations",
...
}
Callable Function (updates firestore):
/**
* Add an conversation to conversations collection
* #type {HttpsFunction & Runnable<any>}
*/
exports.addConversations = functions.https.onCall(async (data, context) => {
// expects conversation & interested state
const {valid, errors} = validateRequiredBody(data, [
"conversation",
]);
if (!valid) {
return {
status: false,
message: "Missing or invalid parameters",
errors: errors,
jwtToken: "",
};
}
// get conversation item
const conversation = {
id: data["conversation"]["id"],
name: data["conversation"]["name"],
}
// create conversation with empty counter
// let writeResult = await collectionRefs.conversationsRef.doc(conversation.id).set({
let writeResult = await admin.firestore().collection(collections.conversations).doc(conversation.id).set({
id: conversation.id,
name: conversation.name,
count: 0
});
console.log(`[function-addConversations] New Conversation [${conversation.name}] added`);
return {
status: true,
message: ""
}
});
Firestore Trigger (not triggering):
/**
* On conversations updated/removed, update corresponding counter
* #type {CloudFunction<Change<QueryDocumentSnapshot>>}
*/
exports.onConversationProfileCollectionCreate = functions.firestore.document(`${collections.conversations}/{id}`)
.onWrite(async snapshot => {
console.log("Conversation Collection Changed");
// let conversation = collectionRefs.conversationsRef.doc(snapshot.id);
// await conversation.update({count: FieldValue.increment(1)});
});
In my mobile application, the user (calls) the addConversations() firebase function, this adds the new conversation to Firestore which is clearly visible, but the counter trigger (trigger function) doesn't run.
Emulator output:
...
{"verifications":{"app":"MISSING","auth":"MISSING"},"logging.googleapis.com/labels":{"firebase-log-type":"callable-request-verification"},"severity":"INFO","message":"Callable request verification passed"}
[function-addConversations] New Conversation [Test Conversation Topic] added
Profile updated
(print profile data)
...
What I SHOULD expect to see:
...
{"verifications":{"app":"MISSING","auth":"MISSING"},"logging.googleapis.com/labels":{"firebase-log-type":"callable-request-verification"},"severity":"INFO","message":"Callable request verification passed"}
[function-addConversations] New Conversation [Test Conversation Topic] added
Conversation Collection Changed // this is output by the trigger
Profile updated
(print profile data)
...
Did I do something wrong?
The issue was one closer to home.
I am developing using the firebase emulators and connecting to them using the emulators addition in Flutter's firebase packages built in emulator feature.
Firebase emulator setup e.g. Firebase Functions can be found here
TL;DR:
When starting your Firebase emulator, you should see something similar to:
C:\Users\User\MyApp\awesome-app-server>firebase emulators:start --only functions
i emulators: Starting emulators: functions
! functions: You are running the functions emulator in debug mode (port=9229). This means that functions will execute in sequence rather than in parallel.
! functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: auth, firestore, database, hosting, pubsub, storage
! Your requested "node" version "12" doesn''t match your global version "14"
i ui: Emulator UI logging to ui-debug.log
i functions: Watching "C:\Users\User\MyApp\awesome-app-server\functions" for Cloud Functions...
> Debugger listening on ws://localhost:9229/03dc1d62-f2a3-418e-a343-bb0b357f7329
> Debugger listening on ws://localhost:9229/03dc1d62-f2a3-418e-a343-bb0b357f7329
> For help, see: https://nodejs.org/en/docs/inspector
! functions: The Cloud Firestore emulator is not running, so calls to Firestore will affect production.
! functions: The Realtime Database emulator is not running, so calls to Realtime Database will affect production.
...
BUT then you see this - THIS is very important!
i functions[us-central1-api-user-onConversationProfileCollectionCreate ]: function ignored because the database emulator does not exist or is not running.
This, since Firestore & (Realtime) Database use triggers which are found in functions, functions expects to find a local firestore/database emulator.
Since no firestore/database emulators were running locally
! functions: The Cloud Firestore emulator is not running, so calls to Firestore will affect production.
! functions: The Realtime Database emulator is not running, so calls to Realtime Database will affect production.
and these functions don't automagically attach to production Firestore/Database(s) (that would be potentially devestating), these triggers didn't run when I expected them to while emulating locally.
Solution:
Emulate firestore & database locally (see this to import your firestore data to a local emulator) with firebase emulators:start --only functions,...,firestore,database
Upload functions to work with Firestore/Database(s) (please do this with care)
More details:
Below I provide details what lead me to the problem, and how I figured out the issue:
What I was doing:
firebase emulators:start --inspect-functions --only functions,auth
[local] Firebase Functions I was developing and testing the backend for my mobile app using Firebase Functions for various interactivity.
Since Firebase Auth is handled locally through the use of custom tokens on my firebase functions app, I used local auth for testing
I had prepopulated my Firestore with data, thus I intended to use Firestore data while emulating locally which had worked as expected, BUT the firebase firestore & database triggers won't work due to emulating them locally.
I knew some of my triggers DID infact trigger correctly, thus it must be a configuration error of some kind.
Extended Solution (import firestore data to local emulator)
firebase emulators:start --inspect-functions --only functions,auth,firestore,database
notice the last 2, firestore & database - this should be added for triggers to work locally
I noticed when reviewing the logs at startup, the text indicating some functions won't run. This made me realize I was making a crucial mistake - the cause.
Import Data from production firestore
The reason for using production (during development) firestore is to not recreate all the data for each test. The solution is to export from firestore and import you data to the emulators.
See this for details - I wasn't able to export from the terminal, I thus went to Google Console to export to a storage bucket, and download it via the console from there.
To start the emulator (and allow debugging of firebase functions), I use:
firebase emulators:start --inspect-functions --import ./functions/{path/to/data} --only functions,auth,firestore,database
details:
firebase emulators:start - start emulator
--inspect-functions allow debugging functions via websockets - e.g. Webstorm NodeJs debugger on port 9229
--import ./functions/{path/to/data} - import data to firestore emulator using project root as ./
--only functions,auth,firestore,database specify the modules to emulate
You are using the wrong value. snapshot works only for onDelete and onCreate method
exports.useWildcard = functions.firestore
.document('users/{userId}')
.onWrite((change, context) => {
// If we set `/users/marie` to {name: "Marie"} then
context.params.userId == "marie"
// ... and ...
change.after.data() == {name: "Marie"}
});
for more info read here feedbackCloud Firestore triggers

Using firebase tools as a node module in firebase function

I am trying to call the export method from the firebase tools node module in a firebase function.
await firebase_tools.auth.export(tempLocalFile);
But i am receiving the following error
FirebaseError: No currently active project.
To run this command, you need to specify a project. You have two options:
- Run this command with [1m--project <alias_or_project_id>[22m.
- Set an active project by running [1mfirebase use --add[22m, then rerun this command.
To list all the Firebase projects to which you have access, run [1mfirebase projects:list[22m.
To learn about active projects for the CLI, visit https://firebase.google.com/docs/cli#project_aliases
Is there a way to set the active project in the nodejs cloud function?
Edit
I am initialising the object like so
const firebase_tools = require('firebase-tools');
I'm not exactly replying to your problem, but note that the auth.export command will not work in a Cloud Function.
As explained in the documentation (See the note at the bottom):
when used in a limited environment like Cloud Functions, not all
firebase-tools commands will work programatically because they require
access to a local filesystem.
This is the case for the auth.export command. An alternative is to use the listUsers() method of the Admin SDK, as explained here.

Can I avoid using live Firebase Storage when using emulators?

as I patiently wait for Firebase storage to be added to the emulators, I was wondering if there is a way I can avoid modifying live storage files and folders when running hosting / functions in the emulator?
For example I use the following code to delete all the files in a folder. Last night someone accidentally deleted all the documents in our emulator as part of a test and it deleted all the LIVE storage folders as we use an import of real documents into our emulator 🤦
async function deleteStorageFolder(path:string) {
const bucket = admin.storage().bucket();
return bucket.deleteFiles({
prefix: path
})
Is there any way I can tell firebase to avoid using the production storage APIs when emulators are running?
I have used the following condition in my function to prevent using firebase storage API when running in emulator:
if (process.env.FUNCTIONS_EMULATOR == "true") {
console.log(`Running in emulator, won't call firebase storage`)
} else {
// Code goes here to run storage APIs
}

GCloud Vision API Permission Denied on Second Request

I've gone through all the setup steps to make calls to the Google Vision API from a Node.js App. Link to the guide: https://cloud.google.com/vision/docs/libraries#setting_up_authentication
I'm using the ImageAnnotatorClient from the #google-cloud/vision package to make some text detections.
At first, it looked like everything was set up correctly but I don't know why it only allows me to do one request.
Further requests will give me the following error:
Error: 7 PERMISSION_DENIED: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the vision.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/
If I restart the Node app it again allows me to do one request to the Vision API but then the subsequent requests keep failing.
Here's my code which is almost the same as in the examples:
const vision = require('#google-cloud/vision');
// Creates a client
const client = new vision.ImageAnnotatorClient();
const detectText = async (imgPath) => {
// console.log(imgPath);
const [result] = await client.textDetection(imgPath);
const detections = result.textAnnotations;
return detections;
}
It is worth to mention that this works every time when I run the Node app in my local machine. The problem is happening on my Ubuntu Droplet from Digital Ocean.
Again, I set everything up as it is in the guides. Created a Service Account, downloaded the Service Account Key JSON file, set up the environment variable like this:
export GOOGLE_APPLICATION_CREDENTIALS="PATH-TO-JSON-FILE"
I'm also setting the environment variable in the .bashrc file.
What could I be missing? Before setting up everything from scratch and go through the whole process again I thought it would be good to ask for some help.
So I found the problem. In my case, it was a problem with PM2 not passing the system env variables to the Node app.
So I had everything set up correctly auth-wise but the Node app wasn't seeing the GOOGLE_APPLICATION_CREDENTIALS env var.
I deleted the PM2 process, created a new one and now it works.

Resources