I'm currently trying to build a cloud function to export my Firestore data to my Storage Bucket.
The only example I've found on the Firebase DOCs on how to do this:
https://googleapis.dev/nodejs/firestore/latest/v1.FirestoreAdminClient.html#exportDocuments
EXAMPLE
const firestore = require('#google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient({
// optional auth parameters.
});
const formattedName = client.databasePath('[PROJECT]', '[DATABASE]');
client.exportDocuments({name: formattedName})
.then(responses => {
const response = responses[0];
// doThingsWith(response)
})
.catch(err => {
console.error(err);
});
From that example, it seems that I need to install #google-cloud/firestore as a dependency to my cloud function.
But I was wondering if I can access these methods using only the firebase-admin package.
I've thought of that because the firebase-admin has the #google-cloud/firestore as a dependency already.
> firebase-admin > package.json
"dependencies": {
"#firebase/database": "^0.4.7",
"#google-cloud/firestore": "^2.0.0", // <---------------------
"#google-cloud/storage": "^3.0.2",
"#types/node": "^8.0.53",
"dicer": "^0.3.0",
"jsonwebtoken": "8.1.0",
"node-forge": "0.7.4"
},
QUESTION:
Is it possible to get an instance of the FirestoreAdminClient and use the exportDocuments method using just the firebase-admin ?
Or do I really need to install the #google-cloud/firestore as a direct dependency and work with it directly?
The way you're accessing the admin client is correct as far as I can tell.
const client = new admin.firestore.v1.FirestoreAdminClient({});
However, you probably won't get any TypeScript/intellisense help beyond this point since the Firestore library does not actually define detailed typings for v1 RPCs. Notice how they are declared with any types: https://github.com/googleapis/nodejs-firestore/blob/425bf3d3f5ecab66fcecf5373e8dd03b73bb46ad/types/firestore.d.ts#L1354-L1364
Here is an implementation I'm using that allows you to do whatever operations you need to do, based on the template provided by firebase here https://firebase.google.com/docs/firestore/solutions/schedule-export
In my case I'm filtering out collections from firestore I don't want the scheduler to automatically backup
const { Firestore } = require('#google-cloud/firestore')
const firestore = new Firestore()
const client = new Firestore.v1.FirestoreAdminClient()
const bucket = 'gs://backups-user-data'
exports.scheduledFirestoreBackupUserData = async (event, context) => {
const databaseName = client.databasePath(
process.env.GCLOUD_PROJECT,
'(default)'
)
const collectionsToExclude = ['_welcome', 'eventIds', 'analyticsData']
const collectionsToBackup = await firestore.listCollections()
.then(collectionRefs => {
return collectionRefs
.map(ref => ref.id)
.filter(id => !collectionsToExclude.includes(id))
})
return client
.exportDocuments({
name: databaseName,
outputUriPrefix: bucket,
// Leave collectionIds empty to export all collections
// or define a list of collection IDs:
// collectionIds: ['users', 'posts']
collectionIds: [...collectionsToBackup]
})
.then(responses => {
const response = responses[0]
console.log(`Operation Name: ${response['name']}`)
return response
})
.catch(err => {
console.error(err)
})
}
firebase-admin just wraps the Cloud SDK and re-exports its symbols. You can use the wrapper, or use the Cloud SDK directly, or even a combination of the two if you want. If you want to use both, you have to declare an explicit dependency on #google-cloud/firestore in order to be able to import it directly into your code.
Here is the full explanation with code (I use it and it works very well) on how to do automated Firestore backups by mixing Cloud Scheduler, PubSub and Firebase Function https://firebase.google.com/docs/firestore/solutions/schedule-export
Related
Im using NodeJs within google cloud functions and I want to increase the value of one parameter in one document
I initiate Firestore:
const Firestore = require('#google-cloud/firestore');
const PROJECTID = 'XXXX';
const firestore = new Firestore({
projectId: PROJECTID,
timestampsInSnapshots: true
});
And my .get() and .set() functions work correctly. But when I try to update a value incrementing it, I get errors with FieldValue.increment
I've tried:
await snapshot.ref.update({ parameter: FieldValue.increment(1) });
adding:
const FieldValue = require('firebase-admin').firestore.FieldValue;
await snapshot.ref.update({ parameter: FieldValue.increment(1) });
as explained in other sites, and:
documentRef.update(
'parameter', Firestore.FieldValue.increment(1)
)
as explained in https://cloud.google.com/nodejs/docs/reference/firestore/latest/firestore/fieldvalue
const admin = require('firebase-admin');
increment = admin.firestore.FieldValue.increment(1);
await snapshotRef.update({ parameter: increment });
But none work.
Error:
TypeError: admin.firestore.FieldValue.increment is not a function
or
TypeError: FieldValue.increment is not a function
"dependencies": {
"firebase-admin": "^6.5.1",
"nodemailer": "^6.6.1"
}
The increment() was added to Firestore SDK in April 2019 as mentioned in the release notes but Firebase Admin 6.5.1 was released in January 2019. Upgrading to latest version should resolved this issue.
Just posting this as a community wiki. Command to update to the latest Firebase Admin version.
npm i firebase-admin#latest
You could also modify the specific package in package.json by changing the version to the latest. See sample below:
"firebase-admin": "latest"
and then run
npm install
Also, for best practice, always remove the node_modules folder and package-lock.json and then run npm install before you deploy your app.
I'm using Next.js for my side project. I have a PostrgeSQL database hosted on ElephantSQL. Inside the Next.js project, I have a GraphQL API set up, using the apollo-server-micro package.
Inside the file where the GraphQL API is set up (/api/graphql), I import a database helper-module. Inside that, I set up a pool connection and export a function which uses a client from the pool to execute a query and return the result. This looks something like this:
// import node-postgres module
import { Pool } from 'pg'
// set up pool connection using environment variables with a maximum of three active clients at a time
const pool = new Pool({ max: 3 })
// query function which uses next available client to execute a single query and return results on success
export async function queryPool(query) {
let payload
// checkout a client
try {
// try executing queries
const res = await pool.query(query)
payload = res.rows
} catch (e) {
console.error(e)
}
return payload
}
The problem I'm running into, is that it appears as though the Next.js API doesn't (always) keep the connection alive but rather opens up a new one (either for every connected user or maybe even for every API query), which results in the database quickly running out of connections.
I believe that what I'm trying to achieve is possible for example in AWS Lambda (by setting context.callbackWaitsForEmptyEventLoop to false).
It is very possible that I don't have a proper understanding of how serverless functions work and this might not be possible at all but maybe someone can suggest me a solution.
I have found a package called serverless-postgres and I wonder if that might be able to solve it but I'd prefer to use the node-postgres package instead as it has much better documentation. Another option would probably be to move away from the integrated API functionality entirely and build a dedicated backend-server, which maintains the database connection but obviously this would be a last resort.
I haven't stress-tested this yet, but it appears that the mongodb next.js example, solves this problem by attaching the database connection to global in a helper function. The important bit in their example is here.
Since the pg connection is a bit more abstract than mongodb, it appears this approach just takes a few lines for us pg enthusiasts:
// eg, lib/db.js
const { Pool } = require("pg");
if (!global.db) {
global.db = { pool: null };
}
export function connectToDatabase() {
if (!global.db.pool) {
console.log("No pool available, creating new pool.");
global.db.pool = new Pool();
}
return global.db;
}
then in, eg, our API route, we can just:
// eg, pages/api/now
export default async (req, res) => {
const { pool } = connectToDatabase();
try {
const time = (await pool.query("SELECT NOW()")).rows[0].now;
res.end(`time: ${time}`);
} catch (e) {
console.error(e);
res.status(500).end("Error");
}
};
I am trying to seed some sample data into my local firestore emulator database. I adapted the example from this github issue
My code looks like this:
const {Firestore} = require('#google-cloud/firestore');
const {credentials} = require('grpc');
const db = new Firestore({
projectId: 'my-project-id',
servicePath: 'localhost',
port: 8100,
sslCreds: credentials.createInsecure(),
customHeaders: {
"Authorization": "Bearer owner"
}
});
async function load_data() {
await db.collection("mycollection").doc("myid").set({ foo: "test" })
}
load_data();
But I receive the error
this.credentials._getCallCredentials is not a function
Tested on node 10 and 12 with same error.
Library versions:
#google-cloud/firestore 3.5.1
grpc 1.24.2
Is there a better approach to writing to local emulated firestore? Or is there something wrong with my code?
The problem here is that you're trying to use two different implementations of gRPC together. Internally firestore uses #grpc/grpc-js, so that is what you should be using. You should only need to change the second line to const {credentials} = require('#grpc/grpc-js'); and switch the dependency to that library.
We're working on an app that will allow users to store templates with images on them, and pull those templates up later. This is for an AR environment using Viro on React Native.
We're trying to dynamically load an image into the component, and receiving errors when we require the filepath, which has been set to a variable:
const exampleUri = '/some/uri'
render() {
return(
<Viro3DObject
source={require(exampleUri)}
/>)
}
The URI for the source prop has to be dynamic, as the URIs are pulled from a Database.
We've tried storing the entire request in the database (in models/element.js):
const Sequelize = require('sequelize');
const db = require('../db');
const Element = db.define('element', {
sourceViro3DObject: {
type: Sequelize.STRING
}
});
sourceViro3DObject: `require('../../assets/emoji_heart/emoji_heart.vrx')`
When we called it in the React Native class component:
getObjectData = async () => {
try {
const {data} = await axios.get(`/api/elements/${this.props.elementId}`)
this.setState({sourceViro3DObject: data.sourceViro3DObject})
} catch (err) {
console.log(err)
}
}
async componentDidMount() {
await this.getObjectData()
}
But this simply sets state.sourceViro3DObject to a string:
'require('../../assets/emoji_heart/emoji_heart.vrx')'
We've tried setting the filepath directly to state as a string:
state.sourceViro3DObject = '../../assets/emoji_heart/emoji_heart.vrx'
and then call require on it:
require(this.state.sourceViro3DObject)
and received the following error:
Invalid prop `source` supplied to `Viro3DObject`
We've seen recommendations of storing the URIs in an object, but that can't work for us as we don't know what image is going to be used until it's pulled from the database. We can't hard-code them anywhere.
We'd appreciate any help with this!
I want to delete a folder in firebase storage with node js because this is a firebase function.
For example :
storageRef.child(child1).child(child2).delete();
something like this, but firebase documentation doesn't tell anything.
One more question:
When initialize storage documentation node js requires my admin json, but realtime database doesn't want this wonder why?
Have a look at the Node.js client API Reference for Google Cloud Storage and in particular at the delete() method for a File.
You can do it like this using Node.js:
const firebase = require('firebase-admin');
async function deleteImageFromFirebase(imageName) {
await firebase.storage().bucket().file("folderName/"+imageName).delete();
}
And like this client side:
// Create a reference to the file to delete
var desertRef = storageRef.child('images/desert.jpg');
// Delete the file
desertRef.delete().then(function() {
// File deleted successfully
}).catch(function(error) {
// Uh-oh, an error occurred!
});
View this info on the Firebase website:
how to delete files Firebase-storage
This might be late but at least on Web (so basically what you need), there is new API to delete the whole folder.
I tested deleting a folder with 2 pictures inside and it works. I then tried a folder-A with contents: folder-B + picture-A. Folder-B also has a picture-B inside; it still deleted folder-A with all of its contents.
Solution:
const bucket = admin.storage().bucket();
return bucket.deleteFiles({
prefix: `posts/${postId}`
);
I couldn't find this on the official documentation (perhaps is really new API) but really cool article where I found the solution:
Automatically delete your Firebase Storage Files from Firestore with Cloud Functions for Firebase
import { storage } from "./firebaseClient";
import { bucket } from "./firebaseServer";
//Let's assume this is the URL of the image we want to delete
const downloadUrl = "https://storage.googleapis.com/storage/v1/b/<projectID>.appspot.com/o/<location>?"
//firebase delete function
const deleteImages = async ({ downloadUrl }) => {
const httpsRef = storage.refFromURL(downloadUrl).fullPath;
return await bucket
.file(httpsRef)
.delete()
.then(() => "success")
.catch(() => "error")
}
//call the deleteImages inside async function
const deleteStatus = await deleteImages({ downloadUrl: oldImage });
console.log(deleteStatus) //=> "success"