for the correct security of my app i have been obliged to set up a small nodejs server for firebase, but the problems are not over.... the firebase server sdk not support the storage apis. i read that for this is neccessary a gcloud storage apis because firebase use the same service.
in the server side is important to get a files custom metadata because i must read and update they. i not find the apposite functions to get a file metadata. in the client sdk is simple
// Create a reference to the file whose metadata we want to retrieve
var forestRef = storageRef.child('images/forest.jpg');
// Get metadata properties
forestRef.getMetadata().then(function(metadata) {
// Metadata now contains the metadata for 'images/forest.jpg'
}).catch(function(error) {
// Uh-oh, an error occurred!
});
and in gcloud storage what function can i use?? thanks
You'll want to use the getMetadata() method in gcloud:
var gcloud = require('gcloud');
// Initialize GCS
var gcs = gcloud.storage({
projectId: 'my-project',
keyFilename: '/path/to/keyfile.json'
});
// Reference an existing bucket
var bucket = gcs.bucket('foo.appspot.com');
// Reference to a file
var file = bucket.file('path/to/my/file');
// Get the file metadata
file.getMetadata(function(err, metadata, apiResponse) {
if (err) {
console.log(err);
} else {
console.log(metadata);
}
});
Related
I'm trying to upload some images to a firebase storage bucket,
and trying to run the code locally in the firebase emulator.
With most examples I'll get an error like:
"Unhandled error FirebaseError: Firebase: Need to provide options, when not being deployed to hosting via source.
(app/no-options). at initializeApp
I tried this: https://firebase.google.com/docs/admin/setup
const defaultApp = initializeApp({ credential: applicationDefault() })
Which may or not be working.
I think this creates an authed app (maybe?) but if I later use
const storage = getStorage() I get the same auth error.
So next, I tried using the defaultApp for next steps, like:
const storage = defaultApp.storage();
that doesn't error, but it returns something different from getStorage, that I can't seem to use for other needed things like getting a ref
as per examples https://firebase.google.com/docs/storage/web/upload-files
So it's not clear how to upload files to firebase from the local simulator.
my full function looks like this:
import cuid from "cuid";
import {
getStorage,
ref,
uploadBytes,
uploadBytesResumable,
getDownloadURL
} from "firebase/storage";
import { admin, defaultApp } from '../utils/firebaseInit'
export function saveToBucket(url: string): string {
const normalStorage = getStorage() // Gets a FirebaseStorage instance for the given Firebase app. but NOT AUTHED
const defaultStorage = defaultApp.storage(); // Gets the default FirebaseStorage instance.
const storageRef = ref(normalStorage)
const bucketPath = `personas-out/img-${cuid()}.jpg`
const bucket = defaultStorage.bucket();
// const bucketRef = bucket.ref(); // Property 'ref' does not exist on type 'Bucket'.ts(2339)
const metadata = {
contentType: 'image/jpeg'
};
const imageRef = ref(storageRef, bucketPath);
const file = new File([url], bucketPath, {
type: "image/jpeg",
});
// will happen in the background?
uploadBytesResumable(imageRef, file, metadata)
.then((snapshot: any) => {
console.log('Uploaded', snapshot);
});
return bucketPath
}
You're not supposed to use the Firebase client SDK (firebase/storage) in nodejs code. The client SDKs are meant to work in a browser only. For nodejs code that runs on a backend (which is what you're doing when you deploy to Cloud Functions), you should use the Firebase Admin SDK. The API it exposes for Storage is essentially a thin wrapper around the Google Cloud Storage nodejs library.
In a MERN + Firebase project, I have an image data string that I want to upload and then get the access token of that file.
The image data string is of the following form:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOoAAAClCAYAAABSmmH3AAAAAXNSR0IArs4c6QAABFJJREFUeF7t1oENg0AQA8F8/6XSAx8pVWTloQIzZwvOvfd+PAQI/LXAMdS/vo9wBH4ChqoIBAIChho4kogEDFUHCAQEDDVwJBEJGKoOEAgIGGrgSCISMFQdIBAQMNTAkUQkYKg6QCAgYKiBI4lIwFB1gEBAwFADRxKRgKHqAIGAgKEGjiQiAUPVAQIBAUMNHElEAoaqAwQCAoYaOJKIBAxVBwgEBAw1cCQRCRiqDhAICBhq4EgiEjBUHSAQEDDUwJFEJGCoOkAgIGCogSOJSMBQdYBAQMBQA0cSkYCh6gCBgIChBo4kIgFD1QECAQFDDRxJRAKGqgMEAgKGGjiSiAQMVQcIBAQMNXAkEQkYqg4QCAgYauBIIhIwVB0gEBAw1MCRRCRgqDpAICBgqIEjiUjAUHWAQEDAUANHEpGAoeoAgYCAoQaOJCIBQ9UBAgEBQw0cSUQChqoDBAIChho4kogEDFUHCAQEDDVwJBEJGKoOEAgIGGrgSCISMFQdIBAQMNTAkUQkYKg6QCAgYKiBI4lIwFB1gEBAwFADRxKRgKHqAIGAgKEGjiQiAUPVAQIBAUMNHElEAoaqAwQCAoYaOJKIBAxVBwgEBAw1cCQRCRiqDhAICBhq4EgiEjBUHSAQEDDUwJFEJGCoOkAgIGCogSOJSMBQdYBAQMBQA0cSkYCh6gCBgIChBo4kIgFD1QECAQFDDRxJRAKGqgMEAgKGGjiSiAQMVQcIBAQMNXAkEQkYqg4QCAgYauBIIhIwVB0gEBAw1MCRRCRgqDpAICBgqIEjiUjAUHWAQEDAUANHEpGAoeoAgYCAoQaOJCIBQ9UBAgGB8zzPDeQUkcC0wHnf11CnK+DlCwJ+fQtXknFewBd1vgIACgK+qIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCBgqIUryTgvYKjzFQBQEDDUwpVknBcw1PkKACgIGGrhSjLOCxjqfAUAFAQMtXAlGecFDHW+AgAKAoZauJKM8wKGOl8BAAUBQy1cScZ5AUOdrwCAgoChFq4k47yAoc5XAEBBwFALV5JxXsBQ5ysAoCDwBaT9ke70WG4vAAAAAElFTkSuQmCC
This is the reference to where the file should be uploaded:
const imageRef: StorageReference = ref(
storage,
`/issueImages/${firebaseImageId}`
);
So far, I have attempted to use the 'put' function with the imageRef, and when I try using uploadBytes() of firebase, I have to upload it as a Buffer, and even then I cannot seem to find the access token in the metadata.
To upload a data URL to Firebase, you would use storageRef.putString(url, 'DATA_URL') (legacy) or uploadString(storageRef, url, 'DATA_URL') (modern) depending on the SDK you are using.
When you upload a file to a Cloud Storage bucket, it will not be issued an access token until a client calls its version of getDownloadURL(). So to fix your issue, you would call getDownloadURL() immediately after upload.
If Node is running on a client's machine, you would use:
// legacy syntax
import * as firebase from "firebase";
// reference to file
const imageStorageRef = firebase.storage()
.ref(`/issueImages/${firebaseImageId}`);
// perform the upload
await imageStorageRef.putString(dataUrl, 'DATA_URL');
// get the download URL
const imageStorageDownloadURL = await imageStorageRef.getDownloadURL();
// modern syntax
import { getStorage, getDownloadURL, ref, uploadString } from "firebase/storage";
// reference to file
const imageStorageRef = ref(
getStorage(),
`/issueImages/${firebaseImageId}`
);
// perform the upload
await uploadString(imageStorageRef, dataUrl, 'DATA_URL');
// get the download URL
const imageStorageDownloadURL = await getDownloadURL(imageStorageRef);
If Node is running on a private server you control, you should opt to use the Firebase Admin SDK instead as it bypasses the rate limits and restrictions applied to the client SDKs.
As mentioned before, the download URLs aren't created automatically. Unfortunately for us, getDownloadURL is a feature of the client SDKs and the Admin SDK doesn't have it. So we can either let a client call getDownloadURL when it is needed or we can manually create the download URL if we want to insert it into a database.
Nico has an excellent write up on how Firebase Storage URLs work, where they collated information from the Firebase Extensions GitHub and this StackOverflow thread. In summary, to create (or recreate) a download URL once it has been uploaded, you can use the following function:
import { uuid } from "uuidv4";
// Original Credit: Nico (#nicomqh)
// https://www.sentinelstand.com/article/guide-to-firebase-storage-download-urls-tokens
// "file" is an instance of the File class from the Cloud Storage SDK
// executing this function more than once will revoke all previous tokens
function createDownloadURL(file) {
const downloadToken = uuid();
await file.setMetadata({
metadata: {
firebaseStorageDownloadTokens: downloadToken
}
});
return `https://firebasestorage.googleapis.com/v0/b/${file.bucket.name}/o/${encodeURIComponent(file.name)}?alt=media&token=${downloadToken}`;
}
The allows us to change the client-side code above into the following so it can run using the Admin SDK:
// assuming firebase-admin is initialized already
import { getStorage } from "firebase-admin/storage";
// reference to file
const imageStorageFile = getStorage()
.bucket()
.file(`/issueImages/${firebaseImageId}`);
// perform the upload
await imageStorageFile.save(dataUrl);
// get the download URL
const imageStorageDownloadURL = await createDownloadURL(imageStorageFile);
In all of the above examples, a download URL is retrieved and saved to the imageStorageDownloadURL variable. You should store this value as-is in your database. However, if you instead want to store only the access token and reassemble the URL on an as-needed basis, you can extract the token from its ?token= parameter using:
const downloadToken = new URL(imageStorageDownloadURL).searchParams.get('token');
I am trying to create a new table in BigQuery. I have followed these instructions https://codelabs.developers.google.com/codelabs/cloud-bigquery-nodejs/index.html?index=..%2F..index#9 and have my user and roles defined properly.
I created a node project, installed the google dependencies and have the following code:
const {BigQuery} = require('#google-cloud/bigquery');
const bigquery = new BigQuery({
projectId: 'myproject-develop-3fcb6',
private_key_id: "11111111111",
client_email: "myuser-bigquery-sa#myproject-develop-3fcb6.iam.gserviceaccount.com",
client_id: "212111112",
});
This is how im creating my dataset and table:
module.exports = {
createTable: ({ datasetId, tableId, schema, partitionBy}) => {
const options = { schema };
if (partitionBy) {
options.timePartitioning = {
field: partitionBy
};
}
return new Promise((resolve, reject) => {
resolve();
bigquery
.dataset(datasetId)
.createTable(tableId, options)
.then(results => resolve(results[0]))
.catch(err => {
handleError(err);
reject(err);
});
});
},
};
When i run my createTableFunction and pass in the data set name, table name, schema I get the following error immediately
ERROR: Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information.
How do I pass my default credentials to BigQuery so i can perform CRUD operations in node.js? Thanks
In the tutorial that you mentioned, this gcloud command creates a key.json:
gcloud iam service-accounts keys create ~/key.json --iam-account my-bigquery-sa#${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
Then you can use the following code:
// Create a BigQuery client explicitly using service account credentials.
// by specifying the private key file.
const {BigQuery} = require('#google-cloud/bigquery');
const options = {
keyFilename: 'path/to/key.json',
projectId: 'my_project',
};
const bigquery = new BigQuery(options);
Authenticating With a Service Account Key File
I do not know where are you running your code, but in the tutorial is a line where you set the env variable therefore you do not need to authenticate using the key.json file in your code:
export GOOGLE_APPLICATION_CREDENTIALS="/home/${USER}/key.json"
GCP client libraries use a strategy called Application Default
Credentials (ADC) to find your application's credentials. When your
code uses a client library, the strategy checks for your credentials
in the following order:
First, ADC checks to see if the environment variable
GOOGLE_APPLICATION_CREDENTIALS is set. If the variable is set, ADC
uses the service account file that the variable points to. The next
section describes how to set the environment variable.
If the environment variable isn't set, ADC uses the default service
account that Compute Engine, Kubernetes Engine, Cloud Run, App Engine,
and Cloud Functions provide, for applications that run on those
services.
If ADC can't use either of the above credentials, an error occurs.
You can also pass credentials directly as parameters.
const {BigQuery} = require('#google-cloud/bigquery');
const bigQuery = new BigQuery({
projectId: "your-prject-id",
credentials: {...}, // content of json file
});
Thanks to #MahendraPatel comment.
I have a Google Cloud Function that works well and I want, once executed before the callback, to connect to Firestore to add a document to my Notifications collection.
const Firestore = require('#google-cloud/firestore');
const firestore = new Firestore({
projectId: 'my-firebase-project',
keyFilename: 'thekey.json',
});
var fsDocument = {
'username': 'theuser',
'organization': 'theorg',
'attr': [
{k:'key1', v:'val1'},
{k:'key2', v:'val2'}
]
};
firestore.collection('Notifications').add(fsDocument).then(documentReference => {
console.log('Added document with name' + documentReference.id);
});
How can I include the key file to my google cloud function? So far, I am creating them in console.cloud.google.com.
All files in your functions directory will be sent to Cloud Functions when you deploy. You could put your credentials in a file under functions, then refer to it with a relative path like this:
const firestore = new Firestore({
projectId: 'my-firebase-project',
keyFilename: './thekey.json',
});
You should only do this if your credentials are for a project different from the one running your Cloud Function. If you're trying to access Firestore in the same project as the one running your function, just use the default credentials using the Admin SDK. There are lots of examples of this in functions-samples.
There is no node.js Firebase Storage client at the moment (too bad...), so I'm turning to gcloud-node with the parameters found in Firebase's console.
I'm trying :
var firebase = require('firebase');
var gcloud = require('gcloud')({
keyFilename: process.env.FB_JSON_PATH,
projectId: process.env.FB_PROJECT_ID
});
firebase.initializeApp({
serviceAccount: process.env.FB_JSON_PATH,
databaseURL: process.env.FB_DATABASE_URL
});
var fb = firebase.database().ref();
var gcs = gcloud.storage();
var bucket = gcs.bucket(process.env.FB_PROJECT_ID);
bucket.exists(function(err, exists) {
console.log('err', err);
console.log('exists', exists);
});
Where :
FB_JSON_PATH is the path to the JSON file generated in order to use the Firebase Server SDK
FB_DATABASE_URL is something like https://app-a36e5.firebaseio.com/
FB_PROJECT_ID is the name of the firebase project in Google's console : "app-a36e5"
The id of the bucket is FB_PROJECT_ID (in Firebase's console the storage tab displays gs://app-a36e5.appspot.com)
When I run this code I get :
err null
exists false
But no other errors.
I'm expecting exists true at least.
Some additional info : I can query the database (so I imagine the JSON file is correct), and I have set the storage rules as follow :
service firebase.storage {
match /b/app-a36e5.appspot.com/o {
match /{allPaths=**} {
allow read: if true;
allow write: if request.auth != null;
}
}
}
So that everything on the storage is readable.
Any ideas how to get it to work ? Thank you.
The issue here is that you aren't naming your storage bucket correctly. The bucket initialization should be:
var bucket = gcs.bucket('app-a36e5.appspot.com'); // full name of the bucket includes the .appspot.com
I would assume that process.env.FB_PROJECT_ID is just the your-bucket part, and you'd need to get the full bucket name, not just the project id (though the bucket name may be process.env.FB_PROJECT_ID + '.appspot.com').
Also, sorry about not providing Storage integrated with Firebase--GCS has a high quality library that you've already found (gcloud-node), and we figured that this provides the best story for developers (Firebase for mobile, Google Cloud Platform for server side development), and didn't want to muddy the waters further.