I am building an App and my objective is every time someone upload an image to firebase storage, the cloud function resize that image.
...
import * as Storage from '#google-cloud/storage'
const gcs = new Storage()
...
exports.resizeImage = functions.storage.object().onFinalize( async object => {
const bucket = gcs.bucket(object.bucket);
const filePath = object.name;
const fileName = filePath.split('/').pop();
const bucketDir = dirname(filePath);
....
And when I tried to deploy this funtion I get this error:
Error: Error occurred while parsing your function triggers.
TypeError: Storage is not a constructor
I tried with "new Storage()" or just "Storage" and nothing works.
I'm newbie around here so if there's anything I forgot for you to debug this just let me know.
Thanks!
google-cloud/storage: 2.0.0
Node js: v8.11.4
The API documentation for Cloud Storage suggests that you should use require to load the module:
const Storage = require('#google-cloud/storage');
This applied to versions of Cloud Storage prior to version 2.x.
In 2.x, there was a breaking API change. You need to do this now:
const { Storage } = require('#google-cloud/storage');
If you would like TypeScript bindings, consider using Cloud Storage via the Firebase Admin SDK. The Admin SDK simply wraps the Cloud Storage module, and also exports type bindings to go along with it. It's easy to use:
import * as admin from 'firebase-admin'
admin.initializeApp()
admin.storage().bucket(...)
admin.storage() gives you a reference to the Storage object that you're trying to work with.
On node 14, using commonJS with babel (although I don't think Babel was interfering here), this is how I eventually got it working on an older project bumping GCS from 1.x to 5.x:
const Storage = require('#google-cloud/storage');
const gcs = new Storage({project_id});
didn't see this captured anywhere on the web
Just for knowledge purpose if you guys are using the ES module approach and node 12, the below snippet would work. I couldn't make none of other syntax working~
import storagePackage from '#google-cloud/storage';
const { Storage } = storagePackage;
const storage = new Storage();
If you are using electron and still don't work above solutions,try this.
const {Storage} = window.require('#google-cloud/storage');
const storage = new Storage({ keyFilename: "./uploader-credentials.json" });
Related
I'm trying to upload some images to a firebase storage bucket,
and trying to run the code locally in the firebase emulator.
With most examples I'll get an error like:
"Unhandled error FirebaseError: Firebase: Need to provide options, when not being deployed to hosting via source.
(app/no-options). at initializeApp
I tried this: https://firebase.google.com/docs/admin/setup
const defaultApp = initializeApp({ credential: applicationDefault() })
Which may or not be working.
I think this creates an authed app (maybe?) but if I later use
const storage = getStorage() I get the same auth error.
So next, I tried using the defaultApp for next steps, like:
const storage = defaultApp.storage();
that doesn't error, but it returns something different from getStorage, that I can't seem to use for other needed things like getting a ref
as per examples https://firebase.google.com/docs/storage/web/upload-files
So it's not clear how to upload files to firebase from the local simulator.
my full function looks like this:
import cuid from "cuid";
import {
getStorage,
ref,
uploadBytes,
uploadBytesResumable,
getDownloadURL
} from "firebase/storage";
import { admin, defaultApp } from '../utils/firebaseInit'
export function saveToBucket(url: string): string {
const normalStorage = getStorage() // Gets a FirebaseStorage instance for the given Firebase app. but NOT AUTHED
const defaultStorage = defaultApp.storage(); // Gets the default FirebaseStorage instance.
const storageRef = ref(normalStorage)
const bucketPath = `personas-out/img-${cuid()}.jpg`
const bucket = defaultStorage.bucket();
// const bucketRef = bucket.ref(); // Property 'ref' does not exist on type 'Bucket'.ts(2339)
const metadata = {
contentType: 'image/jpeg'
};
const imageRef = ref(storageRef, bucketPath);
const file = new File([url], bucketPath, {
type: "image/jpeg",
});
// will happen in the background?
uploadBytesResumable(imageRef, file, metadata)
.then((snapshot: any) => {
console.log('Uploaded', snapshot);
});
return bucketPath
}
You're not supposed to use the Firebase client SDK (firebase/storage) in nodejs code. The client SDKs are meant to work in a browser only. For nodejs code that runs on a backend (which is what you're doing when you deploy to Cloud Functions), you should use the Firebase Admin SDK. The API it exposes for Storage is essentially a thin wrapper around the Google Cloud Storage nodejs library.
I have file which was stored in some Azure blob directory "folder1/folder2/file.txt". This file was soft deleted - I can see it in Azure web console. I need to have function which checks this file existence.
I tried library "azure-storage". It perfectly works with NOT removed files:
const blobService = azure.createBlobService(connectingString);
blobService.doesBlobExist(container, blobPath, callback)
May be anyone knows how use same approach with soft removed files?
I tied with lib "#azure/storage-blob".
But I stuck with endless entities there (BlobServiceClient, ContainerItem, BlobClient, ContainerClient, etc) and couldn't find way to see particular file in particular blob directory.
Following this MSDOC, I got to restore the Soft deleted blobs and their names with the below code snippet.
const { BlobServiceClient } = require('#azure/storage-blob');
const connstring = "DefaultEndpointsProtocol=https;AccountName=kvpstorageaccount;AccountKey=<Storage_Account_Key>;EndpointSuffix=core.windows.net"
if (!connstring) throw Error('Azure Storage Connection string not found');
const blobServiceClient = BlobServiceClient.fromConnectionString(connstring);
async function main(){
const containerName = 'kpjohncontainer';
const blobName = 'TextFile05.txt';
const containerClient = blobServiceClient.getContainerClient(containerName);
undeleteBlob(containerClient, blobName)
}
main()
.then(() => console.log(`done`))
.catch((ex) => console.log(ex.message));
async function undeleteBlob(containerClient, blobName){
const blockBlobClient = await containerClient.getBlockBlobClient(blobName);
await blockBlobClient.undelete(); //to restore the deleted blob
console.log(`undeleted blob ${blobName}`);
}
Output:
To check if the blob exists and if exists but in Soft-deleted state, I found the relevant code but it’s in C# provided by #Gaurav Mantri. To achieve the same in NodeJS refer here.
I need to delete images at eventPoster folder in my cloud storage that has last modified metadata more than 6 months ago.
so I want to make a cron job using Cloud Function, and I need to 'query' that files that has last modified more than 6 months ago
how to do that using Admin SDK?
I think I find the solution
according to this answer, There is no way to query by metadata
so, as for now I use the code below to get out of date files
import * as admin from "firebase-admin";
import * as moment from "moment";
const app = admin.initializeApp(); // I am using Google Cloud Function, much more simple to initialize the app
const bucket = app.storage().bucket();
const response = await bucket.getFiles({prefix: "eventPoster/"}); // set your folder name in prefix here
const files = response[0];
const sixMonthAgo = moment().subtract(6, "months").toDate();
const outOfDateFiles = files.filter((file) => {
const createdTime = new Date(file.metadata.timeCreated);
return moment(createdTime).isBefore(sixMonthAgo);
});
console.log(`number of outdated files: ${outOfDateFiles.length}`);
read the documentation in here for further reading
I need to access a file stored on Google Cloud Storage, this is not a file that is uploaded, instead it is created by a cloud function. I cannot figure out how to access this file using Node.
I have tried many of the things recommended on Stackoverflow.
google-cloud TypeError: gcs.bucket is not a function
Python GAE - How to check if file exists in Google cloud storage
How to read content of JSON file uploaded to google cloud storage using node js
Similarly, I have checked out the sample Google project but that only uses read streams and because I am not uploading the file, I couldn't figure out how to use that.
The closest I have gotten is modifing the first link to get this
var {Storage} = require('#google-cloud/storage');
var gcs = new Storage({
keyFilename: path.join("_file.json"),
projectId: 'ProjectId'
});
const chosenBucket = gcs.bucket("bucket_name");
var file = chosenBucket('file.json');
This causes a TypeError:
TypeError: bucket is not a function
How can I access and read the json located within the file?
const chosenBucket = gcs.bucket("bucket_name");
var file = chosenBucket('file.json');
chosenBucket is not a function. It's a Bucket object. Do something like this:
const chosenBucket = gcs.bucket("bucket_name");
var file = chosenBucket.file('file.json');
const download_options = {
// The path to which the file should be downloaded, e.g. "./file.txt"
destination: destFilename,
};
await file.download(download_options)
See an example: https://github.com/googleapis/nodejs-storage/blob/master/samples/files.js
I am running a Firebase Functions instance like so:
import * as functions from 'firebase-functions'
import * as express from 'express'
import * as admin from 'firebase-admin'
import { MyApi } from './server'
admin.initializeApp(functions.config().firebase)
const firebaseDb: admin.database.Database = admin.database()
const app: express.Application = MyApi.bootstrap(firebaseDb).app
export const myApp = functions.https.onRequest(app)
Everything works fine with my Realtime Database, but I am having trouble integrating Storage properly. According to the docs, I need to set it up like this:
var config = {
apiKey: '<your-api-key>',
authDomain: '<your-auth-domain>',
databaseURL: '<your-database-url>',
storageBucket: '<your-storage-bucket>'
};
firebase.initializeApp(config);
// Get a reference to the storage service, which is used to create references in your storage bucket
var storage = firebase.storage();
var storageRef = storage.ref();
However this doesn't work for me since I'm using the Admin SDK. I'm not importing the firebase library at all. I tried accessing Storage like this:
const fbStorage = admin.storage()
This feels wrong. The interface for the Admin storage method is completely different from the Firebase client SDK's:
node_modules/firebase-admin/lib/index.d.ts
declare namespace admin.storage {
interface Storage {
app: admin.app.App;
bucket(name?: string): Bucket;
}
}
node_modules/firebase/index.d.ts
declare namespace firebase.storage {
interface Storage {
app: firebase.app.App;
maxOperationRetryTime: number;
maxUploadRetryTime: number;
ref(path?: string): firebase.storage.Reference;
refFromURL(url: string): firebase.storage.Reference;
setMaxOperationRetryTime(time: number): any;
setMaxUploadRetryTime(time: number): any;
}
Significantly, the admin Storage interface is missing the ref() and ref().put() methods, so none of the documentation dealing with uploading files applies. I can access my files through admin.storage().bucket().file('path/to/file.jpg'), but that seems rather roundabout and I'm not sure I'm supposed to be doing that.
As a workaround, I tried to initialize a non-admin Firebase app (firebase.initializeApp(config)) on top of admin.initializeApp(). But when I try to launch Functions that gives the fatal error database: Firebase: Firebase service named 'database' already registered (app/duplicate-service).
Right now I made a separate app, and am trying to delegate Storage functionality to that secondary app. Is there a better way?
Thanks.
UDPATE (ANSWER):
Thanks to Renaud's advice, I learned that my initial attempt was actually the correct one. Turns out you ARE supposed to access your Storage instance through admin.storage(). The interface definitions do differ, because the client-side SDK and admin (server-side) SDK are catering to different needs. If you want to include the type definitions, you need to import them from '#google-cloud/storage'.
Below is a sample of how to use the Bucket API based on the official docs:
import { Bucket } from '#google-cloud/storage'
const filename = 'path/to/myfile.mp3'
const bucket: Bucket = admin.storage().bucket()
bucket
.upload(filename, {
destination: 'audio/music/myfile.mp3',
})
.then(() => {
console.log(`${filename} uploaded to ${bucket.name}.`)
})
.catch(err => {
console.error('ERROR:', err)
})
(I hope I understood correctly your question) Here is an example on how to access the Cloud Storage from a Cloud Function in order to stream a file:
const config = {
projectId: '.....',
keyFilename: './......json'
};
const storage = require('#google-cloud/storage')(config);
const yourStorageBucket = 'xyz.appspot.com';
const bucket = storage.bucket(yourStorageBucket);
const file = bucket.file(filename);
const stream = file.createWriteStream({resumable: false});
....
You can have a look for more details at:
https://cloud.google.com/nodejs/getting-started/using-cloud-storage#uploading_to_cloud_storage
and
https://firebase.google.com/docs/storage/extend-with-functions