Check if image exists at storage with firebase cloud function - node.js

i need your help with one function that i createed to manipulate the images that my users send to my app.
What i need is get the image that the user sent, resize and check if the image was changed to avoid the function to do all again. The examples that i saw change the image name and check if the beggined if the name is equals with the name set, but in my case i need keep the original name of the picture, So, how can i do that? Or exists a better way to solve this problem?
My function code is:
import * as functions from 'firebase-functions';
import * as Storage from '#google-cloud/storage';
const gcs = new Storage();
import { tmpdir } from 'os';
import { join, dirname } from 'path';
import * as sharp from 'sharp';
import * as fs from 'fs-extra';
export const generateThumbs = functions.storage
.object()
.onFinalize(async object => {
const bucket = gcs.bucket(object.bucket);
const filePath = object.name;
const fileName = filePath.split('/').pop();
const bucketDir = dirname(filePath);
const workingDir = join(tmpdir(), 'thumbs');
const tmpFilePath = join(workingDir, 'source.png');
if (!object.contentType.includes('image')) {
console.log('exiting function');
return false;
}
// 1. Ensure thumbnail dir exists
await fs.ensureDir(workingDir);
// 2. Download Source File
await bucket.file(filePath).download({
destination: tmpFilePath
});
// 3. Resize the images and define an array of upload promises
const sizes = [64, 128, 256];
const uploadPromises = sizes.map(async size => {
const thumbName = `thumb#${size}_${fileName}`;
const thumbPath = join(workingDir, thumbName);
// Resize source image
await sharp(tmpFilePath)
.resize(size, size)
.toFile(thumbPath);
// Upload to GCS
return bucket.upload(thumbPath, {
destination: join(bucketDir, thumbName)
});
});
// 4. Run the upload operations
await Promise.all(uploadPromises);
// 5. Cleanup remove the tmp/thumbs from the filesystem
return fs.remove(workingDir);
});

If you need to overwrite the original file, and you want to avoid an infinite loop with this function, you could attach custom metadata to the file when you upload it back to Cloud Storage. Then, when the function is invoked again for that file, you can check the metadata on the incoming ObjectMetadata object to know when the function should bail out without making any more changes.

Related

Error uploading image to firebase storage in React Native [ Firebase 9.6.2]

so i'm trying to upload an image to firebase storage (a local photo from the source file directory - path: same path as .js). The problem is that in firebase storage the image appears to be corrupted and every picture have 9 bytes firebase storage image . The authentication and firestore works perfectly, this is my configuration file: firebase config file and this is the code:
const uploadPhoto = async() => {
// console.log(image);
// const uploadUri = image;
// let filename = uploadUri.substring(uploadUri.lastIndexOf('/') + 1);
const metadata = {
//path: '../../firebase_image.jpg',
contentType: 'image/jpeg'
};
const photog = `./photo.jpg`;
console.log(photog);
console.log(uploadUri);
const storageRef;
storageRef=ref(storage, 'photogra.jpg');//filename+
uploadBytes(storageRef, photog, metadata).then((snapshot) => {
console.log('Uploaded a blob or file!');
});
}
I hope you're fine!
I got the information from this amazing tutorial and worked fine for me
https://www.youtube.com/watch?v=H-yXO46WDak&lc=z22ph5dhssfqufkcxacdp430segloszlmvuqlp1seplw03c010c
Try with this:
const uploadImageFirebase = async () =>{
const nameImage = new Date().toISOString();
const img = await fetch(image);
const bytes = await img.blob();
try {
await uploadBytes(ref_con, bytes);
} catch (error) {
console.log(error);
} finally {
//
}
};
If you check the contents of your 9-byte file, it'll like be "photo.jpg".
Since you're passing "photo.jpg" to uploadBytes, it uploads that strings as the contents of the new file. It does not know how to load the file at that path.
You will either need to pass a local File or Buffer (which you'll usually get from a file picker or something like that), or load the data from the file yourself and pass the contents to uploadBytes.

Firebase function: Error: unable to open for write

so I was trying to implement a firebase function. I went to firebase functions repository example and copied it. Everything is working properly "Deploy complete!" with no signs of an error. However, when I'm trying to upload image to the firebase store, firebase functions can't open it?
There is a code that I used:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp()
const {Storage} = require("#google-cloud/storage");
const gcs = new Storage();
const path = require('path');
const os = require('os');
const fs = require('fs');
const sharp = require("sharp");
exports.generateThumbnail = functions.storage.object().onFinalize(async (object) => {
const fileBucket = object.bucket; // The Storage bucket that contains the file.
const filePath = object.name; // File path in the bucket.
const contentType = object.contentType; // File content type.
const metageneration = object.metageneration; // Number of times metadata has been generated. New objects have a value of 1.
if (!contentType.startsWith('image/')) {
return console.log('This is not an image.');
}
const fileName = path.basename(filePath);
if (fileName.startsWith('thumb_')) {
return console.log('Already a Thumbnail.');
}
const bucket = admin.storage().bucket(fileBucket);
const tempFilePath = path.join(os.tmpdir(), fileName);
console.log('Created temporary path',tempFilePath);
const metadata = {
contentType: contentType,
};
await bucket.file(filePath).download({destination: tempFilePath});
console.log('Image downloaded locally to', tempFilePath);
const thumbFileName = `thumb_${fileName}`;
const thumbFilePath = path.join(path.dirname(filePath), thumbFileName);
console.log('Created thumb path',tempFilePath);
const size = 200;
/*await*/ sharp(tempFilePath).resize(size,size).toFile(thumbFilePath);
await bucket.upload(tempFilePath, {
destination: filePath,
metadata: metadata,
});
return fs.unlinkSync(tempFilePath);
});
Error:
Cloud Functions has a read-only filesystem except for the /tmp directory. You have to make sure you are writing your data to a path /tmp/your-file
The only writeable part of the filesystem is the /tmp directory, which
you can use to store temporary files in a function instance. This is a
local disk mount point known as a "tmpfs" volume in which data written
to the volume is stored in memory. Note that it will consume memory
resources provisioned for the function.
Cloud Functions Execution Environment
I guess that this will work also in Firebase, else please comment:
gcsfs
If you put gcsfs in the "requirements.txt" and import gcsfs in the Python code, you can use the module like this (as an example taken from Have a look at an example for saving a csv:
fs = gcsfs.GCSFileSystem(project=MY_PROJECT)
fs.ls(BUCKET_NAME)
# or choose 'w' here:
with fs.open(filepath, 'wb') as outcsv:
...
Further links:
How to open a file from google cloud storage into a cloud function
https://gcsfs.readthedocs.io/en/latest/index.html#examples

Error when trying to count number of files in a specific directory using event driven Google Cloud Functions Node JS 10

I get the following error when using file.length in a Google Function using Node JS 10:
textPayload: "TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument
must be one of type string, Buffer, or URL. Received type object"
My current code is as follows:
const {Storage} = require('#google-cloud/storage');
const {path} = require('path');
var fs = require('fs');
exports.copyRenders = (event, context) => {
const gcsEvent = event;
const sourcePathOnly = gcsEvent.name
const sourceFolder = sourcePathOnly.split('/').slice(-2)
fs.readdir(sourceFolder, (err, files) => {
console.log(files.length);
//console.log(`Files are: ${sourceFolder}`);
});
}
What am I doing wrong here?
Thanks
Try adding .join("") to make your sourceFolder constant a String, it seems to me you made it an Array with .slice
exports.copyRenders = (event, context) => {
const gcsEvent = event;
const sourcePathOnly = gcsEvent.name
const sourceFolder = sourcePathOnly.split('/').slice(-2)
.join("") // <- here
fs.readdir(sourceFolder, (err, files) => {
//console.log(files.length);
console.log(`Files are: ${sourceFolder}`);
});
}
In Cloud Storage there are no folders here It is explained how directories work, as they are only a longer names for the objects that include directory/subdirectory/objectName.
Therefor the usual ways to get all the elements in a directory doesn't work, here is an example of listing objects on a bucket and the way to list object that share a prefix (that are on the same directory).
Here is how you can get the number of files that share a prefix on a bucket.
async function countFilesByPrefix(bucketName, prefix) {
// [START storage_list_files_with_prefix]
// Imports the Google Cloud client library
const storage = require('#google-cloud/storage')();
const options = {
prefix: prefix,
};
const [files] = await storage.bucket(bucketName).getFiles(options);
console.log(` Number of Files ${ files.length}`);
}

Save an audiofile from Google Text-to-Speech to Firebase Storage using Google Cloud Storage?

We're trying to get an audiofile from Google Text-to-Speech and save it to Firebase Storage, using a Google Cloud Function. The documentation for Google Text-to-Speech show how to get an audiofile and save it locally:
// Performs the Text-to-Speech request
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
const writeFile = util.promisify(fs.writeFile);
await writeFile('output.mp3', response.audioContent, 'binary');
console.log('Audio content written to file: output.mp3');
This results in an error message Error: EROFS: read-only file system. Google Cloud Storage doesn't allow writing files locally.
Using Firebase Storage bucket.upload() has a few problems:
const destinationPath = 'Audio/Spanish' + filename.ogg;
// Performs the Text-to-Speech request
const [response] = await client.synthesizeSpeech(request);
// response.audioContent is the downloaded file
await bucket.upload(response.audioContent, {
destination: destinationPath
));
The error message is TypeError: Path must be a string. The first parameter of bucket.upload() is The fully qualified path to the file you wish to upload to your bucket. and is expected to be a string so response.audioContent doesn't work.
The documentation for bucket.upload() suggests that destination: destinationPath is where we should put the path to the Firebase Storage location. Is this correct?
How do we take the audiofile from Google Text-to-Speech (response.audioContent) and save it as a string that bucket.upload() can access? Or should we use something else instead of bucket.upload()?
Here's our full cloud function:
exports.Google_T2S = functions.firestore.document('Users/{userID}/Spanish/T2S_Request').onUpdate((change, context) => {
if (change.after.data().word != undefined) {
// Performs the Text-to-Speech request
async function test() {
try {
const word = change.after.data().word; // the text
const longLanguage = 'Spanish';
const audioFormat = '.mp3';
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const fs = require('fs');
const util = require('util');
const textToSpeech = require('#google-cloud/text-to-speech'); // Imports the Google Cloud client library
const client = new textToSpeech.TextToSpeechClient(); // Creates a client
let myWordFile = word.replace(/ /g,"_"); // replace spaces with underscores in the file name
myWordFile = myWordFile.toLowerCase(); // convert the file name to lower case
myWordFile = myWordFile + audioFormat; // append .mp3 to the file name;
// copied from https://cloud.google.com/blog/products/gcp/use-google-cloud-client-libraries-to-store-files-save-entities-and-log-data
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('myProject-cd99d.appspot.com');
const destinationPath = 'Audio/Spanish/' + myWordFile;
const request = { // Construct the request
input: {text: word},
// Select the language and SSML Voice Gender (optional)
voice: {languageCode: 'es-ES', ssmlGender: 'FEMALE'},
// Select the type of audio encoding
audioConfig: {audioEncoding: 'MP3'},
};
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
const writeFile = util.promisify(fs.writeFile);
await writeFile('output.mp3', response.audioContent, 'binary');
console.log('Audio content written to file: output.mp3')
// response.audioContent is the downloaded file
await bucket.upload(response.audioContent, {
destination: destinationPath
});
}
catch (error) {
console.error(error);
}
}
test();
} // close if
return 0; // prevents an error message "Function returned undefined, expected Promise or value"
});
file.save() was the answer. util.promisify was unnecessary, and causes an error message about original something. Here's the finished cloud function:
const functions = require('firebase-functions');
// // Create and Deploy Your First Cloud Functions
// // https://firebase.google.com/docs/functions/write-firebase-functions
//
// exports.helloWorld = functions.https.onRequest((request, response) => {
// response.send("Hello from Firebase!");
// });
async function textToSpeechRequest()
{
try
{
const word = change.after.data().word; // the text
const longLanguage = 'Spanish';
const audioFormat = '.mp3';
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const util = require('util');
const textToSpeech = require('#google-cloud/text-to-speech'); // Imports the Google Cloud client library
const client = new textToSpeech.TextToSpeechClient(); // Creates a client
let myWordFile = word.replace(/ /g,"_"); // replace spaces with underscores in the file name
myWordFile = myWordFile.toLowerCase(); // convert the file name to lower case
myWordFile = myWordFile + audioFormat; // append .mp3 to the file name;
// copied from https://cloud.google.com/blog/products/gcp/use-google-cloud-client-libraries-to-store-files-save-entities-and-log-data
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
//const bucket = storage.bucket('myProject-cd99d.appspot.com');
var file = bucket.file('Audio/Spanish/' + myWordFile);
const request = { // Construct the request
input: {text: word},
// Select the language and SSML Voice Gender (optional)
voice: {languageCode: 'es-ES', ssmlGender: 'FEMALE'},
// Select the type of audio encoding
audioConfig: {audioEncoding: 'MP3'},
};
const options = { // construct the file to write
metadata: {
contentType: 'audio/mpeg',
metadata: {
source: 'Google Text-to-Speech'
}
}
};
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
// response.audioContent is the downloaded file
return await file.save(response.audioContent, options)
.then(() => {
console.log("File written to Firebase Storage.")
return;
})
.catch((error) => {
console.error(error);
});
} // close try
catch (error) {
console.error(error);
} // close catch
} // close async function declaration
exports.Google_T2S = functions.firestore.document('Users/{userID}/Spanish/T2S_Request').onUpdate((change, context) => {
if (change.after.data().word !== undefined)
{
textToSpeechRequest();
} // close if
}); // close Google_T2S
We're getting an error TypeError: [ERR_INVALID_ARG_TYPE]: The "original" argument must be of type function at Object.promisify. This error doesn't appear to effect the cloud function.
To reiterate the stuff that didn't work, fs.createWriteStream didn't work because Google Cloud Functions can't handle Node file system commands. Instead, Google Cloud Functions have their own methods that wrap the Node file system commands. bucket.upload() will upload a local file to a bucket, but the path to the local file has to be a string, not a buffer or a stream coming from an API. file.save() is documented as
Write arbitrary data to a file.
This is a convenience method which wraps File#createWriteStream.
That's what I want! If there's one thing about my data, it's arbitrary. Or maybe contrary by nature. After that we just had to straighten out the contentType (audio/mpeg, not mp3) and the file path.

Update Firebase database on Storage change using Firebase Functions

I am trying to implement the Profile Picture feature in my Android app. So I have used Generate Thumbnail sample from firebase. So whenever I am uploading an full sized image, it generates and thumbnail for me. But I want to update the URL of thumbnail in my realtime database once the thumbnail is generated.
const functions = require('firebase-functions');
const mkdirp = require('mkdirp-promise');
const gcs = require('#google-cloud/storage')();
const spawn = require('child-process-promise').spawn;
const LOCAL_TMP_FOLDER = '/tmp/';
// Max height and width of the thumbnail in pixels.
const THUMB_MAX_HEIGHT = 100;
const THUMB_MAX_WIDTH = 100;
// Thumbnail prefix added to file names.
const THUMB_PREFIX = 'thumb_';
/**
* When an image is uploaded in the Storage bucket We generate a thumbnail automatically using
* ImageMagick.
*/
exports.generateThumbnail = functions.storage.object().onChange(event => {
const filePath = event.data.name;
const filePathSplit = filePath.split('/');
const fileName = filePathSplit.pop();
const fileDir = filePathSplit.join('/') + (filePathSplit.length > 0 ? '/' : '');
const thumbFilePath = `${fileDir}${THUMB_PREFIX}${fileName}`;
const tempLocalDir = `${LOCAL_TMP_FOLDER}${fileDir}`;
const tempLocalFile = `${tempLocalDir}${fileName}`;
const tempLocalThumbFile = `${LOCAL_TMP_FOLDER}${thumbFilePath}`;
// Exit if this is triggered on a file that is not an image.
if (!event.data.contentType.startsWith('image/')) {
console.log('This is not an image.');
return;
}
// Exit if the image is already a thumbnail.
if (fileName.startsWith(THUMB_PREFIX)) {
console.log('Already a Thumbnail.');
return;
}
// Exit if this is a move or deletion event.
if (event.data.resourceState === 'not_exists') {
console.log('This is a deletion event.');
return;
}
// Create the temp directory where the storage file will be downloaded.
return mkdirp(tempLocalDir).then(() => {
// Download file from bucket.
const bucket = gcs.bucket(event.data.bucket);
return bucket.file(filePath).download({
destination: tempLocalFile
}).then(() => {
console.log('The file has been downloaded to', tempLocalFile);
// Generate a thumbnail using ImageMagick.
return spawn('convert', [tempLocalFile, '-thumbnail', `${THUMB_MAX_WIDTH}x${THUMB_MAX_HEIGHT}>`, tempLocalThumbFile]).then(() => {
console.log('Thumbnail created at', tempLocalThumbFile);
// Uploading the Thumbnail.
return bucket.upload(tempLocalThumbFile, {
destination: thumbFilePath
}).then(() => {
console.log('Thumbnail uploaded to Storage at', thumbFilePath);
// Don't know what to write here.
});
});
});
});
});
Once the final promise of this code completes, I want to take the downloadable URL from the generated profile pic at /users/{userId}/profilePic
You'll need to add a couple things to achieve what you want here. Firstly you need to require firebase-admin and initialize the app.
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
This should be done at the top of the file after you require the other components.
Once you've got that you can add some code into the final promise to update the database with the new reference.
.then(() => {
console.log('Thumbnail uploaded to Storage at', thumbFilePath);
//userID here would be the userID that you want to update. I couldn't see a reference to it in the code you provided.
var databaseRef = admin.database().ref('users').child(userID).child("profilePic");
databaseRef.transaction(function(oldValue) {
return bucket.file(thumbFilePath).getDownloadURL().then(funct‌​ion(url) {
return url;
}
});
});
You can read about the transaction to update the reference here. I've only included the mandatory transactionUpdate
parameter and not the optional parameters that you may want. https://firebase.google.com/docs/reference/admin/node/admin.database.Reference#transaction

Resources