Delete a file from firebase storage using download url with Cloud Functions - node.js

I have a collection of profiles in my Firestore db and a field named "profilePicture" with a downloadUrl as the value.
Im using cloud functions and been trying for a long time to figure out how to delete the profilePicture when the profile is deleted.
I know how to create a trigger when the profile is deleted and get the profile picture downloadUrl, but how do I delete the file from storage with only the downloadUrl?

The firebase storage documentation provides a method refFromURL(url) that can be used on a Storage instance. It states the url argument can be:
A URL in the form:
1) a gs:// URL, for example gs://bucket/files/image.png
2) a download URL taken from object metadata.
Based (2) above, it seems like an HTTP URL should also work. However it probably is better practise to store a path string, as the tokens on the HTTP URLs can get rotated by Firebase.

In Angular I use this to delete file from Cloud Firestore by downloadURL
constructor(private storage: AngularFireStorage) {}
onDeleteAttachment(downloadURL: string) {
this.storage.storage.refFromURL(downloadURL).delete();
}

My understanding is that the node SDK for Cloud Storage can't convert HTTP URLs into file paths within a storage bucket. Instead, you should be storing the file path along with the download URL in document. This will make it possible for to build a File object that can be used to delete the image when it's time to do so.

for admin.storage.Storage is no build in method for get reference from url for storage
but you can extract file path from URL ,by remove baseUrl and do some code replace on URL
im create method for this task to accept url from storage project and return path
function getPathStorageFromUrl(url:String){
const baseUrl = "https://firebasestorage.googleapis.com/v0/b/project-80505.appspot.com/o/";
let imagePath:string = url.replace(baseUrl,"");
const indexOfEndPath = imagePath.indexOf("?");
imagePath = imagePath.substring(0,indexOfEndPath);
imagePath = imagePath.replace("%2F","/");
return imagePath;
}
NOTE : You must replace baseUrl for every project, you can find it by open any image in you storage , and copy it from URL in browser from start to end of last slash '/'
Ex :
Some image link on my storage :
https://firebasestorage.googleapis.com/v0/b/project-80505.appspot.com/o/RequestsScreenshot%2F-M6CA-2bG2aP_WwOF-dR__1i5056O335?alt=media&token=d000fab7
the base URL will be
https://firebasestorage.googleapis.com/v0/b/project-80505.appspot.com/o/
now after get path call file to delete it from storage
const storage = admin.storage();
const imagePath:string = getPathStorageFromUrl(obj.imageUrl);
storage.bucket().file(imagePath).delete().catch((err) => console.error(err));
NOTE : There is no documentation explaining the format of the URL,
which implies that the Firebase team might feel the need to change it
some day , mean maybe will not work in the future if format is change.

confing.js
import firebase from 'firebase/app'
import "firebase/firestore";
import "firebase/storage";
const firebaseConfig = {
apiKey: "XXXX",
authDomain: "XXXXX.firebaseapp.com",
databaseURL: "https://XXXX-app-web.firebaseio.com",
projectId: "XXXX",
storageBucket: "XXXX-app-web.appspot.com",
messagingSenderId: "XXXXXX",
appId: "1:XXX:web:XXXX",
measurementId: "G-XXXX"
};
firebase.initializeApp(firebaseConfig);
export const firestore = firebase.firestore();
export const storageRef = firebase.storage();
export default firebase;
Button.js
import React from 'react';
import {firestore,storageRef} from './Config';
function removeFile(id,downloadUrl) {
const storageRefa = storageRef.refFromURL(downloadUrl);
storageRefa.delete().then(() => {
firestore.collection("All_Files").doc(id).delete().then((response) => {
console.log('delete response', response)
}).catch((error) => {
console.log('delete error', error)
})
}).catch((error) => {
console.log('delete error', error)
});
}
export default function MediaCard(props) {
return (
<>
<Button
onClick={() =>{
removeFile(props.ID,props.downloadUrl)
}}
variant="contained"
color="secondary"
>
Delete
</Button>
</>
);
}

Mahmoud's answer need a little edit .. it works tho .. He is doing the replacements wrongly and might not work if you have nested directories or spaced filenames in your storage
getPathStorageFromUrl(url:String){
const baseUrl = "https://firebasestorage.googleapis.com/v0/b/project-80505.appspot.com/o/";
let imagePath:string = url.replace(baseUrl,"");
const indexOfEndPath = imagePath.indexOf("?");
imagePath = imagePath.substring(0,indexOfEndPath);
imagePath = imagePath.replace(/%2F/g,"/");
imagePath = imagePath.replace(/%20/g," ");
return imagePath;
}

Related

Node.js Google Cloud Storage Get Multiple Files' Metadata

I have several files on Google Cloud Storage that are named as 0.jpg, 1.jpg, 2.jpg, etc. I want to get the metadata for each file without setting file names separately. Then, want to send these metadata information to React application. In React application, when one clicks on an image, the popup displays the metadata information for this clicked image.
For only one file, I used the following code:
const express = require("express");
const cors = require("cors");
// Imports the Google Cloud client library
const { Storage } = require("#google-cloud/storage");
const bucketName = "bitirme_1";
const filename = "detected/0.jpg";
const storage = new Storage();
const app = express();
app.get("/api/metadata", cors(), async (req, res, next) => {
try {
// Gets the metadata for the file
const [metadata] = await storage
.bucket(bucketName)
.file(filename)
.getMetadata();
const metadatas = [
{id: 0, name: `Date: ${metadata.updated.substring(0,10)}, Time: ${metadata.updated.substring(11,19)}`},
{id: 1, name: metadata.contentType}
];
res.json(metadatas);
} catch (e) {
next(e);
}
});
const port = 5000;
app.listen(port, () => console.log(`Server started on port ${port}`));
I first set the bucket name. Then, set filename array as (89 is the number of files):
const filename = Array(89).fill(1).map((_, i) => ('detected/' + i + '.jpg'));
These files are in detected folder. When I try this, it gives me this error:
Error: No such object: bitirme_1/detected/0.jpg, detected/1.jpg, detected/2.jpg, detected/3.jpg, detected/4.jpg,detected/5.jpg,detected/6.jpg, ....
How can I solve the getting multiple files' metadata issue?
Also, I want to get the number of files in a bucket (or, in the detected folder). I searched the API but cannot found anything. I do not want to enter the total number of files as 89, want to get it from the API.
I found the solution for finding the number of files in a bucket or a folder in a bucket. This is the solution:
const [files] = await storage.bucket(bucketName).getFiles();
const fileStrings = files.map(file => file.name);
const fileSliced = fileStrings.map(el => el.slice(9, 11));
for (i = 0; i < fileSliced.length; i++) {
if (fileSliced[i].includes('.')) {
fileSliced[i] = fileSliced[i].slice(0, 1);
}
}
const fileNumbers = fileSliced.map(function(item) {
return parseInt(item, 10);
});
const numOfFiles = Math.max(...fileNumbers) + 1;
console.log(numOfFiles);
First, I got the all files with file names in a string array. In my case, file names are detected/0.jpg, detected/1.jpg, detected/2.jpg, etc. I just only want to the number part of the file name; hence, I sliced the string array starting from 9th index up to 11th index(not included). As a result, I got only the numbers except one digit numbers.
To handle one digit case, which have '.' at the end of the sliced name, I also removed '.' from these one digit file names.
As a result, I got ['0', '1', '2', '3', ...]. Next, I convert this string array to number array using parseInt function. Finally, to get the number of files, I got the maximum of the array and add 1 to this number.
I have an image detail component that includes location of sender ip, download option and exiting the popup page. This detail popup page opens at /#i where i is the name of image file, such as 1.jpg, 2.jpg. So, for example when I click the first image, the popup page opens at /#1. In this popup page, I want to get metadata information for the opened image. But, I could not find a solution for this.

Dynamic file names in react native require()

We're working on an app that will allow users to store templates with images on them, and pull those templates up later. This is for an AR environment using Viro on React Native.
We're trying to dynamically load an image into the component, and receiving errors when we require the filepath, which has been set to a variable:
const exampleUri = '/some/uri'
render() {
return(
<Viro3DObject
source={require(exampleUri)}
/>)
}
The URI for the source prop has to be dynamic, as the URIs are pulled from a Database.
We've tried storing the entire request in the database (in models/element.js):
const Sequelize = require('sequelize');
const db = require('../db');
const Element = db.define('element', {
sourceViro3DObject: {
type: Sequelize.STRING
}
});
sourceViro3DObject: `require('../../assets/emoji_heart/emoji_heart.vrx')`
When we called it in the React Native class component:
getObjectData = async () => {
try {
const {data} = await axios.get(`/api/elements/${this.props.elementId}`)
this.setState({sourceViro3DObject: data.sourceViro3DObject})
} catch (err) {
console.log(err)
}
}
async componentDidMount() {
await this.getObjectData()
}
But this simply sets state.sourceViro3DObject to a string:
'require('../../assets/emoji_heart/emoji_heart.vrx')'
We've tried setting the filepath directly to state as a string:
state.sourceViro3DObject = '../../assets/emoji_heart/emoji_heart.vrx'
and then call require on it:
require(this.state.sourceViro3DObject)
and received the following error:
Invalid prop `source` supplied to `Viro3DObject`
We've seen recommendations of storing the URIs in an object, but that can't work for us as we don't know what image is going to be used until it's pulled from the database. We can't hard-code them anywhere.
We'd appreciate any help with this!

Firebase Storage-How to delete file from storage with node.js?

I want to delete a folder in firebase storage with node js because this is a firebase function.
For example :
storageRef.child(child1).child(child2).delete();
something like this, but firebase documentation doesn't tell anything.
One more question:
When initialize storage documentation node js requires my admin json, but realtime database doesn't want this wonder why?
Have a look at the Node.js client API Reference for Google Cloud Storage and in particular at the delete() method for a File.
You can do it like this using Node.js:
const firebase = require('firebase-admin');
async function deleteImageFromFirebase(imageName) {
await firebase.storage().bucket().file("folderName/"+imageName).delete();
}
And like this client side:
// Create a reference to the file to delete
var desertRef = storageRef.child('images/desert.jpg');
// Delete the file
desertRef.delete().then(function() {
// File deleted successfully
}).catch(function(error) {
// Uh-oh, an error occurred!
});
View this info on the Firebase website:
how to delete files Firebase-storage
This might be late but at least on Web (so basically what you need), there is new API to delete the whole folder.
I tested deleting a folder with 2 pictures inside and it works. I then tried a folder-A with contents: folder-B + picture-A. Folder-B also has a picture-B inside; it still deleted folder-A with all of its contents.
Solution:
const bucket = admin.storage().bucket();
return bucket.deleteFiles({
prefix: `posts/${postId}`
);
I couldn't find this on the official documentation (perhaps is really new API) but really cool article where I found the solution:
Automatically delete your Firebase Storage Files from Firestore with Cloud Functions for Firebase
import { storage } from "./firebaseClient";
import { bucket } from "./firebaseServer";
//Let's assume this is the URL of the image we want to delete
const downloadUrl = "https://storage.googleapis.com/storage/v1/b/<projectID>.appspot.com/o/<location>?"
//firebase delete function
const deleteImages = async ({ downloadUrl }) => {
const httpsRef = storage.refFromURL(downloadUrl).fullPath;
return await bucket
.file(httpsRef)
.delete()
.then(() => "success")
.catch(() => "error")
}
//call the deleteImages inside async function
const deleteStatus = await deleteImages({ downloadUrl: oldImage });
console.log(deleteStatus) //=> "success"

How to create a dynamic sitemap with mongodb, node.js, express and EJS?

I'm trying to create a dynamic sitemap for my website, it has lots of pages that change often.
The sitemap needs to be accessed from www.mywebsite.com/sitemap.xml
My current attempt queries the database for all the pages, get's each pages url and passes it to an EJS template that creates what looks like XML.
I have two problems here
The route to the page cannot have a file suffix. e.g. '.xml'
The page is automatically treated as html
I realise that there are other options for creating a sitemap using modules like "express-sitemap," but I haven't been able to find any easily understood (i am new to this) documentation for them, and this seems like a good way of doing things to me
yes you can use express-sitemap
To generate Sitemap automatically
var sitemap = require('express-sitemap')();
var app = require('express')();
sitemap.generate(app);
To generate dynamically..
for suppose you have products pages and you have specified url for them..
You can create a dynamic file everytime and place it in your public folder.
const Product = require('./model/product')
const sitemap = require('sitemap');
let sitemapData;
const generateSitemap = async () => {
const products = await Product.find({},{path: 1});
const urls = products.map({path} => `/products/${path}`)
sitemapData = sitemap.createSitemap ({
hostname: 'http://example.com',
cacheTime: 600000, // 600 sec - cache purge period
urls
});
}
You can use this function in a routine or with a cron and generate sitemap regularly..
setInterval(generateSitemap, 360000); //running every hour
other thing that you can do is:
use sitemapData variable and do things like this.
sitemapData.add({url: '/product-a/'}); // when some product is added
sitemapData.add({url: '/product-b/', changefreq: 'monthly', priority: 0.7});
sitemapData.del({url: '/product-c/'}); // when something is removed
sitemapData.del('/product-d/');
you can serve it in a route like this:
app.get('/sitemap.xml', function(req, res) {
sitemapData.toXML( function (err, xml) {
if (err) {
return res.status(500).end();
}
res.header('Content-Type', 'application/xml');
res.send( xml );
});
});
Here is how I made a txt sitemap. I find Google Search Console has an easier time fetching txt sitemaps vs xml sitemaps. But if you want to make a xml sitemap, you can look at this blog for the correct formatting. This code uses Mongoose and is saved as /pages/sitemap.txt.js.
// pages/sitemap.txt.js
import dbConnect from "../utils/dbConnect";
import Pduct from "../models/Pduct";
const createSitemap = (posts) => `${posts
.map(({ slug }) => {
return `https://[YOUR DOMAIN]/${slug}`;
})
.join("\n")}
`;
export async function getServerSideProps({ res }) {
await dbConnect();
const request = await Pduct.find({}, "slug").lean();
res.setHeader("Content-Type", "text");
res.write(createSitemap(request));
res.end();
}
export default () => null;

Retrieving a Firebase storage image link via Cloud Function

How do I retrieve the download links to stored images in Firebase via a cloud function?
I've tried all kinds of variations including the next one:
exports.getImgs = functions.https.onRequest((req, res) => {
var storage = require('#google-cloud/storage')();
var storageRef = storage.ref;
console.log(storageRef);
storageRef.child('users/user1/avatar.jpg').getDownloadURL().then(function(url) {
});
});
It annoyed me, so I will put the solution with a straight forward explanation to those who are looking for it.
1st, install GCD Storage using the firebase command line:
npm install --save #google-cloud/storage
Cloud function code:
const gcs = require('#google-cloud/storage')({keyFilename: 'service-account.json'});
const bucket = gcs.bucket('name-of-bucket.appspot.com');
const file = bucket.file('users/user1/avatar.jpg');
return file.getSignedUrl({
action: 'read',
expires: '03-09-2491'
}).then(signedUrls => {
console.log('signed URL', signedUrls[0]); // this will contain the picture's url
});
The name of your bucket can be found in the Firebase console under the Storage section.
The 'service-account.json' file can be created and downloaded from here:
https://console.firebase.google.com/project/_/settings/serviceaccounts/adminsdk
And should be stored locally in your Firebase folder under the functions folder. (or other as long as change the path in the code above)
That's it.

Resources