Let's look at the code below. If I wanted to save a file to /tmp/new_folder should I use node's fs.mkdir() function or can I just give it the path as a string even though the sub-directory does not exist yet?
Also, is it a requirement to use path.join() over concatenating strings to create a the destination path?
// Download file from bucket.
const bucket = gcs.bucket(fileBucket);
const tempFilePath = path.join(os.tmpdir(), fileName);
const metadata = {
contentType: contentType,
};
return bucket.file(filePath).download({
destination: tempFilePath,
})
In the Cloud Functions runtime, /tmp already exists, so there is no need to try to create it before you write a file there. If you want to create a subdirectory under /tmp, you will have to create that on your own (and delete it when your function is done).
Related
Is there a better option for uploading a file?
In my case I need to upload a lot of small files (pdf or text) and I have to decide the best option, but anyway are there differences between these two methods?
Here two examples taken directly from the documentation
Save method: (Docs)
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const myBucket = storage.bucket('my-bucket');
const file = myBucket.file('my-file');
const contents = 'This is the contents of the file.';
file.save(contents, function(err) {
if (!err) {
// File written successfully.
}
});
//-
// If the callback is omitted, we'll return a Promise.
//-
file.save(contents).then(function() {});
Upload method: (Docs)
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('albums');
//-
// Upload a file from a local path.
//-
bucket.upload('/local/path/image.png', function(err, file, apiResponse) {
// Your bucket now contains:
// - "image.png" (with the contents of `/local/path/image.png')
// `file` is an instance of a File object that refers to your new file.
});
At its core, both of these functions do the same thing. They upload a file to a bucket. One is just a function on bucket and the other a function on file. They both end up calling file.createWriteStream, so they have the same performance as well.
The functions behave differently in terms of upload type. file.save will default to a resumable upload unless you specify otherwise (you can set the resumable boolean on SaveOptions to false). bucket.upload will perform a multipart upload if the file is smaller than 5MB and a resumable upload otherwise. For bucket.upload, you can force a resumable or multipart upload by modifying the resumable boolean on UploadOptions.
Note that in the upcoming major version release (https://github.com/googleapis/nodejs-storage/pull/1876), this behavior will be unified. Both functions will default to a resumable upload regardless of file size. The behavior will be the same.
For small files, multipart uploads are recommended.
I am using node js to download azure blob storage files into our local machine. I am able to download it into my project path but not able to download into my local machine. I am using html, Express,and node js. Currently working on localhost only. How to download ?
Below is the code that i am using to download blob file to local folder.
app.get("/downloadImage", function (req, res) {
var fileName = req.query.fileName;
var downloadedImageName = util.format('CopyOf%s', fileName);
blobService.getBlobToLocalFile(containerName, fileName, downloadedImageName, function (error, serverBlob) {
});
});
I am able to to download it to my project folder but i want to download it to my downloads folder. Please help me on this ?
To download a file from Azure blob storage
ConnectionString: Connection string to blob storage
blobContainer: Blob Container Name
sourceFile: Name of file in container i.e sample-file.zip
destinationFilePath: Path to save file i.e ${appRoot}/download/${sourceFile}
const azure = require('azure-storage');
async function downloadFromBlob(
connectionString,
blobContainer,
sourceFile,
destinationFilePath,
) {
logger.info('Downloading file from blob');
const blobService = azure.createBlobService(connectionString);
const blobName = blobContainer;
return new Promise((resolve, reject) => {
blobService.getBlobToLocalFile(blobName, sourceFile, destinationFilePath, (error, serverBlob) => {
if (!error) {
logger.info(`File downloaded successfully. ${destinationFilePath}`);
resolve(serverBlob);
}
logger.info(`An error occured while downloading a file. ${error}`);
reject(serverBlob);
});
});
};
According to the reference of the method blobService.getBlobToLocalFile, as below, the value of parameter localFileName should be the local file path with related dir path.
localFileName string The local path to the file to be downloaded.
So I created a directory named downloadImages and changed your code as below.
var downloadDirPath = 'downloadImages'; // Or the absolute dir path like `D:/downloadImages`
app.get("/downloadImage", function (req, res) {
var fileName = req.query.fileName;
var downloadedImageName = util.format('%s/CopyOf%s', path, fileName);
blobService.getBlobToLocalFile(containerName, fileName, downloadedImageName, function (error, serverBlob) {
});
});
It works for me, the image file was downloaded into my downloadImages directory, not under the path of my node app.js running.
Note: If you want to deploy it on Azure WebApp later, please must use the absolute directory path like D:/home/site/wwwroot/<your defined directory for downloading images>, because the related directory path is always related the path of IIS started up node.
I am using express-file package. I use it like this:
const upload = require('express-fileupload');
app.use(upload({
createParentPath: true
}));
This is how I store:
await req.files.main_image.mv('./public/images/movies/'+main_image);
I already have public/images directory created. I don't have movies directory in public/images directory though.
When It works: If I have /public/images/movies directory already created, it works
When it doesn't work: If I don't have /public/images/movies directory created, but have /public/images directory. Then it says:
ENOENT: no such file or directory, open
'C:\Users\glagh\Desktop\Work\MoviesAdviser\public\images\movies\1554741546720-8485.11521.brussels.the-hotel-brussels.amenity.restaurant-AD3WAP2L-13000-853x480.jpeg
What to do so that it automatically creates /movies directory and put the image there?
Express-fileupload uses the following code to create the directory-path which you pass to the .mv function:
const checkAndMakeDir = function(fileUploadOptions, filePath){
//Check upload options were set.
if (!fileUploadOptions) return false;
if (!fileUploadOptions.createParentPath) return false;
//Check whether folder for the file exists.
if (!filePath) return false;
const parentPath = path.dirname(filePath);
//Create folder if it is not exists.
if (!fs.existsSync(parentPath)) fs.mkdirSync(parentPath);
return true;
};
The problem is that fs.mkdirSync() does not create the full path with the parent directories you specifiy (note that you'd use mkdir -p on the shell to create the whole folder structure) -> checkout How to create full path with node's fs.mkdirSync? for more information.
What you could do is use a module like fs-extra and use its function ensureDir which would create the corresponding parent directories (if they don't exist yet) before calling the .mv() function.
Or in case you're node version is >= 10 use the native {rescursive:true} option which you can pass to fs.mkdirSync.
I am trying to upload image to backblaze-b2 using hapi.js. I am using a plugin called easy-backblaze but while using it I need to mention the path of the file. But While importing the file using Hapijs I am not able to understand how to get that path.
This is the Code I have written and here in b2.uploadFile I need to mention the path of the file on local Drive:
var B2 = require('easy-backblaze');
var b2 = new B2('accountId', 'applicationKey');
b2.uploadFile('C:/Users/Lovika/Desktop/addDriver.png', {
name: 'addDriver.png', // Optional, renames file
bucket: 'testBucket', // Optional, defaults to first bucket
}, function(err, res) {
console.log('Done!', err, res);
});
Do I need to upload the file to the server first and then to backblaze or Is there any way to upload the file directly to backblaze
I am trying to rename or move a folder in google cloud storage using the gcloud api.
A similar question explains how to delete a folder:
Delete folder in Google Cloud Storage using nodejs gcloud api
But how can one rename a folder? or move to another path?
You can try something like this:
'use strict'
var async = require('async')
var storage = require('#google-cloud/storage')()
var bucket = storage.bucket('stephen-has-a-new-bucket')
bucket.renameFolder = function(source, dest, callback) {
bucket.getFiles({ prefix: source }, function(err, files) {
if (err) return callback(err)
async.eachLimit(files, 5, function(file, next) {
file.move(file.name.replace(source, dest), next)
}, callback)
})
}
bucket.renameFolder('photos/cats', 'photos/dogs', console.log)
There are no folders. There is simply a collection of objects that all happen to have the same key prefix, for example photos/animals/cat.png and photos/animals/dog.png both have a common prefix photos/animals/ and that's what makes them appear to be in the same folder.
You will need to copy (or move) each of the objects to its new key, for example move photos/animals/cat.png to photos/pets/cat.png and move photos/animals/dog.png to photos/pets/dog.png.
That said, Google Cloud provides a way to do this from the command line using gsutil mv.