I need to read a content of a password-protected archive (zip is preferred) to a node-js app, without writing the protected content to a file
In addition, the app is cross-platform so solution such this doesn't help
I looked also here but there is no code in the answer
The only library I can find that supports encryption is: https://github.com/rf00/minizip-asm.js
Unfortunately, it isn't well maintained.
This solution will read the file buffer which you can get from base64 or by reading the zip file, after that unzipping and opening the password-protected file is done in-memory. I hope this helps -
const unzipper = require("unzipper");
const unzipAndUnlockZipFileFromBuffer = async (zippedFileBase64, password) => {
try {
const zipBuffer = Buffer.from(zippedFileBase64, "base64"); // Change base64 to buffer
const zipDirectory = await unzipper.Open.buffer(zipBuffer); // unzip a buffered file
const file = zipDirectory.files[0]; // find the file you want
// if you want to find a specific file by path
// const file = zipDirectory.files.find((f) => f.path === "filename");
const extracted = await file.buffer(password); // unlock the file with the password
console.log(extracted.toString()); // file content
} catch (e) {
console.log(e);
}
};
const zippedFileBase64 = "{{BASE64}}";
const password = "1234";
unzipAndUnlockZipFileFromBuffer(zippedFileBase64, password);
Related
I'm following up on this article to download objects from GCP Cloud storage bucket: https://cloud.google.com/storage/docs/downloading-objects#storage-download-object-nodejs
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
async function downloadIntoMemory() {
// Downloads the file into a buffer in memory.
const contents = await storage.bucket(bucketName).file(fileName).download();
return contents;
);
}
downloadIntoMemory().catch(console.error);
I'm currently getting a buffer data in contents. I've this code hooked upto a API on NodeJS backend. I'm using React Typescript on frontend. Calling the API, gives me data buffer. How can I use it to download the file instead of the data buffer?
I tried the above method explicitly providing file destination, but I'm still getting the following error: EISDIR: illegal operation on a directory, open '{file_path_which_i_was_set}. Err: -21
As rightly pointed out by #John Hanley, you are referring to the documentation, where the code sample downloads an object into memory/ buffer in memory. If you want to download an object from a bucket to a file, refer to this code sample, where the ‘options’ parameter has to be passed to the download() method.
The code goes like this :
// The ID of your GCS bucket
const bucketName = 'your-unique-bucket-name';
// The ID of your GCS file
const fileName = 'your-file-name';
// The path to which the file should be downloaded
const destFileName = '/local/path/to/file.txt';
// Imports the Google Cloud client library
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
async function downloadFile() {
const options = {
destination: destFileName,
};
// Downloads the file to the destination file path
await storage.bucket(bucketName).file(fileName).download(options);
console.log(
`gs://${bucketName}/${fileName} downloaded to ${destFileName}.`
);
}
downloadFile().catch(console.error);
I am using google drive API to download an excel file in my Flutter app but I want to store the downloaded file content response in a File and then do some update operations using excel dart package, below is the given code from reading an xlsx file from a path location.
var file = "Path_to_pre_existing_Excel_File/excel_file.xlsx"; //here I want to store the response from drive api
var bytes = File(file).readAsBytesSync();
var excel = Excel.decodeBytes(bytes);
//Do some logic here
for (var table in excel.tables.keys) {
print(table); //sheet Name
print(excel.tables[table].maxCols);
print(excel.tables[table].maxRows);
for (var row in excel.tables[table].rows) {
print("$row");
}
}
//then saving the excel file
// updating the excel sheet to Drive
updateToDrive(excel,fileId);
I have created all the required auth functions, drive scopes and my download function looks like this :
Future<void> downloadFile() async{
String fileId = '1TOa4VKfZBHZe######WLA4M95nOWp';
final response = await driveApi.files.get(
fileId,
downloadOptions: drive.DownloadOptions.fullMedia
);
print(response);
}
This function is executing correctely and giving Media type response, but I could not able to read this response so that I could store it in a file.
Any help would be truly appreciated, Thanks
I changed my download function to this, as drive.files.get() was returning a Future Object so I changed it to return Future<Media?> by type casting.
String fileId = "19jF3lOVW563LU6m########jXVLNQ7poXY1Z";
drive.Media? response = (await driveApi.files.get(
fileId,
downloadOptions: drive.DownloadOptions.fullMedia
)) as drive.Media?;
Now response is a Media on which we can listen to the sream to store the response in a file.
To do that first we need to get the app directory by path_provider
final String path = (await getApplicationSupportDirectory()).path;
final String fileName = '$path/Output.xlsx';
File file = File(fileName);
Now we want to write the stream of response Stream<List> into our file object which I found from this link
List<int> dataStore = [];
await response!.stream.listen((data) {
print("DataReceived: ${data.length}");
dataStore.insertAll(dataStore.length, data);
}, onDone: () {
print("Task Done");
file.writeAsBytes(dataStore);
OpenFile.open(file.path);
print("File saved at ${file.path}");
}, onError: (error) {
print("Some Error");
});
Now we can do whatever we want to make changes through excel package.
I am trying to consume Azure Forms Recognizer API, where I have to provide the body in the form of "[Binary PNG data]" as stated here.
The connection seems the be working fine, however I am getting this response:
{"error":{"code":"InvalidImage","innerError":{"requestId":"73c86dc3-51a3-48d8-853b-b6411f54c51e"},"message":"The input data is not a valid image or password protected."}}
I am using a png that is my local directory and I've tried converting it in many different ways including:
fs.readFile('test.png', function(err, data){
if (err) throw err;
// Encode to base64
let encodedImage = new Buffer(data, 'binary').toString('base64');
// Decode from base64
var decodedImage = new Buffer(encodedImage, 'base64').toString('binary');});
or
let data_string = fs.createReadStream('test.png');
and many others. None of them seem to work and I always get the same response from my post request.
I would appreciate if anyone could share how to convert this png into the correct format. Thank you in advance
To base 64:
const file = fs.readFileSync('/some/place/image.png')
const base64String = Buffer.from(file).toString('base64')
Then pass the base64String to Azure
If you want just a BLOB so a binary file, you can do this
const file = fs.readFileSync('/some/place/image.png')
const blob = Buffer.from(file)
const processFile = (file: any) => {
const reader = new FileReader();
reader.readAsArrayBuffer(file);
reader.onload = function(){
const binaryData = Buffer.from(reader.result as string,'binary');
console.log(binaryData);
};
}
I am following a tutorial to resize images via Cloud Functions on upload and am experiencing two major issues which I can't figure out:
1) If a PNG is uploaded, it generates the correctly sized thumbnails, but the preview of them won't load in Firestorage (Loading spinner shows indefinitely). It only shows the image after I click on "Generate new access token" (none of the generated thumbnails have an access token initially).
2) If a JPEG or any other format is uploaded, the MIME type shows as "application/octet-stream". I'm not sure how to extract the extension correctly to put into the filename of the newly generated thumbnails?
export const generateThumbs = functions.storage
.object()
.onFinalize(async object => {
const bucket = gcs.bucket(object.bucket);
const filePath = object.name;
const fileName = filePath.split('/').pop();
const bucketDir = dirname(filePath);
const workingDir = join(tmpdir(), 'thumbs');
const tmpFilePath = join(workingDir, 'source.png');
if (fileName.includes('thumb#') || !object.contentType.includes('image')) {
console.log('exiting function');
return false;
}
// 1. Ensure thumbnail dir exists
await fs.ensureDir(workingDir);
// 2. Download Source File
await bucket.file(filePath).download({
destination: tmpFilePath
});
// 3. Resize the images and define an array of upload promises
const sizes = [64, 128, 256];
const uploadPromises = sizes.map(async size => {
const thumbName = `thumb#${size}_${fileName}`;
const thumbPath = join(workingDir, thumbName);
// Resize source image
await sharp(tmpFilePath)
.resize(size, size)
.toFile(thumbPath);
// Upload to GCS
return bucket.upload(thumbPath, {
destination: join(bucketDir, thumbName)
});
});
// 4. Run the upload operations
await Promise.all(uploadPromises);
// 5. Cleanup remove the tmp/thumbs from the filesystem
return fs.remove(workingDir);
});
Would greatly appreciate any feedback!
I just had the same problem, for unknown reason Firebase's Resize Images on purposely remove the download token from the resized image
to disable deleting Download Access Tokens
goto https://console.cloud.google.com
select Cloud Functions from the left
select ext-storage-resize-images-generateResizedImage
Click EDIT
from Inline Editor goto file FUNCTIONS/LIB/INDEX.JS
Add // before this line (delete metadata.metadata.firebaseStorageDownloadTokens;)
Comment the same line from this file too FUNCTIONS/SRC/INDEX.TS
Press DEPLOY and wait until it finish
note: both original and resized will have the same Token.
I just started using the extension myself. I noticed that I can't access the image preview from the firebase console until I click on "create access token"
I guess that you have to create this token programatically before the image is available.
I hope it helps
November 2020
In connection to #Somebody answer, I can't seem to find ext-storage-resize-images-generateResizedImage in GCP Cloud Functions
The better way to do it, is to reuse the original file's firebaseStorageDownloadTokens
this is how I did mine
functions
.storage
.object()
.onFinalize((object) => {
// some image optimization code here
// get the original file access token
const downloadtoken = object.metadata?.firebaseStorageDownloadTokens;
return bucket.upload(tempLocalFile, {
destination: file,
metadata: {
metadata: {
optimized: true, // other custom flags
firebaseStorageDownloadTokens: downloadtoken, // access token
}
});
});
On a node server I would like to save uploaded datauri data as an image. To do this I've tried decoding the content of this png-
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAMAAAACCAIAAAFlEcHbAAAAB3RJTUUH1gMWFjk7nUWcXQAAAAlwSFlzAABOIAAATiABFn2Z3gAAAARnQU1BAACxjwv8YQUAAAAeSURBVHjaY7h79y7DhAkTGIA04/Tp0xkYGJ49ewYAgYwLV/R7bDQAAAAASUVORK5CYII=
And saving it as a .png extension. Looks like there is more too it than that. How do I decode the datauri and save it as a file?
I've created a library to be used with Node.js that helps with encoding and decoding of data URI schemes. I believe it can help you, check:
https://github.com/DiegoZoracKy/image-data-uri
Using this library, in your case, the code would be:
'use strict';
const ImageDataURI = require('image-data-uri');
const dataURI = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAMAAAACCAIAAAFlEcHbAAAAB3RJTUUH1gMWFjk7nUWcXQAAAAlwSFlzAABOIAAATiABFn2Z3gAAAARnQU1BAACxjwv8YQUAAAAeSURBVHjaY7h79y7DhAkTGIA04/Tp0xkYGJ49ewYAgYwLV/R7bDQAAAAASUVORK5CYII=';
const fileName = 'decoded-image.png';
ImageDataURI.outputFile(dataURI, filePath);
I was trying to decode the data using atob and saving this as a png file. I'm instead saving it base64 encoded but specifying the encoding in the write buffer.
fs.writeFileSync('tmp/myfile.png', new Buffer(data, 'base64'));
You can convert your data uri to a blob using below code:
function dataURItoBlob(dataURI) {
var byteStr;
if (dataURI.split(',')[0].indexOf('base64') >= 0)
byteStr = atob(dataURI.split(',')[1]);
else
byteStr = unescape(dataURI.split(',')[1]);
var mimeStr = dataURI.split(',')[0].split(':')[1].split(';')[0];
var arr= new Uint8Array(byteStr.length);
for (var i = 0; i < byteStr.length; i++) {
arr[i] = byteStr.charCodeAt(i);
}
return new Blob([arr], {type:mimeStr});
}
and then you can append this blob data to from data and upload it as a file:
var blob = dataURItoBlob(dataURI);
var fd = new FormData(document.forms[0]);
fd.append("image", blob);