Im looking for node package to unzip password protected zip files.
As of now i have looked here
But the above provides functions to open the files and parses the files one by one. I require to just extract the files at one place.
If I understand your question correctly, and if you still are looking for an answer
In the same package check out these functions:
Unzipper File Wise Extracting Options
Method specific extract
These can help you extract the files (all or selected ones) at the desired location
If you want to unzip a password-protected file from a directory -
const unzipper = require("unzipper");
const unzipAndUnlockZipFile = async (filepath, password) => {
try {
const zipDirectory = await unzipper.Open.file(filepath); // unzip a file
const file = zipDirectory.files[0]; // find the file you want
// if you want to find a specific file by path
// const file = zipDirectory.files.find((f) => f.path === "filename");
const extracted = await file.buffer(password); // unlock the file with the password (password is optional, do not pass this if the file is not password-protected)
console.log(extracted.toString()); // file content
} catch (e) {
console.log(e);
}
};
const zipFilePath = "./src/application.zip";
const password = "1234";
unzipAndUnlockZipFile(zipFilePath, password);
If you want to unzip and extract the file from a buffer then the solution would look like below -
const unzipper = require("unzipper");
const unzipAndUnlockZipFileFromBuffer = async (zippedFileBase64, password) => {
try {
const zipBuffer = Buffer.from(zippedFileBase64, "base64"); // Change base64 to buffer
const zipDirectory = await unzipper.Open.buffer(zipBuffer); // unzip a buffered file
const file = zipDirectory.files[0]; // find the file you want
// if you want to find a specific file by path
// const file = zipDirectory.files.find((f) => f.path === "filename");
const extracted = await file.buffer(password); // unlock the file with the password (password is optional, do not pass this if the file is not password-protected)
console.log(extracted.toString()); // file content
} catch (e) {
console.log(e);
}
};
const zippedFileBase64 = "{{BASE64}}";
const password = "1234";
unzipAndUnlockZipFileFromBuffer(zippedFileBase64, password);
Related
i have a problem with node.js, i have a script that download from a sftp server some zip file, every zip file are a compress csv file. My task is to download the file, unzip it and delete the zip file.
I have already a working script to download all the files from the sftp server. Now i would like to add to this script a function to unzip all the files and store only the csv files.
For doing that i started to work to a local script that open directly a single file and it had tried to unzip it. But i can't figured out how to do.
This is the portion of code that i wrote, after this code starts working I would like to put it in a helper function class where i can call from my script after the process of the download from sftp was completed.
Anyone can help me to understand on what i am wrong?
const logger = require("./utils/logger");
const path = require("path");
const fs = require("fs");
const unzipper = require("unzipper");
const { LOCAL_IN_DIR_PATH } = require("./utils/consts");
const unzipAll = async (pathToSearch) => {
console.log("unzipAll");
try {
const compFiles = fs.readdirSync(pathToSearch).forEach(function (file) {
if (file.endsWith(".zip")) {
const path = LOCAL_IN_DIR_PATH + `/${file}`;
fs.createReadStream(path).pipe(
unzipper.Extract({ path: path })
);
}
});
} catch (err) {
console.log(err);
}
};
const run = async () => {
try {
const LOCAL_IN_DIR_PATH = path.resolve(__dirname, "IN");
const result = await unzipAll(LOCAL_IN_DIR_PATH);
console.log(result);
} catch (error) {
console.log(error);
}
};
run();
so i'm trying to upload an image to firebase storage (a local photo from the source file directory - path: same path as .js). The problem is that in firebase storage the image appears to be corrupted and every picture have 9 bytes firebase storage image . The authentication and firestore works perfectly, this is my configuration file: firebase config file and this is the code:
const uploadPhoto = async() => {
// console.log(image);
// const uploadUri = image;
// let filename = uploadUri.substring(uploadUri.lastIndexOf('/') + 1);
const metadata = {
//path: '../../firebase_image.jpg',
contentType: 'image/jpeg'
};
const photog = `./photo.jpg`;
console.log(photog);
console.log(uploadUri);
const storageRef;
storageRef=ref(storage, 'photogra.jpg');//filename+
uploadBytes(storageRef, photog, metadata).then((snapshot) => {
console.log('Uploaded a blob or file!');
});
}
I hope you're fine!
I got the information from this amazing tutorial and worked fine for me
https://www.youtube.com/watch?v=H-yXO46WDak&lc=z22ph5dhssfqufkcxacdp430segloszlmvuqlp1seplw03c010c
Try with this:
const uploadImageFirebase = async () =>{
const nameImage = new Date().toISOString();
const img = await fetch(image);
const bytes = await img.blob();
try {
await uploadBytes(ref_con, bytes);
} catch (error) {
console.log(error);
} finally {
//
}
};
If you check the contents of your 9-byte file, it'll like be "photo.jpg".
Since you're passing "photo.jpg" to uploadBytes, it uploads that strings as the contents of the new file. It does not know how to load the file at that path.
You will either need to pass a local File or Buffer (which you'll usually get from a file picker or something like that), or load the data from the file yourself and pass the contents to uploadBytes.
I am trying to upload file to FTP server, but I see few entries but rest are skipped, even no error is generating. I don't know where exactly I am getting wrong.is it a synchronization issue or an issue from the package itself? I even used jsFtp package which can also put a buffer in the server but not works as expected. below are my code and output.
const unzipper = require("unzipper");
const ftp = require("basic-ftp");
const client = new ftp.Client();
await client.access({...options});
const zip = fs.createReadStream(path.join(filePath, `code.zip`)).pipe(unzipper.Parse({ raw: true, forceStream: true}));
for await (const entry of zip) {
await client.cd("/");
const type = entry.type; // 'Directory' or 'File'
const size = entry.vars.uncompressedSize; // There is also compressedSize;
let fileArray = entry.path.split("/");
if(size > 0 ) {
let fileName = fileArray.pop();
let dir = fileArray.splice(1).join("/");
await client.uploadFrom(entry, dir + "/" + fileName);
}
if(type === 'Directory') {
let dir = fileArray.splice(1).join("/");
await client.ensureDir(`${dir}`);
// await client.clearWorkingDir();
}
}
console.log("Entry Read Finished");
.gitignore
LICENSE
README.md
app/
app/bootstrap.php
app/config.php
composer.json
console.php
src/
src/SuperBlog/
src/SuperBlog/Command/
src/SuperBlog/Command/ArticleDetailComm
src/SuperBlog/Controller/
src/SuperBlog/Controller/ArticleControl
src/SuperBlog/Controller/HomeController
src/SuperBlog/Model/
src/SuperBlog/Model/Article.php
src/SuperBlog/Model/ArticleRepository.p
src/SuperBlog/Persistence/
src/SuperBlog/Persistence/InMemoryArtic
src/SuperBlog/Views/
src/SuperBlog/Views/article.twig
src/SuperBlog/Views/home.twig
src/SuperBlog/Views/layout.twig
web/
web/.htaccess
web/index.php
Creating Directory /
Uploading .gitignore
File: '' '.gitignore'
Uploading LICENSE
File: '' 'LICENSE'
Uploading README.md
File: '' 'README.md'
Creating Directory /app/
Uploading bootstrap.php
File: 'app' 'bootstrap.php'
Uploading config.php
File: 'app' 'config.php'
Entry Read Finished
can any one suggest what wrong with the code. Zip is perfectly fine no error with that.
You can simplify this using basic-ftp's support for uploadFromDir which will automatically create the required folder structure and upload the file contents. Here's a working example (tested successfully with the public ftp-server https://dlptest.com/ftp-test/):
const unzipper = require("unzipper");
const ftp = require("basic-ftp");
const fs = require('fs');
const path = require('path');
(async () => {
try {
await fs.createReadStream(path.resolve('./testzip.zip'))
.pipe(unzipper.Extract({path: './tmp-extract'}))
.promise();
const client = new ftp.Client();
await client.access({
host: "<host>",
user: "<user>",
password: "<password>",
});
await client.uploadFromDir("./tmp-extract");
console.log("upload successful");
} catch (err) {
console.log(err);
}
})();
I have web appication that can upload excel file. If user upload, the app should parse it and will return some rows that file have. So, The application don't need to save file to its filesystem. Parsing file and return rows is a job. But below code, I wrote this morning, it save file to its server and then parse it.. I think it's waste server resource.
I don't know how to read excel file with createReadStream. Without saving file, how can I parse excel directly? I am not familiar with fs, of course, I can delete file after the job finished, but is there any elegant way?
import { createWriteStream } from 'fs'
import path from 'path'
import xlsx from 'node-xlsx'
// some graphql code here...
async singleUpload(_, { file }, context) {
try {
console.log(file)
const { createReadStream, filename, mimetype, encoding } = await file
await new Promise((res) =>
createReadStream()
.pipe(createWriteStream(path.join(__dirname, '../uploads', filename)))
.on('close', res)
)
const workSheetsFromFile = xlsx.parse(path.join(__dirname, '../uploads', filename))
for (const row of workSheetsFromFile[0].data) {
console.log(row)
}
return { filename }
} catch (e) {
throw new Error(e)
}
},
Using express-fileupload library which provides a buffer representation for uploaded files (through data property), combined with excel.js which accepts a buffers will get you there.
see Express-fileupload and Excel.js
// read from a file
const workbook = new Excel.Workbook();
await workbook.xlsx.readFile(filename);
// ... use workbook
// read from a stream
const workbook = new Excel.Workbook();
await workbook.xlsx.read(stream);
// ... use workbook
// load from buffer // this is what you're looking for
const workbook = new Excel.Workbook();
await workbook.xlsx.load(data);
// ... use workbook
Here's a simplified example:
const app = require('express')();
const fileUpload = require('express-fileupload');
const { Workbook } = require('exceljs');
app.use(fileUpload());
app.post('/', async (req, res) => {
if (!req.files || Object.keys(req.files).length === 0) {
return res.status(400).send('No files were uploaded.');
}
// The name of the input field (i.e. "myFile") is used to retrieve the uploaded file
await new Workbook().xlsx.load(req.files.myFile.data)
});
app.listen(3000)
var xlsx = require('xlsx')
//var workbook = xlsx.readFile('testSingle.xlsx')
var workbook = xlsx.read(fileObj);
You just need to use xlsx.read method to read a stream of data.
you can add an event listener before you pipe the data, so you can do something with your file before it uploaded, it look like this
async singleUpload(_, { file }, context) {
try {
console.log(file)
const { createReadStream, filename, mimetype, encoding } = await file
await new Promise((res) =>
createReadStream()
.on('data', (data)=>{
//do something with your data/file
console.log({data})
//your code here
})
.pipe(createWriteStream(path.join(__dirname, '../uploads', filename)))
.on('close', res)
)
},
you can see the documentation
stream node js
We're trying to get an audiofile from Google Text-to-Speech and save it to Firebase Storage, using a Google Cloud Function. The documentation for Google Text-to-Speech show how to get an audiofile and save it locally:
// Performs the Text-to-Speech request
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
const writeFile = util.promisify(fs.writeFile);
await writeFile('output.mp3', response.audioContent, 'binary');
console.log('Audio content written to file: output.mp3');
This results in an error message Error: EROFS: read-only file system. Google Cloud Storage doesn't allow writing files locally.
Using Firebase Storage bucket.upload() has a few problems:
const destinationPath = 'Audio/Spanish' + filename.ogg;
// Performs the Text-to-Speech request
const [response] = await client.synthesizeSpeech(request);
// response.audioContent is the downloaded file
await bucket.upload(response.audioContent, {
destination: destinationPath
));
The error message is TypeError: Path must be a string. The first parameter of bucket.upload() is The fully qualified path to the file you wish to upload to your bucket. and is expected to be a string so response.audioContent doesn't work.
The documentation for bucket.upload() suggests that destination: destinationPath is where we should put the path to the Firebase Storage location. Is this correct?
How do we take the audiofile from Google Text-to-Speech (response.audioContent) and save it as a string that bucket.upload() can access? Or should we use something else instead of bucket.upload()?
Here's our full cloud function:
exports.Google_T2S = functions.firestore.document('Users/{userID}/Spanish/T2S_Request').onUpdate((change, context) => {
if (change.after.data().word != undefined) {
// Performs the Text-to-Speech request
async function test() {
try {
const word = change.after.data().word; // the text
const longLanguage = 'Spanish';
const audioFormat = '.mp3';
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const fs = require('fs');
const util = require('util');
const textToSpeech = require('#google-cloud/text-to-speech'); // Imports the Google Cloud client library
const client = new textToSpeech.TextToSpeechClient(); // Creates a client
let myWordFile = word.replace(/ /g,"_"); // replace spaces with underscores in the file name
myWordFile = myWordFile.toLowerCase(); // convert the file name to lower case
myWordFile = myWordFile + audioFormat; // append .mp3 to the file name;
// copied from https://cloud.google.com/blog/products/gcp/use-google-cloud-client-libraries-to-store-files-save-entities-and-log-data
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('myProject-cd99d.appspot.com');
const destinationPath = 'Audio/Spanish/' + myWordFile;
const request = { // Construct the request
input: {text: word},
// Select the language and SSML Voice Gender (optional)
voice: {languageCode: 'es-ES', ssmlGender: 'FEMALE'},
// Select the type of audio encoding
audioConfig: {audioEncoding: 'MP3'},
};
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
const writeFile = util.promisify(fs.writeFile);
await writeFile('output.mp3', response.audioContent, 'binary');
console.log('Audio content written to file: output.mp3')
// response.audioContent is the downloaded file
await bucket.upload(response.audioContent, {
destination: destinationPath
});
}
catch (error) {
console.error(error);
}
}
test();
} // close if
return 0; // prevents an error message "Function returned undefined, expected Promise or value"
});
file.save() was the answer. util.promisify was unnecessary, and causes an error message about original something. Here's the finished cloud function:
const functions = require('firebase-functions');
// // Create and Deploy Your First Cloud Functions
// // https://firebase.google.com/docs/functions/write-firebase-functions
//
// exports.helloWorld = functions.https.onRequest((request, response) => {
// response.send("Hello from Firebase!");
// });
async function textToSpeechRequest()
{
try
{
const word = change.after.data().word; // the text
const longLanguage = 'Spanish';
const audioFormat = '.mp3';
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const util = require('util');
const textToSpeech = require('#google-cloud/text-to-speech'); // Imports the Google Cloud client library
const client = new textToSpeech.TextToSpeechClient(); // Creates a client
let myWordFile = word.replace(/ /g,"_"); // replace spaces with underscores in the file name
myWordFile = myWordFile.toLowerCase(); // convert the file name to lower case
myWordFile = myWordFile + audioFormat; // append .mp3 to the file name;
// copied from https://cloud.google.com/blog/products/gcp/use-google-cloud-client-libraries-to-store-files-save-entities-and-log-data
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
//const bucket = storage.bucket('myProject-cd99d.appspot.com');
var file = bucket.file('Audio/Spanish/' + myWordFile);
const request = { // Construct the request
input: {text: word},
// Select the language and SSML Voice Gender (optional)
voice: {languageCode: 'es-ES', ssmlGender: 'FEMALE'},
// Select the type of audio encoding
audioConfig: {audioEncoding: 'MP3'},
};
const options = { // construct the file to write
metadata: {
contentType: 'audio/mpeg',
metadata: {
source: 'Google Text-to-Speech'
}
}
};
// copied from https://cloud.google.com/text-to-speech/docs/quickstart-client-libraries#client-libraries-usage-nodejs
const [response] = await client.synthesizeSpeech(request);
// Write the binary audio content to a local file
// response.audioContent is the downloaded file
return await file.save(response.audioContent, options)
.then(() => {
console.log("File written to Firebase Storage.")
return;
})
.catch((error) => {
console.error(error);
});
} // close try
catch (error) {
console.error(error);
} // close catch
} // close async function declaration
exports.Google_T2S = functions.firestore.document('Users/{userID}/Spanish/T2S_Request').onUpdate((change, context) => {
if (change.after.data().word !== undefined)
{
textToSpeechRequest();
} // close if
}); // close Google_T2S
We're getting an error TypeError: [ERR_INVALID_ARG_TYPE]: The "original" argument must be of type function at Object.promisify. This error doesn't appear to effect the cloud function.
To reiterate the stuff that didn't work, fs.createWriteStream didn't work because Google Cloud Functions can't handle Node file system commands. Instead, Google Cloud Functions have their own methods that wrap the Node file system commands. bucket.upload() will upload a local file to a bucket, but the path to the local file has to be a string, not a buffer or a stream coming from an API. file.save() is documented as
Write arbitrary data to a file.
This is a convenience method which wraps File#createWriteStream.
That's what I want! If there's one thing about my data, it's arbitrary. Or maybe contrary by nature. After that we just had to straighten out the contentType (audio/mpeg, not mp3) and the file path.