I want to unzip gzip files in Node.js I've tried [some] packages but nothing is working. Can you provide a package with sample code which can decompress gzip files in Node.js?
gunzip-file node package worked fine!
Do:
npm install gunzip-file
Then:
'use strict'
const gunzip = require('gunzip-file')
// 'sitemap.xml.gz' - source file
// 'sitemap.xml' - destination file
// () => { ... } - notification callback
gunzip('sitemap.xml.gz', 'sitemap.xml', () => {
console.log('gunzip done!')
})
Finally, run with Node at your shell.
I would suggest to use zlib.Gunzip.
Function prototype is zlib.Gunzip(buf, callback). The first argument is the raw archive data as a buffer that you want to extract, the second one is a callback which accept two arguments (result and error).
An implementation would be:
zlib.Gunzip(raw_data, function (error, result) {
if (error) throw error;
// Access data here through result as a Buffer
})
You can use tar.gz npm package to take care of this problem.
write following method to extract it to specific path:
targz().extract('/bkp/backup.tar.gz', '/home/myuser')
.then(function(){
console.log('Job done!');
})
.catch(function(err){
console.log('Something is wrong ', err.stack);
});
For more details you can follow this link
To unzip files in a directory:
To zip files in a directory:
More information:
https://medium.com/#harrietty/zipping-and-unzipping-files-with-nodejs-375d2750c5e4
Related
can someone suggest npm package for extract 7z file for node.js.
I can see some npm package available for ZIP file but that does not work for the 7Z file.
I'm basically looking to extract 7z password protect file on S3 and read the data from 7z file.
Give node-7z package a try:
npm i node-7z
import Seven from 'node-7z'
// myStream is a Readable stream
const myStream = Seven.extractFull('./archive.7z', './output/dir/', {
$progress: true
})
myStream.on('data', function (data) {
doStuffWith(data) //? { status: 'extracted', file: 'extracted/file.txt" }
})
myStream.on('progress', function (progress) {
doStuffWith(progress) //? { percent: 67, fileCount: 5, file: undefinded }
})
myStream.on('end', function () {
// end of the operation, get the number of folders involved in the operation
myStream.info.get('Folders') //? '4'
})
myStream.on('error', (err) => handleError(err))
It also supports password feature you were requesting mate.
As you can see, there is a file at the path. But fs says no such file or directory. I can't understand why?
In another file, I can remove with the same code.
My boat.js file:
boat.findById(req.params.id,function(err, foundBoat) {
if(err){
console.log(err);
}else{
foundBoat.boatsFoto.forEach(function(path){
console.log(typeof(path));
fs.unlink("../public"+path,function(err){
if(err) throw err;
console.log('File deleted!');
});
});
}
});
And it is my error:
Error: ENOENT: no such file or directory, unlink '../public/uploads/akingokay/BoatsFoto/1524411110335kiralik-tekne.jpg'
at Error (native)
And you can see my file system
Can you try this instead:
fs.unlink("public"+path,function(err){
if(err) throw err;
console.log('File deleted!');
});
You should first install the path module via CLI:
npm install path --save
and use it:
fs.unlink(path.join("public/" + path, photo.id + ".jpg"), function(response) {
// handle the callback
});
Check if the path variable you are adding to the "/public" has a "/" at the beginning. If there is no/ seperating it it will treat the path as one.
Install and import path module. this helps you https://nodejs.org/api/path.html
Instead of "../public"+path use path.join("../", "public", path);
It depends where you host the server. If it is on a local machine then you will probably need to specify the complete path to the file(s) that you want to delete or manipulate. If it is on a web server that is live then you will need to specify the complete path to it.
on a "local machine" it might look something like this:
fs.unlink('/home/user/project/someothername/'+filename, err => {
// handler
});
Consider like you are in app.js file and give the relative path of deleting file to the fs.unlink('path/to/file'). it will work!
I have a very simple node lambda function which reads the contents of packaged file in it. I upload the code as zip file. The directory structure is as follows.
index.js
readme.txt
Then have in my index.js file:
fs.readFile('/var/task/readme.txt', function (err, data) {
if (err) throw err;
});
I keep getting the following error NOENT: no such file or directory, open '/var/task/readme.txt'.
I tried ./readme.txt also.
What am I missing ?
Try this, it works for me:
'use strict'
let fs = require("fs");
let path = require("path");
exports.handler = (event, context, callback) => {
// To debug your problem
console.log(path.resolve("./readme.txt"));
// Solution is to use absolute path using `__dirname`
fs.readFile(__dirname +'/readme.txt', function (err, data) {
if (err) throw err;
});
};
to debug why your code is not working, add below link in your handler
console.log(path.resolve("./readme.txt"));
On AWS Lambda node process might be running from some other folder and it looks for readme.txt file from that folder as you have provided relative path, solution is to use absolute path.
What worked for me was the comment by Vadorrequest to use process.env.LAMBDA_TASK_ROOT. I wrote a function to get a template file in a /templates directory when I'm running it locally on my machine with __dirname or with the process.env.LAMBDA_TASK_ROOT variable when running on Lambda:
function loadTemplateFile(templateName) {
const fileName = `./templates/${templateName}`
let resolved
if (process.env.LAMBDA_TASK_ROOT) {
resolved = path.resolve(process.env.LAMBDA_TASK_ROOT, fileName)
} else {
resolved = path.resolve(__dirname, fileName)
}
console.log(`Loading template at: ${resolved}`)
try {
const data = fs.readFileSync(resolved, 'utf8')
return data
} catch (error) {
const message = `Could not load template at: ${resolved}, error: ${JSON.stringify(error, null, 2)}`
console.error(message)
throw new Error(message)
}
}
This is an oldish question but comes up first when attempting to sort out whats going on with file paths on Lambda.
Additional Steps for Serverless Framework
For anyone using Serverless framework to deploy (which probably uses webpack to build) you will also need to add the following to your webpack config file (just after target: node):
// assume target: 'node', is here
node: {
__dirname: false,
},
Without this piece using __dirname with Serverless will STILL not get you the desired absolute directory path.
I went through this using serverless framework and it really was the file that was not sent in the compression. Just add the following line in serverless.yml:
package:
individually: false
include:
- src/**
const filepath = path.resolve('../../filename.text');
const fileData2 = fs.readFileSync(process.env.LAMBDA_TASK_ROOT + filepath, 'utf-8');
I was using fs.promises.readFile(). Couldn't get it to error out at out. The file was there, and LAMBDA_TASK_ROOT seemed right to me as well. After I changed to fs.readFileSync(), it worked.
I hade the same problem and I tried applying all these wonderful solutions above - which didn't work.
The problem was that I setup one of the folder name with one letter in upper case which was really lowercase.
So when I tried to fetch the content of /src/SOmething/some_file.txt
While the folder was really /src/Something/ - I got this error...
Windows (local environment) is case insensitive while AWS is not!!!....
I have a file(data.file an image), I would like to save this image. Now an image with the same name could exist before it. I would like to overwrite if so or create it if it does not exist since before. I read that the flag "w" should do this.
Code:
fs.writeFile('/avatar/myFile.png', data.file, {
flag: "w"
}, function(err) {
if (err) {
return console.log(err);
}
console.log("The file was saved!");
});
Error:
[Error: ENOENT: no such file or directory, open '/avatar/myFile.png']
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/avatar/myFile.png'
This is probably because you are trying to write to root of file system instead of your app directory '/avatar/myFile.png' -> __dirname + '/avatar/myFile.png' should do the trick, also check if folder exists. node.js won't create parent folder for you.
Many of us are getting this error because parent path does not exist. E.g. you have /tmp directory available but there is no folder "foo" and you are writing to /tmp/foo/bar.txt.
To solve this, you can use mkdirp - adapted from How to write file if parent folder doesn't exist?
Option A) Using Callbacks
const mkdirp = require('mkdirp');
const fs = require('fs');
const getDirName = require('path').dirname;
function writeFile(path, contents, cb) {
mkdirp(getDirName(path), function (err) {
if (err) return cb(err);
fs.writeFile(path, contents, cb);
});
}
Option B) Using Async/Await
Or if you have an environment where you can use async/await:
const mkdirp = require('mkdirp');
const fs = require('fs');
const writeFile = async (path, content) => {
await mkdirp(path);
fs.writeFileSync(path, content);
}
I solved a similar problem where I was trying to create a file with a name that contained characters that are not allowed. Watch out for that as well because it gives the same error message.
I ran into this error when creating some nested folders asynchronously right before creating the files. The destination folders wouldn't always be created before promises to write the files started. I solved this by using mkdirSync instead of 'mkdir' in order to create the folders synchronously.
try {
fs.mkdirSync(DestinationFolder, { recursive: true } );
} catch (e) {
console.log('Cannot create folder ', e);
}
fs.writeFile(path.join(DestinationFolder, fileName), 'File Content Here', (err) => {
if (err) throw err;
});
Actually, the error message for the file names that are not allowed in Linux/ Unix system comes up with the same error which is extremely confusing. Please check the file name if it has any of the reserved characters. These are the reserved /, >, <, |, :, & characters for Linux / Unix system. For a good read follow this link.
It tells you that the avatar folder does not exist.
Before writing a file into this folder, you need to check that a directory called "avatar" exists and if it doesn't, create it:
if (!fs.existsSync('/avatar')) {
fs.mkdirSync('/avatar', { recursive: true});
}
you can use './' as a prefix for your path.
in your example, you will write:
fs.writeFile('./avatar/myFile.png', data.file, (err) => {
if (err) {
return console.log(err);
}
console.log("The file was saved!");
});
I had this error because I tried to run:
fs.writeFile(file)
fs.unlink(file)
...lots of code... probably not async issue...
fs.writeFile(file)
in the same script. The exception occurred on the second writeFile call. Removing the first two calls solved the problem.
In my case, I use async fs.mkdir() and then, without waiting for this task to complete, I tried to create a file fs.writeFile()...
As SergeS mentioned, using / attempts to write in your system root folder, but instead of using __dirname, which points to the path of the file where writeFile is invoked, you can use process.cwd() to point to the project's directory. Example:
writeFile(`${process.cwd()}/pictures/myFile.png`, data, (err) => {...});
If you want to avoid string concatenations/interpolations, you may also use path.join(process.cwd(), 'pictures', 'myFile.png') (more details, including directory creation, in this digitalocean article).
OK so I am creating a gulp plugin to create a table of contents and imports for my sass setup.
I have the gulp task all setup like this
var sgc = require('./tasks/sass-generate-contents');
gulp.task('sass-generate-contents', function(){
gulp.src(config.tests + '/**/*.scss')
.pipe(sgc(config.tests + '/_main.scss'));
});
and then my plugin code
function sassGenerateContents(){
return through.obj(function(file, enc, cb){
if (file.isNull()) {
cb(null, file);
return;
}
if (file.isStream()) {
cb(new gutil.PluginError(PLUGIN_NAME, 'Streaming not supported'));
return;
}
console.log(file);
});
}
module.exports = sassGenerateContents;
in my file system (the test directory) are 3 x *.scss files
the console log in the plugin only ever returns the first.
I'm probably missing somehting really obvious here, but I thought the idea of **/* was to loop through the folder structure specified and pull out the files.
Can someone explain to me why it's only returning the first file?
Thanks
I was missing the callback function, which called back to itself. At the end of the function i needed to ad
return cb();