Running gulp tasks in sequence throws `Unhandled stream error in pipe` - node.js

I have a gulp task as following :
gulp.task('build-files, function(cb) {
runSequence('run-build-cmd',
'delete-dest', 'copy-files',
cb);
});
This task is running whenever something changed in a source folder as following :
gulp.watch(pathToSrcFolder, ['build-files']);
So this task runs 3 other gulp tasks in the specified order, the first one it runs a build command, the second one will delete a folder as following :
gulp.task('delete-dest', (cb) => {
del([pathToDestFolder], {force: true});
cb();
});
and the third one will copy files from a source into two destinations :
gulp.task('copy-files', () => {
return gulp.src(pathToSrcFolder)
.pipe(gulp.dest(pathToDestFolder))
.pipe(gulp.dest(anotherPath));
});
Please notice that the pathToDestFolder is the same folder in both delete-source and copy-files commands.
The problem I had running this sequence, is this error:
internal/streams/legacy.js:59
throw er; // Unhandled stream error in pipe.
^
Error: ENOENT: no such file or directory, chmod 'pathToDestFolder\path\to\some\file\file.ext'
I don't know why I'm getting this error.
And when I run the gulp delete-dest in cmd prompt (which will clear the pathToDestFolder) and then gulp copy-files (which will copy the source folder into two folders pathToDestFolder and anotherPath) it works as expected .
So I guess runSequence didn't work as expected ? if so, how can I solve this ?
Edit:
I tried to use rimraf instead of del, and all seems to work just fine, I know that rimraf is depricated, and it's better to use del instead, but why del in this case results in an exception ?
Edit 2:
Instead of using rimraf as a solution, I tried this solution:
gulp.task('delete-dest', (cb) => {
del([pathToDestFolder], {force: true})
.then(paths => {cb();});
});
And it worked like magic.
Why this worked instead ? I don't know !
If someone can clarify things I would be greatful.

If you look here, you'll find that del returns a promise. It's an async method.
In your code, you call the cb() right after calling del(...), but before del actually finishes deleting the directory.
That's why you have to execute the callback after chaining with then:
gulp.task('delete-dest', (cb) => {
del([pathToDestFolder], {force: true})
.then(paths => {cb();});
});
And then your operations will run in the correct order.
The error you were getting before is the result of things being run in a weird order, leading to weird behavior. Gulp is trying to copy to a directory while the file system is deleting it, and that conflict leads to a file system error.
As an experiment, you can try running the synchronous version of del like so:
gulp.task('delete-dest', (cb) => {
del.sync([pathToDestFolder], {force: true});
cb();
});
See if that works (though you should prefer the async version).
As a side note, unless this is a feature of Gulp 4 though probably not, rather than calling a callback in your task, you can just return a promise like so:
gulp.task('delete-dest', () => {
return del([pathToDestFolder], {force: true});
});
That will tell Gulp that your task is finished, and it can move on to the next.

Related

How to decompress gzip files in Node.js

I want to unzip gzip files in Node.js I've tried [some] packages but nothing is working. Can you provide a package with sample code which can decompress gzip files in Node.js?
gunzip-file node package worked fine!
Do:
npm install gunzip-file
Then:
'use strict'
const gunzip = require('gunzip-file')
// 'sitemap.xml.gz' - source file
// 'sitemap.xml' - destination file
// () => { ... } - notification callback
gunzip('sitemap.xml.gz', 'sitemap.xml', () => {
console.log('gunzip done!')
})
Finally, run with Node at your shell.
I would suggest to use zlib.Gunzip.
Function prototype is zlib.Gunzip(buf, callback). The first argument is the raw archive data as a buffer that you want to extract, the second one is a callback which accept two arguments (result and error).
An implementation would be:
zlib.Gunzip(raw_data, function (error, result) {
if (error) throw error;
// Access data here through result as a Buffer
})
You can use tar.gz npm package to take care of this problem.
write following method to extract it to specific path:
targz().extract('/bkp/backup.tar.gz', '/home/myuser')
.then(function(){
console.log('Job done!');
})
.catch(function(err){
console.log('Something is wrong ', err.stack);
});
For more details you can follow this link
To unzip files in a directory:
To zip files in a directory:
More information:
https://medium.com/#harrietty/zipping-and-unzipping-files-with-nodejs-375d2750c5e4

Using grunt to run a node server and do cleanup after

So basically this is what I want to do. Have a grunt script that compiles my coffee files to JS. Then run the node server and then, either after the server closes or while it's still running, delete the JS files that were the result of the compilation and only keep the .coffee ones.
I'm having a couple of issues getting it to work. Most importantly, the way I'm currently doing it is this:
grunt.loadNpmTasks("grunt-contrib-coffee");
grunt.registerTask("node", "Starting node server", function () {
var done = this.async();
console.log("test");
var sp = grunt.util.spawn({
cmd: "node",
args: ["index"]
}, function (err, res, code) {
console.log(err, res, code);
done();
});
});
grunt.registerTask("default", ["coffee", "node"]);
The problem here is that the node serer isn't run in the same process as grunt. This matters because I can't just CTRL-C once to terminate JUST the node server.
Ideally, I'd like to have it run in the same process and have the grunt script pause while it's waiting for me to CTRL-C the server. Then, after it's finished, I want grunt to remove the said files.
How can I achieve this?
Edit: Note that the snippet doesn't have the actual removal implemented since I can't get this to work.
If you keep the variable sp in a more global scope, you can define a task node:kill that simply checks whether sp === null (or similar), and if not, does sp.kill(). Then you can simply run the node:kill task after your testing task. You could additionally invoke a separate task that just deletes the generated JS files.
For something similar I used grunt-shell-spawn in conjunction with a shutdown listener.
In your grunt initConfig:
shell: {
runSuperCoolJavaServer:{
command:'java -jar mysupercoolserver.jar',
options: {
async:true //spawn it instead!
}
}
},
Then outside of initConfig, you can set up a listener for when the user ctrl+c's out of your grunt task:
grunt.registerTask("superCoolServerShutdownListener",function(step){
var name = this.name;
if (step === 'exit') process.exit();
else {
process.on("SIGINT",function(){
grunt.log.writeln("").writeln("Shutting down super cool server...");
grunt.task.run(["shell:runSuperCoolJavaServer:kill"]); //the key!
grunt.task.current.async()();
});
}
});
Finally, register the tasks
grunt.registerTask('serverWithKill', [
'runSuperCoolJavaServer',
'superCoolServerShutdownListener']
);

How to use jasmine with gulp.watch

I'm trying to make my tests run each time I'm saving some files. Here is the gulp watch:
gulp.task('jasmine', function() {
gulp.src('spec/nodejs/*Spec.js')
.pipe(jasmine({verbose:true, includeStackTrace: true}));
});
gulp.task('watch', function () {
gulp.watch(['app/*.js', 'app/!(embed)**/*.js','spec/nodejs/*.js'], ['jasmine']);
});
To test for example app/maps.js I'm creating a spec/nodejs/mapsSpec.js file like this:
'use strict';
var maps = require('../../app/maps');
describe('/maps related routes', function(){
it('should ...', function(){...}
...
If I change a spec file everything is working well, if I modify app/maps.js file the change trigger the test. if I modify it again tests are tiggered but the modifications do not taking effect. For example if I add a console.log('foo') in a second time, I will not see it until I relaunch gulp watch and save it again. So only one run of jasmine is ok when using it with gulp.watch.
I guess it's because require is cached by nodejs in the gulp process. So how should I do ?
I took a look at the code of gulp-jasmine. The problem is that the only file from the cache is the Specs.js file. The cache of the children(the reqquired files to test) aren't cleared.
Within the index.js of gulp-jasmine is a row which deletes the cache:
delete require.cache[require.resolve(path.resolve(file.path))];
If you put the next block of code before the delete, you will delete all the children's cache and will it run correctly after every time you save your file.
var files = require.cache[require.resolve(path.resolve(file.path))];
if( typeof files !== 'undefined' ) {
for( var i in files.children ) {
delete require.cache[ files.children[i].id ];
}
}
You can change this in the node_modules.
I will go for a pull request, so maybe in the near future this will be solved permanently.
Also wrote a post about it on: http://navelpluisje.nl/entry/fix-cache-problem-jasmine-tests-with-gulp
I haven't found a fix for this issue, but you can work around it via the gulp-shell task.
npm install gulp-shell --save-dev
then
var shell = require('gulp-shell');
...
gulp.task('jasmine', function() {
gulp.src('spec/nodejs/*Spec.js')
.pipe(shell('minijasminenode spec/*Spec.js'));
});
You'll also need jasmine installed as a direct dependency (gulp-jasmine uses minijasminenode)

gulp-sass, watch stops when invalid property name

watch stops when error messages occur.
stream.js:94
throw er; // Unhandled stream error in pipe.
^
source string:51: error: invalid property name
How I can keep watch running and just to tell me where is the error located.
grunt could deal with errors and doesn't need to stop,
styleSheet.scss:41: error: invalid property name
otherwise, I need to keep typing "gulp" in the command-line when an error occurs.
This answer has been appended to reflect recent changes to Gulp. I've retained the original response, for relevance to the OPs question. If you are using Gulp 2.x, skip to the second section
Original response, Gulp 1.x
You may change this default behavior by passing errLogToConsole: true as an option to the sass() method.
Your task might look something like this, right now:
gulp.task('sass', function () {
gulp.src('./*.scss')
.pipe(sass())
.pipe(gulp.dest('./'));
});
Change the .pipe(sass()) line to include the errLogToConsole: true option:
.pipe(sass({errLogToConsole: true}))
This is what the task, with error logging, should look like:
gulp.task('sass', function () {
gulp.src('./*.scss')
.pipe(sass({errLogToConsole: true}))
.pipe(gulp.dest('./'));
});
Errors output will now be inline, like so:
[gulp] [gulp-sass] source string:1: error: invalid top-level expression
You can read more about gulp-sass options and configuration, on nmpjs.org
Gulp 2.x
In Gulp 2.x errLogToConsole may no longer be used. Fortunately, gulp-sass has a method for handling errors. Use on('error', sass.logError):
gulp.task('sass', function () {
gulp.src('./sass/**/*.scss')
.pipe(sass().on('error', sass.logError))
.pipe(gulp.dest('./css'));
});
If you need more fine-grained control, feel free to provide a callback function:
gulp.task('sass', function () {
gulp.src('./sass/**/*.scss')
.pipe(sass()
.on('error', function (err) {
sass.logError(err);
this.emit('end');
})
)
.pipe(gulp.dest('./css'));
});
This is a good thread to read if you need more information on process-control: https://github.com/gulpjs/gulp/issues/259#issuecomment-55098512
Actually above anwsers doesn't work for me (Im using gulp-sass 3.XX). What really worked:
gulp.task('sass', function () {
return gulp.src(config.scssPath + '/styles.scss')
.pipe(sourcemaps.init())
.pipe(sass({ outputStyle: 'compressed' })
.on('error', sass.logError)
)
.pipe(sourcemaps.write('./'))
.pipe(gulp.dest(config.cssPath))
});
In gulp-sass 3.x.x when I was using "sass.logError(err);" I constantly recive error that "this.emit('end'); is not a function". Now when I'm using:
.pipe(sass({ outputStyle: 'compressed' })
.on('error', sass.logError)
)
everything is working like a charm
In gulp "^2.0.0" the option errLogToConsole will no longer work. Instead gulp-sass has a built in error logging callback that uses gulp-util under the hood. Also, because gulp has some problems with killing the process on errors, if you are using with watch you will have to call this.emit('end')
https://github.com/gulpjs/gulp/issues/259#issuecomment-55098512
var sass = require('gulp-sass');
//dev
sass(config.sassDev)
.on('error', function(err) {
sass.logError(err);
this.emit('end'); //continue the process in dev
})
)
//prod
sass(config.sassProd).on('error', sass.logError)
A heads up for Gulp 3 users:
I liked #dtotheftp solution above, regarding gulp 2.x. Interestingly, it doesn't work unter Gulp3, at least not under #3.9.1:
.on('error', function(err){
sass.logError(err);
this.emit('end'); //continue the process in dev
})
gets me
TypeError: this.emit is not a function
at Function.logError (/depot/myproject/node_modules/gulp-sass/index.js:181:8)
Note, that the complaint is not coming from his this.emit() in the gulpfile but rather from the sass node-module, hence from the prior line.
This works for me:
.on('error', function(err){
gutil.log(err);
this.emit('end');
})
I do get all errors², and the watch never ends ;) (I am also using plumber() right after gulp.src(), which might help with that).
(Yes, the fix might be highly illogical, since sass.logError is said to be based on gutil...)
—
²also on undefined macros which went silent before on my setup for whatever reason.

Is it possible to run the grunt command after a yeoman generator install?

Basically I'd like to run grunt after my generator finishes installing dependencies, I found that you can add a callback function to the installDependencies method to run after everything has been installed like this:
this.on('end', function () {
this.installDependencies({
skipInstall: options['skip-install'],
callback: function () {
console.log('All done!');
}
});
});
However I'm not sure how to run the grunt task (as in going to the terminal and running "grunt")
After this.on('end') add this lines
// Now you can bind to the dependencies installed event
this.on('dependenciesInstalled', function() {
this.spawnCommand('grunt', ['build']);
});
check this topic for more details.
But if you're using the latest update of yeomen, you'll need to make it like this
this.on('end', function () {
if (!this.options['skip-install']) {
this.npmInstall();
this.spawnCommand('grunt', ['prepare']); // change 'prepare' with your task.
}
});

Resources