This issue...
... remembers me to another one, but seems different. (scroll down after code, to see what I mean)
I don't have really a problem with gulp-watch in other projects, especially since I fixed a similar issue. (which you can find on the end of this post)
But in this project (bigger web-application) I think nodejs/gulp-watch hitting his limit.
The JS-file which I get from gulp (via gulp-watch) seems to be corrupted when it was fired more then once.
The Javascript-file...
...isn't corrupted, when I use the default-task via gulp command.
...isn't corrupted, on the first recognized change by the watch-event.
...is often corrupted, after all following fired tasks of the watch-event.
I compared the good and bad (corrupted) resulting JS-files and recognized, that one file lost the half part of his content.
MEANS:
It stops in the middle of the JS-file and continues with the next JS-File. (concat only a half of the js-file)
All other JS-files before and after are concatenated without lost.
Does anyone knows this issue and know how to fix?
I already tried...
...to add gulp-wait with a 500ms delay, but that doesn't had a effect.
...check if the code on this position is bad, but it seems valid.
...restart the machine.
Environment: NodeJS 4.4.7 on Debian Jessie.
terminal output: These 8 starting script-tasks aren't fired at once. This are different detected changes when saving file and uploading this into the server and directory where Gulp is watching. (the arrows, worked & failed text aren't part of the output)
user#host: /path/to/vhost/public$ gulp watch
[20:09:02] Using gulpfile /path/to/vhost/public/gulpfile.js
[20:09:02] Starting 'watch'...
[20:09:02] Finished 'watch' after 25 ms
[20:09:32] Starting 'scripts'...
[20:09:32] Finished 'scripts' after 84 ms <- worked
[20:09:49] Starting 'scripts'...
[20:09:49] Finished 'scripts' after 51 ms <- failed
[20:09:51] Starting 'scripts'...
[20:09:51] Finished 'scripts' after 38 ms <- failed
[20:09:53] Starting 'scripts'...
[20:09:54] Finished 'scripts' after 68 ms <- worked
[20:10:01] Starting 'scripts'...
[20:10:02] Finished 'scripts' after 63 ms <- worked
[20:10:18] Starting 'scripts'...
[20:10:19] Finished 'scripts' after 59 ms <- failed
[20:10:35] Starting 'scripts'...
[20:10:35] Finished 'scripts' after 52 ms <- worked
[20:10:53] Starting 'scripts'...
[20:10:54] Finished 'scripts' after 58 ms <- failed
gulpfile.js:
/////////////
// INCLUDE //
/////////////
var gulp = require('gulp');
// Plugins
var autoprefixer = require('gulp-autoprefixer');
var concat = require('gulp-concat');
var rename = require('gulp-rename');
var sass = require('gulp-ruby-sass');
var uglify = require('gulp-uglify');
var expect = require('gulp-expect-file');
// Globals
var jsFiles = ['js/custom1.js',
'js/custom2.js',
'js/custom3.js',
'js/custom4.js',
'js/custom5.js',
'js/custom6.js',
'js/custom7.js',
'js/custom8.js',
'js/custom9.js',
'js/custom10.js',
'js/custom11.js',
'js/custom12.js',
'js/custom13.js',
'js/custom14.js',
'js/custom15.js',
'js/custom16.js',
'js/custom17.js',
'js/custom18.js'];
var cssFiles = ['scss/style.scss'];
///////////
// TASKS //
///////////
// FILES
gulp.task('files', function() {
var files = cssFiles.concat(jsFiles);
return gulp.src(files)
.pipe(expect(files));
});
// SCRIPTS
gulp.task('scripts', function() {
return gulp.src(jsFiles)
.pipe(concat('script.js'))
.pipe(rename({suffix: '.min'}))
// .pipe(uglify())
.pipe(gulp.dest(''));
});
// SASS
gulp.task('sass', function() {
return sass(cssFiles, {style: 'compressed'})
.pipe(autoprefixer({
browsers: ['last 2 versions'],
cascade: false
}))
.pipe(rename({suffix: '.min'}))
.pipe(gulp.dest(''));
});
// DEFAULT
gulp.task('default', ['files','scripts', 'sass']);
// WATCH
gulp.task('watch', function() {
// Watch .js files
gulp.watch(jsFiles, ['scripts']);
// Watch .scss files
gulp.watch(cssFiles, ['sass']);
});
REMEMBERS ME TO ANOTHER BUG
Two month ago I had a similar bug like this, which only worked once when the watch-task was fired. After that, you had to restart the command. Strange was that in the terminal wasn't any error message and it responded as if it would work. But the result was an empty css-file.
I fixed this OLD issue by updating my nodejs-installation from 0.12.x to 4.x.x.
But the current issue seems different.
From your log, it looks like scripts is being run too often, which could cause conflicts. Are you outputting your concatted file to the source folder, retriggering the task?
.pipe(gulp.dest(''));
What happens if you output to a different folder?
.pipe(gulp.dest('dist/'));
Found the reason and solution:
gulp-watch fired to fast and executes while the upload via FTP wasn't finished.
So gulp-watch can't differ your upload methode... and just firing when it recognizing a change. Instead to wait until the full file is uploaded sucessfully.
The solution would be to change the upload-methode of your FTP client or just selecting another watcher which can recognize this. (I use when-changed * ) at the moment.
This thing toggles two times when I upload a file on save. That is a good indicator that SFTP seems to upload data in parts. So I will read now how to change the upload-methode in Sublime SFTP. (so that I can switch back to gulp-watch again.
Related
The following Gulp task does almost what I want.
const gulp = require('gulp');
const browserify = require('browserify');
const vinylStream = require('vinyl-source-stream');
const vinylBuffer = require('vinyl-buffer');
const watchify = require('watchify');
const glob = require('glob');
const jasmineBrowser = require('gulp-jasmine-browser');
gulp.task('test', function() {
let testBundler = browserify({
entries: glob.sync('src/**/*-test.js'),
cache: {},
packageCache: {},
}).plugin(watchify);
function updateSpecs() {
return testBundler.bundle()
.pipe(vinylStream(jsBundleName))
.pipe(vinylBuffer())
.pipe(jasmineBrowser.specRunner({console: true}))
.pipe(jasmineBrowser.headless({driver: 'phantomjs'}));
}
testBundler.on('update', updateSpecs);
updateSpecs();
});
It bundles all my Jasmine specs using Browserify and has them tested through gulp-jasmine-browser. It also watches all specs and all modules that they depend on and re-runs the tests if any of these modules changes.
The only ugly bit, which I'd really like to see solved, is that a new PhantomJS instance and a new Jasmine server are created every time updateSpecs is run. I was hoping to avoid that with code like the following:
gulp.task('test', function() {
let testBundler = browserify({
entries: glob.sync('src/**/*-test.js'),
cache: {},
packageCache: {},
}).plugin(watchify);
// persist the Jasmine server and PhantomJS browser
let testServer = jasmineBrowser.headless({driver: 'phantomjs'});
function updateSpecs() {
return testBundler.bundle()
.pipe(vinylStream(jsBundleName))
.pipe(vinylBuffer())
.pipe(jasmineBrowser.specRunner({console: true}))
.pipe(testServer);
}
testBundler.on('update', updateSpecs);
updateSpecs();
});
Alas, this doesn't work. Right after starting the task, all tests run fine, but the next time updateSpecs is called, I get a write after end error and the task exits with status 1. This error originates from the readable-stream Node module.
As I understand it, the end event during the first run of updateSpecs leaves testServer in a state in which it doesn't accept any new inputs. Unfortunately, the Node.js streams documentation isn't very clear on how to remedy this.
I have tried breaking the pipe chain at a different place, but I got the same result, which seems to indicate this is universal behaviour for streams. I also tried stopping the end event from propagating by inserting a through-stream that didn't re-emit that event, but this prevented the tests from being run at all. Finally, I tried returning the testServer stream from the task; this stopped the error, but although the updateSpecs function gets called every time the sources change, the tests are only being run the first time the task starts. This time, the testServer simply seems to ignore the new input.
The gulp-jasmine-browser documentation suggests that the following code would work:
var watch = require('gulp-watch');
gulp.task('test', function() {
var filesForTest = ['src/**/*.js', 'spec/**/*-test.js'];
return gulp.src(filesForTest)
.pipe(watch(filesForTest))
.pipe(jasmineBrowser.specRunner())
.pipe(jasmineBrowser.server());
});
And it goes on to suggest that you can also make this work with Browserify, but this isn't illustrated. Apparently, gulp-watch does something which causes the follow-up pipes to accept updated inputs later. How can I imitate this behaviour with watchify?
GitHub issue
As it turns out, it is a hard rule in Node.js that you cannot write after the end event. In addition, jasmineBrowser.specRunner(), .server() and .headless() must receive the end signal in order to actually test anything. This restriction is inherited from the official Jasmine test runner.
The example with gulp-watch from the README doesn't actually work, either, for the same reason. In order to make it work, one would have to do something similar to the working version of my watchify code in the question:
gulp.task('test', function() {
var filesForTest = ['src/**/*.js', 'spec/**/*-test.js'];
function runTests() {
return gulp.src(filesForTest)
.pipe(jasmineBrowser.specRunner())
.pipe(jasmineBrowser.server());
}
watch(filesForTest).on('add change unlink', runTests);
});
(I didn't test it, but something very close to this should work.)
So whatever watching mechanism you're using, you'll always need to call .specRunner() and .server() again for every cycle. The good news is that apparently, the Jasmine server will be reused if you explicitly pass a port number:
.pipe(jasmineBrowser.server({port: 8080}));
this also applies to .headless().
I need to alter a stream of files to contain a different base folder name. I thought the gulp-rename plugin would allow for this, but it only seems to replace the glob portion.
Example:
gulp.task("test", function() {
gulp.src("bower_components/**/*", { base: "bower_components", read:false })
.pipe($.rename(function (p) { p.dirname = "X/" + p.dirname; }))
.pipe($.print());
});
outputs:
[gulp] bower_components\X\jquery\test\data\offset\scroll.html
[gulp] bower_components\X\jquery\test\data\offset\static.html
[gulp] bower_components\X\jquery\test\data\offset\table.html
...
I want
[gulp] X\jquery\test\data\offset\scroll.html
[gulp] X\jquery\test\data\offset\static.html
[gulp] X\jquery\test\data\offset\table.html
...
Is there a way to do this with gulp-replace, or some other plugin?
I believe you could do this with gulp-tap to get a hold of the the file instances and alter properties on them before they get printed or use it to print them.
Out of curiosity what are you aiming to do?
Hope that helps!
EDIT-1::
The following is a slightly modified version of the example in the gulp-tap documentation which may work for your use case.
gulp.src("src/**/*.{coffee,js}")
.pipe(tap(function(file, t) {
file.path = 'X/' + file.path;
}))
.pipe($.print())
.pipe(gulp.dest('build'));
EDIT-2::
This is a common task I have set up in my projects for handling external scripts (note; I am using gulp-load-plugins hence invoking my plugins with plugins.<NAME>);
gulp.task('vendor:scripts:publish', function() {
return gulp.src(sources.vendor.js)
.pipe(plugins.plumber())
.pipe(plugins.concat('vendor.js'))
.pipe(gulp.dest(destinations.js))
.pipe(plugins.uglify())
.pipe(plugins.rename(pluginOpts.rename))
.pipe(gulp.dest(destinations.js));
});
destinations and sources are two variables that I have defined in a config file for my gulpfile.
But for clarity, sources.vendor.js points at an array much like the following;
js: [
'src/vendor/jquery/dist/jquery.js',
'src/vendor/lodash/lodash.js',
'src/vendor/backbone/backbone.js'
],
The reason my folder is named vendor and not bower_components is because I've made use of a .bowerrc file to point my bower installation at a different folder.
In addition if you have discrete scripts that you may not want to include all of the time you can look to make use of gulp-utils and gulp-filter to filter out certain scripts when an option is passed or not passed when gulp is invoked on the CLI.
For example; having gulp vendor:scripts:publish include all scripts but gulp vendor:scripts:publish --release omitting discrete scripts.
This then requires modifying your task to declare a filter that is piped in based on an option flag being picked up by gulp-utils.
var isRelease = (plugins.utils.env.release) ? true: false;
gulp.task('vendor:scripts:publish', function() {
var discreteFilter = plugins.filter([
'**/*.js',
'!**/discrete.min.js'
]);
return gulp.src(sources.vendor.js)
.pipe(plugins.plumber())
.pipe(isRelease ? discreteFilter: plugins.utils.noop())
.pipe(plugins.concat('vendor.js'))
.pipe(gulp.dest(destinations.js))
.pipe(plugins.uglify())
.pipe(plugins.rename(pluginOpts.rename))
.pipe(gulp.dest(destinations.js));
});
Hope that helps you out!
The below code seems to work just fine until I go to 1ocalhost:8081...
then I get the message..
<pre>{"tinylr":"Welcome","version":"0.0.5"}</pre>
My directory structure is....
____gulp
| |____build
| | |____images
| | |____index.html
| | |____scripts
| | |____styles
| |____gulpfile.js
| |____node_modules
| |____src
| | |____images
| | |____index.html
| | |____scripts
| | |____styles
Why isn't my html page loading? If I try to browse to
1ocalhost:8081/build/index.html The page wont load and I get the msg
{"error":"not_found","reason":"no such route"}
I've also tried the chrome plugin but I get the below msg when I hit the plugin Could not connect to LiveReload server. Please make sure that LiveReload 2.3 (or later) or another compatible server is running.
I checked the plugin settings from the plugin in Chrome and check the option for file urls
Heres my commented code.....
//sudo npm install gulp -g
// install chrome extension from https://chrome.google.com/webstore/detail/livereload/jnihajbhpnppcggbcgedagnkighmdlei
//Go into the settings from the plugin in Chrome and check the option for file urls: chrome://extensions/
// include gulp
var gulp = require('gulp');
// include plug-ins
var jshint = require('gulp-jshint');
var changed = require('gulp-changed');
var imagemin = require('gulp-imagemin');
var minifyHTML = require('gulp-minify-html');
var concat = require('gulp-concat');
var stripDebug = require('gulp-strip-debug');
var uglify = require('gulp-uglify');
var autoprefix = require('gulp-autoprefixer');
var minifyCSS = require('gulp-minify-css');
var livereload = require('gulp-livereload');
var lr = require('tiny-lr');
var server = lr();
// JS hint task
gulp.task('jshint', function() {
gulp.src('./src/scripts/*.js')
.pipe(jshint())
.pipe(jshint.reporter('default'))
.pipe(livereload(server));
});
// minify new images
gulp.task('imagemin', function() {
var imgSrc = './src/images/**/*',
imgDst = './build/images';
gulp.src(imgSrc)
.pipe(changed(imgDst))
.pipe(imagemin())
.pipe(gulp.dest(imgDst))
.pipe(livereload(server));
});
// minify new or changed HTML pages
gulp.task('htmlpage', function() {
var htmlSrc = './src/*.html',
htmlDst = './build';
gulp.src(htmlSrc)
.pipe(changed(htmlDst))
.pipe(minifyHTML())
.pipe(gulp.dest(htmlDst))
.pipe(livereload(server));
});
// JS concat, strip debugging and minify
gulp.task('scripts', function() {
gulp.src(['./src/scripts/lib.js','./src/scripts/*.js'])
.pipe(concat('script.js'))
.pipe(stripDebug())
.pipe(uglify())
.pipe(gulp.dest('./build/scripts/'))
.pipe(livereload(server));
});
// CSS concat, auto-prefix and minify
gulp.task('styles', function() {
gulp.src(['./src/styles/*.css'])
.pipe(concat('styles.css'))
.pipe(autoprefix('last 2 versions'))
.pipe(minifyCSS())
.pipe(gulp.dest('./build/styles/'))
.pipe(livereload(server));
});
// default gulp task
gulp.task('default', ['imagemin', 'htmlpage', 'scripts', 'styles'], function() {
server.listen(8081, function (err) { if (err) return console.log(err);
// watch for HTML changes
gulp.watch('./src/*.html', function() {
gulp.run('htmlpage');
});
// watch for JS changes
gulp.watch('./src/scripts/*.js', function() {
gulp.run('jshint', 'scripts');
});
// watch for IMG changes
gulp.watch('./src/images/*.png', function() {
gulp.run('imagemin');
});
// watch for CSS changes
gulp.watch('./src/styles/*.css', function() {
gulp.run('styles');
});
});
});
</pre>
And the output from gulp looks good...
Bills-MacBook-Pro:gulp Bill$ gulp
[gulp] Using file /Users/Bill/gulp/gulpfile.js
[gulp] Working directory changed to /Users/Bill/gulp
[gulp] Running 'imagemin'...
[gulp] Finished 'imagemin' in 77 ms
[gulp] Running 'htmlpage'...
[gulp] Finished 'htmlpage' in 2.47 ms
[gulp] Running 'scripts'...
[gulp] Finished 'scripts' in 4.05 ms
[gulp] Running 'styles'...
[gulp] Finished 'styles' in 1.09 ms
[gulp] Running 'default'...
[gulp] Finished 'default' in 1.38 ms
gulp.run() has been deprecated. Use task dependencies or gulp.watch task triggering instead.
[gulp] Running 'htmlpage'...
[gulp] Finished 'htmlpage' in 3.5 ms
[gulp] index.html was reloaded.
[gulp] Running 'htmlpage'...
[gulp] Finished 'htmlpage' in 712 μs
[gulp] Running 'htmlpage'...
[gulp] Finished 'htmlpage' in 1.05 ms
[gulp] index.html was reloaded.
That's not how livereload works. It doesn't run a server to load your content — it runs a separate server to notify when content changes.
When you enable livereload*, a small javascript is embedded in your page which listens to the LR server. When you notify the server that a resource was modified, it tells any listeners, which in turn reload the resource from where ever they originally loaded the resource.
If your webapp/site/page is entirely self contained, you can simply open the file:// url to the page you want in your browser, enable livereload, and it should work.
However, if you are dealing with external resources, you should fire up a server of some sort. There's far too many ways for me to select one for you, but you can use connect, express, or some other node library, you could run python -m SimpleHTTPServer in your directory if you have python installed, etc.
If you want to integrate a connect server into your build process, I have a recipe at the bottom of this article.
* You can enable livereload via a browser plugin or using the gulp-embedlr plugin during development, which I prefer since it works across multiple browsers and devices.
1ocalhost:8081 or localhost:8081 ? Perhaps a spelling error on the first letter.
I'm trying to make my tests run each time I'm saving some files. Here is the gulp watch:
gulp.task('jasmine', function() {
gulp.src('spec/nodejs/*Spec.js')
.pipe(jasmine({verbose:true, includeStackTrace: true}));
});
gulp.task('watch', function () {
gulp.watch(['app/*.js', 'app/!(embed)**/*.js','spec/nodejs/*.js'], ['jasmine']);
});
To test for example app/maps.js I'm creating a spec/nodejs/mapsSpec.js file like this:
'use strict';
var maps = require('../../app/maps');
describe('/maps related routes', function(){
it('should ...', function(){...}
...
If I change a spec file everything is working well, if I modify app/maps.js file the change trigger the test. if I modify it again tests are tiggered but the modifications do not taking effect. For example if I add a console.log('foo') in a second time, I will not see it until I relaunch gulp watch and save it again. So only one run of jasmine is ok when using it with gulp.watch.
I guess it's because require is cached by nodejs in the gulp process. So how should I do ?
I took a look at the code of gulp-jasmine. The problem is that the only file from the cache is the Specs.js file. The cache of the children(the reqquired files to test) aren't cleared.
Within the index.js of gulp-jasmine is a row which deletes the cache:
delete require.cache[require.resolve(path.resolve(file.path))];
If you put the next block of code before the delete, you will delete all the children's cache and will it run correctly after every time you save your file.
var files = require.cache[require.resolve(path.resolve(file.path))];
if( typeof files !== 'undefined' ) {
for( var i in files.children ) {
delete require.cache[ files.children[i].id ];
}
}
You can change this in the node_modules.
I will go for a pull request, so maybe in the near future this will be solved permanently.
Also wrote a post about it on: http://navelpluisje.nl/entry/fix-cache-problem-jasmine-tests-with-gulp
I haven't found a fix for this issue, but you can work around it via the gulp-shell task.
npm install gulp-shell --save-dev
then
var shell = require('gulp-shell');
...
gulp.task('jasmine', function() {
gulp.src('spec/nodejs/*Spec.js')
.pipe(shell('minijasminenode spec/*Spec.js'));
});
You'll also need jasmine installed as a direct dependency (gulp-jasmine uses minijasminenode)
I'm writing a node.js program that will watch a directory filled with a large (300-ish) amount of scss projects. Grunt-watch (run either through the node module or on its own, whatever works) will be configured so that whenever a scss file is changed, it will be compiled with compass, the output file moved to a separate directory, for example:
./1234/style.scss was changed >> grunt-watch runs grunt-compass >> /foo/bar/baz/1234/style.css updated
The project directory that the file was in is obviously very important (if grunt-compass sent all the compiled files to the same directory, they would be jumbled and unusable and the grunt automation would be purposeless). I order to make sure all files are routed to the correct place, I am dynamically changing the grunt-compass settings every time a css file is updated.
Sample gruntfile:
module.exports = function(grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
watch: {
files: './*/*.scss',
tasks: ['compass']
},
compass: {
origin:{
options: {
//temportary settings to be changed later
sassDir: './',
cssDir: './bar',
specify: './foo.scss'
}
}
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-compass');
grunt.event.on('watch', function(action, filepath, target) {
var path = require('path');
grunt.log.writeln(target + ': ' + filepath + ' might have ' + action);
var siteDirectory = path.dirname(filepath);
//changes sass directory to that of the changed file
var option = 'compass.origin.options.sassDir';
var result = __dirname + '/' + siteDirectory;
grunt.log.writeln(option + ' changed to ' + result);
grunt.config(option, result);
//customizes css output directory so that file goes to correct place
option = 'compass.origin.options.cssDir';
result = path.resolve(__dirname, '../', siteDirectory);
grunt.log.writeln(option + ' changed to ' + result);
grunt.config(option, result);
//grunt.task.run(['compass']);
});
};
However this doesn't work. If you run 'grunt watch' in verbose mode, you will see that grunt runs both the grunt.event.on function and the watch task in separate processes. The second parsing of the gruntfile reverts all my event.on config changes to the defaults above, and compass fails to run.
As seen in the event.on comments, I attempted to add a grunt.task.run() to make sure that compass was run in the same process as the event.on function, which would preserve my config changes. However the task refused to run, likely because I'm doing it wrong.
Unfortunately, the grunt.event.on variables are not sent to the defined grunt-watch task, otherwise I could write a custom function that would change the compass settings and then run compass in the same process.
I've tried implementing this without grunt, using the watch function build into compass, however compass can only store one static output path per project and can only watch one project at once.
I have currently gotten around this issue by adding a node program that takes the site name as a parameter, rewrites the grunfile.js by running using fs, and then running 'grunt watch' via an exec function. This however has it's own drawbacks (I can't view the grunt.log data) and is horribly convoluted, so I'd like to change it.
Thank you so much for any insight.
You need to specify
options : { nospawn : true }
in your watch task config to have the watch run in the same context:
watch: {
files: './*/*.scss',
tasks: ['compass'],
options : { nospawn : true }
}
See this section of documentation for more info on this.