gulp.dest() does not result in file updates - node.js

Given the following dir structure:
<project>
|- src
|- gen
|- target
We have a Gulp build chain in place which performs the entire front-end build for us. Intermediate results are put in the gen directory and the final results are put in the target directory.
During development, we want to monitor the target directory and synchronize changes to a separate directory that contains our Grails based application. We use the following snippet for this:
'use strict';
var debug = require('gulp-debug');
var config = require('../config');
var gulp = require('gulp');
var watch = require('gulp-watch');
module.exports = {
//command: 'prod',
//desc: 'Builds "production" version',
run: function(){
gulp.task('watch', ['server'], function() {
// Copy files from the 'target' directory to the framework directory while developing
var source = config.dist.root,
destination = config.fw.root;
gulp.src(source + '/**/*', {base: source})
.pipe(debug({title: 'unicorn:'}))
.pipe(watch(source + '/**/*', {base: source}))
.pipe(debug({title: 'minotaur:'}))
.pipe(gulp.dest(destination))
.pipe(debug({title: 'centaur:'}));
});
}
};
When I update a source file, the build chain fires and puts the updated results in the target directory. But the updates are NOT synced to separate Grails directory. When I check the logging, I see this:
[14:29:42] Rebundle...
[14:29:42] minotaur: target\web-app\portal\js\appLoader.js
[14:29:43] minotaur: target\web-app\portal\js\appLoader.js
It seems that the file IS regenerated in the target directory and the regeneration IS picked up by the gulp-watch package. But the file is NOT written by the gulp.dest() function?!
What's might be going on here?

After some trial and error, it seems that you cannot use watch() in the middle of your pipeline. Instead, you should use it as the head of your pipeline (instead of gulp.src()). Splitting the single pipeline into two separate pipelines solved the issue.
So this (removed debug() statements for brevity):
gulp.src(source + '/**/*', {base: source})
.pipe(watch(source + '/**/*', {base: source}))
.pipe(gulp.dest(destination));
Becomes this:
gulp.src(source + '/**/*', {base: source})
.pipe(gulp.dest(destination));
watch(source + '/**/*', {base: source})
.pipe(gulp.dest(destination));

Related

Node.js Gulp no outputfile created

I have a Gulp script to concatenate, and minimize javascript.
It seems to be working but doesn't output the combined file.
The script is (complete - including extra debug bits):
// include plug-ins
var fs = require('fs');
var gulp = require('gulp');
var count = require('gulp-count');
var debug = require('gulp-debug');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var del = require('del');
var config = {
src: 'dist/libraries/',
dest: 'dist/js/',
outputfile: 'libraries.min.js'
}
gulp.task('read', (done) => {
fs.readdir(config.src, (err, items) => {
console.log(items);
});
done();
});
//delete the output file(s)
gulp.task('clean', gulp.series('read'), (done) => {
//del is an async function and not a gulp plugin (just standard nodejs)
//It returns a promise, so make sure you return that from this task function
// so gulp knows when the delete is complete
return del([config.dest + config.outputfile]);
});
// Combine and minify all files from the app folder
// This tasks depends on the clean task which means gulp will ensure that the
// Clean task is completed before running the scripts task.
gulp.task('scripts', gulp.series('clean'), (done) => {
//Include all js files but exclude any min.js files
var files = [config.src + '*.js', '!' + config.src + '*.min.js'];
return gulp.src(files)
.pipe(debug())
.pipe(count('## files selected'))
.pipe(uglify())
.pipe(concat(config.outputfile))
.pipe(gulp.dest(config.dest));
});
//Set a default tasks
gulp.task('default', gulp.series('scripts'), (done) => {
});
Which produces the output - including file list for verification there are src files:
[07:46:25] Using gulpfile <path>\gulpfile.js
[07:46:25] Starting 'default'...
[07:46:25] Starting 'scripts'...
[07:46:25] Starting 'clean'...
[07:46:25] Starting 'read'...
[07:46:25] Finished 'read' after 996 μs
[07:46:25] Finished 'clean' after 2.73 ms
[07:46:25] Finished 'scripts' after 4.26 ms
[07:46:25] Finished 'default' after 6.9 ms
[ 'bootstrap-datetimepicker.js',
'bootstrap.min.js',
'chart.min.js',
'cycle.js',
'farbtastic.js',
'jquery-3.2.1.min.js',
'jquery-sortable-min.js',
'moment.min.js',
'ol.min.js',
'pablo.min.js',
'popper.min.js',
'proj4.js',
'promisedIndexDB.js',
'qunit-2.6.1.js',
'toastr.js' ]
If I create an empty file, at dist/js/libraries.min.js it isn't deleted as part of the gulp tasks, however if i move the call to del() outside the gulp tasks it is deleted, so that leads me to assume that its not as simple as a permissions issue, or path issues.
Any idea what I've done wrong?
PS: its on a windows box, running in an admin cmd window.
You were using the wrong signature for the task. The correct one is :
task([taskName], taskFunction)
see task signature
But your tasks look like this:
gulp.task('scripts', gulp.series('clean'), (done) => { // 3 parameters
Merely changing that to:
gulp.task('scripts', gulp.series('clean', (done) => {
...
}));
makes it work - I tested it. So now that task has only two parameters: a task name and a function. Yours had a task name plus two functions.
You would also need to change your default and clean tasks to this proper signature. Also you should call done() at the end of the task as you did with your cb().
Your new code uses task functions, which are better than named tasks for a number of reasons - but now you know what was wrong with your original code. The main body of your scripts task was never being run.
I never worked out what was wrong, but went direct to the doc's and started again (previous version was from a example)..
Works with the below (much simpler) script.
// // include plug-ins
var gulp = require('gulp');
var count = require('gulp-count');
var debug = require('gulp-debug');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var del = require('del');
var config = {
src: 'jspa-scada/dist/libraries/',
dest: 'jspa-scada/dist/js/',
outputfile: 'libraries.min.js'
}
function defaultTask(cb) {
del([config.dest + config.outputfile]);
// Include all js files but exclude any min.js files
var globs = [
config.src + '*.js',
'!' + config.src + '*.min.js'
];
return gulp.src(globs)
.pipe(debug())
.pipe(count('## files selected'))
.pipe(uglify())
.pipe(concat(config.outputfile))
.pipe(gulp.dest(config.dest));
cb();
}
exports.default = defaultTask

Ionic / bower / cordova - ignore files for build

My project structure is the following:
MyApp
- hooks
- platforms
- android
- ios
- www
- js / css / templates..
- lib (including all bower components)
Right now, the www/lib directory is taking up 21,8 Mb. (I have a large set of bower components added to my project.)
When building each project, the entire www folder is copied to the platform/android (for instance) folder for build, including of course www/lib.
This leads to a very big build, as lots of files included into bower
components are useless for production.
Manually managing all bower dependencies is clearly not an option. So how do you guys manage to clean your project platform directory for build?
I was thinking about creating a hook for that but before writing lines of code in a language that i do not know (nodeJS), I was hoping for your return and advises.
According to Cordova workflow you can add a hook script that removes unnecessary files.
A detailed example of a cleanup script can be found here: https://www.thepolyglotdeveloper.com/2015/01/hooks-apache-cordova-mobile-applications/
But to give a quick step by step summary:
Add to the after_prepare hook folder (/hooks/after_prepare) a script (01_junk_cleanup.js - 01 to be run first, the rest whatever you want) and in the file specify the files and folders you want to delete. For example, here is how you can delete a test folder and relevant files just change to you lib directory and to the files there. Note that this example is a bit different from the example in the link i gave earlier so you might want to take a look there as well.
01_junk_cleanup.js:
#!/usr/bin/env node
var fs = require('fs');
var path = require('path');
var foldersToProcess = [
"js",
"css"
];
var foldersToDelete = [
"test"
];
var filesToDelete = [
"karmaOnBrowser.conf.js",
"karmaOnEmulators.conf.js",
"SpecRunner.html"
];
var iosPlatformsDir = "platforms/ios/www/";
var androidPlatformsDir = "platforms/android/assets/www/";
filesToDelete.forEach(function(file) {
var filePathIOS = iosPlatformsDir + file;
var filePathAndroid = androidPlatformsDir + file;
if(fs.existsSync(filePathIOS)){
fs.unlinkSync(filePathIOS);
};
if(fs.existsSync(filePathAndroid)){
fs.unlinkSync(filePathAndroid);
};
});
foldersToProcess.forEach(function(folder) {
processFiles(iosPlatformsDir + folder);
processFiles(androidPlatformsDir + folder);
});
foldersToDelete.forEach(function(folder) {
deleteFolderRecursive(iosPlatformsDir + folder);
deleteFolderRecursive(androidPlatformsDir + folder);
});
function deleteFolderRecursive(path){
if( fs.existsSync(path) ) {
fs.readdirSync(path).forEach(function(file,index){
var curPath = path + "/" + file;
if(fs.lstatSync(curPath).isDirectory()) { // recurse
deleteFolderRecursive(curPath);
} else { // delete file
fs.unlinkSync(curPath);
}
});
fs.rmdirSync(path);
}
}
function processFiles(dir) {
fs.readdir(dir, function(err, list) {
if(err) {
console.log('processFiles err: ' + err);
return;
}
list.forEach(function(file) {
file = dir + '/' + file;
fs.stat(file, function(err, stat) {
if(!stat.isDirectory()) {
switch(path.basename(file)) {
case ".DS_Store":
fs.unlink(file, function(error) {
console.log("Removed file " + file);
});
break;
case "Thumbs.db":
fs.unlink(file, function(error) {
console.log("Removed file " + file);
});
break;
default:
console.log("Skipping file " + file);
break;
}
}
});
});
});
}
Aside to above, A bit more obvious but I feel worth mentioning anyhow, After having the www/lib bloat as well I always try to keep the folder lean and add only libraries required for deployment, the other dev. dependencies such as jasmine I either hold in the 'node_modules' folder or 'bower_components' as I only install today through them.
Hope this helps,
Good luck
I think the best approach would be to do this:
Move the bower_components folder and your index.html file to the project root, outside the /www folder
Install gulp and gulp-usemin
Wrap all of the .js files and .css files from bower components in usemin <build:js> and <build:css> sections
Configure a task in your gulpfile to concatenate all those files into a lib.js and a lib.css file. Make sure that those two files as well as the rewritten index.html are output to the /www folder
Execute the gulp task before your next build, and each time you add a new bower component.
This will keep your /www folder tidy and only containing the files you need in your cordova build.
With Bower you need to use npm preen to remove unnecessary files
See my example using Gulp with Ionic Framework: https://github.com/jdnichollsc/Ionic-Starter-Template
Basically you can set your bower.json file to indicate the path which files you need, for example:
"preen": {
//... More libraries
"ionic-datepicker": [
"dist/*.js" //You only need these files
//Other files and folders will be deleted to reduce the size of your app
],
"ion-floating-menu": [
"dist/*" //Keep all the files (.html, .css, .js, etc) of the directory.
]
}
Regards, Nicholls
This is an improvement over this answer. I've applied it to my own project.
Move the bower_components folder to the project root, outside the www folder.
Rename index.html to _index.html. We will later make sure that Gulp automatically generates index.html.
Install gulp and gulp-useref.
Edit your _index.html so that it looks something like this:
<!-- build:js dist/js/vendor.js -->
<script src="../bower_components/ionic/release/js/ionic.bundle.min.js"></script>
<script src="../bower_components/ngstorage/ngStorage.min.js"></script>
<script src="../bower_components/ngCordova/dist/ng-cordova.min.js"></script>
<!-- endbuild -->
Configure your gulp watch task to build new index.html file in the www folder with the concatenated files.
var entrypoint = './www/_index.html';
gulp.task('watch', function() {
gulp.watch(entrypoint, ['concat-js-and-css']);
});
gulp.task('concat-js-and-css', function () {
return gulp.src(entrypoint)
.pipe(useref())
.pipe(rename(function (path) {
// rename _index.html to index.html
if (path.basename == '_index' && path.extname == '.html') {
path.basename = "index";
}
}))
.pipe(gulp.dest('./www'))
});
gulp.task('build', ['concat-js-and-css']);
When that task runs, your index.html file will contain just this:
<script src="dist/js/vendor.js"></script>
Edit your ionic.project file so that it looks like the following. This will make sure that gulp watch is run before ionic serve.
{
"watchPatterns": [
"www/_index.html",
],
"gulpStartupTasks": [
"watch"
]
}

Browserify + Remapify (Gulp workflow)

I've successfully got Browserify to compile my JavaScript entry files, but I want to utilise the Remapify plugin so as to not have to specify the full relative path upon requiring a module every time.
For example:
require('components/tabs.js')
Rather than:
require('../../components/tabs/tabs.js').
But I cannot get the shorter module references to map to the corresponding file... "Error: Cannot find module [specified_ref] from [file]".
Have I misconfigured Remapify, or is there something wrong with my wider Browserify setup? I am new to Broswerify and Gulp having previously used Require.js and Grunt. Any help would be greatly appreciated. Please let me know if you need any more information about my setup.
If alternatively you can recommend an alternative Gulp task file that will do all of this, thereby throwing my current task out the window, by all means. I wasn't able to find many Browserify + Remapify examples.
Directory Structure
I have my modules (components) in the following directory: './src/components', so for example: './src/components/tabs/tabs.js'.
I am requiring these modules in a JS file for a given page of the app, which are in: './src/pages', so for example, './src/pages/portfolio/portfolio.js'.
Gulp Browserify Task
var gulp = require('gulp');
var config = require('../config');
var browserify = require('browserify');
var remapify = require('remapify');
var source = require('vinyl-source-stream');
var glob = require('glob');
var browserSync = require('browser-sync');
gulp.task('browserify', function(){
var entries = glob.sync(config.src.pages + '/**/*.js');
return browserify({
entries: entries,
debug: true
})
// (Remapify:)
.plugin(remapify, [{ src: config.src.components + '/**/*.js', expose: 'components', cwd: config.srcDir }])
.bundle()
.pipe(source('app.js'))
.pipe(gulp.dest(config.build.js))
.pipe(browserSync.reload({ stream: true }));
});
Page.js
'use strict';
var tabs = require('components/tabs.js'); // (Doesn't work, but I want it to)
// var tabs = require('../../components/tabs/tabs.js'); // (Does work)
Remapify has all sorts of problems. I suggest giving my pathmodify plugin a shot.
For your situation usage would look something like:
var pathmod = require('pathmodify');
// ...
.plugin(pathmod(), {mods: [
pathmod.mod.dir('components', '/path/to/src/components'),
]})

Gulp, livereload, jade

Need help.
I use gulp-conect and it livereload method. But if I build a few template in time, get a lot of page refresh. Is any solution, I want to build few templates with single page refresh?
So, I reproduce the problem you have and came accross this working solution.
First, lets check gulp plugins you need:
gulp-jade
gulp-livereload
optional: gulp-load-plugins
In case you need some of them go to:
http://gulpjs.com/plugins/
Search for them and install them.
Strategy: I created a gulp task called live that will check your *.jade files, and as you are working on a certain file & saving it, gulp will compile it into html and refresh the browser.
In order to accomplish that, we define a function called compileAndRefresh that will take the file returned by the watcher. It will compile that file into html and the refesh the browser (test with livereload plugin for chrome).
Notes:
I always use gulp-load-plugin to load plugins, so thats whay I use plugins.jad and plugins.livereload.
This will only compile files that are saved and while you have the task live exucting on the command line. Will not compile other files that are not in use. In order to accomplish that, you need to define a task that compiles all files, not only the ones that have been changed.
Assume .jade files in /jade and html output to /html
So, here is the gulpfile.js:
var gulp = require('gulp'),
gulpLoadPlugins = require('gulp-load-plugins'),
plugins = gulpLoadPlugins();
gulp.task('webserver', function() {
gulp.src('./html')
.pipe(plugins.webserver({
livereload: true
}));
gulp.watch('./jade/*.jade', function(event) {
compileAndRefresh(event.path);
});
});
function compileAndRefresh(file) {
gulp.src(file)
.pipe(plugins.jade({
}))
.pipe(gulp.dest('./html'))
}
Post edit notes:
Removed liveReload call from compileAndRefresh (webserver will do that).
Use gulp-server plugin insted of gulp-connect, as they suggest on their repository: "New plugin based on connect 3 using the gulp.src() API. Written in plain javascript. https://github.com/schickling/gulp-webserver"
Something you can do is to watch only files that changes, and then apply a function only to those files that have been changed, something like this:
gulp.task('live', function() {
gulp.watch('templates/folder', function(event) {
refresh_templates(event.path);
});
});
function refresh_templates(file) {
return
gulp.src(file)
.pipe(plugins.embedlr())
.pipe(plugins.livereload());
}
PS: this is not a working example, and I dont know if you are using embedlr, but the point, is that you can watch, and use a callback to call another function with the files that are changing, and the manipulate only those files. Also, I supposed that your goal is to refresh the templates for your browser, but you manipulate as you like, save them on dest or do whatever you want.
Key point here is to show how to manipulate file that changes: callback of watch + custom function.
var jadeTask = function(path) {
path = path || loc.jade + '/*.jade';
if (/source/.test(path)) {
path = loc.jade + '/**/*.jade';
}
return gulp.src(path)
.pipe(changed(loc.markup, {extension: '.html'}))
.pipe(jade({
locals : json_array,
pretty : true
}))
.pipe(gulp.dest(loc.markup))
.pipe(connect.reload());
}
First install required plugins
gulp
express
gulp-jade
connect-livereload
tiny-lr
connect
then write the code
var gulp = require('gulp');
var express = require('express');
var path = require('path');
var connect = require("connect");
var jade = require('gulp-jade');
var app = express();
gulp.task('express', function() {
app.use(require('connect-livereload')({port: 8002}));
app.use(express.static(path.join(__dirname, '/dist')));
app.listen(8000);
});
var tinylr;
gulp.task('livereload', function() {
tinylr = require('tiny-lr')();
tinylr.listen(8002);
});
function notifyLiveReload(event) {
var fileName = require('path').relative(__dirname, event.path);
tinylr.changed({
body: {
files: [fileName]
}
});
}
gulp.task('jade', function(){
gulp.src('src/*.jade')
.pipe(jade())
.pipe(gulp.dest('dist'))
});
gulp.task('watch', function() {
gulp.watch('dist/*.html', notifyLiveReload);
gulp.watch('src/*.jade', ['jade']);
});
gulp.task('default', ['livereload', 'express', 'watch', 'jade'], function() {
});
find the example here at GitHub

How to modify grunt watch tasks based on the file changed?

I'm writing a node.js program that will watch a directory filled with a large (300-ish) amount of scss projects. Grunt-watch (run either through the node module or on its own, whatever works) will be configured so that whenever a scss file is changed, it will be compiled with compass, the output file moved to a separate directory, for example:
./1234/style.scss was changed >> grunt-watch runs grunt-compass >> /foo/bar/baz/1234/style.css updated
The project directory that the file was in is obviously very important (if grunt-compass sent all the compiled files to the same directory, they would be jumbled and unusable and the grunt automation would be purposeless). I order to make sure all files are routed to the correct place, I am dynamically changing the grunt-compass settings every time a css file is updated.
Sample gruntfile:
module.exports = function(grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
watch: {
files: './*/*.scss',
tasks: ['compass']
},
compass: {
origin:{
options: {
//temportary settings to be changed later
sassDir: './',
cssDir: './bar',
specify: './foo.scss'
}
}
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-compass');
grunt.event.on('watch', function(action, filepath, target) {
var path = require('path');
grunt.log.writeln(target + ': ' + filepath + ' might have ' + action);
var siteDirectory = path.dirname(filepath);
//changes sass directory to that of the changed file
var option = 'compass.origin.options.sassDir';
var result = __dirname + '/' + siteDirectory;
grunt.log.writeln(option + ' changed to ' + result);
grunt.config(option, result);
//customizes css output directory so that file goes to correct place
option = 'compass.origin.options.cssDir';
result = path.resolve(__dirname, '../', siteDirectory);
grunt.log.writeln(option + ' changed to ' + result);
grunt.config(option, result);
//grunt.task.run(['compass']);
});
};
However this doesn't work. If you run 'grunt watch' in verbose mode, you will see that grunt runs both the grunt.event.on function and the watch task in separate processes. The second parsing of the gruntfile reverts all my event.on config changes to the defaults above, and compass fails to run.
As seen in the event.on comments, I attempted to add a grunt.task.run() to make sure that compass was run in the same process as the event.on function, which would preserve my config changes. However the task refused to run, likely because I'm doing it wrong.
Unfortunately, the grunt.event.on variables are not sent to the defined grunt-watch task, otherwise I could write a custom function that would change the compass settings and then run compass in the same process.
I've tried implementing this without grunt, using the watch function build into compass, however compass can only store one static output path per project and can only watch one project at once.
I have currently gotten around this issue by adding a node program that takes the site name as a parameter, rewrites the grunfile.js by running using fs, and then running 'grunt watch' via an exec function. This however has it's own drawbacks (I can't view the grunt.log data) and is horribly convoluted, so I'd like to change it.
Thank you so much for any insight.
You need to specify
options : { nospawn : true }
in your watch task config to have the watch run in the same context:
watch: {
files: './*/*.scss',
tasks: ['compass'],
options : { nospawn : true }
}
See this section of documentation for more info on this.

Resources