Looking to use Gulp to create individual distributions of standalone HTML files - node.js

Basically I'm looking for a Gulp plugin to turn a directory like this:
/app
- htmlfile1.html
- htmlfile2.html
- htmlfile3.html
- /css
-cssmain.css
-/js
-global.js
And turn that into this:
/dist
-/htmlfile1
- htmlfile1.html
- /css
-cssmain.css
-/js
-global.js
- /htmlfile2
- htmlfile2.html
- /css
-cssmain.css
-/js
-global.js
- /htmlfile3
- htmlfile3.html
- /css
-cssmain.css
-/js
-global.js
Any thoughts on how to do accomplish a build system like this?

The code allows common files to be added to every page distribution as well as unique dependencies defined as an array in the pages object.
The following Gulp file relies on gulp-foreach, parse-filepath, and event-stream: npm install gulp gulp-foreach parse-filepath event-stream --save-dev
gulpfile.js:
// Command:
// npm install gulp gulp-foreach parse-filepath event-stream --save-dev
// Include gulp
var gulp = require('gulp');
var foreach = require('gulp-foreach'); // https://www.npmjs.org/package/gulp-foreach
var parsePath = require('parse-filepath'); // https://www.npmjs.org/package/parse-filepath
var es = require('event-stream'); // https://www.npmjs.org/package/event-stream
// The pages that each make a distribution
// Unique dependencies are defined as an array value for each page.
var pages = {
'./app/htmlfile1.html': [
'./app/images/1.png',
'./app/images/1-another.png',
],
'./app/htmlfile2.html': [],
'./app/htmlfile3.html': []
};
// Files added to each page distribution
var common = [
'./app/css/cssmain.css',
'./app/js/global.js',
];
function makeDistributionStream(page)
{
var gulpStream = gulp.src(page)
.pipe(foreach(function(stream, file) {
var pathParts = parsePath(file.path);
// Assemble the distribution path
var destinationPath = './dist/' + pathParts.name + '/';
// Pipe the html into the distribution folder
stream.pipe(gulp.dest(destinationPath));
// Move all of the unique and common files into the distibution
var uniqueDependencies = pages[page];
// Merge the common files to the unique ones
var distFiles = uniqueDependencies.concat(common);
gulp.src(distFiles, {base: './app/'})
.pipe(gulp.dest(destinationPath));
}));
return gulpStream;
}
// Assemble the distribution directories for each page
gulp.task('make-distributions', function() {
var mergedStream = null;
for(var page in pages)
{
var stream = makeDistributionStream(page);
// Merge the streams, if there is already one
if(mergedStream)
{
mergedStream = es.merge(mergedStream, stream);
}
// Otherwise, just make it this one
else
{
mergedStream = stream;
}
}
return mergedStream;
});
// Rerun the task when a file changes
gulp.task('watch', function() {
// If the html pages change, re-make the distributions
gulp.watch(Object.keys(pages), ['make-distributions']);
});
// Default Task
gulp.task('default', ['make-distributions', 'watch']);

Related

Node.js Gulp no outputfile created

I have a Gulp script to concatenate, and minimize javascript.
It seems to be working but doesn't output the combined file.
The script is (complete - including extra debug bits):
// include plug-ins
var fs = require('fs');
var gulp = require('gulp');
var count = require('gulp-count');
var debug = require('gulp-debug');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var del = require('del');
var config = {
src: 'dist/libraries/',
dest: 'dist/js/',
outputfile: 'libraries.min.js'
}
gulp.task('read', (done) => {
fs.readdir(config.src, (err, items) => {
console.log(items);
});
done();
});
//delete the output file(s)
gulp.task('clean', gulp.series('read'), (done) => {
//del is an async function and not a gulp plugin (just standard nodejs)
//It returns a promise, so make sure you return that from this task function
// so gulp knows when the delete is complete
return del([config.dest + config.outputfile]);
});
// Combine and minify all files from the app folder
// This tasks depends on the clean task which means gulp will ensure that the
// Clean task is completed before running the scripts task.
gulp.task('scripts', gulp.series('clean'), (done) => {
//Include all js files but exclude any min.js files
var files = [config.src + '*.js', '!' + config.src + '*.min.js'];
return gulp.src(files)
.pipe(debug())
.pipe(count('## files selected'))
.pipe(uglify())
.pipe(concat(config.outputfile))
.pipe(gulp.dest(config.dest));
});
//Set a default tasks
gulp.task('default', gulp.series('scripts'), (done) => {
});
Which produces the output - including file list for verification there are src files:
[07:46:25] Using gulpfile <path>\gulpfile.js
[07:46:25] Starting 'default'...
[07:46:25] Starting 'scripts'...
[07:46:25] Starting 'clean'...
[07:46:25] Starting 'read'...
[07:46:25] Finished 'read' after 996 μs
[07:46:25] Finished 'clean' after 2.73 ms
[07:46:25] Finished 'scripts' after 4.26 ms
[07:46:25] Finished 'default' after 6.9 ms
[ 'bootstrap-datetimepicker.js',
'bootstrap.min.js',
'chart.min.js',
'cycle.js',
'farbtastic.js',
'jquery-3.2.1.min.js',
'jquery-sortable-min.js',
'moment.min.js',
'ol.min.js',
'pablo.min.js',
'popper.min.js',
'proj4.js',
'promisedIndexDB.js',
'qunit-2.6.1.js',
'toastr.js' ]
If I create an empty file, at dist/js/libraries.min.js it isn't deleted as part of the gulp tasks, however if i move the call to del() outside the gulp tasks it is deleted, so that leads me to assume that its not as simple as a permissions issue, or path issues.
Any idea what I've done wrong?
PS: its on a windows box, running in an admin cmd window.
You were using the wrong signature for the task. The correct one is :
task([taskName], taskFunction)
see task signature
But your tasks look like this:
gulp.task('scripts', gulp.series('clean'), (done) => { // 3 parameters
Merely changing that to:
gulp.task('scripts', gulp.series('clean', (done) => {
...
}));
makes it work - I tested it. So now that task has only two parameters: a task name and a function. Yours had a task name plus two functions.
You would also need to change your default and clean tasks to this proper signature. Also you should call done() at the end of the task as you did with your cb().
Your new code uses task functions, which are better than named tasks for a number of reasons - but now you know what was wrong with your original code. The main body of your scripts task was never being run.
I never worked out what was wrong, but went direct to the doc's and started again (previous version was from a example)..
Works with the below (much simpler) script.
// // include plug-ins
var gulp = require('gulp');
var count = require('gulp-count');
var debug = require('gulp-debug');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var del = require('del');
var config = {
src: 'jspa-scada/dist/libraries/',
dest: 'jspa-scada/dist/js/',
outputfile: 'libraries.min.js'
}
function defaultTask(cb) {
del([config.dest + config.outputfile]);
// Include all js files but exclude any min.js files
var globs = [
config.src + '*.js',
'!' + config.src + '*.min.js'
];
return gulp.src(globs)
.pipe(debug())
.pipe(count('## files selected'))
.pipe(uglify())
.pipe(concat(config.outputfile))
.pipe(gulp.dest(config.dest));
cb();
}
exports.default = defaultTask

gulp - wrap plugin (which uses through2) output with string

I would like to know how exactly can I manipulate the output of my Gulp plugin so, for example, no matter how many files are passed to the plugin, it will wrap the output with a string. Currently I cannot know when does the last file is done.
The super simplified example below will iterate on 3 files and will create a new file named output.js and in it there will be three times the string xxx (xxxxxxxxx).
I would like the plugin itself to wrap the contents so the output will
be: +xxxxxxxxx+.
How can I do this?
Thanks!
Gulpfile
var gulp = require('gulp');
var concat = require('gulp-concat');
var foo = require('./index');
gulp.task('default', function() {
gulp.src([a.html, b.html, c.html])
.pipe(foo())
.pipe(concat('output.js'))
.pipe(gulp.dest('./test/output'))
});
The most basic gulp plugin (index.js):
var through2 = require('through2'),
gutil = require('gulp-util');
var PLUGIN_NAME = 'foo';
module.exports = function( options ){
// through2.obj(fn) is a convenience wrapper around
// through2({ objectMode: true }, fn)
return through2.obj(function( file, enc, callback ){
file.contents = new Buffer( 'xxx' );
this.push(file);
callback();
});
}
I understand the files are currently simply returned modified, but what I don't understand is how to append text and return the concatenated result that I want, while keeping it OK with Gulp working standards.
The "real" plugin should actually wrap the files results with:
var foo = { FILES_CONTENT }
where FILES_CONTENT will actually be a a concatenated string of all the files:
"file_name" : "file_content",
"file_name" : "file_content",
...
I would make the following changes to your gulpfile.js:
var gulp = require('gulp');
var foo = require('./index.js');
gulp.task('default', function() {
return gulp.src(['a.html', 'b.html', 'c.html'])
.pipe(foo({fileName:'output.js', varName:'bar'}))
.pipe(gulp.dest('./test/output'))
});
Since your foo() plugin itself will concatenate all the files, there's no need to use gulp-concat at all. Instead your plugin should accept an option fileName that provides the name of the generated file. I've also added another option varName that will provide the name of the var in the output file.
I'll assume that a.html, b.html and c.html are simple HTML files, something like this:
<h1 class="header">a</h1>
As you've already realized you need to concat all the files in the plugin itself. That's not really difficult however and doesn't require a lot of code. Here's a index.js which does exactly that:
var through2 = require('through2'),
gutil = require('gulp-util'),
path = require('path'),
File = require('vinyl');
var PLUGIN_NAME = 'foo';
module.exports = function(options) {
var files = { };
var outputFile = null;
return through2.obj(function(file, enc, callback){
outputFile = outputFile || file;
var filePath = path.relative(file.base, file.path);
files[filePath] = file.contents.toString();
callback();
}, function(callback) {
outputFile = outputFile ? outputFile.clone() : new File();
outputFile.path = path.resolve(outputFile.base, options.fileName);
outputFile.contents = new Buffer(
'var ' + options.varName + ' = ' +
JSON.stringify(files, null, 2) + ';'
);
this.push(outputFile);
callback();
});
}
Since you want to output a key/value mapping from file names to file contents our transformFunction just stores both of those things in a regular JavaScript object files. None of the input files themselves are emitted. Their names and contents are just stored until we have all of them.
The only tricky part is making sure that we respect the .base property of each file as is customary for gulp plugins. This allows the user to provide a custom base folder using the base option in gulp.src().
Once all files have been processed through2 calls the flushFunction. In there we create our output file with the provided fileName (once again making sure we respect the .base property).
Creating the output file contents is then just a matter of serializing our files object using JSON.stringify() (which automatically takes care of any escaping that has to be done).
The resulting ./test/output/output.js will then look like this:
var bar = {
"a.html": "<h1 class=\"header\">a</h1>\n",
"b.html": "<h1 class=\"header\">b</h1>\n",
"c.html": "<h1 class=\"header\">c</h1>\n"
};
You should use the gulp pipeline technique (standard).
This means that you can use the gulp-insert package in order
to add the string xxx.
var insert = require('gulp-insert');
.pipe(insert.append('xxx')); // Appends 'xxx' to the contents of every file
You can also prepend, append and wrap with this package and it support of course the gulp standards.
So the full example will be:
var gulp = require('gulp');
var concat = require('gulp-concat');
var foo = require('./index');
var insert = require('gulp-insert');
gulp.task('default', function() {
gulp.src([a.html, b.html, c.html])
.pipe(foo()
.pipe(insert.append('xxx'))
.pipe(concat('output.js'))
.pipe(gulp.dest('./test/output'))
});

Copy specified files with condition using gulp

I am trying copy my vendor files to my dev folder using gulp. When I was in development mode, I want copy only the unminified files, if unminified is not present copy minified files. And in production mode I want copy minifed files if files are not present minify the normal files.
my folder structure
js
app.js
jquery
jquery.min.js
jquery.js
fontawesome
fontawesome.min.js
fontawesome.min.css
fonts.ttf...
Here my basic I had written.
var scriptsPath = '../vendor/';
function getFolders(dir) {
return fs.readdirSync(dir)
.filter(function(file) {
return fs.statSync(path.join(dir, file)).isDirectory();
});
}
gulp.task('vendor', function() {
var folders = getFolders(scriptsPath);
var cssFilter = $.filter('**/*.css')
var tasks = folders.map(function(folder) {
var jsFilter;
if (isProduction) {
jsFilter = $.filter('**/*.min.js');
} else {
jsFilter = $.filter(['**/*.js', '!**/*.min.js']);
}
return gulp.src(path.join(scriptsPath, '**/'))
.pipe(jsFilter)
.pipe($.if(useSourceMaps, $.sourcemaps.init()))
.pipe($.if(isProduction, $.uglify({preserveComments: 'some'})))
.on('error', handleError)
.pipe(jsFilter.restore())
.pipe(cssFilter)
.pipe($.if( isProduction, $.minifyCss() ))
.on('error', handleError)
.pipe(cssFilter.restore())
.on('error', handleError)
.pipe(gulp.dest(build.vendor.js));
});
return es.concat.apply(null, tasks);
});
I am trying the last two days using gulp-if& some methods. But not yet get the solution.Thanks in advance.
You are trying to cram way to much into your vendor task. The stuff you do with your JS files is completely unrelated to the stuff you do with your CSS files. That's hard to read.
Instead of using gulp-filter try splitting vendor up into smaller tasks like vendor-js, vendor-css, etc... and then declare them as dependencies for your vendor task:
gulp.task('vendor', ['vendor-js', 'vendor-css' /* etc ... */]);
Your vendor-js task could then look like this:
var glob = require('glob');
gulp.task('vendor-js', function () {
var js = glob.sync('../vendor/**/*.js');
if (isProduction) {
// use <file>.min.js, unless there is only <file>.js
js = js.filter(function(file) {
return file.match(/\.min\.js$/) ||
js.indexOf(file.replace(/\.js$/, '.min.js')) < 0;
});
} else {
// use <file>.js, unless there is only <file>.min.js
js = js.filter(function(file) {
return !file.match(/\.min\.js$/) ||
js.indexOf(file.replace(/\.min\.js$/, '.js')) < 0;
});
}
gulp.src(js, { base: '../vendor' })
.pipe($.if(isProduction, // only minify for prod and when
$.if("!**/*.min.js", uglify()))) // the file isn't minified already
.pipe(gulp.dest('build'));
});
Adapting this to you specific needs should be fairly trivial from here on.

Why Gulp does not ignore the file?

My project structure:
build
public
assets
css
app.css
vendor.css
js
app.js
vendor.js
index.html
src
assets
styles
main.styl
coffee
main.coffee
index.jade
I have a task to build a library of Bower in two files : vendor.css and vendor.js:
var uglify = require('gulp-uglify');
var minifyCss = require('gulp-minify-css');
var mainBowerFiles = require('gulp-main-bower-files');
var filter = require('gulp-filter');
var concat = require('gulp-concat');
var connect = require('gulp-connect');
gulp.task('bower', ['clean'], function () {
var jsFilter = filter('**/*.js', { restore: true });
var cssFilter = filter('**/*.css', { restore: true });
return gulp.src('./bower.json')
.pipe(mainBowerFiles())
.pipe(cssFilter)
.pipe(concat('vendor.css'))
.pipe(minifyCss())
.pipe(gulp.dest(path.dist.css))
.pipe(cssFilter.restore)
.pipe(jsFilter)
.pipe(concat('vendor.js'))
.pipe(uglify())
.pipe(gulp.dest(path.dist.js))
.pipe(jsFilter.restore)
.pipe(connect.reload());
});
Here's my task on cleaning of temporary folders when you change files in the src folder:
var del = require('del');
...
gulp.task('clean', function () {
return del(['build', 'public/**', '!public/**/vendor.{js,css}']);
});
That must delete the build folder and all the contents of the public directory, except the vendor.css and vendor.js that are in the folder public/assets/css and public/assets/js.
This is done in order to not build a library of bower at each change of the file, because it takes 3-4 seconds (by the way, is this normal?), but only at the first start or change bower.json ( running separate watcher).
But somehow, the files vendor.js and vendor.css also removed. With gulp-tap I got a list of files:
List of files
What am i doing wrong?
According to https://github.com/sindresorhus/del#beware you need also exclude sub-directories, otherwise del will delete all.
So, you can use something like this:
del([.., 'public/**/*', '!public/{dir_1,dir_2}', '!test/**/vendor{.css,.js}']);

Browserify + browserify-ngannotate + Tsify not working

I'm using gulp with browserify and tsify. This has been working quite well. Then I decided to add ng-annotate using browserify-ngannotate.
I've added the ng-annotate browserify transform but it seems that if tsify is added as a plugin the ng-annotate transform is never called.
If I remove the tsify plugin then ng-annote gets called. I've played around and switched around the plugin/transform registration. Am I missing something here, or should I go and log an issue at browserify/tsify?
var browserify = require('browserify');
var browserSyncConfig = require('../config').browserSync;
var browserSync = require('browser-sync').get(browserSyncConfig.instance);
var watchify = require('watchify');
var tsify = require('tsify');
var ngAnnotate = require('browserify-ngannotate');
var mergeStream = require('merge-stream');
var bundleLogger = require('../util/bundleLogger');
var gulp = require('gulp');
var handleErrors = require('../util/handleErrors');
var source = require('vinyl-source-stream');
var config = require('../config').browserify;
var _ = require('lodash');
var browserifyTask = function (devMode) {
var browserifyThis = function (bundleConfig) {
if (devMode) {
// Add watchify args and debug (sourcemaps) option
_.extend(bundleConfig, watchify.args, {debug: true});
// A watchify require/external bug that prevents proper recompiling,
// so (for now) we'll ignore these options during development. Running
// `gulp browserify` directly will properly require and externalize.
bundleConfig = _.omit(bundleConfig, ['external', 'require']);
}
var b = browserify(bundleConfig);
if (bundleConfig.tsify) {
b = b.plugin(tsify, {
noImplicitAny: false,
target: 'ES5',
noExternalResolve: false,
module: 'commonjs',
removeComments: false
});
}
if (bundleConfig.ngAnnotate) {
b = b.transform(ngAnnotate);
}
var bundle = function () {
// Log when bundling starts
bundleLogger.start(bundleConfig.outputName);
return b
.bundle()
// Report compile errors
.on('error', handleErrors)
// Use vinyl-source-stream to make the
// stream gulp compatible. Specify the
// desired output filename here.
.pipe(source(bundleConfig.outputName))
// Specify the output destination
.pipe(gulp.dest(bundleConfig.dest))
.pipe(browserSync.stream());
};
if (devMode) {
// Wrap with watchify and rebundle on changes
b = watchify(b, {
poll: true
});
// Rebundle on update
b.on('update', bundle);
bundleLogger.watch(bundleConfig.outputName);
} else {
// Sort out shared dependencies.
// b.require exposes modules externally
if (bundleConfig.require) b.require(bundleConfig.require);
// b.external excludes modules from the bundle, and expects
// they'll be available externally
if (bundleConfig.external) b.external(bundleConfig.external);
}
return bundle();
};
// Start bundling with Browserify for each bundleConfig specified
return mergeStream.apply(gulp, _.map(config.bundleConfigs, browserifyThis));
};
gulp.task('browserify', function () {
return browserifyTask()
});
// Exporting the task so we can call it directly in our watch task, with the 'devMode' option
module.exports = browserifyTask;
You can solve it by specify extensions in ng-annotate options.
bundler.transform(ngAnnotate, { ext: ['.ts', '.js'] });
I realized I had this problem too, when I added uglifyify to the bundle transforms to produce minified builds.
An important aspect of my solution is that the missing, explicit $inject statements, that ng-annotate should have inserted, doesn't matter until the code is actually minified. Luckily, UglifyJS2, which does the actual minification in uglifyify, got support for handling ng-annotate's ngInject comments in version 2.4.9 (in January, 2014).
So, the solution that worked for me was to install uglifyify:
npm install --save-dev uglifyify
and add the following uglifyify transform to the Browserify bundle:
b.transform({
global: true,
mangle: false,
comments: true,
compress: {
angular: true
}
}, 'uglifyify');
This will make UglifyJS2 insert the appropriate $inject statements into your code before it is minified.
So, to summarize, I did not have a solution for only using ng-annotate, but my solution will add the necessary $inject statements before the code is minified, which is what matters in most cases.

Resources