Related
My project structure is the following:
MyApp
- hooks
- platforms
- android
- ios
- www
- js / css / templates..
- lib (including all bower components)
Right now, the www/lib directory is taking up 21,8 Mb. (I have a large set of bower components added to my project.)
When building each project, the entire www folder is copied to the platform/android (for instance) folder for build, including of course www/lib.
This leads to a very big build, as lots of files included into bower
components are useless for production.
Manually managing all bower dependencies is clearly not an option. So how do you guys manage to clean your project platform directory for build?
I was thinking about creating a hook for that but before writing lines of code in a language that i do not know (nodeJS), I was hoping for your return and advises.
According to Cordova workflow you can add a hook script that removes unnecessary files.
A detailed example of a cleanup script can be found here: https://www.thepolyglotdeveloper.com/2015/01/hooks-apache-cordova-mobile-applications/
But to give a quick step by step summary:
Add to the after_prepare hook folder (/hooks/after_prepare) a script (01_junk_cleanup.js - 01 to be run first, the rest whatever you want) and in the file specify the files and folders you want to delete. For example, here is how you can delete a test folder and relevant files just change to you lib directory and to the files there. Note that this example is a bit different from the example in the link i gave earlier so you might want to take a look there as well.
01_junk_cleanup.js:
#!/usr/bin/env node
var fs = require('fs');
var path = require('path');
var foldersToProcess = [
"js",
"css"
];
var foldersToDelete = [
"test"
];
var filesToDelete = [
"karmaOnBrowser.conf.js",
"karmaOnEmulators.conf.js",
"SpecRunner.html"
];
var iosPlatformsDir = "platforms/ios/www/";
var androidPlatformsDir = "platforms/android/assets/www/";
filesToDelete.forEach(function(file) {
var filePathIOS = iosPlatformsDir + file;
var filePathAndroid = androidPlatformsDir + file;
if(fs.existsSync(filePathIOS)){
fs.unlinkSync(filePathIOS);
};
if(fs.existsSync(filePathAndroid)){
fs.unlinkSync(filePathAndroid);
};
});
foldersToProcess.forEach(function(folder) {
processFiles(iosPlatformsDir + folder);
processFiles(androidPlatformsDir + folder);
});
foldersToDelete.forEach(function(folder) {
deleteFolderRecursive(iosPlatformsDir + folder);
deleteFolderRecursive(androidPlatformsDir + folder);
});
function deleteFolderRecursive(path){
if( fs.existsSync(path) ) {
fs.readdirSync(path).forEach(function(file,index){
var curPath = path + "/" + file;
if(fs.lstatSync(curPath).isDirectory()) { // recurse
deleteFolderRecursive(curPath);
} else { // delete file
fs.unlinkSync(curPath);
}
});
fs.rmdirSync(path);
}
}
function processFiles(dir) {
fs.readdir(dir, function(err, list) {
if(err) {
console.log('processFiles err: ' + err);
return;
}
list.forEach(function(file) {
file = dir + '/' + file;
fs.stat(file, function(err, stat) {
if(!stat.isDirectory()) {
switch(path.basename(file)) {
case ".DS_Store":
fs.unlink(file, function(error) {
console.log("Removed file " + file);
});
break;
case "Thumbs.db":
fs.unlink(file, function(error) {
console.log("Removed file " + file);
});
break;
default:
console.log("Skipping file " + file);
break;
}
}
});
});
});
}
Aside to above, A bit more obvious but I feel worth mentioning anyhow, After having the www/lib bloat as well I always try to keep the folder lean and add only libraries required for deployment, the other dev. dependencies such as jasmine I either hold in the 'node_modules' folder or 'bower_components' as I only install today through them.
Hope this helps,
Good luck
I think the best approach would be to do this:
Move the bower_components folder and your index.html file to the project root, outside the /www folder
Install gulp and gulp-usemin
Wrap all of the .js files and .css files from bower components in usemin <build:js> and <build:css> sections
Configure a task in your gulpfile to concatenate all those files into a lib.js and a lib.css file. Make sure that those two files as well as the rewritten index.html are output to the /www folder
Execute the gulp task before your next build, and each time you add a new bower component.
This will keep your /www folder tidy and only containing the files you need in your cordova build.
With Bower you need to use npm preen to remove unnecessary files
See my example using Gulp with Ionic Framework: https://github.com/jdnichollsc/Ionic-Starter-Template
Basically you can set your bower.json file to indicate the path which files you need, for example:
"preen": {
//... More libraries
"ionic-datepicker": [
"dist/*.js" //You only need these files
//Other files and folders will be deleted to reduce the size of your app
],
"ion-floating-menu": [
"dist/*" //Keep all the files (.html, .css, .js, etc) of the directory.
]
}
Regards, Nicholls
This is an improvement over this answer. I've applied it to my own project.
Move the bower_components folder to the project root, outside the www folder.
Rename index.html to _index.html. We will later make sure that Gulp automatically generates index.html.
Install gulp and gulp-useref.
Edit your _index.html so that it looks something like this:
<!-- build:js dist/js/vendor.js -->
<script src="../bower_components/ionic/release/js/ionic.bundle.min.js"></script>
<script src="../bower_components/ngstorage/ngStorage.min.js"></script>
<script src="../bower_components/ngCordova/dist/ng-cordova.min.js"></script>
<!-- endbuild -->
Configure your gulp watch task to build new index.html file in the www folder with the concatenated files.
var entrypoint = './www/_index.html';
gulp.task('watch', function() {
gulp.watch(entrypoint, ['concat-js-and-css']);
});
gulp.task('concat-js-and-css', function () {
return gulp.src(entrypoint)
.pipe(useref())
.pipe(rename(function (path) {
// rename _index.html to index.html
if (path.basename == '_index' && path.extname == '.html') {
path.basename = "index";
}
}))
.pipe(gulp.dest('./www'))
});
gulp.task('build', ['concat-js-and-css']);
When that task runs, your index.html file will contain just this:
<script src="dist/js/vendor.js"></script>
Edit your ionic.project file so that it looks like the following. This will make sure that gulp watch is run before ionic serve.
{
"watchPatterns": [
"www/_index.html",
],
"gulpStartupTasks": [
"watch"
]
}
Currently in our Sass files we have something like the following:
#import "../../node_modules/some-module/sass/app";
This is bad, because we're not actually sure of the path: it could be ../node_modules, it could be ../../../../../node_modules, because of how npm installs stuff.
Is there a way in Sass that we can search up until we find node_modules? Or even a proper way of including Sass through npm?
If you are looking for a handy answer in 2017 and are using Webpack, this was the easiest I found.
Suppose your module path is like:
node_modules/some-module/sass/app
Then in your main scss file you can use:
#import "~some-module/sass/app";
Tilde operator shall resolve any import as a module.
As Oncle Tom mentioned, the new version of Sass has this new importer option, where every "import" you do on your Sass file will go first through this method. That means that you can then modify the actual url of this method.
I've used require.resolve to locate the actual module entry file.
Have a look at my gulp task and see if it helps you:
'use strict';
var path = require('path'),
gulp = require('gulp'),
sass = require('gulp-sass');
var aliases = {};
/**
* Will look for .scss|sass files inside the node_modules folder
*/
function npmModule(url, file, done) {
// check if the path was already found and cached
if(aliases[url]) {
return done({ file:aliases[url] });
}
// look for modules installed through npm
try {
var newPath = path.relative('./css', require.resolve(url));
aliases[url] = newPath; // cache this request
return done({ file:newPath });
} catch(e) {
// if your module could not be found, just return the original url
aliases[url] = url;
return done({ file:url });
}
}
gulp.task("style", function() {
return gulp.src('./css/app.scss')
.pipe(sass({ importer:npmModule }))
.pipe(gulp.dest('./css'));
});
Now let's say you installed inuit-normalize using node. You can simply "require" it on your Sass file:
#import "inuit-normalize";
I hope that helps you and others. Because adding relative paths is always a pain in the ass :)
You can add another includePaths to your render options.
Plain example
Snippet based on example from Oncle Tom.
var options = {
file: './sample.scss',
includePaths: [
path.join(__dirname, 'bower_components'), // bower
path.join(__dirname, 'node_modules') // npm
]
};
sass.render(options, function(err, result){
console.log(result.css.toString());
});
That should do. You can include the files from package using #import "my-cool-package/super-grid
Webpack and scss-loader example
{
test: /\.scss$/,
loader: 'style!css!autoprefixer?browsers=last 2 version!sass?outputStyle=expanded&sourceMap=true&sourceMapContents=true&includePaths[]=./node_modules'
},
Notice the last argument, includePaths has to be array. Keep in mind to use right format
You can use a Sass importer function to do so. Cf. https://github.com/sass/node-sass#importer--v200.
The following example illustrates node-sass#3.0.0 with node#0.12.2:
Install the bower dependency:
$ bower install sass-mq
$ npm install sass/node-sass#3.0.0-pre
The Sass file:
#import 'sass-mq/mq';
body {
#include mq($from: mobile) {
color: red;
}
#include mq($until: tablet) {
color: blue;
}
}
The node renderer file:
'use strict';
var sass = require('node-sass');
var path = require('path');
var fs = require('fs');
var options = {
file: './sample.scss',
importer: function bowerModule(url, file, done){
var bowerComponent = url.split(path.sep)[0];
if (bowerComponent !== url) {
fs.access(path.join(__dirname, 'bower_components', bowerComponent), fs.R_OK, function(err){
if (err) {
return done({ file: url });
}
var newUrl = path.join(__dirname, 'bower_components', url);
done({ file: newUrl });
})
}
else {
done({ file: url });
}
}
};
sass.render(options, function(err, result){
if (err) {
console.error(err);
return;
}
console.log(result.css.toString());
});
This one is simple and not recursive. The require.resolve function could help to deal with the tree – or wait until npm#3.0.0 to benefit from the flat dependency tree.
I made the sass-npm module specifically for this.
npm install sass-npm
In your SASS:
// Since node_modules/npm-module-name/style.scss exists, this will be imported.
#import "npm-module-name";
// Since just-a-sass-file isn't an installed npm module, it will be imported as a regular SCSS file.
#import "just-a-sass-file";
I normally use gulp-sass (which has the same 'importer' option as regular SASS)
var gulp = require('gulp'),
sass = require('gulp-sass'),
sassNpm = require('sass-npm')();
Then, in your .pipe(sass()), add the importer as an option:
.pipe(sass({
paths: ['public/scss'],
importer: sassNpm.importer,
}))
For dart-sass and commandline user at 2022, just use the --load-path option:
$ npx sass --load-path=node_modules
Important: the whole node_modules folder contains so much, just set it launch extremely slow in watch mode. Your should only set your package paths, eg:
$npx sass -w --load-path=node_modules/foo --load-path=node_modules/bar/scss
From offical docuumentation of Sass, adding ~ to imports should do the job.
However, for some reason it did'nt work for me, and sass compiler still complains that the module cannot be found.
Hence, I tried another method which worked for me without any issues. Here's the solution:
If you are compiling sass files directly from CLI try this:
sass src/main.scss dist/main.css --load-path=node_modules
If you are using npm and/or webpack for compiling sass files, add something like this to the scripts of package.json:
"scripts": {
...
"build": "sass src/main.scss dist/main.css --load-path=node_modules",
...
}
Then Run:
npm run build
Finally, import your modules like this:
#import "some-module/sass/app";
To wrap it up, adding --load-path=node_modules flag solved the issue permanently. For more information you can check:
sass --help
I'm new to Grunt and I'm trying to use the grunt-bower-concat node module to concat all my bower components into a single js file and css file respectively. It's working great, except that I want to force grunt-bower-concat to use the minified versions of my bower components instead of the uncompressed versions.
Luckily, it comes with a callback feature where I can customize this:
callback: function(mainFiles, component) {
return _.map(mainFiles, function(filepath) {
// Use minified files if available
var min = filepath.replace(/\.js$/, '.min.js');
return grunt.file.exists(min) ? min : filepath;
});
}
And I added it to my Gruntfile:
module.exports = function (grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
bower_concat: {
all: {
dest: "src/js/<%= pkg.name %>-bower.js",
cssDest: "src/css/<%= pkg.name %>-bower.css",
callback: function(mainFiles) {
return _.map(mainFiles, function(filepath) {
var min = filepath.replace(/\.js$/, '.min.js');
return grunt.file.exists(min) ? min : filepath;
});
}
}
},
...
And it fails with the following error:
$ /usr/local/bin/grunt --gruntfile /Applications/MAMP/htdocs/proj/Gruntfile.js bower_concat
Running "bower_concat:all" (bower_concat) task
Fatal error: _ is not defined
Process finished with exit code 3
This example is trying to use underscore's map function and it's clear Grunt does not have access to this library.
How can I load underscore or use it's functions inside of my Gruntfile?
Instead of requiring an extra library, simply replace
return _.map(mainFiles, function(filepath) {
With this:
return mainFiles.map(function(filepath) {
Doesn't look like you required underscore anywhere, unless you're not showing the whole file.
Any file in which you want to use underscore you need to do:
var _ = require('underscore');
before making use of _.
Oh, and of course you need to npm install underscore --save in the folder the gruntfile is in as well, to have the library there.
I am trying to use gulp-requirejs to build a demo project. I expect result to be a single file with all js dependencies and template included. Here is my gulpfile.js
var gulp = require('gulp');
var rjs = require('gulp-requirejs');
var paths = {
scripts: ['app/**/*.js'],
images: 'app/img/**/*'
};
gulp.task('requirejsBuild', function() {
rjs({
name: 'main',
baseUrl: './app',
out: 'result.js'
})
.pipe(gulp.dest('app/dist'));
});
// The default task (called when you run `gulp` from cli)
gulp.task('default', ['requirejsBuild']);
The above build file works with no error, but the result.js only contains the content of main.js and config.js. All the view files, jquery, underscore, backbone is not included.
How can I configure gulp-requirejs to put every js template into one js file?
If it is not the right way to go, can you please suggest other method?
Edit
config.js
require.config({
paths: {
"almond": "/bower_components/almond/almond",
"underscore": "/bower_components/lodash/dist/lodash.underscore",
"jquery": "/bower_components/jquery/dist/jquery",
"backbone": "/bower_components/backbone/backbone",
"text":"/bower_components/requirejs-text/text",
"book": "./model-book"
}
});
main.js
// Break out the application running from the configuration definition to
// assist with testing.
require(["config"], function() {
// Kick off the application.
require(["app", "router"], function(app, Router) {
// Define your master router on the application namespace and trigger all
// navigation from this instance.
app.router = new Router();
// Trigger the initial route and enable HTML5 History API support, set the
// root folder to '/' by default. Change in app.js.
Backbone.history.start({ pushState: false, root: '/' });
});
});
The output is just a combination this two files, which is not what I expected.
gulp-requirejs has been blacklisted by the gulp folks. They see the RequireJS optimizer as its own build system, incompatible with gulp. I don't know much about that, but I did find an alternative in amd-optimize that worked for me.
npm install amd-optimize --save-dev
Then in your gulpfile:
var amdOptimize = require('amd-optimize');
var concat = require('gulp-concat');
gulp.task('bundle', function ()
{
return gulp.src('**/*.js')
.pipe(amdOptimize('main'))
.pipe(concat('main-bundle.js'))
.pipe(gulp.dest('dist'));
});
The output of amdOptimize is a stream which contains the dependencies of the primary module (main in the above example) in an order that resolves correctly when loaded. These files are then concatenated together via concat into a single file main-bundle.js before being written into the dist folder.
You could also minify this file and perform other transformations as needed.
As an aside, in my case I was compiling TypeScript into AMD modules for bundling. Thinking this through further I realized that when bundling everything I don't need the asynchronous loading provided by AMD/RequireJS. I am going to experiment with having TypeScript compile CommonJS modules instead, then bundling them using webpack or browserify, both of which seem to have good support within gulp.
UPDATE
My previous answer always reported taskReady even if requirejs reported an error. I reconsidered this approach and added error logging. Also I try to fail the build completely as described here gulp-jshint: How to fail the build? because a silent fail really eats your time.
See updated code below.
Drew's comment about blacklist was very helpfull and gulp folks suggest using requirejs directly. So I post my direct requirejs solution:
var DIST = './dist';
var requirejs = require('requirejs');
var requirejsConfig = require('./requireConfig.js').RJSConfig;
gulp.task('requirejs', function (taskReady) {
requirejsConfig.name = 'index';
requirejsConfig.out = DIST + 'app.js';
requirejsConfig.optimize = 'uglify';
requirejs.optimize(requirejsConfig, function () {
taskReady();
}, function (error) {
console.error('requirejs task failed', JSON.stringify(error))
process.exit(1);
});
});
The file at ./dist/app.js is built and uglified. And this way gulp will know when require has finished building. So the task can be used as a dependency.
My solution works like this:
./client/js/main.js:
require.config({
paths: {
jquery: "../vendor/jquery/dist/jquery",
...
},
shim: {
...
}
});
define(["jquery"], function($) {
console.log($);
});
./gulpfile.js:
var gulp = require('gulp'),
....
amdOptimize = require("amd-optimize"),
concat = require('gulp-concat'),
...
gulp.task('scripts', function(cb) {
var js = gulp.src(path.scripts + '.js')
.pipe(cached('scripts'))
.pipe(jshint())
.pipe(jshint.reporter('default'))
.pipe(remember('scripts'))
.pipe(amdOptimize("main",
{
name: "main",
configFile: "./client/js/main.js",
baseUrl: './client/js'
}
))
.pipe(concat('main.js'));
.pipe(gulp.dest(path.destScripts));
}
...
This part was important:
configFile: "./client/js/main.js",
baseUrl: './client/js'
This allowed me to keep my configuration in one place. Otherwise I was having to duplicate my paths and shims into gulpfile.js.
This works for me. I seems that one ought to add in uglification etc via gulp if desired. .pipe(uglify()) ...
Currently I have to duplicate the config in main.js to run asynchronously.
....
var amdOptimize = require("amd-optimize");
...
var js = gulp.src(path.scripts + '.js')
.pipe(cached('scripts'))
.pipe(jshint())
.pipe(jshint.reporter('default'))
.pipe(remember('scripts'))
.pipe(amdOptimize("main",
{
name: "main",
paths: {
jquery: "client/vendor/jquery/dist/jquery",
jqueryColor: "client/vendor/jquery-color/jquery.color",
bootstrap: "client/vendor/bootstrap/dist/js/bootstrap",
underscore: "client/vendor/underscore-amd/underscore"
},
shim: {
jqueryColor : {
deps: ["jquery"]
},
bootstrap: {
deps: ["jquery"]
},
app: {
deps: ["bootstrap", "jqueryColor", "jquery"]
}
}
}
))
.pipe(concat('main.js'));
Try this code in your gulpfile:
// Node modules
var
fs = require('fs'),
vm = require('vm'),
merge = require('deeply');
// Gulp and plugins
var
gulp = require('gulp'),
gulprjs= require('gulp-requirejs-bundler');
// Config
var
requireJsRuntimeConfig = vm.runInNewContext(fs.readFileSync('app/config.js') + '; require;'),
requireJsOptimizerConfig = merge(requireJsRuntimeConfig, {
name: 'main',
baseUrl: './app',
out: 'result.js',
paths: {
requireLib: 'bower_modules/requirejs/require'
},
insertRequire: ['main'],
// aliases from config.js - libs will be included to result.js
include: [
'requireLib',
"almond",
"underscore",
"jquery",
"backbone",
"text",
"book"
]
});
gulp.task('requirejsBuild', ['component-scripts', 'external-scripts'], function (cb) {
return gulprjs(requireJsOptimizerConfig)
.pipe(gulp.dest('app/dist'));
});
Sorry for my english. This solution works for me. (I used gulp-requirejs at my job)
I think you've forgotten to set mainConfigFile in your gulpfile.js. So, this code will be work
gulp.task('requirejsBuild', function() {
rjs({
name: 'main',
mainConfigFile: 'path_to_config/config.js',
baseUrl: './app',
out: 'result.js'
})
.pipe(gulp.dest('app/dist'));
});
In addition, I think when you run that task in gulp, require can not find its config file and
This is not gulp-requirejs fault.
The reason why only main.js and config.js is in the output is because you're not requiring/defining any other files. Without doing so, the require optimizer wont understand which files to add, the paths in your config-file isn't a way to require them!
For example you could load a main.js file from your config file and in main define all your files (not optimal but just a an example).
In the bottom of your config-file:
// Load the main app module to start the app
requirejs(["main"]);
The main.js-file: (just adding jquery to show the technique.
define(["jquery"], function($) {});
I might also recommend gulp-requirejs-optimize instead, mainly because it adds the minification/obfuscation functions gulp-requirejs lacks: https://github.com/jlouns/gulp-requirejs-optimize
How to implement it:
var requirejsOptimize = require('gulp-requirejs-optimize');
gulp.task('requirejsoptimize', function () {
return gulp.src('src/js/require.config.js')
.pipe(requirejsOptimize(function(file) {
return {
baseUrl: "src/js",
mainConfigFile: 'src/js/require.config.js',
paths: {
requireLib: "vendor/require/require"
},
include: "requireLib",
name: "require.config",
out: "dist/js/bundle2.js"
};
})).pipe(gulp.dest(''));
});
In my gulp build I've made a task that runs after all compiling, uglifying and minification has occurred. This task simply copies everything from the src into the dest directory that hasn't been touched/processed by earlier tasks. The little issue I'm having is that this results in empty directories in the dest directory.
Is there a way to tell the gulp.src glob to only include files in the pattern matching (like providing the 'is_file' flag)?
Thanks.
Fixed it by adding a filter to the pipeline:
var es = require('event-stream');
var onlyDirs = function(es) {
return es.map(function(file, cb) {
if (file.stat.isFile()) {
return cb(null, file);
} else {
return cb();
}
});
};
// ...
var s = gulp.src(globs)
.pipe(onlyDirs(es))
.pipe(gulp.dest(folders.dest + '/' + module.folder));
// ...
I know I'm late to the party on this one, but for anyone else stumbling upon this question, there is another way to do this that seems pretty elegant in my eyes. I found it in this question
To exclude the empty folders I added { nodir: true } after the glob pattern.
Your general pattern could be such (using the variables from Nick's answer):
gulp.src(globs, { nodir: true })
.pipe(gulp.dest(folders.dest + '/' + module.folder));
Mine was as follows:
gulp.src(['src/**/*', '!src/scss/**/*.scss', '!src/js/**/*.js'], { nodir: true })
.pipe(gulp.dest('dev/'));
This selects all the files from the src directory that are not scss or js files, and does not copy any empty folders from those two directories either.