The difficulty I see is with all the require calls, and the dependency tree. Is there a way to iterate through a project, including dependencies where needed, and produce a single, fully contained javascript file?
I am hoping to convert some server side only libraries to client side apps.
Alternatively, is there another method to achieve this...
If there is a file lib/_third_party_main.js in the node source when you compile, it will run that on start. See src/node.js. You might be able to compile your sources with e.g. UglifyJS or Google Closure.
Edit: Also, you can require any modules you put in lib as if they were native modules. Example:
lib/_third_party_main.js
var foo = require('foo');
foo();
lib/foo.js
module.exports = function() {
console.log('O hai');
}
Compile and run, and it will print O hai.
Edit: You might be able to use Ender.js, Browserify or a similar browser packaging tool to build a single file.
Related
I am building a custom binary of NodeJS from the latest code base for an embedded system. I have a couple modules that I would like to ship as standard with the binary - or even run a custom script the is compiled into the binary and can be invoked through a command line option.
So two questions:
1) I vaguely remember that node allowed to include custom modules during build time but I went through the latest 5.9.0 configure script and I can't see anything related - or maybe I am missing it.
2) Did someone already do something similar? If yes, what were the best practices you came up with?
I am not looking for something like Electron or other binary bundlers but actually building into the node binary.
Thanks,
Andy
So I guess I figure it out much faster that I thought.
For anyone else, you can add any NPM module to it and just add the actual source files to the node.gyp configuration file.
Compile it and run the custom binary. It's all in there now.
> var cmu = require("cmu");
undefined
> cmu
{ version: [Function] }
> cmu.version()
'It worked!'
> `
After studying this for quite a while, I have to say that the flyandi's answer is not quite true. You cannot add any NPM module just by adding it to the node.gyp.
You can only add pure JavaScript modules this way. To be able to embed a C++ module (I deliberately don't use the word "native", because that one is quite ambiguous in nodeJS terminology - just look at the sources).
To summarize this:
To embed a JS module to your custom nodejs, just add it in the library_files section of the node.gyp file. Also note that it should be placed within the lib folder, otherwise you'll have troubles requiring the module. That's because the name/path listed in node.gyp / library_files is used to encode the id of the module in the node_javascript.cc intermediate file which is then used when searching for the built-in modules.
To embed a native module is much more difficult. The best way I have found so far is to build the module as a static library instead of dynamic, which for cmake(-js) based module you can achieve by changing the SHARED parameter to STATIC like this:
add_library(${PROJECT_NAME} STATIC ${SRC})
instead of:
add_library(${PROJECT_NAME} SHARED ${SRC})
And also changing the suffix:
set_target_properties(
${PROJECT_NAME}
PROPERTIES
PREFIX ""
SUFFIX ".lib") /* instead of .node */
Then you can link it from node.gyp by adding this section:
'link_settings': {
'libraries' : [
"path/to/my/library.lib",
#...add other static dependencies
],
},
(how to do this with node-gyp based project should be quite ease to google)
This allows you to build the module, but you won't be able to require it, because require() function in node can only be used to load built-in JS modules, external JS modules or external dynamic node modules. But now we have a built-in C++ module. Well, lot of node integrated modules are C++, but they always have a JS wrapper in /lib, and those wrappers they use process.binding() to load the C++ module. That is, process.binding() is sort of a require() function for integrated C++ modules.
That said, we also need to call require.binding() instead of require to load our integrated module. To be able to do that, we have to make our module "built-in" first.
We can do that by replacing
NODE_MODULE(mymodule, InitAll)
int the module definition with
NODE_BUILTIN_MODULE_CONTEXT_AWARE(mymodule, InitAll)
which will register it as internal module and from now on we can process.binding() it.
Note that NODE_BUILTIN_MODULE_CONTEXT_AWARE is not defined in node.h as NODE_MODULE but in node_internals.h so you either have to include that one, or copy the macro definition over to your cpp file (the first one is of course better because the nodejs API tends to change quite often...).
The last thing we need to do is to list our newly integrated module among the others so that the node knows to initialize them (that is include them within the list of modules used when searching for the modules loaded with process.binding()). In node_internals.h there is this macro:
#define NODE_BUILTIN_STANDARD_MODULES(V) \
V(async_wrap) \
V(buffer) \
V(cares_wrap) \
...
So just add the your module to the list the same way as the others V(mymodule).
I might have forgotten some step, so ask in the comments if you think I have missed something.
If you wonder why would anyone even want to do this... You can come up with several reasons, but here's one most important to me: Those package managers used to pack your project within one executable (like pkg or nexe) work only with node-gyp based modules. If you, like me, need to use cmake based module, the final executable won't work...
I've got a very crude implementation working:
var screens = {
a: require('./../react_components/screens/a.jsx'),
b: require('./../react_components/screens/b.jsx'),
c: require('./../react_components/screens/c.jsx'),
d: require('./../react_components/screens/d.jsx'),
e: require('./../react_components/screens/e.jsx')
};
Which works fine, however, I'd like to make it a little more scalable, so that when I change a filename, or add a new file to the /screens folder, I don't really have to keep updating this list of require statements.
I'm using Browserify, so I do have a build step I can hook into if need be (this will be ran in the browser)
So for node/commonjs itself there are modules such as requireindex that will automate this kind of thing. However, the fact that these approaches determine the dependencies at runtime by looking at the filesystem defeats browserify's static analysis so I'm not aware of any of them that are browserify-compatible. Therefore, I'd suggest a code generation route where you use a module such as glob to discover the files you want to export, but then write out a full .js file with the exports all statically coded, and point browserify at that file, which you can regenerate every time during your build step.
I have a project and installed for example underscore and marked via npm install. When I launch node in my project root folder, I can require both with var __ = require("underscore"), similar for marked. This is how I use both libraries in my backend, if needed. Now I want to write a myscript.js which is delivered to the browser, it's referenced for example in my index.html file. In this script, I want to use both mentioned libraries as well.
For this to work, I think there are two methods:
inline the required files (I think I prefer this)
copy the required js files into a separate directory, and use require.js in the browser.
In neither case I want to point a program to where the libs are found, similar to node, which knows what to do by just saying require("lib").
My questions are:
Did I understand this correctly?
Is grunt-contrib-requirejs exactly the the thing I need to automate this process?
Can someone provide me with a snippet of code or point me to a blog post or something? I think I lost the overview, I found the documentation but many things are confusing.
Here is what I have so far:
Configuration for my Grunt task:
requirejs: {
compile: {
options: {
baseUrl: "./",
name: "src/main.js",
out: "build/result.js"
}
}
}
My main.js:
var _ = require("underscore")
function foo(values)
{
_.find(values, function(value){ return false; });
}
The result:
function foo(e){_.find(e,function(e){return!1})}var _=require("underscore");define("src/main.js",function(){});
... not exactly what I thought.
While this does not answer my original question, I found out that using node modules is not exactly the way to go for frontend stuff, I found bower. Anyway, this whole thing is very frustrating.
Bower allows me to install required packages, and then there are modules for grunt which allow the JS files either to be "plugged" into the HTML document within a special section, or to be concatenated into one file. Bower-ready modules should ship with a bower.json file. This reliefes me fron the work of manually collecting all needed js files from the node_modules folder like I was afraid of in the question.
Is there a way I can get grunt-closure-compiler to apply minimization to each file separately in a directory (overriding the original) instead of producing a single file as the output. If I can't override the original I am happy to place output files in a separate output directory.
https://github.com/gmarty/grunt-closure-compiler
Normally the procedue would be like this producing a single file:
grunt.initConfig({
'closure-compiler': {
frontend: {
closurePath: '/src/to/closure-compiler',
js: 'static/src/frontend.js',
jsOutputFile: 'static/js/frontend.min.js',
maxBuffer: 500,
options: {
compilation_level: 'ADVANCED_OPTIMIZATIONS',
language_in: 'ECMASCRIPT5_STRICT'
}
}
}
});
You can use the module option of the Closure Compiler to produce multiple output files. You would have to list each JavaScript file as its own module, so if you have many JavaScript files this could be pretty tedious.
The module option is not very well documented, but see the posts below to see how it works:
Using the Module Option in Closure Compiler to Create Multiple
Output Files
How do I split my javascript into modules using Google's Closure
Compiler?
As the other answer suggests, you need the modules option. However grunt-closure-compiler doesn't actually support this.
There's a fork of it which supports modules. It doesn't use the standard grunt file config so you can't use globbing patterns to get it to take all of the files in a folder. I've gotten around this by writing a grunt task to create the modules object and pass it into the config for the closure-compiler task.
I'm enjoying emberjs a lot and would like to take the next step in a couple of my small, mobile apps and precompile my Ember/Handlebars templates as part of my build process.
I'd prefer to stay away from messing with Ruby and would like to use node.js as I'm more comfortable with using it.
I believe what I want to use is Ember.Handlebars.precompile, but unfortunately I'm unable to load the canonical ember.js file in a node environment. Example of a naive attempt from the node repl:
> var e = require('./ember');
ReferenceError: window is not defined
at /Users/jeremyosborne/git/projects/ldls/client/lib/emberjs/src/ember.js:5:1
at Object.<anonymous> (/Users/jeremyosborne/git/projects/ldls/client/lib/emberjs/src/ember.js:1596:2)
--- stack trace, you get the idea ---
I think I've already figured out how to set them up in my code so that they work correctly with my views, I just want to compile them in an environment outside of a browser DOM.
In lieu of getting the canonical ember.js to load in node, are there a specific set of files that I can pluck from the ember repo and use to compile my templates?
EDIT
I did a kluge fix that works great but gets an F for maintainability. I grabbed all the Handlebars code minus the reference to the window object. Then I followed with the Ember.Handlebars.Compiler code, replacing Ember.create with Object.create, exporting my Ember object, and viola things work seemingly great in node (as in it works and the functions produced are templates). But I don't consider this an answer to my own question due to the aforementioned maintainafail, so still open for answers.
EDIT 2
The above turns out to be a total fail. Perhaps there's something wrong in the procedure, but using Ember.Handlebars.precompile or Ember.Handlebars.compile doesn't work. The templates get made, but when I use the precompiled templates attached to Ember.TEMPLATES in my code, they do not work. I only seem to be able to get templates to work when they are explicitly passed in the modified script tags, as suggested on the emberjs.com site.
EDIT 3
I figured out what I was doing wrong. My answer is below.
I've written a grunt plugin called grunt-ember-handlebars that does exactly this. It pretty much mimics Garth's script, with one important difference:
It uses lib/headless-ember.js and lib/ember.js, which are maintained (at least for now) by ember.js to precompile default templates. If you don't want to use grunt, you can extract the relevant code from the precompile_handlebars helper in tasks/ember-handlebars.js.
Found a good enough solution to my problem that seems easy enough to maintain that I'll consider my problem solved.
Here's how I solved things:
Grab the minimal amount of code I need to precompile the ember templates.
Copy the contents of the ember handlebars into a file. This file is located at the emberjs repo: https://github.com/emberjs/ember.js/blob/master/packages/handlebars/lib/main.js
Within the code, find the one instance of window.Handlebars = Handlebars; and remove it.
Append the contents of the Ember template compiler and handlebar overrides. The file is located at the emberjs repo: https://github.com/emberjs/ember.js/blob/master/packages/ember-handlebars/lib/ext.js
Find all instances of Ember.create and change to Object.create.
Outline of the simple precompile procedure:
Load up the code with a var Ember = require('./my_ember_precompiler').Ember.
Get your templates as strings and compile them with var templateString = Ember.Handlebars.precompile(str).toString().
This will be different from app to app, but Ember seems to require registration of precompiled templates. After loading, for every created template, we need to register our templates. Basically wrap templateString in a call to Handlebars.template() and make sure this wrapped function is added to the Ember.TEMPLATES object.
The above is painless when it's automated in a script.
I've published a version of ember-precompiler with a similar interface to the handlebars command line utility. You can install it from NPM:
npm install -g ember-precompile
and then run it as:
ember-precompile template... [-f OUTPUT_FILE]
It does essentially what you describe (and what the gist versions do too): mock out the missing components needed to run Ember and Handlebars, compile the templates and add them to Ember.TEMPLATES.
Take a look at the npm package Ember-Runner
The following gist contains a nodejs build script that will precompile all .handlebars files in a given folder:
https://gist.github.com/1622236
I wrote an official precompiler npm module for anyone else who might be wanting to do this w/ a recent version of ember.js
https://npmjs.org/package/ember-template-compiler
It's simple to install and use (example below)
npm install ember-template-compiler
var compiler = require('ember-template-compiler');
var template = fs.readFileSync('foo.handlebars').toString();
var input = compiler.precompile(template).toString();
var output = "Ember.TEMPLATES['foo'] = Ember.Handlebars.template(" + input + ");";
Look at the code for the official ember-rails gem at https://github.com/emberjs/ember-rails
While it's not a node.js project, it does show you how to precompile the templates using the Rails 3.1+ asset pipeline, and it includes all the necessary Javascript code that you would need to do it in Node without having to hack together a solution that you'd have to maintain on your own.
More specifically, look at vendor/assets/javascripts/ember-precompiler.js and lib/ember-rails/hjs_template.rb
I'm far from an expert on Node (obviously, Rails is more my thing).. but I think those two files should point you in the right direction. Basically, you're going to want to concatenate ember-precompiler.js (which acts as a "shim" for lack of a better word) with ember.js and then call EmberRails.precompile to compile your templates.
window object can be mocked by jsdom
var jsdom = require("jsdom").jsdom;
global.document = jsdom("<html><head></head><body></body></html>");
global.window = document.createWindow();
global.$ = global.jQuery = window.$ = window.jQuery = require("jquery");
global.Handlebars = window.Handlebars = require('handlebars');
global.Application = window.Application = {};
require('ember.js');
and now you can run anything from Ember including Ember.Handlebars.compile