How to share code between node.js apps? - node.js

I have several apps in node that all share a few modules that I've written. These modules are not available via npm. I would like to be able to share freely between apps, but I don't want to copy directories around, nor rely on Git to do so. And I'm not really big on using symlinks to do this either.
I would like to arrange directories something like this:
app1
server.js
node_modules
(public modules from npm needed for app1)
lib
(my own modules specific to app1)
app2
server.js
node_modules
(public modules from npm needed for app2)
lib
(my own modules specific to app2)
shared_lib
(my own modules that are used in both app1 and app2)
The problem I'm seeing is that the modules in shared_lib seem to get confused as to where to find the modules that will be in the node_modules directory of whichever app they are running in. At least I think that is the problem.
So....what is a good way to do this that avoids having duplicates of files? (note that I don't care about duplicates of things in node_modules, since those aren't my code, I don't check them into Git, etc)

The npm documentation recommends using npm-link to create your own Node.js packages locally, and then making them available to other Node.js applications. It's a simple four-step process.
A typical procedure would be to first create a package with the following structure:
hello
| index.js
| package.json
A typical implementation of these files would be:
index.js
exports.world = function() {
return('Hello World');
}
package.json
{
"name": "hello",
"version": "0.0.1",
"private": true,
"main": "index.js",
"dependencies": {
},
"engines": {
"node": "v0.6.x"
}
}
"private:true" ensures that npm will refuse to publish the package. This is a way to prevent accidental publication of private packages.
Next, navigate to the root of your Node.js package folder and run npm link to link the package globally so it can be used in other applications.
To use this package in another application, e.g., "hello-world", with the following directory structure:
hello-world
| app.js
Navigate to the hello-world folder and run:
npm link hello
Now you can use it like any other npm package like so:
app.js
var http = require('http');
var hello = require('hello');
var server = http.createServer(function(req, res) {
res.writeHead(200);
res.end(hello.world());
});
server.listen(8080);

I've got this working by having node_modules folders at different levels - node then automatically traverses upwards until it finds the module.
Note you don't have to publish to npm to have a module inside of node_modules - just use:
"private": true
Inside each of your private package.json files - for your project I would have the following:
app1
server.js
node_modules
(public modules from npm needed for app1)
(private modules locally needed for app1)
app2
server.js
node_modules
(public modules from npm needed for app2)
(private modules locally needed for app2)
node_modules
(public modules from npm needed for app1 & app2)
(private modules locally for app1 & app2)
The point is node.js has a mechanism for dealing with this already and it's awesome. Just combine it with the 'private not on NPM' trick and you are good to go.
In short a:
require('somemodule')
From app A or B would cascade upwards until it found the module - regardless if it lived lower down or higher up. Indeed - this lets you hot-swap the location without changing any of the require(...) statements.
node.js module documentation

Just use the correct path in your require call
For example in server.js that would be:
var moduleName = require('../shared_lib/moduleName/module.js');
Its Important to know that as soon as your path is prefixed with '/', '../', or './' the path is relative to the calling file.
For further information about nodes module loading visit:
http://nodejs.org/docs/latest/api/modules.html

Yes, you can reference shared_lib from app1, but then you run into a problem if you want to package and deploy app1 to some other environment, such as a web server on AWS.
In this case, you're better off installing your modules in shared_lib to app1 and app2 using "npm install shared_lib/module". It will also install all the dependencies of the shared_lib modules in app1 and app2 and deal with conflicts/duplicates.
See this:
How to install a private NPM module without my own registry?

If you check out the node.js docs, you'll see that Node.js understands the package.json file format as well, at least cursorily.
Basically, if you have a directory named foo, and in that directory is a package.json file with the key-value pair: "main": "myCode.js", then if you try to require("foo") and it finds this directory with a package.json file inside, it will then use foo/myCode.js for the foo module.
So, with your directory structure, if each shared lib has it's own directory with such a simple package.json file inside, then your apps can get the shared libs by:
var lib1 = require('../shared_lib/lib1');
var lib2 = require('../shared_lib/lib2');
And that should work for both of these apps.

Another solution can be cloning files from the other places into this repo:
clone.js:
const path = require('path')
const fs = require('fs')
const shared = [
{
type: 'file',
source: '../app1',
files: [
'src/file1',
'src/file2',
'...'
],
},
]
function cloneFiles(source, files) {
const Reset = '\x1b[0m'
const FgGreen = '\x1b[32m'
console.log(`---------- Cloning ${files.length} files from ${source} ----------`)
for (const file of files) {
const sourceFile = path.join(__dirname, '..', source, file)
const targetFile = path.join(__dirname, '..', file)
process.stdout.write(`📁 ${file} ... `)
fs.copyFileSync(sourceFile, targetFile)
console.log(`${FgGreen}Done!${Reset}`)
}
console.log(`---------- All done successfully ----------\n`)
}
;(() => {
for (const item of shared) {
switch (item.type) {
case 'file':
cloneFiles(item.source, item.files)
break
}
}
})()
Then, in the package.json you can add this script and call it when you want to clone / sync files:
"clone": "node clone.js"

Related

NOde.js/Express App can't find some node_modules

I use several Node/Express modules in my app, and everything works fine for every module as long as I do const module = require('module');. I don't need to define a static path for these modules as app.use(express.static(path.join(__dirname, 'public')));.
However, for the sweetalert module, if I define in my layout.pug (base pug file) script(src="/node_modules/sweetalert/dist/sweetalert.min.js"), I get a 404 Error (not found) unless I include in app.js the following static path: app.use("/node_modules", express.static(__dirname + "/node_modules"));.
My question is: is this the normal behaviour or is it something I'm not doing right? (I'm kinda confused why I have to define a static path just for one of several modules I use.
Here's whats going on:
app.use(express.static(path.join(__dirname, 'public'))); is declaring that the public directory is accessible to the browser. You should put all your front end resources in that folder. This will help separate what can be accessed from the server and what can be accessed from the client.
When you reference script(src="/node_modules/sweetalert/dist/sweetalert.min.js") the browser throws a 404 because that file is not located in the public directory, therefore off limits to the browser.
Adding this line app.use("/node_modules", express.static(__dirname + "/node_modules")); "fixes" your issue but now exposes all your node_modules to the browser. This probably isn't a good idea and I'm sure a security expert could elaborate why this shouldn't be done.
How I would resolve this issue: Go through your .pug code and look at any resources your front end requires. Then copy them over to the public folder and fix your references to use the copy of the resource.
Here's an example of a script I use to move resources from the node_module directory to a public/assets directory:
build.js:
const path = require('path');
var fs = require('fs');
const ASSETS = [
'jquery/dist/jquery.min.js',
'sweetalert/dist/sweetalert.min.js'
];
if (!fs.existsSync('./public/assets')){
fs.mkdirSync('./public/assets');
}
ASSETS.map(asset => {
let filename = asset.substring(asset.lastIndexOf("/") + 1);
let from = path.resolve(__dirname, `./node_modules/${asset}`)
let to = path.resolve(__dirname, `./public/assets/${filename}`)
if (fs.existsSync(from)) {
fs.createReadStream(from).pipe(fs.createWriteStream(to));
} else {
console.log(`${from} does not exist.\nUpdate the build.js script with the correct file paths.`)
process.exit(1)
}
});
then I update my package.json to include this in the scripts:
package.json:
"scripts": {
"build": "node ./build.js || true",
"start": "node ./bin/www"
}
then in any of my views pages I reference the resource by using the new path
random.pug:
script(src="/assets/jquery.min.js")
script(src="/assets/sweetalert.min.js")
Finally before you deploy your app you now must run the following command:
npm run build then npm start
You will only need to run the build command if your front end resources change. So if you only ever use sweetalert.min.js you will only need to run the build the first time you run your app. If later on you add another resource aNewResource.js you will need to update the build.js file and run npm run build again.

Sharing code between React Native + Node

I am using React Native and Node.js. I want to share code between the two. My folder structure is as so.
myreactnativeapp/
mynodeserver/
myshared/
In the react native and node apps I have included the
package.json
"dpendencies" : {
"myshared": "git+https://myrepository/ugoshared.git"
}
This can then be included in each project via require/import etc. This all works fine and for production I'm happy with it. (Though I'd love to know a better way?)
The issue I'm facing is in development it's really slow.
The steps for a change to populate are:
Make changes in Shared
Commit Changes to git
Update the npm module
In development, I really want the same codebase to be used rather than this long update process. I tried the following:
Adding a symlink in node_models/shared - doesn't work in react-native package mangaer
Using relative paths ../../../shared - doesn't work in react-native package mangaer
Any other ideas?
Update 1
I created a script.sh which I run to copy the files into a local directory before the package manager starts. It's not ideal but at least I only have to restart the packager instead of messing with git etc.
#myreactnativeapp/start.sh
SOURCE=../myshared
MODULE=myshared
rm -rf ./$MODULE
mkdir ./$MODULE
find $SOURCE -maxdepth 1 -name \*.js -exec cp -v {} "./$MODULE/" \;
# create the package.json
echo '{ "name": "'$MODULE'" }' > ./$MODULE/package.json
# start the packager
node node_modules/react-native/local-cli/cli.js start
Then in my package.json I update the script to
"scripts": {
"start": "./start.sh",
},
So, the process is now.
Make a change
Start/Resetart the packager
Automatic:
Script copies all .js files under myshared/ -> myreactnativeapp/myshared/
Script creates a package.json with the name of the module
Because I've added the package.json to the copied files with the name of the module, in my project I can just include the items the same as I would if the module was included via the package manager above. In theory when I switch to using the package in production I wont have to change anything.
Import MyModule from 'myshared/MyModule'
Update 2
My first idea got tiresome restarting the package manager all the time. Instead i created a small node script in the shared directory to watch for changes. Whenever there is a change it copies it to the react native working directory.
var watch = require('node-watch')
var fs = require('fs')
var path = require('path')
let targetPath = '../reactnativeapp/myshared/'
watch('.', { recursive: false, filter: /\.js$/ }, function(evt, name) {
console.log('File changed: '+name+path.basename(__filename))
// don't copy this file
if(path.basename(__filename) === name) {
return
}
console.log(`Copying file: ${name} --> ${targetPath+name}`);
fs.copyFile(name, targetPath+name, err => {
if(err) {
console.log('Error:', err)
return;
}
console.log('Success');
})
});
console.log(`Starting to watch: ${__dirname}. All files to be copied to: ${targetPath}`)

NPM CLI application relative path doesn't work

I'm creating a CLI application in NodeJS and the package is going to be published on NPM. The application is very simple as it has only two files. Here is the structure of application:
package.json
{
"name": "mycliapp",
"version": "1.0.0",
"description": "Some description",
"main": "./bin/cli.js",
"preferGlobal": true,
"bin": {
"mycliapp": "bin/cli.js"
},
}
bin/cli.js
const nodePlop = require('node-plop');
const configPath = './bin/config.js';
const plop = nodePlop(configPath, {
force: argv.force || argv.f
});
bin/config.js
{
// some configuration
}
Now if I create symlink with npm install -g from this directory and run the command mycliapp from the same development directory, it works absolutely fine but if I run this mycliapp command from any other directory in my computer, the const configPath = './bin/config.js' is tried to be taken from the current working directory not from the actual npm package and hence the config file is not found.
How can I solve this issue? I tried using __dirname and __filename with path.join but nothing seems to be working.
I also published this package on npm and installed from there, the same issue is occurring.
The JSON in your package.json is malformed -- you should remove the trailing comma after you set the bin parameter.
Also, if your configuration file is in the same directory as your script, you should reference it within your script as ./config.js.
Finally, you need to include a shebang (#!/usr/bin/env node) at the top of your cli.js file or any file you intend to use as your point of entry into the app so that your system knows what interpreter to use to execute the file.
See this post on the npm blog for more information.

How to pass config options to a global node package?

I have a CLI (using Commander) node module with the following directory structure
|--myProject
| |--lib
| |--config.js
| |--main.js
The config file takes the form
var config = {};
config.host = 'localhost';
config.user = 'admin';
config.password = 'password';
config.database = 'doge';
module.exports = config;
and then in main.js I require the config file and set things up accordingly.
It works great for me locally but I'd love to publish it to npm, having never published a package (and I think it would eventually be useful for others). I just don't know how I would then set the config variables.
Since it's a CLI, I set the following in package.json so that ideally it can be run from anywhere by just using myProject --whateverOptions
"preferGlobal": "true",
"bin": {
"myProject" : "lib/main.js"
}
Say I publish it, and then someone comes along and runs npm install -g myProject. They could go to their global node_modules directory, into myProject/lib and add their own config.js.
I have two issues with this approach though
It seems overly messy for anyone that wants to use my package
If I update the package, and they then npm update will their config.js be erased?

How to make node.js require absolute? (instead of relative)

I would like to 'require' my files always by the root of my project and not relative to the current module.
For example, if you look at Express.js' app.js line 6, you will see
express = require('../../')
That's really bad, IMO. Imagine I would like to put all my examples closer to the root only by one level. That would be impossible, because I would have to update more than 30 examples and many times within each example. To this:
express = require('../')
My solution would be to have a special case for root based: if a string starts with an $ then it's relative to the root folder of the project.
What can I do?
Update 2
Now I'm using RequireJS which allows you to write in one way and works both on client and on server. RequireJS also allows you to create custom paths.
Update 3
Now I moved to Webpack and Gulp.js and I use enhanced-require to handle modules on the server side. See here for the rationale: http://hackhat.com/p/110/module-loader-webpack-vs-requirejs-vs-browserify/
Use:
var myModule = require.main.require('./path/to/module');
It requires the file as if it were required from the main JavaScript file, so it works pretty well as long as your main JavaScript file is at the root of your project... and that's something I appreciate.
There's a really interesting section in the Browserify Handbook:
avoiding ../../../../../../..
Not everything in an application properly belongs on the public npm
and the overhead of setting up a private npm or git repo is still
rather large in many cases. Here are some approaches for avoiding the
../../../../../../../ relative paths problem.
node_modules
People sometimes object to putting application-specific modules into
node_modules because it is not obvious how to check in your internal
modules without also checking in third-party modules from npm.
The answer is quite simple! If you have a .gitignore file that
ignores node_modules:
node_modules
You can just add an exception with ! for each of your internal
application modules:
node_modules/*
!node_modules/foo
!node_modules/bar
Please note that you can't unignore a subdirectory, if the parent is
already ignored. So instead of ignoring node_modules, you have to
ignore every directory inside node_modules with the
node_modules/* trick, and then you can add your exceptions.
Now anywhere in your application you will be able to require('foo')
or require('bar') without having a very large and fragile relative
path.
If you have a lot of modules and want to keep them more separate from
the third-party modules installed by npm, you can just put them all
under a directory in node_modules such as node_modules/app:
node_modules/app/foo
node_modules/app/bar
Now you will be able to require('app/foo') or require('app/bar')
from anywhere in your application.
In your .gitignore, just add an exception for node_modules/app:
node_modules/*
!node_modules/app
If your application had transforms configured in package.json, you'll
need to create a separate package.json with its own transform field in
your node_modules/foo or node_modules/app/foo component directory
because transforms don't apply across module boundaries. This will
make your modules more robust against configuration changes in your
application and it will be easier to independently reuse the packages
outside of your application.
symlink
Another handy trick if you are working on an application where you can
make symlinks and don't need to support windows is to symlink a lib/
or app/ folder into node_modules. From the project root, do:
ln -s ../lib node_modules/app
and now from anywhere in your project you'll be able to require files
in lib/ by doing require('app/foo.js') to get lib/foo.js.
custom paths
You might see some places talk about using the $NODE_PATH
environment variable or opts.paths to add directories for node and
browserify to look in to find modules.
Unlike most other platforms, using a shell-style array of path
directories with $NODE_PATH is not as favorable in node compared to
making effective use of the node_modules directory.
This is because your application is more tightly coupled to a runtime
environment configuration so there are more moving parts and your
application will only work when your environment is setup correctly.
node and browserify both support but discourage the use of
$NODE_PATH.
I like to make a new node_modules folder for shared code. Then let Node.js and 'require' do what they do best.
For example:
- node_modules // => these are loaded from your *package.json* file
- app
- node_modules // => add node-style modules
- helper.js
- models
- user
- car
- package.json
- .gitignore
For example, if you're in car/index.js you can require('helper') and Node.js will find it!
How node_modules Work
Node.js has a clever algorithm for resolving modules that is unique among rival platforms.
If you require('./foo.js') from /beep/boop/bar.js, Node.js will look for ./foo.js in /beep/boop/foo.js. Paths that start with a ./ or ../ are always local to the file that calls require().
If, however, you 'require' a non-relative name such as require('xyz') from /beep/boop/foo.js, Node.js searches these paths in order, stopping at the first match and raising an error if nothing is found:
/beep/boop/node_modules/xyz
/beep/node_modules/xyz
/node_modules/xyz
For each xyz directory that exists, Node.js will first look for a xyz/package.json to see if a "main" field exists. The "main" field defines which file should take charge if you require() the directory path.
For example, if /beep/node_modules/xyz is the first match and /beep/node_modules/xyz/package.json has:
{
"name": "xyz",
"version": "1.2.3",
"main": "lib/abc.js"
}
then the exports from /beep/node_modules/xyz/lib/abc.js will be returned by require('xyz').
If there is no package.json or no "main" field, index.js is assumed:
/beep/node_modules/xyz/index.js
The big picture
It seems "really bad" but give it time. It is, in fact, really good. The explicit require()s give a total transparency and ease of understanding that is like a breath of fresh air during a project life cycle.
Think of it this way: You are reading an example, dipping your toes into Node.js and you've decided it is "really bad IMO." You are second-guessing leaders of the Node.js community, people who have logged more hours writing and maintaining Node.js applications than anyone. What is the chance the author made such a rookie mistake? (And I agree, from my Ruby and Python background, it seems at first like a disaster.)
There is a lot of hype and counter-hype surrounding Node.js. But when the dust settles, we will acknowledge that explicit modules and "local first" packages were a major driver of adoption.
The common case
Of course, node_modules from the current directory, then the parent, then grandparent, great-grandparent, etc. is searched. So packages you have installed already work this way. Usually you can require("express") from anywhere in your project and it works fine.
If you find yourself loading common files from the root of your project (perhaps because they are common utility functions), then that is a big clue that it's time to make a package. Packages are very simple: move your files into node_modules/ and put a package.json
there. Voila! Everything in that namespace is accessible from your entire project. Packages are the correct way to get your code into a global namespace.
Other workarounds
I personally don't use these techniques, but they do answer your question, and of course you know your own situation better than I.
You can set $NODE_PATH to your project root. That directory will be searched when you require().
Next, you could compromise and require a common, local file from all your examples. That common file simply re-exports the true file in the grandparent directory.
examples/downloads/app.js (and many others like it)
var express = require('./express')
examples/downloads/express.js
module.exports = require('../../')
Now when you relocate those files, the worst-case is fixing the one shim module.
If you are using yarn instead of npm you can use workspaces.
Let's say I have a folder services I wish to require more easily:
.
├── app.js
├── node_modules
├── test
├── services
│   ├── foo
│   └── bar
└── package.json
To create a Yarn workspace, create a package.json file inside the services folder:
{
"name": "myservices",
"version": "1.0.0"
}
In your main package.json add:
"private": true,
"workspaces": ["myservices"]
Run yarn install from the root of the project.
Then, anywhere in your code, you can do:
const { myFunc } = require('myservices/foo')
instead of something like:
const { myFunc } = require('../../../../../../services/foo')
Have a look at node-rfr.
It's as simple as this:
var rfr = require('rfr');
var myModule = rfr('projectSubDir/myModule');
I use process.cwd() in my projects. For example:
var Foo = require(process.cwd() + '/common/foo.js');
It might be worth noting that this will result in requireing an absolute path, though I have yet to run into issues with this.
IMHO, the easiest way is to define your own function as part of GLOBAL object.
Create projRequire.js in the root of you project with the following contents:
var projectDir = __dirname;
module.exports = GLOBAL.projRequire = function(module) {
return require(projectDir + module);
}
In your main file before requireing any of project-specific modules:
// init projRequire
require('./projRequire');
After that following works for me:
// main file
projRequire('/lib/lol');
// index.js at projectDir/lib/lol/index.js
console.log('Ok');
#Totty, I've comed up with another solution, which could work for case you described in comments. Description gonna be tl;dr, so I better show a picture with structure of my test project.
There's a good discussion of this issue here.
I ran into the same architectural problem: wanting a way of giving my application more organization and internal namespaces, without:
mixing application modules with external dependencies or bothering with private npm repos for application-specific code
using relative requires, which make refactoring and comprehension harder
using symlinks or changing the node path, which can obscure source locations and don't play nicely with source control
In the end, I decided to organize my code using file naming conventions rather than directories. A structure would look something like:
npm-shrinkwrap.json
package.json
node_modules
...
src
app.js
app.config.js
app.models.bar.js
app.models.foo.js
app.web.js
app.web.routes.js
...
Then in code:
var app_config = require('./app.config');
var app_models_foo = require('./app.models.foo');
or just
var config = require('./app.config');
var foo = require('./app.models.foo');
and external dependencies are available from node_modules as usual:
var express = require('express');
In this way, all application code is hierarchically organized into modules and available to all other code relative to the application root.
The main disadvantage is of course that in a file browser, you can't expand/collapse the tree as though it was actually organized into directories. But I like that it's very explicit about where all code is coming from, and it doesn't use any 'magic'.
Assuming your project root is the current working directory, this should work:
// require built-in path module
path = require('path');
// require file relative to current working directory
config = require( path.resolve('.','config.js') );
I have tried many of these solutions. I ended up adding this to the top of my main file (e.g. index.js):
process.env.NODE_PATH = __dirname;
require('module').Module._initPaths();
This adds the project root to the NODE_PATH when the script is loaded. The allows me to require any file in my project by referencing its relative path from the project root such as var User = require('models/user'). This solution should work as long as you are running a main script in the project root before running anything else in your project.
Some of the answers is saying that the best way is to add the code to the node_module as a package, i agree and its probably the best way to lose the ../../../ in require but none of them actually give a way to do so.
from version 2.0.0 you can install a package from local files, which means you can create folder in your root with all the packages you want,
-modules
--foo
--bar
-app.js
-package.json
so in package.json you can add the modules (or foo and bar) as a package without publishing or using external server like this:
{
"name": "baz",
"dependencies": {
"bar": "file: ./modules/bar",
"foo": "file: ./modules/foo"
}
}
After that you do npm install, and you can access the code with var foo = require("foo"), just like you do with all the other packages.
more info can be found here :
https://docs.npmjs.com/files/package.json#local-paths
and here how to create a package :
https://docs.npmjs.com/getting-started/creating-node-modules
You could use a module I made, Undot. It is nothing advanced, just a helper so you can avoid those dot hell with simplicity.
Example:
var undot = require('undot');
var User = undot('models/user');
var config = undot('config');
var test = undot('test/api/user/auth');
Another answer :
Imagine this folders structure :
node_modules
lodash
src
subdir
foo.js
bar.js
main.js
tests
test.js
Then in test.js, you need to require files like this :
const foo = require("../src/subdir/foo");
const bar = require("../src/subdir/bar");
const main = require("../src/main");
const _ = require("lodash");
and in main.js :
const foo = require("./subdir/foo");
const bar = require("./subdir/bar");
const _ = require("lodash");
Now you can use babel and the babel-plugin-module-resolver with this .babelrc file to configure 2 root folders:
{
"plugins": [
["module-resolver", {
"root": ["./src", "./src/subdir"]
}]
]
}
Now you can require files in the same manner in tests and in src:
const foo = require("foo");
const bar = require("bar");
const main = require("main");
const _ = require("lodash");
and if you want use the es6 module syntax:
{
"plugins": [
["module-resolver", {
"root": ["./src", "./src/subdir"]
}],
"transform-es2015-modules-commonjs"
]
}
then you import files in tests and src like this :
import foo from "foo"
import bar from "bar"
import _ from "lodash"
You could define something like this in your app.js:
requireFromRoot = (function(root) {
return function(resource) {
return require(root+"/"+resource);
}
})(__dirname);
and then anytime you want to require something from the root, no matter where you are, you just use requireFromRoot instead of the vanilla require. Works pretty well for me so far.
Imho the easiest way to achieve this is by creating a symbolic link on app startup at node_modules/app (or whatever you call it) which points to ../app. Then you can just call require("app/my/module"). Symbolic links are available on all major platforms.
However, you should still split your stuff in smaller, maintainable modules which are installed via npm. You can also install your private modules via git-url, so there is no reason to have one, monolithic app-directory.
In your own project you could modify any .js file that is used in the root directory and add its path to a property of the process.env variable. For example:
// in index.js
process.env.root = __dirname;
Afterwards you can access the property everywhere:
// in app.js
express = require(process.env.root);
Manual Symlinks (and Windows Junctions)
Couldn't the examples directory contain a node_modules with a symbolic link to the root of the project project -> ../../ thus allowing the examples to use require('project'), although this doesn't remove the mapping, it does allow the source to use require('project') rather than require('../../').
I have tested this, and it does work with v0.6.18.
Listing of project directory:
$ ls -lR project
project:
drwxr-xr-x 3 user user 4096 2012-06-02 03:51 examples
-rw-r--r-- 1 user user 49 2012-06-02 03:51 index.js
project/examples:
drwxr-xr-x 2 user user 4096 2012-06-02 03:50 node_modules
-rw-r--r-- 1 user user 20 2012-06-02 03:51 test.js
project/examples/node_modules:
lrwxrwxrwx 1 user user 6 2012-06-02 03:50 project -> ../../
The contents of index.js assigns a value to a property of the exports object and invokes console.log with a message that states it was required. The contents of test.js is require('project').
Automated Symlinks
The problem with manually creating symlinks is that every time you npm ci, you lose the symlink. If you make the symlink process a dependency, viola, no problems.
The module basetag is a postinstall script that creates a symlink (or Windows junction) named $ every time npm install or npm ci is run:
npm install --save basetag
node_modules/$ -> ..
With that, you don't need any special modification to your code or require system. $ becomes the root from which you can require.
var foo = require('$/lib/foo.js');
If you don't like the use of $ and would prefer # or something else (except #, which is a special character for npm), you could fork it and make the change.
Note: Although Windows symlinks (to files) require admin permissions, Windows junctions (to directories) do not need Windows admin permissions. This is a safe, reliable, cross-platform solution.
Here is the actual way I'm doing for more than 6 months. I use a folder named node_modules as my root folder in the project, in this way it will always look for that folder from everywhere I call an absolute require:
node_modules
myProject
index.js I can require("myProject/someFolder/hey.js") instead of require("./someFolder/hey.js")
someFolder which contains hey.js
This is more useful when you are nested into folders and it's a lot less work to change a file location if is set in absolute way. I only use 2 the relative require in my whole app.
Just came across this article which mentions app-module-path. It allows you to configure a base like this:
require('app-module-path').addPath(baseDir);
I was looking for the exact same simplicity to require files from any level and I found module-alias.
Just install:
npm i --save module-alias
Open your package.json file, here you can add aliases for your paths, for e.g.
"_moduleAliases": {
"#root" : ".", // Application's root
"#deep" : "src/some/very/deep/directory/or/file",
"#my_module" : "lib/some-file.js",
"something" : "src/foo", // Or without #. Actually, it could be any string
}
And use your aliases by simply:
require('module-alias/register')
const deep = require('#deep')
const module = require('something')
If anyone's looking for yet another way to get around this problem, here's my own contribution to the effort:
https://www.npmjs.com/package/use-import
The basic idea: you create a JSON file in the root of the project that maps your filepaths to shorthand names (or get use-automapper to do it for you). You can then request your files/modules using those names. Like so:
var use = require('use-import');
var MyClass = use('MyClass');
So there's that.
I wrote this small package that lets you require packages by their relative path from project root, without introducing any global variables or overriding node defaults
https://github.com/Gaafar/pkg-require
It works like this
// create an instance that will find the nearest parent dir containing package.json from your __dirname
const pkgRequire = require('pkg-require')(__dirname);
// require a file relative to the your package.json directory
const foo = pkgRequire('foo/foo')
// get the absolute path for a file
const absolutePathToFoo = pkgRequire.resolve('foo/foo')
// get the absolute path to your root directory
const packageRootPath = pkgRequire.root()
Just want to follow up on the great answer from Paolo Moretti and Browserify. If you are using a transpiler (e.g., babel, typescript) and you have separate folders for source and transpiled code like src/ and dist/, you could use a variation of the solutions as
node_modules
With the following directory structure:
app
node_modules
... // normal npm dependencies for app
src
node_modules
app
... // source code
dist
node_modules
app
... // transpiled code
you can then let babel etc to transpile src directory to dist directory.
symlink
Using symlink we can get rid some levels of nesting:
app
node_modules
... // normal npm dependencies for app
src
node_modules
app // symlinks to '..'
... // source code
dist
node_modules
app // symlinks to '..'
... // transpiled code
A caveat with babel --copy-files The --copy-files flag of babel does not deal with symlinks well. It may keep navigating into the .. symlink and recusively seeing endless files. A workaround is to use the following directory structure:
app
node_modules
app // symlink to '../src'
... // normal npm dependencies for app
src
... // source code
dist
node_modules
app // symlinks to '..'
... // transpiled code
In this way, code under src will still have app resolved to src, whereas babel would not see symlinks anymore.
I had the same problem many times. This can be solved by using the basetag npm package. It doesn't have to be required itself, only installed as it creates a symlink inside node_modules to your base path.
const localFile = require('$/local/file')
// instead of
const localFile = require('../../local/file')
Using the $/... prefix will always reference files relative to your apps root directory.
Source: How I created basetag to solve this problem
What I like to do is leverage how Node.js loads from the node_modules directory for this.
If one tries to load the module "thing", one would do something like
require('thing');
Node.js will then look for the 'thing' directory in the 'node_modules' directory.
Since the node_modules folder is normally at the root of the project, we can leverage this consistency. (If node_modules is not at the root, then you have other self-induced headaches to deal with.)
If we go into the directory and then back out of it, we can get a consistent path to the root of the Node.js project.
require('thing/../../');
Then if we want to access the /happy directory, we would do this:
require('thing/../../happy');
Though it is quite a bit hacky, however I feel if the functionality of how node_modules load changes, there will be bigger problems to deal with. This behavior should remain consistent.
To make things clear, I do this, because the name of module does not matter.
require('root/../../happy');
I used it recently for Angular 2. I want to load a service from the root.
import {MyService} from 'root/../../app/services/http/my.service';
If your app's entry point js file (i.e. the one you actually run "node" on) is in your project root directory, you can do this really easily with the rootpath npm module. Simply install it via
npm install --save rootpath
...then at the very top of the entry point js file, add:
require('rootpath')();
From that point forward all require calls are now relative to project root - e.g. require('../../../config/debugging/log'); becomes require('config/debugging/log'); (where the config folder is in the project root).
If you're using ES5 syntax you may use asapp. For ES6 you may use babel-plugin-module-resolver using a config file like this:
.babelrc
{
"plugins": [
["module-resolver", {
"root": ["./"],
"alias": {
"app": "./app",
"config": "./app/config",
"schema": "./app/db/schemas",
"model": "./app/db/models",
"controller": "./app/http/controllers",
"middleware": "./app/http/middleware",
"route": "./app/http/routes",
"locale": "./app/locales",
"log": "./app/logs",
"library": "./app/utilities/libraries",
"helper": "./app/utilities/helpers",
"view": "./app/views"
}
}]
]
}
I created a node module called rekuire.
It allows you to 'require' without the use of relative paths.
It is super easy to use.
We are about to try a new way to tackle this problem.
Taking examples from other known projects like Spring Framework and Guice, we will define a "context" object which will contain all the "require" statement.
This object will then be passed to all other modules for use.
For example,
var context = {}
context.module1 = require("./module1")( { "context" : context } )
context.module2 = require("./module2")( { "context" : context } )
This requires us to write each module as a function that receives opts, which looks to us as a best practice anyway...
module.exports = function(context){ ... }
And then you will refer to the context instead of requiring stuff.
var module1Ref = context.moduel1;
If you want to, you can easily write a loop to do the 'require' statements
var context = {};
var beans = {"module1" : "./module1","module2" : "./module2" };
for ( var i in beans ){
if ( beans.hasOwnProperty(i)){
context[i] = require(beans[i])(context);
}
};
This should make life easier when you want to mock (tests) and also solves your problem along the way while making your code reusable as a package.
You can also reuse the context initialization code by separating the beans declaration from it.
For example, your main.js file could look like so
var beans = { ... }; // like before
var context = require("context")(beans); // This example assumes context is a node_module since it is reused..
This method also applies to external libraries, and there isn't any need to hard code their names every time we require them. However, it will require a special treatment as their exports are not functions that expect context...
Later on, we can also define beans as functions—which will allow us to require different modules according to the environment—but that it out of this question's scope.

Resources