Node.js - Are package.json dependencies necessary when installing modules locally? - node.js

I know that doing something like this in package.json :
....
...
"dependencies" : {
"some-node-module" : "*"
}
is a bad idea since you're basically telling node to always update this module to its latest version, even though your code might not be able to handle any other version other than the current one for this particular module.
So I should instead do something like this :
....
...
"dependencies" : {
"some-node-module" : "3.4.1"
}
Which basically tells node to always use the version of the module that my code was built around.
Question
I have an app which I've first tested locally. The app has now been built, and using the package.json dependencies, npm has installed all of the appropriate node modules locally under my app's root folder (as opposed to globally, in some obscure folder I don't have immediate access to and which is irrelevant to this app - I simply don't like global installations of node modules - I find them to.. "abstract").
Given that all of the node modules are now installed locally isn't the node modules dependencies part in my package.json now redundant ?
I mean, what if something happens and npm is not available or the specific version of a module can't be found?
Isn't it best to be independent of dynamic node module installations and just have everything installed locally the first time without having to use the package.json dependencies ?

npm install & update
"you're basically telling node to always update this module to its latest version"
Packages won't be automatically updated. The only time the "*" will be an issue is when you are installing the project for the first time via npm install or when you manually run an update via npm update.
I personally prefer to pick a specific version of a module rather than use any wildcards, but even then there are some gotchas...which is why npm shrinkwrap exists.
npm shrinkwrap
Next gotcha:
basically tells node to always use the version of the module that my
code was built around
Sorta true. Let's say you use version 1.2.3 of your favorite module and package.json reflects that, but in the module itself is a package.json dependency on another module and that uses "*"...so when you install, the new internal dependency and the wildcard can wind up breaking the module you thought was 'locked down'.
See the gotcha? Hard coding a version controls for the top level versions but does not enforce anything beneath that...and if a module author you depend upon (or a module they depend upon) uses wildcards, you can't be 100% sure things will be copacetic.
To strictly enforce a version, you'll want to use npm shrinkwrap. (The link there to the docs provides more background, which is good to understand if your project uses more than a few very simple modules.)
And now...your question.
You say:
I mean, what if something happens and npm is not available or the
specific version of a module can't be found?
Based on the first two parts of this answer, it should now be clear that it doesn't hurt to have the dependencies explicitly listed in the package.json because node isn't checking things every time the app runs. npm uses package.json when specific actions (install, update, etc) are called but even then, it is a manual trigger.
While situations vary, there are very few that I can imagine where omitting dependencies in package.json is a good idea. If you ever wind up having to rebuild the project, you'll be in trouble. If the project is so good you want to share it, you'll be in trouble. Heck, if this is something for work and you want to go on vacation and need to install it on another machine...you'll be in trouble.
So given the fact that after the initial install, dependencies have no negative impact...use --save or add the dependencies to your package.json. Your future self will thank you. :)

Related

Should core Node modules be listed in package.json

My code uses core Node modules such as fs and path. Is there any reason to include them in package.json (npm i fs path)? The README for npm path says "This is an exact copy of the NodeJS ’path’ module published to the NPM registry." Why do these packages have around a million downloads a week?
Is there any reason to include them in package.json (npm i fs path)?
No. These packages have been bundled as part of every major Node release since its inception. They do not need to be installed separately or included in your package.json file.
Why do these packages have around a million downloads a week?
I suppose you could really only speculate, but it's likely that a nonzero number of newcomers aren't aware these modules are available as part of the core of Node and are running npm install fs, etc. following the same idea as some of the documentation or tutorials they're reviewing.
It's prudent to ensure that not only if one does include these modules that they work as intended, but to also prevent unscrupulous actors from namesquatting and enabling dependency confusion vulnerabilities. The npm page for the fs module even states explicitly why they've elected to publish it (emphasis mine):
This package name is not currently in use, but was formerly occupied by another package. To avoid malicious use, npm is hanging on to the package name, but loosely, and we'll probably give it to you if you want it.
As mentioned in similar question, you don't need install it, so you don't need specify it in package.json. Also fs, path are Node.js core modules.
Sorry for my bad english. Have a nice day!

npm update unlinks linked packages

I have a project, which consists of one root node package containing subpackages linked together by npm link - these subpackages depend on each other (listed in package.json dependencies) and the structure basically looks like this:
-rootpackage
--subpackageA
--subpackageB
Lets say subpackageA has dependency on subpackageB, so I link them to avoid publishing/reinstalling subpackageB in subpackageA after every change in the source of subpackageB.
The link works just fine until I run npm update in subpackageA, which causes the subpackageB to be unlinked.
Now, I see two options:
I can theoretically run the npm link operation after each npm install or npm update to ensure the links are always present. This works with postinstall in case of installation, but in case of an update the postinstall is not called. I don't know any postupdate command for npm, which is to be called after update.
Maybe there is a way to do this more cleverly, perhaps with yarn, which I am also using, in a way, that it kind of prevents unlinking or excludes the update for my subpackages, so I don't lose the links between my subpackages, but right now I am not aware of such a way.
Is there any way to make one of those options work or any other way to solve this problem ? I need to keep this and other links so we don't have to run npm link after every installation/update. I can't really find information about this issue anywhere. Btw I am using Node 6.4.0 and NPM 3.10.3.
So the solution is to use Yarn Workspaces or maybe project like Lerna.
Yarn Workspaces is a utility that expects a structure similar to what was described in the question and which maintains the linking subpackages and root automatically. It is very easy to set up (just 2 lines in root package.json and executing yarn for the first time) and after it you don't have to worry about upgrade or install at all, the links stay in place unless you delete them manually.
Lerna expands on that and provides you with additional tooling for managing multipackage projects. It can use Yarn Workspaces internally for the linking if you use yarn but it is not a requirement and works fine with npm. Just make sure to have Git because last time I checked Lerna didn't work with SVN or other VCSs.

Substitute a package in NPM/Node

Is it possible to force an external npm dependency to use a different node.js package that offers the same API but a different implementation?
If you're willing to do that and that module is open source you could fork that on github, change their package.json to include the module you want and use github url for your own package.json like this:
"modulename": "git+https://git#github.com/user/repo.git"
You should be able to download the source of whatever module you would prefer and put that folder within your node_modules folder. From that point you simply require it within your Node.js app like any other NPM module.
I recommend downloading the code for the API you want, creating an src/assets folder, placing it in there, changing the package name in package.json to something not used in npm, then using 'require('newPackageName')' within your code.
If you decide to use some of package.json's capabilities to point towards a specific version (like using "1.4.7" as opposed to "^1.4.7") or if you point to a github address, be careful when you run npm update. It will replace your URL with the latest version in npmjs.org with that specific name. I don't know if it still does this in newer versions of npm, but in the version that works with Node.js 0.12, this is the default behavior.
I can tell you that node shrinkwrap will work, but it will prevent any other packages from being updated as well. No, you cannot just have one shrinkwrapped dependency, it has to be all of them, or npm update won't work.

Installing an npm module as a devDependency of itself

Let's say I've made a TDD tool called foo, and I want to use foo v1 to help me develop foo v2.
But when I npm install --save-dev foo#^1.0.0, npm says "Refusing to install foo as a dependency of itself".
Why does npm refuse to do this?
What can I do instead?
Workarounds I've got so far (and why they're not good enough):
Workaround 1: just require the relevant script directly using a relative require, e.g. require('../lib') (this is how mocha does it, and it's how I've been doing it so far).
but if you're working on a new version of your module, adding features, perhaps even removing old features, then you're constantly having to alter not just the content of your tests but also their format, because you're actually using the thing you're working on to test the thing you're working on. If it breaks, you have to fix it in the dark with no tests to guide you. It would be much better to use the settled v1 API for tdd-ing the new v2 API.
Workaround 2: publish a duplicate of your module to npm under a different name, like "foo-clone". (Then you can just install foo-clone as a devDependency of foo.)
but that seems messy and a misuse of npm. Anyway, if installing an exact clone would work, then what would be the harm in npm allowing me to install [an old version of] foo as a devDependency of foo?
Here is a better alternative to your Workaround 2.
Let's assume that you'll only need this dependency on early stage of development. So, before publishing first production-ready version you'll get rid of it, e.g. by adopting mocha solution (using current stable version to test itself).
In this case, instead of publishing duplicate package you could temporary rename your package (i.e. postfix it with -dev).
To guarantee that this renamed package will not be published, you could also add private flag.
So, your dev package.json will look something like:
{
"name": "mytdd-dev",
"version": "2.0.0-dev",
"private": true,
...
"devDependencies": {
"mytdd": "1.x.x",
...
},
...
}
Then, when your package will be ready for the first release, you'll remove all -dev postfixes, private flag and dev dependency on previous version.
The only problem with this solution is that you won't be able to publish early dev versions of your TDD tool to npm (as long as you'll depend on previous version).
if installing an exact clone would work, then what would be the harm in npm allowing me to install [an old version of] foo as a devDependency of foo
I think it's a safety precautions against circular dependencies.
If you believe that npm should make an exception for devDependencies here, which sounds reasonable to me, then you should post your suggestion to npm issues tracker.

Advantages of bundledDependencies over normal dependencies in npm

npm allows us to specify bundledDependencies, but what are the advantages of doing so? I guess if we want to make absolutely sure we get the right version even if the module we reference gets deleted, or perhaps there is a speed benefit with bundling?
Anyone know the advantages of bundledDependencies over normal dependencies?
For the quick reader : this QA is about the package.json bundledDependencies field, not about the package.
What bundledDependencies do
"bundledDependencies" are exactly what their name implies. Dependencies that should be inside your project. So the functionality is basically the same as normal dependencies. They will also be packed when running npm pack.
When to use them
Normal dependencies are usually installed from the npm registry.
Thus bundled dependencies are useful when:
you want to re-use a third party library that doesn't come from the npm registry or that was modified
you want to re-use your own projects as modules
you want to distribute some files with your module
This way, you don't have to create (and maintain) your own npm repository, but get the same benefits that you get from npm packages.
When not to use bundled dependencies
When developing, I don't think that the main point is to prevent accidental updates though. We have better tools for that, namely code repositories (git, mercurial, svn...) or now lock files.
To pin your package versions, you can use:
Option1: Use the newer NPM version 5 that comes with node 8. It uses a package-lock.json file (see the node blog and the node 8 release)
Option2: use yarn instead of npm.
It is a package manager from facebook, faster than npm and it uses a yarn.lock file. It uses the same package.json otherwise.
This is comparable to lockfiles in other package managers like Bundler
or Cargo. It’s similar to npm’s npm-shrinkwrap.json, however it’s not
lossy and it creates reproducible results.
npm actually copied that feature from yarn, amongst other things.
Option3: this was the previously recommended approach, which I do not recommend anymore. The idea was to use npm shrinkwrap most of the time, and sometimes put the whole thing, including the node_module folder, into your code repository. Or possibly use shrinkpack. The best practices at the time were discussed on the node.js blog and on the joyent developer websites.
See also
This is a bit outside the scope of the question, but I'd like to mention the last kind of dependencies (that I know of): peer dependencies. Also see this related SO question and possibly the docs of yarn on bundledDependencies.
One of the biggest problems right now with Node is how fast it is changing. This means that production systems can be very fragile and an npm update can easily break things.
Using bundledDependencies is a way to get round this issue by ensuring, as you correctly surmise, that you will always deliver the correct dependencies no matter what else may be changing.
You can also use this to bundle up your own, private bundles and deliver them with the install.
Other advantage is that you can put your internal dependencies (application components) there and then just require them in your app as if they were independent modules instead of cluttering your lib/ and publishing them to npm.
If/when they are matured to the point they could live as separate modules, you can put them on npm easily, without modifying your code.
I'm surprised I didn't see this here already, but when carefully selected, bundledDependencies can be used to produce a distributable package from npm pack that will run on a system where npm is not configured. This is helpful if you have e.g. a system that's not networked / not on the internet: bring your package over on a thumb drive (or whatever) and unpack the tarball, then npm run or node index.js and it Just Works.
Maybe there's a better way to bundle up your application to run "offline", but if there is I haven't found it.
Operationally, I look at bundledDependencies as a module's private module store, where dependencies is more public, resolved among your module and its dependencies (and sub-dependencies). Your module may rely on an older version of, say, react, but a dependency requires latest-and-greatest. Your package/install will result in your pinned version in node_modules/$yourmodule/node_modules/react, while your dependency will get their version in node_modules/react (or node_modules/$dependency/node_modules/react if they're so inclined).
A caveat: I recently ran into a dependency that did not properly configure its dependency on react, and having react in bundledDependencies caused that dependent module to fail at runtime.

Resources