I've been working on a lot of different node.js projects. All of them have their own package.json file with their own needed packages. Every time I run node <mainfile>.js, npm installs all the packages to the project directory. Like so: C:/Users/me/Projects/<project-name>/node_modules.
This isn't a very big problem, but is there a way to make npm use/install to the global packages? Like in C:/Users/me/node_modules?
One of the advantages I could see this having is less storage being taken up, although it isn't a huge advantage.
I would assume that if it is possible, it would require you to add/modify something in the package.json file.
While looking into answers for this question, I've seen people saying that you should avoid installing packages globally. Can you also explain why this is a bad practice andy why I should avoid it?
Install Package Globally
NPM installs global packages into //local/lib/node_modules folder.
Apply -g in the install command to install package globally.
npm install -g express
To answer your other question
The obvious short answer is that your project depends on them. If your
project depends on a package, it should be documented in package.json
so that you can guarantee that it is installed when someone types npm
install. Otherwise, you’ll need to add extra steps in your README file
to inform anyone else who clones your project that they need to
install each of your global dependencies as well
Finally, even if someone installs the correct version of Browserify
for your project, they may be working on a different project that
requires a different version of that same tool, which would cause
conflicts. Several of your own projects might even use different
versions of Browserify because you updated it when you started a new
project and didn’t go back to make sure that earlier projects were
updated to work with the new version. These conflicts can be avoided.
You can only have one version installed globally. This causes problems if you have different projects that rely on different versions of a package.
Why not to install all packages globally
It's not really you shouldn't install a package globally it's more knowing what packages to install globally. The packages to install globally are ones that your project/application does not depend on.
How to identify a package that my project depends on
A package that your project is depended on is a package that your application could not run without like axios or express (an express API could not run without express installed or a web page that makes API requests with axios cant make those requests without axios) but something like http-server or minify is not needed to run the application so it can be installed globally.
Why is it important to have locally installed packages
It's important/good practice because if you are working with a group of developers or someone gets your work from Github they can just run npm install and get all the packages with out having to find all the packages them selfs.
How can I remove the node modules folder
You could technically globally install every package but I would sudjest not to. Node and many other developers know this is an issue that they have created a solution for in deno "the node killer".
I would recommend not installing all packages globally and if the node modules folder really annoys you try deno it fixes a lot of things that node developers hate.
I have a monorepo where I have a /packages folder with many packages.
I can use npm i ./packages or npm i if they are already specified using using the file pointer.
Looks something like this:
"dependencies": {
"#reggi/command": "file:packages/command",
"#reggi/dep-merge": "file:packages/dep-merge",
"#reggi/dep-merge-cli": "file:packages/dep-merge-cli",
"#reggi/dep-pointer": "file:packages/dep-pointer"
}
The issue is that if I install these packages I don't get dev dependencies.
What I really want is to also install the devDependencies of these dependencies.
lerna a popular tool that has pioneered the usage of monorepos, suggests that you should add all the devDependencies for these packages in the root package. The issue with this is that it eliminates the ability for two packages to depend on different versions of a given dev dependency.
What I have done is created a script that merges all the devDependencies into dependencies at preinstall then undoes the changes. This works but can be kind of wonky at times, especially when explaining all this to shrinkwrap.
It would be nice if I could just npm i --allDevDepsFromDeps and it would install all of my dependencies dev dependencies.
Is there any other solution I am missing?
I don't see what you're trying to achieve there, aren't the devDepencies used for ... development?
If you want different version for different package just don't put them in the root but in each package.
The issue is that if I install these packages I don't get dev dependencies.
You should consider those packages as 'production'/'bundled' packages, you don't need dev dependencies in this case.
For example, when you are working on #pkg/A, it will have its own devDep but then if you work on #pkg/B that depends on #pkg/A, the #pkg/A should be the production/bundled version (without devDeps).
Maybe you should have a look at bundledDependencies or peerDependencies, that might help you.
I am new to nodejs. And I found that there are many unknown modules in node_modules folder after I installed three modules (express, jade, gulp) in my local project.
Unknown module examples in the node_modules:
vary
statuses
send
promise
From the tutorials I see from others, after they installed gulp, there will be only one "gulp" folder in their node_modules folder, but this is not my case. Why? Thank you.
The node package manager (npm) updated recently, as part of that update, all modules are installed in the top level node_modules folder. This includes modules that your dependencies need to install. In the past these modules would be nested inside another node_modules folder in express (for example).
This is why the tutorials you read say different, likely they were written before this update.
It is mentioned in the npm changelog here
Your dependencies will now be installed maximally flat. Insofar as is
possible, all of your dependencies, and their dependencies, and THEIR
dependencies will be installed in your project's node_modules folder
with no nesting. You'll only see modules nested underneath one another
when two (or more) modules have conflicting dependencies.
#3697 This will hopefully eliminate most cases where windows users ended up with paths that were too long for Explorer and other standard
tools to deal with.
#6912 (#4761 #4037) This also means that your installs will be deduped from the start.
#5827 This deduping even extends to git deps.
#6936 (#5698) Various commands are dedupe aware now. This has some implications for the behavior of other commands:
npm uninstall removes any dependencies of the module that you
specified that aren't required by any other module. Previously, it
would only remove those that happened to be installed under it,
resulting in left over cruft if you'd ever deduped. npm ls now shows
you your dependency tree organized around what requires what, rather
than where those modules are on disk.
#6937 npm dedupe now flattens the tree in addition to deduping. And bundling of dependencies when packing or publishing changes too:
#2442 bundledDependencies no longer requires that you specify deduped sub deps. npm can now see that a dependency is required by something
bundled and automatically include it. To put that another way,
bundledDependencies should ONLY include things that you included in
dependencies, optionalDependencies or devDependencies.
#5437 When bundling a dependency that's both a devDependency and the child of a regular dependency, npm bundles the child dependency. As a
demonstration of our confidence in our own work, npm's own
dependencies are now flattened, deduped, and bundled in the npm#3
style. This means that npm#3 can't be packed or published by npm#2,
which is something to be aware of if you're hacking on npm.
This documentation answers my question very poorly. I didn't understand those explanations. Can someone say in simpler words? Maybe with examples if it's hard to choose simple words?
EDIT also added peerDependencies, which is closely related and might cause confusion.
Summary of important behavior differences:
dependencies are installed on both:
npm install from a directory that contains package.json
npm install $package on any other directory
devDependencies are:
also installed on npm install on a directory that contains package.json, unless you pass the --production flag (go upvote Gayan Charith's answer), or if the NODE_ENV=production environment variable is set
not installed on npm install "$package" on any other directory, unless you give it the --dev option.
are not installed transitively.
peerDependencies:
before 3.0: are always installed if missing, and raise an error if multiple incompatible versions of the dependency would be used by different dependencies.
expected to start on 3.0 (untested): give a warning if missing on npm install, and you have to solve the dependency yourself manually. When running, if the dependency is missing, you get an error (mentioned by #nextgentech) This explains it nicely: https://flaviocopes.com/npm-peer-dependencies/
in version 7 peerDependencies are automatically installed unless an upstream dependency conflict is present that cannot be automatically resolved
Transitivity (mentioned by Ben Hutchison):
dependencies are installed transitively: if A requires B, and B requires C, then C gets installed, otherwise, B could not work, and neither would A.
devDependencies is not installed transitively. E.g. we don't need to test B to test A, so B's testing dependencies can be left out.
Related options not discussed here:
bundledDependencies which is discussed on the following question: Advantages of bundledDependencies over normal dependencies in npm
optionalDependencies (mentioned by Aidan Feldman)
devDependencies
dependencies are required to run, devDependencies only to develop, e.g.: unit tests, CoffeeScript to JavaScript transpilation, minification, ...
If you are going to develop a package, you download it (e.g. via git clone), go to its root which contains package.json, and run:
npm install
Since you have the actual source, it is clear that you want to develop it, so by default, both dependencies (since you must, of course, run to develop) and devDependency dependencies are also installed.
If however, you are only an end user who just wants to install a package to use it, you will do from any directory:
npm install "$package"
In that case, you normally don't want the development dependencies, so you just get what is needed to use the package: dependencies.
If you really want to install development packages in that case, you can set the dev configuration option to true, possibly from the command line as:
npm install "$package" --dev
The option is false by default since this is a much less common case.
peerDependencies
(Tested before 3.0)
Source: https://nodejs.org/en/blog/npm/peer-dependencies/
With regular dependencies, you can have multiple versions of the dependency: it's simply installed inside the node_modules of the dependency.
E.g. if dependency1 and dependency2 both depend on dependency3 at different versions the project tree will look like:
root/node_modules/
|
+- dependency1/node_modules/
| |
| +- dependency3 v1.0/
|
|
+- dependency2/node_modules/
|
+- dependency3 v2.0/
Plugins, however, are packages that normally don't require the other package, which is called the host in this context. Instead:
plugins are required by the host
plugins offer a standard interface that the host expects to find
only the host will be called directly by the user, so there must be a single version of it.
E.g. if dependency1 and dependency2 peer depend on dependency3, the project tree will look like:
root/node_modules/
|
+- dependency1/
|
+- dependency2/
|
+- dependency3 v1.0/
This happens even though you never mention dependency3 in your package.json file.
I think this is an instance of the Inversion of Control design pattern.
A prototypical example of peer dependencies is Grunt, the host, and its plugins.
For example, on a Grunt plugin like https://github.com/gruntjs/grunt-contrib-uglify, you will see that:
grunt is a peer-dependency
the only require('grunt') is under tests/: it's not actually used by the program.
Then, when the user will use a plugin, he will implicitly require the plugin from the Gruntfile by adding a grunt.loadNpmTasks('grunt-contrib-uglify') line, but it's grunt that the user will call directly.
This would not work then if each plugin required a different Grunt version.
Manual
I think the documentation answers the question quite well, maybe you are just not familiar enough with node / other package managers. I probably only understand it because I know a bit about Ruby bundler.
The key line is:
These things will be installed when doing npm link or npm install from the root of a package and can be managed like any other npm configuration parameter. See npm-config(7) for more on the topic.
And then under npm-config(7) find dev:
Default: false
Type: Boolean
Install dev-dependencies along with packages.
If you do not want to install devDependencies you can use npm install --production
As an example, mocha would normally be a devDependency, since testing isn't necessary in production, while express would be a dependency.
dependencies
Dependencies that your project needs to run, like a library that provides functions that you call from your code.
They are installed transitively (if A depends on B depends on C, npm install on A will install B and C).
Example: lodash: your project calls some lodash functions.
devDependencies
Dependencies you only need during development or releasing, like compilers that take your code and compile it into javascript, test frameworks or documentation generators.
They are not installed transitively (if A depends on B dev-depends on C, npm install on A will install B only).
Example: grunt: your project uses grunt to build itself.
peerDependencies
Dependencies that your project hooks into, or modifies, in the parent project, usually a plugin for some other library or tool. It is just intended to be a check, making sure that the parent project (project that will depend on your project) has a dependency on the project you hook into. So if you make a plugin C that adds functionality to library B, then someone making a project A will need to have a dependency on B if they have a dependency on C.
They are not installed (unless npm < 3), they are only checked for.
Example: grunt: your project adds functionality to grunt and can only be used on projects that use grunt.
This documentation explains peer dependencies really well: https://nodejs.org/en/blog/npm/peer-dependencies/
Also, the npm documentation has been improved over time, and now has better explanations of the different types of dependencies: https://github.com/npm/cli/blob/latest/docs/content/configuring-npm/package-json.md#devdependencies
To save a package to package.json as dev dependencies:
npm install "$package" --save-dev
When you run npm install it will install both devDependencies and dependencies. To avoid install devDependencies run:
npm install --production
There are some modules and packages only necessary for development, which are not needed in production. Like it says it in the documentation:
If someone is planning on downloading and using your module in their program, then they probably don't want or need to download and build the external test or documentation framework that you use. In this case, it's best to list these additional items in a devDependencies hash.
peerDependencies didn't quite make sense for me until I read this snippet from a blog post on the topic Ciro mentioned above:
What [plugins] need is a way of expressing these “dependencies” between plugins and their host package. Some way of saying, “I only work when plugged in to version 1.2.x of my host package, so if you install me, be sure that it’s alongside a compatible host.” We call this relationship a peer dependency.
The plugin does expect a specific version of the host...
peerDependencies are for plugins, libraries that require a "host" library to perform their function, but may have been written at a time before the latest version of the host was released.
That is, if I write PluginX v1 for HostLibraryX v3 and walk away, there's no guarantee PluginX v1 will work when HostLibraryX v4 (or even HostLibraryX v3.0.1) is released.
... but the plugin doesn't depend on the host...
From the point of view of the plugin, it only adds functions to the host library. I don't really "need" the host to add a dependency to a plugin, and plugins often don't literally depend on their host. If you don't have the host, the plugin harmlessly does nothing.
This means dependencies isn't really the right concept for plugins.
Even worse, if my host was treated like a dependency, we'd end up in this situation that the same blog post mentions (edited a little to use this answer's made up host & plugin):
But now, [if we treat the contemporary version of HostLibraryX as a dependency for PluginX,] running npm install results in the unexpected dependency graph of
├── HostLibraryX#4.0.0
└─┬ PluginX#1.0.0
└── HostLibraryX#3.0.0
I’ll leave the subtle failures that come from the plugin using a different [HostLibraryX] API than the main application to your imagination.
... and the host obviously doesn't depend on the plugin...
... that's the whole point of plugins. Now if the host was nice enough to include dependency information for all of its plugins, that'd solve the problem, but that'd also introduce a huge new cultural problem: plugin management!
The whole point of plugins is that they can pair up anonymously. In a perfect world, having the host manage 'em all would be neat & tidy, but we're not going to require libraries herd cats.
If we're not hierarchically dependent, maybe we're intradependent peers...
Instead, we have the concept of being peers. Neither host nor plugin sits in the other's dependency bucket. Both live at the same level of the dependency graph.
... but this is not an automatable relationship. <<< Moneyball!!!
If I'm PluginX v1 and expect a peer of (that is, have a peerDependency of) HostLibraryX v3, I'll say so. If you've auto-upgraded to the latest HostLibraryX v4 (note that's version 4) AND have Plugin v1 installed, you need to know, right?
npm can't manage this situation for me --
"Hey, I see you're using PluginX v1! I'm automatically downgrading HostLibraryX from v4 to v3, kk?"
... or...
"Hey I see you're using PluginX v1. That expects HostLibraryX v3, which you've left in the dust during your last update. To be safe, I'm automatically uninstalling Plugin v1!!1!
How about no, npm?!
So npm doesn't. It alerts you to the situation, and lets you figure out if HostLibraryX v4 is a suitable peer for Plugin v1.
Coda
Good peerDependency management in plugins will make this concept work more intuitively in practice. From the blog post, yet again...
One piece of advice: peer dependency requirements, unlike those for regular dependencies, should be lenient. You should not lock your peer dependencies down to specific patch versions. It would be really annoying if one Chai plugin peer-depended on Chai 1.4.1, while another depended on Chai 1.5.0, simply because the authors were lazy and didn’t spend the time figuring out the actual minimum version of Chai they are compatible with.
A simple explanation that made it more clear to me is:
When you deploy your app, modules in dependencies need to be installed or your app won't work. Modules in devDependencies don't need to be installed on the production server since you're not developing on that machine.
link
I found a simple explanation.
Short Answer:
dependencies
"...are those that your project really needs to be able to work in production."
devDependencies
"...are those that you need during development."
peerDependencies
"if you want to create and publish your own library so that it can be used as a dependency"
More details in this post:
https://code-trotter.com/web/dependencies-vs-devdependencies-vs-peerdependencies
I'd like to add to the answer my view on these dependencies explanations
dependencies are used for direct usage in your codebase, things that usually end up in the production code, or chunks of code
devDependencies are used for the build process, tools that help you manage how the end code will end up, third party test modules, (ex. webpack stuff)
In short
Dependencies - npm install <package> --save-prod installs packages required by your application in production environment.
DevDependencies - npm install <package> --save-dev installs
packages required only for local development and testing
Just typing npm install installs all packages mentioned in the
package.json
so if you are working on your local computer just type npm install and continue :)
Dependencies vs dev dependencies
Dev dependencies are modules which are only required during development whereas dependencies are required at runtime. If you are deploying your application, dependencies has to be installed, or else your app simply will not work. Libraries that you call from your code that enables the program to run can be considered as dependencies.
Eg- React , React - dom
Dev dependency modules need not be installed in the production server since you are not gonna develop in that machine .compilers that covert your code to javascript , test frameworks and document generators can be considered as dev-dependencies since they are only required during development .
Eg- ESLint , Babel , webpack
#FYI,
mod-a
dev-dependents:
- mod-b
dependents:
- mod-c
mod-d
dev-dependents:
- mod-e
dependents:
- mod-a
----
npm install mod-d
installed modules:
- mod-d
- mod-a
- mod-c
----
checkout the mod-d code repository
npm install
installed modules:
- mod-a
- mod-c
- mod-e
If you are publishing to npm, then it is important that you use the correct flag for the correct modules. If it is something that your npm module needs to function, then use the "--save" flag to save the module as a dependency. If it is something that your module doesn't need to function but it is needed for testing, then use the "--save-dev" flag.
# For dependent modules
npm install dependent-module --save
# For dev-dependent modules
npm install development-module --save-dev
dependencies: packages that your project/package needs to work in production.
devDependencies: packages that your project/package needs to work while development but are not needed on production (eg: testing packages)
peerDependencies: packages that your project/package needs to work in tandem with (“colaborating” with them) or as a base, useful mainly when you are developing a plugin/component to let know with which version of the “main” package your plugin/component is supposed to work with (eg: React 16)
Dependencies
These are the packages that your package needs to run, so they will be installed when people run
npm install PACKAGE-NAME
An example would be if you used jQuery in your project. If someone doesn't have jQuery installed, then it wouldn't work. To save as a dependency, use
npm install --save
Dev-Dependencies
These are the dependencies that you use in development, but isn't needed when people are using it, so when people run npm install, it won't install them since the are not necessary. For example, if you use mocha to test, people don't need mocha to run, so npm install doesn't install it. To save as a dev dependency, use
npm install PACKAGE --save-dev
Peer Dependencies
These can be used if you want to create and publish your own library so that it can be used as a dependency. For example, if you want your package to be used as a dependency in another project, then these will also be installed when someone installs the project which has your project as a dependency. Most of the time you won't use peer dependencies.
When trying to distribute an npm package you should avoid using dependencies. Instead you need to consider adding it into peerDependencies.
Update
Most of the time dependencies are just a bunch of libraries that describes your ecosystem. Unless, you're really using a specific version of a library you should instead let the user choose whether or not to install that library and which version to choose by adding it into the peerDependencies.
dependencies are required to run, devDependencies only to develop
When using Webpack to bundle a frontend application, the distinction between dependencies and devDependencies is not so clear. For the final bundle, it doesn't matter where you place the dependencies (but it may be important for other tools). That's why the documentation seems confusing.
I found the explanation here: Do "dependencies" and "devDependencies" matter when using Webpack?