I'm working on a larger project that is split into a number of npm packages. There are several dependencies between the packages. The whole code base is stored in a main directory like this:
main/
pkg1/
pkg2/
...
Suppose that pkg2 depends on pkg1, so in main/pkg2/package.json I have:
"dependencies": {
"pkg1": "^0.1.0"
}
I have linked my packages together using npm link. However when I start development on a new machine or for some reason I have to reinstall the packages I can't simply say npm install in pkg2/. It would fail because pkg1 couldn't be found. (It's not published, but anyway, I want the local version because I'm developing both packages).
Of course I can do all the linking manually than call npm install but it's a hassle. Is there any way to do it in a single step?
My previous research:
This question proposes writing a preinstall script, but I don't want to keep linking in production, only in the development environment, as another answer points it out.
I've also tried npm link in pkg1/ then npm install --link in pkg2/. According to the manual,
The --link argument will cause npm to link global installs into the local space in some cases.
Not in my case though.
You can use zelda for this. Written by Feross, it was designed for exactly this purpose.
I'm not a fan of doing it this way; I'd generally prefer running a local repository or using git URLs for dependencies like this.
That said, if you want to keep using npm link you can always use the preinstall script approach but not use the preinstall key.
"autolink": "cd ../project1 && npm link && cd ../project2 && npm link project1_name",
Then in your cli you can do $ npm run autolink when you first setup a dev env.
Whenever I run npm install <package> it installs the package alright, but then it automatically runs the prepare script.
It's worth mentioning that I've already checked that there is no postinstall script in the package.json.
From https://docs.npmjs.com/misc/scripts:
prepare: Run both BEFORE the package is packed and published, and on
local npm install without any arguments (See below). This is run AFTER
prepublish, but BEFORE prepublishOnly.
Since NPM v5, prepare script is executed when you run npm install
The other answers are fine, but for some additional context, this is to support a workflow where you can use devDependencies to build assets or other generated content for your project.
For example, say you want to use node-sass (CSS preprocessor). You add "node-sass" as a devDependency, then you run the sass command in your "prepare" script, which generates your CSS.
So, when you run npm install, the following happens:
dependencies and devDependencies get installed
your "prepare" script generates your CSS
your project is ready to go with all necessary CSS
And when you run npm publish, something similar happens:
your "prepare" script generates your CSS
your code and generated CSS are published to the npm repo
So now when someone comes along and installs your package, they don't need node-sass or any of your devDependencies. They only need to runtime deps.
The prepare script runs on local install and when installing git dependencies:
prepare: Run both BEFORE the package is packed and published, on local npm install without any arguments, and when installing git dependencies (See below). This is run AFTER prepublish, but BEFORE prepublishOnly.
https://docs.npmjs.com/misc/scripts
You can avoid it with the --ignore-scripts flag:
$ npm install <package> --ignore-scripts
source: https://docs.npmjs.com/cli/install
From the doc https://docs.npmjs.com/misc/scripts
prepare: Run both BEFORE the package is packed and published, and on local npm install without any arguments (See below). This is run AFTER prepublish, but BEFORE prepublishOnly.
prepare script run before publishing and after npm install.
Now if you make an npm install and one of the packages has a prepare script, like for building, and it fails the whole install will fail.
We have two options:
Ignore scripts
npm install --ignore-scripts
That will run the ignore to all packages, which might be not the desired behavior. Imagine a third party package that needs to run prepare and build. If you run with --ignore-scripts this will get skipped.
Make the script optional (better option)
Add a package to the optionalDependencies:
{
optionalDependencies: {
"myPackage": "^1.0.0"
}
}
If a dependency can be used, but you would like npm to proceed if it cannot be found or fails to install, then you may put it in the optionalDependencies object. This is a map of package name to version or url, just like the dependencies object. The difference is that build failures do not cause installation to fail.
Entries in optionalDependencies will override entries of the same name in dependencies, so it's usually best to only put in one place.
Check the doc:
https://docs.npmjs.com/cli/v7/configuring-npm/package-json#optionaldependencies
Note: With this, only the chosen package is concerned. And if it fails the installation will continue. That's usually what you want.
Go with optionalDependencies
As per this answer in this thread:
https://github.com/npm/npm/issues/2817#issuecomment-368661749
the problem with --ignore-scripts is that is ignores all scripts. I just need to be able to ignore a script(s) for a particular package (the one where a build fails to compile on certain platforms). This option usually breaks my code because it has ignored ALL scripts in other packages that actually do need to run.
Anyway, to make this work like the OP I make the offending package optional. Then do a regular install, then a second install with --ignore-scripts. That way I get the scripts of other packages run first before ignoring them all (including the intended) the second time which then "fetches" the source of that package.
It's generally better to go with optionalDependencies. That will most likely suit your needs.
The Node.js npm (Node Package Manager) has a -g command line argument, which I often see referenced. For example, the documentation for the Microsoft Azure x-plat (cross-platform) CLI tool says to install it by using npm install -g azure-cli.
Question: What does the -g option do?
What options do I have to install node modules?
After writing this I quickly found and old but still applicable post by Isaac (yes, the npm #isaacs). But I still think the below post is informational.
You can install npm modules globally or locally - you already know that, but why?
Globally: npm install -g some-module-a: This module is intended to be used as an executable (i.e. CLI, file watcher, code minifier, logger, etc.).
Locally: npm install some-module-b: To be imported and used in your app via import, var someModule = require('some-module)
global modules are one of the best ideas of npm. We can easily create executables using node/javascript. If your node app is meant to be run as an executable, then you will want others to install it globally. If it's a utility, helper, application, etc. then you usually don't want it installed globally. So, unless the module explicitly states that you should install it with -g, then don't.
One more time: if you are wanting to use some module called some-module in your node app - var someModule = require('some-module'), then npm install some-module from the root of your node app to pull it into your local node_modules directory. If you've installed some-module globally and not locally, it will usually not load and will show you an error about not finding the module (even though it can be made to load the global module - hint: just don't!)
So what exactly happens when you install globally?
npm install -g [some module] installs the specified node module in a directory higher up in your file system (i.e. usually /usr/local/lib/node_modules in unix systems). The biggest use case for global modules is for CLIs written using node (think npm, bower, gulp, grunt, et. al.).
Let's look at what happens when you install bower globally:
*follow these steps in your command line/terminal
step: npm install -g bower
explanation: the module - all of it's files and dependencies - are saved in your global directory (e.g. /usr/local/lib/node_modules/bower).
Something else happened here. Somehow you can now run bower in your command line. Awesome!
step: bower -v --> results in the installed bower version (i.e. 1.6.5)
explanation: It's now a fully executable node app using bower as the keyword. Inside bower's package.json file you'll find a bin property:
"bin": {
"bower": "bin/bower"
}
So how did that all work?
npm will create a symlink from where most executables live, /usr/local/bin/bower over to /usr/local/lib/node_modules/bower/bin/bower, where the module lives. That symlink makes it so when the executable runs, it can reference other files in the original module, including it's local node_modules. Pretty cool, huh?
*Note on executables: If you create a file called awesomeness in /usr/local/bin/ and chmod u+x (user + executable) it. Then write some scripting in it (in this case javascript using #!/usr/bin/env node at the top). Then you can run it anywhere in your command line/terminal just by typing awesomeness.
Hope that helped. I know doing a deeper dive into it helped me early on.
Node.js packages can be installed one of two ways:
Globally
Locally
The -g option instructs npm to install the package globally. You would install a Node.js package globally, if you want to be able to call the command directly from the terminal.
From the documentation:
There are two ways to install npm packages: locally or globally. You choose which kind of installation to use based on how you want to use the package.
If you want to use it as a command line tool, something like the grunt CLI, then you can want to install it globally. On the other hand, if you want to depend on the package from your own module using something like Node's require, then you want to install locally.
To download packages globally, you simply use the command npm install -g , e.g.:
So, in writing command-line utilities with node, if my command requires certain modules, should I include a package.json file to list those dependencies? Obviously this will require running > npm install on the package file.
Without doing so, packages are not found, even though I've installed them globally. How are others distributing these types of cli scripts?
if my command requires certain modules, should I include a package.json file to list those dependencies?
Yes. This is standard.
Obviously this will require running > npm install on the package file.
This isn't exactly the case. You have to run npm install during development to initialize your environment. However, if someone installs your script through npm (e.g. npm install -g your-script-name), the dependencies for your script will automatically be installed. This is a recursive operation, meaning the dependencies of your dependencies will be installed as well, and so on.
What is the difference between:
npm install [package_name]
and:
npm install [package_name] --save
and:
npm install [package_name] --save-dev
What does this mean? And what is really the effect of --save and -dev keywords?
The difference between --save and --save-dev may not be immediately noticeable if you have tried them both on your own projects. So here are a few examples...
Let's say you were building an app that used the moment package to parse and display dates. Your app is a scheduler so it really needs this package to run, as in: cannot run without it. In this case you would use
npm install moment --save
This would create a new value in your package.json
"dependencies": {
...
"moment": "^2.17.1"
}
When you are developing, it really helps to use tools such as test suites and may need jasmine-core and karma. In this case you would use
npm install jasmine-core --save-dev
npm install karma --save-dev
This would also create a new value in your package.json
"devDependencies": {
...
"jasmine-core": "^2.5.2",
"karma": "^1.4.1",
}
You do not need the test suite to run the app in its normal state, so it is a --save-dev type dependency, nothing more. You can see how if you do not understand what is really happening, it is a bit hard to imagine.
Taken directly from NPM docs docs#dependencies
Dependencies
Dependencies are specified in a simple object that maps a package name
to a version range. The version range is a string that has one or
more space-separated descriptors. Dependencies can also be identified
with a tarball or git URL.
Please do not put test harnesses or transpilers in your dependencies
object. See devDependencies, below.
Even in the docs, it asks you to use --save-dev for modules such as test harnesses.
--save-dev is used to save the package for development purpose.
Example: unit tests, minification..
--save is used to save the
package required for the application to run.
By default, NPM simply installs a package under node_modules. When you're trying to install dependencies for your app/module, you would need to first install them, and then add them to the dependencies section of your package.json.
--save-dev adds the third-party package to the package's development dependencies. It won't be installed when someone runs npm install directly to install your package. It's typically only installed if someone clones your source repository first and then runs npm install in it.
--save adds the third-party package to the package's dependencies. It will be installed together with the package whenever someone runs npm install package.
Dev dependencies are those dependencies that are only needed for developing the package. That can include test runners, compilers, packagers, etc.
Both types of dependencies are stored in the package's package.json file. --save adds to dependencies, --save-dev adds to devDependencies
npm install documentation can be referred here.
--
Please note that --save is now the default option, since NPM 5. Therefore, it is not explicitly needed anymore. It is possible to run npm install without the --save to achieve the same result.
Let me give you an example,
You are a developer of a very SERIOUS npm library which uses different testing libraries to test the package.
Users download your library and want to use it in their code. Do they need to download your testing libraries as well? Maybe you use jest for testing and they use mocha. Do you want them to install jest as well? Just To run your library?
No. right? That's why they are in devDependencies.
When someone does, npm i yourPackage only the libraries required to RUN your library will be installed. Other libraries you used to bundle your code with or testing and mocking will not be installed because you put them in devDependencies. Pretty neat right?
So, Why do the developers need to expose the devDependancies?
Let's say your package is an open-source package and 100s of people are sending pull requests to your package. Then how they will test the package? They will git clone your repo and when they would do an npm i the dependencies as well as devDependencies.
Because they are not using your package. They are developing the package further, thus, in order to test your package they need to pass the existing test cases as well write new. So, they need to use your devDependencies which contain all the testing/building/mocking libraries that YOU used.
A perfect example of this is:
$ npm install typescript --save-dev
In this case, you'd want to have Typescript (a javascript-parseable coding language) available for development, but once the app is deployed, it is no longer necessary, as all of the code has been transpiled to javascript. As such, it would make no sense to include it in the published app. Indeed, it would only take up space and increase download times.
As suggested by #andreas-hultgren in this answer and according to the npm docs:
If someone is planning on downloading and using your module in their program, then they probably don't want or need to download and build the external test or documentation framework that you use.
However, for webapp development, Yeoman (a scaffolding tool that installs a peer-reviewed, pre-written package.json file amongst other things) places all packages in devDependencies and nothing in dependencies, so it appears that the use of --save-dev is a safe bet in webapp development, at least.
--save-dev saves semver spec into "devDependencies" array in your package descriptor file, --save saves it into "dependencies" instead.
--save-dev is used for modules used in development of the application,not require while running it in production environment
--save is used to add it in package.json and it is required for running of the application.
Example: express,body-parser,lodash,helmet,mysql all these are used while running the application use --save to put in dependencies while mocha,istanbul,chai,sonarqube-scanner all are used during development ,so put those in dev-dependencies .
npm link or npm install will also install the dev-dependency modules along with dependency modules in your project folder
Read Complete And Forget --save-dev Headache
Simplest answer is that --save-dev is useful when you are creating packages for other developers and want to host your package at NPM Registry like lodash, mongoose, express etc. When you are building or writing a Node Server there is no difference between --save and --save-dev because your Node Server implementation is private to you and you will never publish it on NPM.
How NPM Install Works
Whenever we install a new package using npm like npm install express then NPM installs that package to our system and put it into node_modules folder, now NPM will analyze the package.json file of newly installed package i.e express in this case, after analyzing NPM will install all those packages which were mentioned in dependencies section of package.json file of express package. After installing those packages on which express was dependent NPM again analyze the package.json file of all newly installed packages and again install the packages for them, this cycle goes on until all packages are available into node_modules folder to function properly. You can check package dependencies by running npm list in terminal where terminal should point location of your project directory.
How --save-dev Is Related To Above Explained Stuff
Suppose you want to create a new package like express, now while development of this new package you probably want to write some unit testing code and test the package with any other available testing package let's assume mocha in this case. Now you know mocha is only required to test the package not required to use the package. In this case you should install mocha using --save-dev flag, otherwise NPM will install it whenever a developer install your package using NPM. So if we want a dependency not installed when someone install our package from NPM we must install that package using --save-dev in development phase.
Last Thing
Do not mix --save-dev with collaboration development, if someone cloned your package code from a source version control system like github then NPM will surely install all devDependencies i.e package installed using --save-dev also.
Clear answers are already provided. But it's worth mentioning how devDependencies affects installing packages:
By default, npm install will install all modules listed as dependencies in package.json . With the --production flag (or when the NODE_ENV environment variable is set to production ), npm will not install modules listed in devDependencies .
See: https://docs.npmjs.com/cli/install
When you install an npm package using npm install <package-name>, you are installing it as a dependency.
The package is automatically listed in the package.json file, under the dependencies list (as of npm 5: before you had to manually specify --save).
ex. npm install lodash
After pressing enter check your package.json file.
"dependencies": {
"lodash": "4.x",
},
When you add the -D flag, or --save-dev, you are installing it as a development dependency, which adds it to the devDependencies list.
ex. npm install --save-dev lite-server
After pressing enter check your package.json file
"devDependencies": {
"lite-server": "^2.6.1"
},
Development dependencies are intended as development-only packages, that are unneeded in production. For example testing packages, webpack, or Babel.
When you go in production, if you type npm install and the folder contains a package.json file, they are installed, as npm assumes this is a development deploy.
You need to set the --production flag (npm install --production) to avoid installing those development dependencies.
All explanations here are great, but lacking a very important thing: How do you install production dependencies only? (without the development dependencies).
We separate dependencies from devDependencies by using --save or --save-dev.
To install all we use:
npm i
To install only production packages we should use:
npm i --only=production
You generally don't want to bloat production package with things that you only intend to use for Development purposes.
Use --save-dev (or -D) option to separate packages such as Unit Test frameworks (jest, jasmine, mocha, chai, etc.)
Any other packages that your app needs for Production, should be installed using --save (or -S).
npm install --save lodash //prod dependency
npm install -S moment // " "
npm install -S opentracing // " "
npm install -D jest //dev only dependency
npm install --save-dev typescript //dev only dependency
If you open the package.json file then you will see these entries listed under two different sections:
"dependencies": {
"lodash": "4.x",
"moment": "2.x",
"opentracing": "^0.14.1"
},
"devDependencies": {
"jest": "22.x",
"typescript": "^2.8.3"
},
--save-dev (only used in the development, not in production)
--save (production dependencies)
--global or -g (used globally i.e can be used anywhere in our local system)
People use npm on production to do wicked cool stuff, Node.js is an example of this, so you don't want all your dev tools being run.
If you are using gulp (or similar) to create build files to put on your server then it doesn't really matter.
Basically We Write
npm install package_name
But specially for Testing Purpose we don't need to run some package while Application is Running in Normal State so that Node introduce good way to solve this problem. Whenever we write
npm install package_name --save-dev
at that time this package is only installed for development purpose.
I want to add some of my ideas as
I think all differences will appear when someone uses your codes instead of using by yourself
For example, you write an HTTP library called node's request
In your library,
you used lodash to handle string and object, without lodash, your codes cannot run
If someone uses your HTTP library as a part of his code. Your codes will be compiled with his.
your codes need lodash, So you need to put in dependencies to compile
If you write a project like monaco-editor, which is a web editor,
you have bundled all your codes and your product env library using webpack, when build completed, only have a monaco-min.js
So someone doesn't care whether --save or --save-dependencies, only he needs is monaco-min.js
Summary:
If someone wants to compile your codes (use as a library),
put lodash which used by your codes into dependencies
If someone want to add more feature to your codes, he needs unit test and compiler, put these into dev-dependencies
as --save is default option for npm, so I use
npm i package
and for --save-dev, I use
npm i package -D
default option will install package as project dependency where as -D is for development dependencies like testing, lint etc. and install package for development process
you can find all the flags here https://docs.npmjs.com/cli/v8/commands/npm-install