Recently I start committing my application node_modules folder into VCS to speed up deployments and fix dependencies.
I noticed that many npm packages contain a bunch of stuff unnecessary to me like tests and various builds that I'll never use and I wrinkle every time when I put it in my repo.
So, what should one put into npm package?
The tests and other items are usually a good item to include in your devDependencies.
You can install packages without them by using npm install --production or setting the configuration flag to production using npm config set production
I would recommend looking at this page and reading the information in the different types of dependencies to get an understanding of what each does.
That being said the bare minimum to include is just what it takes for your module to run but that varies based on the module you're creating. Although a README.md is almost essential if you're sharing your package publicly so users can git a quick overview of your package on npm and github.
Related
I work at a largish project with ~10 devs. We have package.json and the resulting package-lock.json committed, and our ci pipeline does npm ci to restore packages according to package-lock.json.
Currently, the developers are instructed to clone the repo and run npm install. However, I found that npm install will install different versions that match the version spec in package.json - for example, ^5.0.5 might cause npm install to install version 5.1.1, or to keep 5.0.5 if it was already in there.
So, I want to change the instructions for developers to:
(common case) If you don't want to change packages or package versions, only use npm ci
If you do, use npm install and/or npm update (possibly with --save-dev), test locally, and then commit the resulting package.json and pacakge-lock.json.
Are these instructions sound? Am I missing something?
Per documentation "this command is similar to npm install, except it's meant to be used in automated environments such as test platforms, continuous integration, and deployment -- or any situation where you want to make sure you're doing a clean install of your dependencies." (emphasis mine).
I prefer using it instead of "install", because it gives some insurances about state of node_modules folder.
It will remove modules folder, if it is present, which will remove everything that is not in lock file, but may accidentally be present from previous install.
It will throw an error if someone changed dependencies by hand and didn't updated lock file.
It will be faster than install, because it doesn't need to build new dependency tree, and it will preserve versions of dependencies which were installed by tag (like latest or next) or by wild card (*). And sometimes this is a very good thing - recent colors incident is a good illustration.
Basically it means that me and all my colleagues will get identical node_modules folder contents. One of the advantages of Yarn in early days were reproducible installs with lock-file, and it is considered a good practice.
I am using angular-cli and building a SPA using Angular2. I have a Jenkins build system for my application where in every time there is a change in my project repository, a build is triggered which basically deletes the entire node_modules folder and then does npm install followed by my build process. All this is done remotely on a Linux machine.
Problem:
Now the issue I am facing is that of the secondary and tertiary dependencies. Most of the dependencies (if not all) I am installing using npm have their own packagae.json file which in turn have their own, so on & so forth. So even if I freeze the versions in my main package.json file by removing carets or tildes, there is no way I can control the version of the secondary and tertiary dependencies. This is resulting into a lot of UNMET PEER DEPENDENCY errors as one dependency needs one version of the same component while the other one needs another!
Question:
So my question is, how do I make sure that this does not happen and achieve a stable dependency installation?
You can keep your package.json as is and run npm shrinkwrap, which will create a new file npm-shrinkwrap.json with the exact versions of all package hierarchy installed at the time you ran it.
If you commit this file, the next time you run npm install, npm should detect the file and respect it.
Documentation:
https://docs.npmjs.com/cli/shrinkwrap
P.S.
Another option that works similarly is Facebook's yarn npm client (a tool similar to the local npm tool).
It uses its own yarn.lock file, and it's much faster as it caches dependencies in its own shared cache making next installs much faster.
But for your use case on the build server, it might be harder to set it up or so. That's why I emphasized the answer on npm itself.
I'm quite new to ReactNative so sorry if it's obvious, but..
Each RN project init-ed via CLI has a large number of node modules stored in project_root/node_modules. Not that I would mind, but if you have several projects it seems redundant and takes up time/space to move it to the source versioning system.
Wouldn't it be possible to retrieve all these same modules from the general node_modules on the machine instead ?
You never want to store dependencies nested in node_modules in your source control... it defeats the whole purpose of versioning and dependencies in general. Your package.json file will specify the versions so when you run npm install it knows exactly which dependencies to grab.
As an alternative, Yarn is an up and rising package client that Facebook developed that does a much better job of caching your packages locally so that way if multiple projects reuse the same depencencies, it will still satisfy the need to keep them in node_modules but doesn't need to perform http requests for each one.
Yarn doesn't replace NPM as a package registry, just a better client to download, maintain, and cache those packages.
Yarn also adds a yarn.lock file (similar to Ruby's Gemfile.lock) that allows you to lock in the specific versions used in your app, regardles of the package.json. This file can be stored in version control, which is probably what you were wanting to achieve by saving the node_modules in version control.
Some good reads...
Yarn vs NPM
Scotch.io Yarn Tutorial
Why I'm working on Yarn (Yehuda Katz)
I would echo Brad's answer: Don't put node_modules in version control. npm install will install the correct versions from the package.json. Just put package.json in version control, not node_modules.
However, if you still want to save disk space, you can install some of your dependencies in a general node_modules folder by using the link option:
npm config set link true -g
You can read more about link here: https://docs.npmjs.com/misc/config#link.
Note that you must not include node_modules in your version control when using this option since npm will put symlinks to the globally installed packages in node_modules. The global install location varies from machine to machine, so if node_modules is in version control, it may link to non-existent locations.
This started happening just recently, but every time I run npm install I end up getting dozens of node modules beyond what's listed in package.json.
This answer shows that this is a new feature of npm 3 where the dependencies are being "flattened" instead of nested. However, I don't want to look at a bazillion modules every time I venture into the folder. Is there any way I can disable this setting?
No, that cannot be disabled.
https://github.com/npm/npm/issues/10079
is there anyway that I can force npm#3 to install new package for me, but old way? So without calculating project-wise tree. I just want the new package to be placed in node_modules with its dependencies in its node_modules?
No. The new installer is pretty much a complete rewrite, and while there is some special-case code to install packages into siloed subdirectories, that's only available when doing global installs, to simplify packaging and managing shared tools.
I'm not regular node user, so my apologies if this is a stupid newbie question, but I haven't been able to find any clear documentation on this, and my feeble newbie node skills don't let me dig into myself.
I'm following along with these instructions for installing the Ghost blogging system, (a system built with NodeJS).
After telling me to open a terminal window in the just downloaded package folder, yhe instructions include the following line
In the new terminal tab type npm install --production
This confuses me. My understanding of npm is it's a package manager that, like perl's CPAN
Fetches packages from The Internet
Installs them into my local node system
That's clearly not what's happening above, but I don't know what is happening when I run that command, and since I don't run with a NodeJS crowd I don't know who to ask.
I'd like to know what NPM is doing. Specific questions
When I run npm install, it looks like it's downloading a number of packages (lots of npm http GET in the console). How does NPM know what to download?
Where is it downloading these module files to? How does npm know where to download the files?
What effect does the --production flag have on NPM's behavior?
Happy to have specific answers, or a meta-answer that points out where I can learn how npm works with (what appears to be) a application installs (vs. a system install, which is how I normally think of it)
npm has a few different installation modes. From within a module (with a package.json file) npm install installs the dependencies listed in the dependencies and devDependencies fields of the package.json file. Installation means that files the modules are downloaded, placed in the node_modules folder, then npm installed themselves, (but only their dependencies) placing modules their own node_modules folders. This continues until everything needed is installed. Use npm ls to see the tree of installed packages.
Most of the time this is what you want, because running npm install from within a module is what you would do when developing on it, and you'll want to run tests etc. (which is what devDependencies is for).
Occasionally though, you'll be coding a service that consumes modules, but should not necessarily be treated like one (not intended to be require'd). Ghost is such a case. In these cases, you need npm install --production, which only installs the dependencies, leaving the devDependencies.
When I run npm install, it looks like it's downloading a number of
packages (lots of npm http GET in the console). How does NPM know what
to download?
It reads the package.json configuration file in the current directory.
Where is it downloading these module files to? How does npm know where to download the files?
It will create and populate a node_modules directory within the current directory. The file structure is designed in to npm/node and is (mostly) intentionally not configurable.
What effect does the --production flag have on NPM's behavior?
Install just the dependencies without the devDependencies from package.json, meaning "give me what I need to run this app, but I don't intend do do development on this app so I don't need dev-only stuff".
npmjs.org has some docs, FAQ, and man pages, which are pretty good although they are mostly lacking basic introductory material.