If I want to copy a node project: Does it make any difference if I just copy node_modules or install all the modules again from scratch via npm?
2017-05-12
I've updated this answer to reflect changes since the release of npm 3.x and new tools that are available.
npm v3 dependency installation is now non-deterministic meaning you may get different packages depending on the order in which packages have been installed over time. This isn't necessarily a bad thing, just something to be aware of.
Given this change I personally don't copy my node_modules directory around too much (it's still possible though!) and instead opt for a clean install most of the time.
There are new tools like Yarn Package Manager which can speed up the installation process if you are doing that a lot (but as of 2017-05-12 it's unclear how well it handles private npm organisations and private scoped packages).
So the takeaway is still pretty much the same: it won't hurt, but maybe err on the side of a clean install. If something weird does happen and you run into problems then you can just delete node_modules and run npm install.
Original answer from 2014-06-08:
In general it should be fine - I copy the node_modules directory sometimes from my other projects to speed up the setup process.
You can always copy node_modules and then run npm install or npm update in the new project to make sure you've got up-to-date versions. npm will use the files in node_modules as a cache and should only bring down newer content if required.
In short: it won't hurt. If something weird does happen and you run into problems then you can just delete node_modules and run npm install.
Linux command
To copy projects without node modules directory:
In the directory of your projects run this:
rsync -av --progress --exclude="node_modules" <source directory> <target directory>
for example:
rsync -av --progress --exclude="node_modules" . /mnt/d/omer/new-directory
Related
I work at a largish project with ~10 devs. We have package.json and the resulting package-lock.json committed, and our ci pipeline does npm ci to restore packages according to package-lock.json.
Currently, the developers are instructed to clone the repo and run npm install. However, I found that npm install will install different versions that match the version spec in package.json - for example, ^5.0.5 might cause npm install to install version 5.1.1, or to keep 5.0.5 if it was already in there.
So, I want to change the instructions for developers to:
(common case) If you don't want to change packages or package versions, only use npm ci
If you do, use npm install and/or npm update (possibly with --save-dev), test locally, and then commit the resulting package.json and pacakge-lock.json.
Are these instructions sound? Am I missing something?
Per documentation "this command is similar to npm install, except it's meant to be used in automated environments such as test platforms, continuous integration, and deployment -- or any situation where you want to make sure you're doing a clean install of your dependencies." (emphasis mine).
I prefer using it instead of "install", because it gives some insurances about state of node_modules folder.
It will remove modules folder, if it is present, which will remove everything that is not in lock file, but may accidentally be present from previous install.
It will throw an error if someone changed dependencies by hand and didn't updated lock file.
It will be faster than install, because it doesn't need to build new dependency tree, and it will preserve versions of dependencies which were installed by tag (like latest or next) or by wild card (*). And sometimes this is a very good thing - recent colors incident is a good illustration.
Basically it means that me and all my colleagues will get identical node_modules folder contents. One of the advantages of Yarn in early days were reproducible installs with lock-file, and it is considered a good practice.
Every time I create a new project do I have to run ( npm install) for the node_modules folder and dependencies? Or could I just copy the node_modules folder from the previous project on my laptop?
If the dependencies of the projects are identical, then YES - you can copy the node_modules folder across projects.
However, in the project building setup, it is orthodox to run npm install in the target container or target directory.
Look at this answer for more opinion on this.
You must run npm install to install the packages included in package.json. If these packages are exactly the same I think in theory you could just copy the node_modules folder but it should be highly recommended to run npm isntall instead.
Just run npm install, it faster and safer than copy many times, and each project maybe have different dependencies.
yes you need to install node_modules node does not give you permission for each end every library or dependencies npm_install
I am learning Angular and each time it takes like 5 minutes to create a new project because of 100MB folder "node_modules" which CLI creates. And the files in this folder are always the same (unless you add some dependencies which I never do). Is there a way to use one node_modules folder for every project?
Have a look in to https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/
Yarn Workspaces is a feature that allows users to install dependencies
from multiple package.json files in subfolders of a single root
package.json file, all in one go.
Making Workspaces native to Yarn enables faster, lighter installation
by preventing package duplication across Workspaces. Yarn can also
create symlinks between Workspaces that depend on each other, and will
ensure the consistency and correctness of all directories.
npm install -g yarn
You can install all dependencies globally or create a symlink from one place to every project.
BUT it is bad practice, correct way is to use separate node_modules for each project, even if you are using same packages. Once you will need use different versions of same package in different projects and common node_modules will cause a lot headache.
Try to use npm cache and npm install --prefer-offline if you just want to install package faster and don't care about version very match. I didn't use it but believe it should work.
Only packages installed by node (npm install) can be in the node_modules folder. This is because, if someone wants to install your project, instead of downloading the whole project with the node_modules included. They type npm install.
Based on the packag.json the node_modules will now be downloaded to the node_modules folder.
So you can put angular in the node_modules folder if it is an npm package. No you cannot put your own files in this folder.
So you can just copy your package.json to every project and run npm install. Then all the node_modules will be the same.
I would like to install the dependencies for some_project. I know I could cd into some_project and then run yarn install
But I was wondering if it's possible without changing the directory?
There's a bug with yarn that prevents the --modules-folder option from working as intended, however I personally don't like the fact that the option has to be provided in the command line; how would you make sure that on future installs they get installed to your chosen folder?
That's why I came up with this sneaky solution for npm since I wanted full control of the install path, not just into a prefixed version of node_modules. It will work just as well with yarn, in fact we use it in production and we haven't had a problem yet (fingers crossed).
In a nutshell, you need to symlink node_modules to your desired folder in the preinstall event (to trick npm or yarn) and then delete the symlink in the postinstall event.
There's one caveats however, which I didn't mention in the linked answer (since I didn't think of it at the time) and that is, things might not go as planned if the install goes horribly wrong; you'll end up with the symlink in place since the postinstall event might not have been triggered, and then the next install might fail since the symlink already exists.
I'm working in a web application (JavaScript/C#, version controlled by TFS) and our team wants to start using Visual Studio 2015. Microsoft is moving developers to use existing popular tools like Gulp for automated tasks, so I've written a few Gulp tasks that will run on the server.
My problem is that our automated builds generate new project folders on the build server, so I can't run gulp myBuildTask without first running npm install. The npm install adds over 2 minutes to the build process, and it seems very inefficient to download the same dependencies for every build (since they will change rarely).
Is there anyway I can run a Gulp task on a new project folder without first running npm install?
Options I've considered:
Include node_modules in TFS. I couldn't add the node_modules folder to TFS (which would cause it to exist in each new build folder) because bower's nested dependencies have file paths that are too long for Windows. I could go this route without bower, but I'm not certain I want all those files in my solution (much of which is not needed, like readme's and test files).
Run npm install after each automated build.
As already mentioned, I don't want to do this because it adds several minutes to the build process.
Install NPM modules globally.
I'm not sure if this is even possible, but I'm wondering if I can install all project dependencies globally on the build server (avoiding having to install at the project level). My concern with an approach like this is that I don't want to have to manually update the build server's globally installed NPM modules every time we add a gulp plugin.
Ideally, the solution would be something like #3. The modules would install globally, but every build could run an npm install which would verify every module is installed. If a new npm module was added to the package.json, it would be downloaded. This npm install would be pretty fast since in most cases, all modules would already exist (globally installed on the build server).
There are a few things you might do:
Make npm install run faster. For this purpose, use newest npm (if possible) or use npm dedupe. Running dedupe may result in having less dependencies than with plain npm install. Then run npm shrinkwrap which creates npm-shrinkwrap.json file which contain 'freezed' info about what exactly gets installed (and in which version) during npm install.
Remember, node_modules is just a directory, if you can copy / rsync it to your installation, you can skip the npm install phase altogether
Node package resolution approach is to first try local node_modules directory and if not successful, (node_modules not there or dependency missing in node_modules) check out node_modules of the parent directory, then grandparent directory and so on. This means, you don't have to install packages globally, semi-global installation is quite sufficient
:
my_project
node_modules/
dependency1
dependency2
build_001/
build_002/
build_00x/
no node_modules here,
no deps here
Note however, that this, naturally, works only if your dependencies are really not changing. Since in real life you install something new from time to time, slightly enhanced approach might be helpful: organize your directories as follows:
my_project
ver_af729b
node_modules
build_001
build_002
ver_82b5f3
node_modules
build_003
build_004
af729b and 82b5f3 being (prefixes of) sha hashes of your npm-shrinkwrap.json file. If you then add new dependency, shrinkwrap file gets updated, build script creates new ver_something directory and executes npm install in it. Doing all this would naturally require extra work, but it should work great.
------------------ EDIT -------------------
If you are not trying to avoid npm install completely (you just want it to be quick) you can stick to the typical scenario: you checkout the sources always to the same directory, and let npm install re-use the old node_modules as much as possible.
If you want always to create a new directory for your build, you may still create a node_modules symlink to the older version of node_modules - also in this scenario, npm will reuse as much as possible from symlinked folder.