is there any way to route npm install to a specific part of hard drive and when i do npm install it make node_module folder in that part of drive, and when i run any project it look for dependencies in that part of drive,
just like single pool for every project.
then if i have two projects with similar dependencies then i only need to npm install in one project so dependencies become available in pool, and no need to do npm install in another project just npm start
Thank you,
Inzamam Malik
You can achieve something close to what you are describing with the link option.
From https://docs.npmjs.com/misc/config#link:
If true, then local installs will link if there is a suitable globally installed package.
Note that this means that local installs can cause things to be installed into the global space at the same time. The link is only done if one of the two conditions are met:
The package is not already installed globally, or
the globally installed version is identical to the version that is being installed locally.
So you will still have some files in each project's node_modules, but you shouldn't have as large a folder.
To turn this behavior on, run:
npm config set link -g
Edit: There is no way you can avoid running npm install and having a node_modules folder. Node.js always looks in node_modules for dependencies (this behavior pre-dates npm itself). The link option will make npm create symlinks in node_modules, pointing to a common pool. That will reduce disc usage, but you cannot do away with node_modules.
You can use PNPM Package manager, It uses a global pool for dependencies.
Related
I am starting to learn React Native and I am very new to npm package manager. I read that npm can install packages localy or globaly but I am trying to understand what does that mean.
I am reading this page https://docs.npmjs.com/getting-started/installing-npm-packages-locally, can someone explain to me what does this mean please.
If you want to depend on the package from your own module using something like Node.js' require, then you want to install locally, which is npm install's default behavior. On the other hand, if you want to use it as a command line tool, something like the grunt CLI, then you want to install it globally.
Since I am very new to npm, React Native, Node (never used it), I am confused by the very first sentence in this quote. What does it mean "my own module?
If I want to use CRNA, I guess, I would have to install it globally?
If I am to install a package, say CRNA locally or globally, where do I see it installed on my MacBook Pro?
The difference between local and global install is that local install puts it into the node_modules directory of your project (this is what is referred to as "your own module") while global puts it into a system directory (the exact location depends on your OS, on OSX it should be /usr/local/lib/node_modules).
Basically:
Local install ties the installed module to your project: other projects on your computer do not get it but if your project is copied to another computer the module will be installed there too
global install ties it to your computer: you can use it on all of
your projects on your computer but if your project is copied to
another computer the installed module will not be there
And yes, CRNA should be installed globally as it is a general tool not a project's library dependence.
When you install package globally npm install -g <package name> modules drops in {prefix}/lib/node_modules.
Locally - npm install <package name> - drops package in the current working directory.
If you are going to require module in your project you have to install it locally.
If you want to run in from command line you need to install it globally.
If you need more extenden explanation take a look
Since I am very new to npm, React Native, Node (never used it), I am confused by the very first sentence in this quote. What does it mean "my own module?
If you have package.json file, then everything in the same folder is treated as "module". You add dependencies to it by doing npm install --save foo (--save option adds it under dependencies in your package.json).
If I want to use CRNA, I guess, I would have to install it globally?
Not sure what "CRNA" is. But general rule is that mostly everything (libraries...) are installed locally. Which means that they are added to your package.json and installed in same folder under node_modules.
Only case when you want to install something globally (can be added to package.json but is NOT installed in the same folder under node_modules but probably in your home directory), by doing npm install --global bar (--global installs it globally). Is when tool (not library) is project independent, as you can access it from everywhere. Something like create-react-app.
TLDR:
Local are dependencies (libraries) installed in same folder and (usually) added in your package.json as dependencie.
Global are tools installed in your user home folder and (usually) NOT added in your package.json as dependencie.
Let first start with how nodejs finds package.
Suppose you have some folder structure like-
root
-pixel
-project1
-project2
So, if your are working on project1 and required some npm package, nodejs tries to find a folder named node_modules in current directory. If fails, it goes parent(pixel folder) and tries to find node_modules and goes recursively upto root(which is global).
So, if there any package installed globally, you don't need to install it in your current working directory.
So, why don't we install all packages globally? Isn't it saves our harddisk memory?
Yes, true. But as npm packages are updating and changing its version everytime, its necessary to use specific package in your current working package to avoid collusion.
Then how global packages is useful?
Its good idea to install some cli packages to run directly from command line i.e webpack to easy our task.
I am learning Angular and each time it takes like 5 minutes to create a new project because of 100MB folder "node_modules" which CLI creates. And the files in this folder are always the same (unless you add some dependencies which I never do). Is there a way to use one node_modules folder for every project?
Have a look in to https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/
Yarn Workspaces is a feature that allows users to install dependencies
from multiple package.json files in subfolders of a single root
package.json file, all in one go.
Making Workspaces native to Yarn enables faster, lighter installation
by preventing package duplication across Workspaces. Yarn can also
create symlinks between Workspaces that depend on each other, and will
ensure the consistency and correctness of all directories.
npm install -g yarn
You can install all dependencies globally or create a symlink from one place to every project.
BUT it is bad practice, correct way is to use separate node_modules for each project, even if you are using same packages. Once you will need use different versions of same package in different projects and common node_modules will cause a lot headache.
Try to use npm cache and npm install --prefer-offline if you just want to install package faster and don't care about version very match. I didn't use it but believe it should work.
Only packages installed by node (npm install) can be in the node_modules folder. This is because, if someone wants to install your project, instead of downloading the whole project with the node_modules included. They type npm install.
Based on the packag.json the node_modules will now be downloaded to the node_modules folder.
So you can put angular in the node_modules folder if it is an npm package. No you cannot put your own files in this folder.
So you can just copy your package.json to every project and run npm install. Then all the node_modules will be the same.
I'm quite new to ReactNative so sorry if it's obvious, but..
Each RN project init-ed via CLI has a large number of node modules stored in project_root/node_modules. Not that I would mind, but if you have several projects it seems redundant and takes up time/space to move it to the source versioning system.
Wouldn't it be possible to retrieve all these same modules from the general node_modules on the machine instead ?
You never want to store dependencies nested in node_modules in your source control... it defeats the whole purpose of versioning and dependencies in general. Your package.json file will specify the versions so when you run npm install it knows exactly which dependencies to grab.
As an alternative, Yarn is an up and rising package client that Facebook developed that does a much better job of caching your packages locally so that way if multiple projects reuse the same depencencies, it will still satisfy the need to keep them in node_modules but doesn't need to perform http requests for each one.
Yarn doesn't replace NPM as a package registry, just a better client to download, maintain, and cache those packages.
Yarn also adds a yarn.lock file (similar to Ruby's Gemfile.lock) that allows you to lock in the specific versions used in your app, regardles of the package.json. This file can be stored in version control, which is probably what you were wanting to achieve by saving the node_modules in version control.
Some good reads...
Yarn vs NPM
Scotch.io Yarn Tutorial
Why I'm working on Yarn (Yehuda Katz)
I would echo Brad's answer: Don't put node_modules in version control. npm install will install the correct versions from the package.json. Just put package.json in version control, not node_modules.
However, if you still want to save disk space, you can install some of your dependencies in a general node_modules folder by using the link option:
npm config set link true -g
You can read more about link here: https://docs.npmjs.com/misc/config#link.
Note that you must not include node_modules in your version control when using this option since npm will put symlinks to the globally installed packages in node_modules. The global install location varies from machine to machine, so if node_modules is in version control, it may link to non-existent locations.
I'm working in a web application (JavaScript/C#, version controlled by TFS) and our team wants to start using Visual Studio 2015. Microsoft is moving developers to use existing popular tools like Gulp for automated tasks, so I've written a few Gulp tasks that will run on the server.
My problem is that our automated builds generate new project folders on the build server, so I can't run gulp myBuildTask without first running npm install. The npm install adds over 2 minutes to the build process, and it seems very inefficient to download the same dependencies for every build (since they will change rarely).
Is there anyway I can run a Gulp task on a new project folder without first running npm install?
Options I've considered:
Include node_modules in TFS. I couldn't add the node_modules folder to TFS (which would cause it to exist in each new build folder) because bower's nested dependencies have file paths that are too long for Windows. I could go this route without bower, but I'm not certain I want all those files in my solution (much of which is not needed, like readme's and test files).
Run npm install after each automated build.
As already mentioned, I don't want to do this because it adds several minutes to the build process.
Install NPM modules globally.
I'm not sure if this is even possible, but I'm wondering if I can install all project dependencies globally on the build server (avoiding having to install at the project level). My concern with an approach like this is that I don't want to have to manually update the build server's globally installed NPM modules every time we add a gulp plugin.
Ideally, the solution would be something like #3. The modules would install globally, but every build could run an npm install which would verify every module is installed. If a new npm module was added to the package.json, it would be downloaded. This npm install would be pretty fast since in most cases, all modules would already exist (globally installed on the build server).
There are a few things you might do:
Make npm install run faster. For this purpose, use newest npm (if possible) or use npm dedupe. Running dedupe may result in having less dependencies than with plain npm install. Then run npm shrinkwrap which creates npm-shrinkwrap.json file which contain 'freezed' info about what exactly gets installed (and in which version) during npm install.
Remember, node_modules is just a directory, if you can copy / rsync it to your installation, you can skip the npm install phase altogether
Node package resolution approach is to first try local node_modules directory and if not successful, (node_modules not there or dependency missing in node_modules) check out node_modules of the parent directory, then grandparent directory and so on. This means, you don't have to install packages globally, semi-global installation is quite sufficient
:
my_project
node_modules/
dependency1
dependency2
build_001/
build_002/
build_00x/
no node_modules here,
no deps here
Note however, that this, naturally, works only if your dependencies are really not changing. Since in real life you install something new from time to time, slightly enhanced approach might be helpful: organize your directories as follows:
my_project
ver_af729b
node_modules
build_001
build_002
ver_82b5f3
node_modules
build_003
build_004
af729b and 82b5f3 being (prefixes of) sha hashes of your npm-shrinkwrap.json file. If you then add new dependency, shrinkwrap file gets updated, build script creates new ver_something directory and executes npm install in it. Doing all this would naturally require extra work, but it should work great.
------------------ EDIT -------------------
If you are not trying to avoid npm install completely (you just want it to be quick) you can stick to the typical scenario: you checkout the sources always to the same directory, and let npm install re-use the old node_modules as much as possible.
If you want always to create a new directory for your build, you may still create a node_modules symlink to the older version of node_modules - also in this scenario, npm will reuse as much as possible from symlinked folder.
I'm not regular node user, so my apologies if this is a stupid newbie question, but I haven't been able to find any clear documentation on this, and my feeble newbie node skills don't let me dig into myself.
I'm following along with these instructions for installing the Ghost blogging system, (a system built with NodeJS).
After telling me to open a terminal window in the just downloaded package folder, yhe instructions include the following line
In the new terminal tab type npm install --production
This confuses me. My understanding of npm is it's a package manager that, like perl's CPAN
Fetches packages from The Internet
Installs them into my local node system
That's clearly not what's happening above, but I don't know what is happening when I run that command, and since I don't run with a NodeJS crowd I don't know who to ask.
I'd like to know what NPM is doing. Specific questions
When I run npm install, it looks like it's downloading a number of packages (lots of npm http GET in the console). How does NPM know what to download?
Where is it downloading these module files to? How does npm know where to download the files?
What effect does the --production flag have on NPM's behavior?
Happy to have specific answers, or a meta-answer that points out where I can learn how npm works with (what appears to be) a application installs (vs. a system install, which is how I normally think of it)
npm has a few different installation modes. From within a module (with a package.json file) npm install installs the dependencies listed in the dependencies and devDependencies fields of the package.json file. Installation means that files the modules are downloaded, placed in the node_modules folder, then npm installed themselves, (but only their dependencies) placing modules their own node_modules folders. This continues until everything needed is installed. Use npm ls to see the tree of installed packages.
Most of the time this is what you want, because running npm install from within a module is what you would do when developing on it, and you'll want to run tests etc. (which is what devDependencies is for).
Occasionally though, you'll be coding a service that consumes modules, but should not necessarily be treated like one (not intended to be require'd). Ghost is such a case. In these cases, you need npm install --production, which only installs the dependencies, leaving the devDependencies.
When I run npm install, it looks like it's downloading a number of
packages (lots of npm http GET in the console). How does NPM know what
to download?
It reads the package.json configuration file in the current directory.
Where is it downloading these module files to? How does npm know where to download the files?
It will create and populate a node_modules directory within the current directory. The file structure is designed in to npm/node and is (mostly) intentionally not configurable.
What effect does the --production flag have on NPM's behavior?
Install just the dependencies without the devDependencies from package.json, meaning "give me what I need to run this app, but I don't intend do do development on this app so I don't need dev-only stuff".
npmjs.org has some docs, FAQ, and man pages, which are pretty good although they are mostly lacking basic introductory material.