I recently started using pnpm as a package manager to for my node modules. This download allows for just one version of a package to be saved only once on a disk. This saves my hard drive space by only downloading the packages that the aforementioned download doesn't already have in a global scope. I was wondering if there are any other viable options that would assist in not having to run 'npm install' for every new project. Thanks in advance.
There is a new beta command in pnpm that allows doing installation in many projects at the same time. It is called pnpm recursive install.
Run pnpm help recursive to see the docs. As of pnpm#1.25, it prints:
pnpm recursive [concurrency] install
Experimental! Concurrently runs installation in all subdirectories with a package.json (excluding node_modules).
Options: same as for pnpm install
pnpm recursive [concurrency] update
Experimental! Concurrently runs update in all subdirectories with a package.json (excluding node_modules).
Options: same as for pnpm update
As far as I know, pnpm is currently the only Node.js package manager that has this feature.
Alternatively, you can use some monorepo-managing tool like lerna. It will run the package manager in each project for you. It will be a lot slower than pnpm recursive though.
You can try Yarn by Facebook: https://yarnpkg.com/en/
From their website:
Yarn caches every package it downloads so it never needs to download it again. It also parallelizes operations to maximize resource utilization so install times are faster than ever.
However, you still have to run yarn install for each package. I don't think it's ever really ideal to "share" packages between different installs. What if you had to make a one-off change to a package file? That would then affect all your pojects.
What exactly are you trying to avoid/do with your package manager? Yarn is incredibly easy to use and builds out all dependency relations based upon the package.json.
Everytime you make a new project npm install has to be run as it builds the node-modules that will be used for the node_modules. Whether you use Yarn, or pnpm, you still will have to do the equivalent of yarn add, etc.
For managing many dependencies, yarn is really nice because you can yarn add [insert package here] and yarn start.
Related
Is there an easy way to share one node_modules folders with all angular projects and thus avoiding to download same dependencies every-time when we create a new project ?
if yes , is the method recommended ( what are pros/cons)
Thanks for your time
Package Managers namely npm and yarn have caching mechanism.
The packages are download for the first time only and when you run the install command for the already downloaded package, they won't be downloaded again.
That being said even if you don't have an internet connection you can still install previously cached packages.
Yarn is particularly awesome on handling this and npm is now catching up.
This solves your problem of downloading the packages again. Do lock your package dependencies version. This might help in using the same node_modules for other projects too. But I don't recommend using same node_modules folder for all projects personally as this might mess the entire project. Check out my article about dependency management in JS apps
My team is working on a react native project using GitHub for version control and we have been having a lot of issues dealing with npm dependencies. Every time we push to the master branch we keep getting conflicts with the package-lock.json file ... any best practices around that?
When you install the dependencies of libraries, you need to lock down the version of them. So You should get used to the package manager like npm, yarn, ...
I recommend use Yarn This is the document for how to using it. https://yarnpkg.com/en/docs
ie: install lib react-native-button. you just need run command yarn add react-native-button and commit the yarn.lock with package.json files. Then the guys in your team can pull these changes and run 'yarn install' to keep the synchronize for you projects.
Check it out and try. it is simple!
when running npm install --no-optional, it takes around 3 mins every time to complete. It installs ~ 200MB of files. I would like to speed the build process, but I cannot find any ways to really speed it up.
Doesn't npm install by default cache dependencies (like any other decent tool e.g. maven, sbt or nuget) by default? If yes, shouldn't it be much faster than that? If no, then WHY and how to work around that?
I found npm-cache package, but it seems to .tar all the dependencies and when neither of them changes, npm-cache will reuse the tar file. The downside of this is that, whenever a small change in dependencies occurs, it won't be able to reuse the cache (from what I understand).
Are there any nice resources on why this is slow and how to speed it up and how caching works with npm in general? Other tools that I have used (sbt, maven, nuget) are much faster, therefore my expectations are high for npm as well.
Another option I looked into is npm install -g, but it seems not to solve any problems here, as it is meant to be used for installing some cli tools like grunt, npm-cache and etc., as it adds them to a path. So this definitely doesn't solve the problem.
npm -v: 4.0.5
node -v: 6.8.1
The problem with node was that coming from sbt background, where sbt uses a local ivy cache to cache dependencies, I expect the same behaviour from Node. So at least up to V 5.0, Node didn't have a proper dependency caching mechanism, so you basically needed to redownload all of the dependencies every time you do a node install with a clean node_modules folder.
There were some tool developed to work around that, but none of them were satisfactory.
But it seems that this might have been fixed in Node V 5.0 with some caching strategy, therefore if you have a similar issue, please take a look at the changes for the 5th version.
It's better to install pnpm package using the following command:
npm i -g pnpm
pnpm uses hard links and symlinks to save one version of a module only ever once on a disk. When using npm or Yarn for example, if you have 100 projects using the same version of lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be saved in a single place on the disk and a hard link will put it into the node_modules where it should be installed.
As an example I can mention that whenever you want to install the dependencies of package.json file, what you should do is simply that enter the pnpm i and it handles the other things by itself. Its speed is faster than the npm, because it will reuse the dependencies that you've installed them before!
I'm quite new to ReactNative so sorry if it's obvious, but..
Each RN project init-ed via CLI has a large number of node modules stored in project_root/node_modules. Not that I would mind, but if you have several projects it seems redundant and takes up time/space to move it to the source versioning system.
Wouldn't it be possible to retrieve all these same modules from the general node_modules on the machine instead ?
You never want to store dependencies nested in node_modules in your source control... it defeats the whole purpose of versioning and dependencies in general. Your package.json file will specify the versions so when you run npm install it knows exactly which dependencies to grab.
As an alternative, Yarn is an up and rising package client that Facebook developed that does a much better job of caching your packages locally so that way if multiple projects reuse the same depencencies, it will still satisfy the need to keep them in node_modules but doesn't need to perform http requests for each one.
Yarn doesn't replace NPM as a package registry, just a better client to download, maintain, and cache those packages.
Yarn also adds a yarn.lock file (similar to Ruby's Gemfile.lock) that allows you to lock in the specific versions used in your app, regardles of the package.json. This file can be stored in version control, which is probably what you were wanting to achieve by saving the node_modules in version control.
Some good reads...
Yarn vs NPM
Scotch.io Yarn Tutorial
Why I'm working on Yarn (Yehuda Katz)
I would echo Brad's answer: Don't put node_modules in version control. npm install will install the correct versions from the package.json. Just put package.json in version control, not node_modules.
However, if you still want to save disk space, you can install some of your dependencies in a general node_modules folder by using the link option:
npm config set link true -g
You can read more about link here: https://docs.npmjs.com/misc/config#link.
Note that you must not include node_modules in your version control when using this option since npm will put symlinks to the globally installed packages in node_modules. The global install location varies from machine to machine, so if node_modules is in version control, it may link to non-existent locations.
I work in a team of about 20 other developers. All of our projects utilize npm packages and currently all of our developers are running npm to manage those packages. I'm very curious about Yarn and have it currently installed on my machine. However I'm nervous to actually use it to install packages in case it screws up a project for other developers.
My question is can one developer utilize Yarn on a project while other devs are using npm in the same repo? From what I've read, Yarn uses the same package.json file to get its dependencies. We ignore the node_modules in our repo, however we use npm shrinkwrap to lock dependencies. I know Yarn has a Yarn.lock file and that is where my concern lies. Has anyone attempted to run Yarn independently from their team and what issues have you run into?
Yarn doesn't read npm-shrinkwrap.json. It generates its own yarn.lock. While only you use yarn, the project in your environment might have dependencies version different then that your teammates have. However it's safe to try yarn in your own environment since it doesn't overwrite shrinkwrap file and won't impact other developers.