Cache NPM library of different versions - node.js

For my use case, I want to install the NPM package Playwright(https://www.npmjs.com/package/playwright) of different versions in the same node project in zero time.
e.g. for executing one script, I will specify the package version as 1.21.0, for other, I will specify it as 1.26.0, etc. I want his installation to be in zero time. Playwright also installs the browser binaries in the system according to the version installed(https://playwright.dev/docs/browsers).
I tried using yarn zero installs(https://yarnpkg.com/features/zero-installs) feature for this, but it was removing the previous version of Playwright in the .yarn/cache/ folder. Thus, as the cache wasn't found, it was downloading the other version again.
Is there any way to resolve this? Please experiment with the Playwright package as it has the browser files with it. Experimenting with any other npm package might not help in this specific case.

Related

Can you prevent node.js from installing packages locally? (Use global packages)

I've been working on a lot of different node.js projects. All of them have their own package.json file with their own needed packages. Every time I run node <mainfile>.js, npm installs all the packages to the project directory. Like so: C:/Users/me/Projects/<project-name>/node_modules.
This isn't a very big problem, but is there a way to make npm use/install to the global packages? Like in C:/Users/me/node_modules?
One of the advantages I could see this having is less storage being taken up, although it isn't a huge advantage.
I would assume that if it is possible, it would require you to add/modify something in the package.json file.
While looking into answers for this question, I've seen people saying that you should avoid installing packages globally. Can you also explain why this is a bad practice andy why I should avoid it?
Install Package Globally
NPM installs global packages into //local/lib/node_modules folder.
Apply -g in the install command to install package globally.
npm install -g express
To answer your other question
The obvious short answer is that your project depends on them. If your
project depends on a package, it should be documented in package.json
so that you can guarantee that it is installed when someone types npm
install. Otherwise, you’ll need to add extra steps in your README file
to inform anyone else who clones your project that they need to
install each of your global dependencies as well
Finally, even if someone installs the correct version of Browserify
for your project, they may be working on a different project that
requires a different version of that same tool, which would cause
conflicts. Several of your own projects might even use different
versions of Browserify because you updated it when you started a new
project and didn’t go back to make sure that earlier projects were
updated to work with the new version. These conflicts can be avoided.
You can only have one version installed globally. This causes problems if you have different projects that rely on different versions of a package.
Why not to install all packages globally
It's not really you shouldn't install a package globally it's more knowing what packages to install globally. The packages to install globally are ones that your project/application does not depend on.
How to identify a package that my project depends on
A package that your project is depended on is a package that your application could not run without like axios or express (an express API could not run without express installed or a web page that makes API requests with axios cant make those requests without axios) but something like http-server or minify is not needed to run the application so it can be installed globally.
Why is it important to have locally installed packages
It's important/good practice because if you are working with a group of developers or someone gets your work from Github they can just run npm install and get all the packages with out having to find all the packages them selfs.
How can I remove the node modules folder
You could technically globally install every package but I would sudjest not to. Node and many other developers know this is an issue that they have created a solution for in deno "the node killer".
I would recommend not installing all packages globally and if the node modules folder really annoys you try deno it fixes a lot of things that node developers hate.

Webpack failed to compile react code after upgrade to node v8 (on Ubuntu 16)

After upgrading from node v7 to node v8, webpack failed to compile my react code.
It said that some node_modules are not found. Note: This issue does not happen on Windows.
I tried several ways from refreshing node cache & npm_modules to double-checking $NODE_PATH, but still failed to make it work. Looks like an compatibility issue of node v8 and webpack (or some ES6 syntax).
Temporarily I reverted back to node v7 to make it work.
Does anyone encounter this issue?
When you install a new Node version then you need to make sure that you install all modules that you want installed globally using that new version (you need to have both node and npm in your PATH with that new version first before any other version of node and npm that you might have in your PATH or this will likely not work as expected). Then you need to rebuild all locally installed modules (you may need to remove node_modules directory in some cases to make sure that everything is built from scratch).
One issue that is quite common when you have multiple Node versions installed from the binary packages (using either binary tarballs or nvm etc.) is that the npm that gets installed has a general shebang line #!/usr/bin/env node instead of a specific path like #!/opt/node-v8.0.0/bin/node which would be the case if you built Node from the sources yourself.
This means that you can sometimes go into trouble by running npm with global path that would use the wrong version of Node, leading to some issues that can be hard to notice.
The least problematic Node installation in my experience is building it from sources, installing in a versioned directory and explicitly adding it to PATH before installing the modules globally, before local npm install and before running the project that you want to be using that specific Node version. That way you can make sure that you're not using any files that could be left overs from older versions, like would be the case with installing under a common /usr/local prefix.
You can take a look at my tutorial for more details and other options to install Node:
https://gist.github.com/rsp/edf756a05b10f25ee305cc98a161876a
In general what you described is usually caused by clashing installations of Node.

npm install is really slow every time

when running npm install --no-optional, it takes around 3 mins every time to complete. It installs ~ 200MB of files. I would like to speed the build process, but I cannot find any ways to really speed it up.
Doesn't npm install by default cache dependencies (like any other decent tool e.g. maven, sbt or nuget) by default? If yes, shouldn't it be much faster than that? If no, then WHY and how to work around that?
I found npm-cache package, but it seems to .tar all the dependencies and when neither of them changes, npm-cache will reuse the tar file. The downside of this is that, whenever a small change in dependencies occurs, it won't be able to reuse the cache (from what I understand).
Are there any nice resources on why this is slow and how to speed it up and how caching works with npm in general? Other tools that I have used (sbt, maven, nuget) are much faster, therefore my expectations are high for npm as well.
Another option I looked into is npm install -g, but it seems not to solve any problems here, as it is meant to be used for installing some cli tools like grunt, npm-cache and etc., as it adds them to a path. So this definitely doesn't solve the problem.
npm -v: 4.0.5
node -v: 6.8.1
The problem with node was that coming from sbt background, where sbt uses a local ivy cache to cache dependencies, I expect the same behaviour from Node. So at least up to V 5.0, Node didn't have a proper dependency caching mechanism, so you basically needed to redownload all of the dependencies every time you do a node install with a clean node_modules folder.
There were some tool developed to work around that, but none of them were satisfactory.
But it seems that this might have been fixed in Node V 5.0 with some caching strategy, therefore if you have a similar issue, please take a look at the changes for the 5th version.
It's better to install pnpm package using the following command:
npm i -g pnpm
pnpm uses hard links and symlinks to save one version of a module only ever once on a disk. When using npm or Yarn for example, if you have 100 projects using the same version of lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be saved in a single place on the disk and a hard link will put it into the node_modules where it should be installed.
As an example I can mention that whenever you want to install the dependencies of package.json file, what you should do is simply that enter the pnpm i and it handles the other things by itself. Its speed is faster than the npm, because it will reuse the dependencies that you've installed them before!

How is this npm switch fixing Windows path length issues

On Windows NPM has issues due to its deep nesting of dependencies. In order to fix this a friend suggested the following command
npm install <dep> -g --no-bin-link
The man pages say about this command
The --no-bin-links argument will prevent npm from creating symlinks for any binaries the package might contain."
Could anyone explain, in plain language, what the impact this flag has on allowing dependencies to be installed that would usually causing deep path issues?
Could anyone explain, in plain language, what the impact this flag has on allowing dependencies to be installed that would usually causing deep path issues?
Sure. Many packages published on npm can be used both as a command-line tool and programmatically. For example the jslint package publishes both a command-line tool for linting files and an API that can be required, so you can write code that uses jslint
The deep path issues usually become visible when creating the files that go into the bin directory, for command-line use. The deep paths usually do not affect packages used programmatically with require.
So for "regular" dependencies of a package, it is usually harmless to omit the bin links, because those dependencies are consumed with require.
For "dev" dependencies or packages installed globally, it is usually necessary to keep the bin links, because those packages are more likely to be used as command-line tools.
Incidentally you should update to the latest npm if you haven't yet -- latest is 2.1.16 at this writing, and a guide for updating npm on windows is here: https://github.com/npm/npm/wiki/Troubleshooting#upgrading-on-windows

How to upgrade node.js on Windows?

I already have Node.js v0.8.0 running on Windows. Can I just run the latest installer to upgrade it to v0.8.4? I am afraid it will break existing third party modules on my machine.
Yes, you just install the latest version. Generally you shouldn't have any compatibility problems if you are already using the same major version (e.g. Version 0.8.x). If you are concerned about changes, you can always check the changelog for each version (link to changelog is on node.js download page at nodejs.org). That should tell you of any big changes (i.e API changes, etc).
For the record, I have just gone through the process, and it is painless even if you upgrade to another major version.
I have moved from 0.8 to 0.10, using the .msi package, overwriting the one installed on my system. Package problems were all fixed with npm update -g. Worked like a charm.
In case it does not work like a charm:
npm cache clean usually fixes the problem. Once the cache is empty, just run npm update -g again.
In case you really run into trouble:
Delete the modules you have installed globally, then reinstall them. Here's how:
Take stock of what you have:
npm list -g --depth=0 lists all top-level packages, with version numbers.
npm list -g --parseable --depth=0 > npm-global-modules.txt writes them to a file in your cwd.
Any strange stuff you didn't install yourself has probably been installed by another module (rare, but I have seen it happen). Remove those modules from the list. Also remove the module "npm".
In an editor, format the output for the command line by replacing \n?[^\n]+[\\/] (regex) with a single space.
(I didn't get this to work with findstr in a pipe, hence the roundtrip to the editor. You can also do it manually, of course ;)
Delete all modules. On Windows, delete (or rename) the %appdata%\npm directory. For other OS, see Command to remove all npm modules globally?
Reinstall the modules with npm install -g [your module list here]. Don't forget to npm cache clean before you do it.
I don't have experience with node on Windows, but I have just upgraded node & modules on my Mac, so this is just a general answer:
If you install v0.8, you might break your existing node modules, if they use deprecated functions, etc. The problem is that npm only checks your version of node while modules are being installed, not at run-time.
To be on the safe side, you need to find the global node_modules folder on your machine, back it up to somewhere, then delete and reinstall the modules. You will need to do the same thing for the node_modules folders in the apps you are using. (Assuming you have package.json files, reinstalling these should be easy.)
In practice, I don't think any of the modules I was using were actually incompatible. Good luck.
Yes. You can upgrade your node.js version to the latest by running the installer for latest node.js version at https://nodejs.org/en/. I upgraded mine from 4.4.4 to 8.11.2 running the installer.
Unless you're using a module that relies on an actual bug that was present in 0.8.0 and was fixed by 0.8.4, you're OK. There were no API changes in between those two versions (and the node team is too smart to introduce such changes in a minor release).
Currently upgrading to 4.4.*. I just used to installer from the nodejs.org website and that upgraded everything works just fine.
Just go to nodejs site & download it. You can install it directly without any hesitation. If you have any dependency on earlier version then check change logs.
JUST GO TO
nodejs.org
INSTALL THE LATEST STABLE VERSION (Recommended For Most Users)
and then run
npm install node
Now, you are good to go
You can even check the version of NodeJS using command
node --version
Best way to install node on windows is by using nvm-windows, so you can quickly switch between versions if you need to. This is analogous to the best way of installing node on linux and max, ie with nvm.
But Benjen is right (how could he not be with all his scouting experience) that you can just install a different version of node, and your version will update. Your npm dependencies may have to be reinstalled, and any extensions that aren't managed by npm may need to be recompiled, but this will be true no matter how you change your node version.

Resources