Here is what's given:
part of the packages from project dependencies are located in the private registry. They're completely independent;
correctly-filled package-lock.json left from the previous npm install;
empty node_modules folder;
Now, when I run npm install <package-name> (package already listed in package.json and package-lock.json) npm tries to restore all packages listed on the lock file, stumbles on private packages, and fails to install one specific package because of an authorization error that unrelated to it.
With the --no-shrinkwrap option or with filled node_modules everything working as expected.
I first thought that it connected with the audit, but --no-audit option didn't change anything.
My question is, why npm behaves like this when there are no visible (at least for me) reasons for it?
npm version: 6.9.0 but I also was able to reproduce it on the latest version
The answer, as usual, was in the docs. According to the https://docs.npmjs.com/cli/v6/commands/npm-install#algorithm npm takes package.json, adds requested dependency, and then tries to restore everything that is missing in comparison with the existing node_modules tree.
The other part of the doc says that if package-lock.json exists npm install uses it by default. So in the result, we get an empty node_modules, hence no tree exists but package-lock claims that there should be packages and we ends up restoring all dependency tree for nothing.
If we omit usage of package-lock.json the only difference between expected and actual tree will be the package we were trying to install.
Related
I'm trying to set up a development environment with several packages, and as a result I need to manually install some dependencies. More specifically, I have some local changes in several packages which I need to test before I can push them to github, so I can't just npm install the top level because it won't pick up those change. So I run the first npm install manually on packages which are missing, and then try to run my node code and see which package it is still missing, then try to npm install what it says is missing.
However, when I go to install the second package, it ends up with this message:
added 3 packages from 4 contributors, removed 799 packages and audited 3 packages in 4.197s
The second install removed practically every package that was already installed! I didn't notice this until about the third time, when I realized that I seemed to be installing the same thing over and over.
However can I prevent this particularly naughty behavior and force npm to only install what I tell it to and leave everything else alone?
Have a look at npm link if you need to test against modified packages.
From npm link:
This is handy for installing your own stuff, so that you can work on it and test it iteratively without having to continually rebuild.
Say b is a dependency of a. You made changes to b and want to check if a still works with those changes. Instead of using b in node_modules installed from npm, use your local, modified version:
cd ~/projects/b # go into the package directory
npm link # creates global link
cd ~/projects/a # go into some other package directory.
npm link b # link-install the package
Now, any changes to ~/projects/b will be reflected in ~/projects/a/node_modules/b/.
If your development flow involves updating in parallel packages which depend on one another, you might consider switching your project's package manager to from npm to yarn to take advantage of yarn's workspaces feature.
Yarns's workspaces allow you to easily setup a single monorepo containing all your interconnected dependencies, and let yarn thinking how to link them together in your dev environment.
i had a similar problem today , & thought this might help someone in the future and l have found out that if you install simultaneouly it
npm install --save package1 package2 package3 ...
it worked as l had
npm install xlsx angular-oauth2-oidc
but if you install separately it will have issues
Edit 2 More infor by #Michael
installing multiple packages in the same command also prevents hooks from being installed multiple times
Remove "package-lock.json" file befor installing the new package.
Are you saving the dependencies to package.json?
To Save : npm install --save {package_name}. This will save the package to package.json and install using npm install.
You can't particularly control the dependencies(fully). The dependencies which you have installed might be using dependencies themselves.So when you remove a package, npm deletes all the package's dependencies and the package.
Node.js newbie here, Windows 10. I npm install-ed some packages (without -g) while inside a directory that didn't have package.json. npm placed the packages in C:\Users\{MyName}\node_modules\.
Now I'm seeing some weird behavior:
When I'm in my project directory (has package.json but no node_modules/ yet), npm list and npm list -g both show an empty list
When I'm in a non-project directory (no package.json)...
npm list -g still shows an empty list
However, npm list shows everything in C:\Users\{MyName}\node_modules\
Question 1. What is going on here? Apparently, npm's default global path should be C:\Users\{MyName}\AppData\Roaming\npm\. If so, why is it using C:\Users\{MyName}\node_modules\?
Question 2. How do I get out of this mess? Node.js has no problem importing packages from C:\Users\{MyName}\node_modules\, but I want npm to list them properly. How can I delete the semi-global packages, reinstall them correctly, and ensure that this doesn't happen again?
Welp, turns out I've been mistakenly npm install-ing packages without package.json. The first time I did this, I was in my home directory(C:\Users\{MyName}\). This caused npm to create node_modules/ and package-lock.json in the home directory. Further (mistaken) attempts to install packages in my projects--which were still missing package.json--caused npm to traverse upwards, until it found the initial node_modules/ dir, and install everything there. Because my home directory is among the places Node.js looks for modules, I didn't notice my mistake until now. :P
Not sure why it’s doing it, but the way to avoid it is to initialize your project directory using:
npm init
or if you don’t want to answer the questions:
npm init -y
That will setup the directory with the package.json and node_modules will be put there.
Ok, a couple of tips then...
when you install a package that you are going to use in production then add --save, e.g.
npm install --save some-package
this will automatically add the dependency to your package.json. If you are installing a package for use purely in development, e.g. chai, then use--save-devand it will add it to the development dependencies.
Also, git is your friend, even if you are only messing :)
Happy noding :)
For me the solution here was:
Go to c:\users[me]\AppData\Roaming\npm and delete the node_modules folder completely
Make sure I had the package.json file for the project
Delete the project package-lock.json file
Run npm init
Run npm install
Project then worked, not sure why the node_modules got to be in the folder above, ain't got time to find out.
I'm using npm-shrinkwrap to lock down my project dependencies.
However I'm running into one issue. It seems as though if you remove a dependency from the npm-shrinkwrap.json then node backs on to the original tree dependency resolution via package.json. I tested this by deleting an entry in my npm-shrinkwrap.json and noticing that it still turned up in node_modules after an npm install.
My concern here is that someone unwittingly use npm intall --save-dev to add a new dependency to the project but forgets to update the npm-shrinkwrap.json. You're then in a position where potentially you're introducing variable dependencies into the project that may change from developer to developer or CI server, which it seems is what npm shrinkwrap is designed to prevent.
My question is whether I'm right in this assertion, or misunderstanding the behaviour somehow, or if correct, is there something I can do to force npm install to use only the npm-shrinkwrap.json dependencies and ignore what's in package.json. It occurs to me I could work around this by deleting the dependencies in package.json after producing the npm-shrinkwrap.json, but obviously this is not ideal.
My node_modules has packages that are not listed in my package.json's dependencies, so I'm guessing that those packages are dependencies of my dependencies. How would I be able to check this? I want to make sure that there aren't any unnecessary packages in my node_modules directory.
If your dependency list won't take too long to reinstall, a simple option is a table-flip: remove the node_modules directory entirely and run npm install to re-create it.
If you don't want to do that, you can try tools that inspect your dependencies, like depcheck as #sagar-gopale suggests in their answer.
Related: Run npm -v to find out if you are running npm v2 or v3. Like #cartant says in their answer, with v3, your node_modules directory will be maximally flat, which means things that used to appear as subdirectories of other modules (when installed with npm v2) will now appear at the top level of node_modules itself. That may be the reason you see more modules than you expect.
If you are using NPM 3, you will likely see a large number of modules that you were not expecting to see in the node_modules directory, as NPM 3 flattens the dependency hierarchy.
Whichever version you are using, if you run the npm list command, NPM should highlight any extraneous modules that are not required.
Please checkout this package.
https://www.npmjs.com/package/depcheck
Since packages can require other packages, just because there are packages in the node_modules folder that don't exist in your packages.json file doesn't mean they aren't needed by one of your specified packages.
If you run an npm prune command on the root directory of your solution it will read the dependency tree and remove the packages that are truly no longer needed.
Background
We're having issues with a Windows build system hitting the file path too long error when the node modules folder has items within it that have paths which are over 260 characters.
We've discovered adding a deeply nested dependency to the top of the devDependencies section fixes this issue. The assumption is that when npm sees a nested dependency C.1 require package A, which is already declared and available in devDependencies, npm will not add dependency A to dependency C.1's node_modules directory.
Issue
The problem I'm seeing on my local machine is that running npm install <package> --save-dev reorders the packages in devDependencies alphabetically, but the order npm process packages and their dependencies matters. If I check this in, then the build system will hit the same file path too long error.
ie If package A comes after package C and dependency C.1 requires package A, then npm will add package A to the node_modules folder of dependency C.1.
I'm not sure if this reordering is only on my machine since I haven't seen npm reorder dependencies on my home machine before.
Has anyone seen this before or know how to stop this behavior?
Versions
Node: v0.10.32
NPM: v1.4.28
Side note: I've read that npm 2.0 or future versions will analyze the dependency hierarchy, find duplicated packages, and only reference them once on the file system, but the upgrade to npm 2.0 is not in the picture at this time.
The only way I see this working is to have some sort of preinstall script which [hopefully] will run after the dependencies file has been updated but before the package is installed. From the npm site:
In the current version of node, the standard way to do this is using a
.gyp file. If you have a file with a .gyp extension in the root of
your package, then npm will run the appropriate node-gyp commands
automatically at install time
If that doesn't work, you will need to use MakeFile and rewrite the package.json file. This is not too out of the ordinary as some projects require some sort of pre-compilation - you would just instruct your team to run a separate command for installing npm packages.