I'm new to NPM. I'm getting an alert from github dependabot saying my json-schema package is vulnerable. My current json-schema version is 0.2.3, it's required by my current npm#7.20.6 via transitive dependency on jsprim#1.4.1. I can't update my npm version right now since that involves lots of testing which I don't have time to do right now. Wondering what's a good way to force npm to use a newer version of json-schema. (0.4.0 is the version I should use.) I was thinking updating package-lock.json to use json-schema#0.2.3, but if I run npm install, that change will be erased right? Some examples would be appreciated!
react-dev-utils#11.0.4 installing a vulnerable version of browserlist, browserslist#4.14.2, although we have updated package on github. https://github.com/facebook/create-react-app/blob/main/packages/react-dev-utils/package.json#L57
[to test out, you may simple create any folder and do npm i react-dev-utils and then check it using npm ls browserlist]
I dont get, what all are the constraint for this. (I dont see any package-lock.json for this package, which could be a potential reason for the vuln). older version has reported vulnerability CVE-2021-23364.
react-dev-utils#11.0.4 installing a vulnerable version of browserlist, browserslist#4.14.2, although we have updated package on github
This is because that package.json file resides in the default branch which usually contains the latest or development code. At the time you posted the question, that change was not published to the npm registry.
react-dev-utils#11.0.4 has browserslist#4.14.2 listed in its package so that's the version that will be installed. Reference: https://cdn.jsdelivr.net/npm/react-dev-utils#11.0.4/package.json
You need atleast react-dev-utils#12.0.0 to fix that vulnerability. See the versions tab.
[to test out, you may simple create any folder and do npm i react-dev-utils and then check it using npm ls browserlist]
Running that command will install the latest version of react-dev-utils, which now has no vulnerability. So it will fix your issue.
I dont see any package-lock.json for this package, which could be a potential reason for the vuln
package-lock.json cannot be published to registry, only the top level lock file is respected. Reference: Should package-lock.json also be published?
this may be an example package, but how in general we update to latest package? have tried npm update as well.
npm update respects the semver range that you've set in your package.json. If its like "react-dev-utils": "11.0.4" that command won't do anything. If its "react-dev-utils": "^11.0.4", it will try to update to the latest 11.x.x version which you are already on, so again it won't do anything. Reference: npm update does not do anything
In general if you want to upgrade every direct dependency to latest version you can use npm-check-updates before running npm update. Refer https://nodejs.dev/learn/update-all-the-nodejs-dependencies-to-their-latest-version for detailed guide. Related: How to update each dependency in package.json to the latest version?
Now, if it is not a direct dependency, as was in your case, you can force resolutions. This is natively supported in Yarn and NPM v8.3.0 and above. In older NPM versions you need to use a dependency like npm-force-resolutions. Related thread: npm equivalent of yarn resolutions?
There are much more related threads that you can easily find by searching on the web.
How to safely npm install/update when deploying/upgrading ?
Problem 1 : npm install is a statefull operation that depends on the latest versions of dependencies in the time when the command is executed. This causes surprises when deploying since package.json file indicates ranges but not specific versions.
Problem 2 : everytime I make npm update or use ncu, I spend hours/days trying to handle incoherences between modules. Why would this happens in 2018 ?
Problem 3 : How to have package.json file that describes exactly the state of installed packages instead of ranges so that I can deploy without surprises ?
NB: I use Angular
If you use yarn or a more recent version of npm, it will generate for you a yarn.lock or package-lock.json.
This will keep exactly the version of any package when it's first installed, so further calls to yarn or npm install will fetch and install exactly those versions.
Of course you should add these lock files to your repository so anyone doing a fresh clone get the same dependencies installed.
See the npm docs: https://docs.npmjs.com/files/package-lock.json
And the yarn docs: https://yarnpkg.com/lang/en/docs/yarn-lock/
package.json file indicates ranges but not specific versions : Re-read the documentation, you can specify specific versions. See point 3 for an example.
Why would this happens in 2018 <= I/we can't speculate as to problems where you did not include any specific details, it might be a valid general gripe you have but StackOverflow is not the correct place to vent it.
Again, see the documentation. You just have to include the version number with an = sign. Example below would get only the version 5.0.0 of #angular/cdk.
"#angular/cdk": "5.0.0"
be advised that any call to npm update will update your package.json with the latest minor version, so setting a strict constraint version ex (5.0.0) will only work with npm install when no package-lock.json is present. When doing npm update, your 5.0.0 constraint will be replaced by a ^5.5.0 constraint (or whatever the next minor release is). It's fine if the packages you are using implement semver correctly, but you can have a lot of issues with breaking changes on minor release.
I just recently upgraded to npm#5. I now have a package-lock.json file with everything from package.json. I would expect that, when I run npm install that the dependency versions would be pulled from the lock file to determine what should be installed in my node_modules directory. What's strange is that it actually ends up modifying and rewriting my package-lock.json file.
For example, the lock file had typescript specified to be at version 2.1.6. Then, after the npm install command, the version was changed to 2.4.1. That seems to defeat the whole purpose of a lock file.
What am I missing? How do I get npm to actually respect my lock file?
Update 3: As other answers point out as well, the npm ci command got introduced in npm 5.7.0 as additional way to achieve fast and reproducible builds in the CI context. See the documentation and npm blog for further information.
Update 2: The issue to update and clarify the documentation is GitHub issue #18103.
Update 1: The behaviour that was described below got fixed in npm 5.4.2: the currently intended behaviour is outlined in GitHub issue #17979.
Original answer (pre-5.4.2): The behaviour of package-lock.json was changed in npm 5.1.0 as discussed in issue #16866. The behaviour that you observe is apparently intended by npm as of version 5.1.0.
That means that package.json can override package-lock.json whenever a newer version is found for a dependency in package.json. If you want to pin your dependencies effectively, you now must specify the versions without a prefix, e.g., you need to write them as 1.2.0 instead of ~1.2.0 or ^1.2.0. Then the combination of package.json and package-lock.json will yield reproducible builds. To be clear: package-lock.json alone no longer locks the root level dependencies!
Whether this design decision was good or not is arguable, there is an ongoing discussion resulting from this confusion on GitHub in issue #17979. (In my eyes it is a questionable decision; at least the name lock doesn't hold true any longer.)
One more side note: there is also a restriction for registries that don’t support immutable packages, such as when you pull packages directly from GitHub instead of npmjs.org. See this documentation of package locks for further explanation.
I've found that there will be a new version of npm 5.7.1 with the new command npm ci, that will install from package-lock.json only
The new npm ci command installs from your lock-file ONLY. If your package.json and your lock-file are out of sync then it will report an error.
It works by throwing away your node_modules and recreating it from scratch.
Beyond guaranteeing you that you'll only get what is in your lock-file it's also much faster (2x-10x!) than npm install when you don't start with a node_modules.
As you may take from the name, we expect it to be a big boon to continuous integration environments. We also expect that folks who do production deploys from git tags will see major gains.
Short Answer:
npm install honors package-lock.json only if it satisfies the requirements of package.json.
If it doesn't satisfy those requirements, packages are updated & package-lock is overwritten.
If you want the install to fail instead of overwriting package-lock when this happens, use npm ci.
Here is a scenario that might explain things (Verified with NPM 6.3.0)
You declare a dependency in package.json like:
"depA": "^1.0.0"
Then you do, npm install which will generate a package-lock.json with:
"depA": "1.0.0"
Few days later, a newer minor version of "depA" is released, say "1.1.0", then the following holds true:
npm ci # respects only package-lock.json and installs 1.0.0
npm install # also, respects the package-lock version and keeps 1.0.0 installed
# (i.e. when package-lock.json exists, it overrules package.json)
Next, you manually update your package.json to:
"depA": "^1.1.0"
Then rerun:
npm ci # will try to honor package-lock which says 1.0.0
# but that does not satisfy package.json requirement of "^1.1.0"
# so it would throw an error
npm install # installs "1.1.0" (as required by the updated package.json)
# also rewrites package-lock.json version to "1.1.0"
# (i.e. when package.json is modified, it overrules the package-lock.json)
Use the newly introduced
npm ci
npm ci promises the most benefit to large teams. Giving developers the ability to “sign off” on a package lock promotes more efficient collaboration across large teams, and the ability to install exactly what is in a lockfile has the potential to save tens if not hundreds of developer hours a month, freeing teams up to spend more time building and shipping amazing things.
Introducing npm ci for faster, more reliable builds
Use the npm ci command instead of npm install.
"ci" stands for "clean install".
It will install the project dependencies based on the package-lock.json file instead of the lenient package.json file dependencies.
It will produce identical builds to your team mates and it is also much faster.
You can read more about it in this blog post:
https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable
It appears this issue is fixed in npm v5.4.2
https://github.com/npm/npm/issues/17979
(Scroll down to the last comment in the thread)
Update
Actually fixed in 5.6.0. There was a cross platform bug in 5.4.2 that was causing the issue to still occur.
https://github.com/npm/npm/issues/18712
Update 2
See my answer here:
https://stackoverflow.com/a/53680257/1611058
npm ci is the command you should be using when installing existing projects now.
In the future, you will be able to use a --from-lock-file (or similar) flag to install only from the package-lock.json without modifying it.
This will be useful for CI, etc. environments where reproducible builds are important.
See https://github.com/npm/npm/issues/18286 for tracking of the feature.
Probably you should use something like this
npm ci
Instead of using npm install
if you don't want to change the version of your package.
According to the official documentation, both npm install and npm ci install the dependencies which are needed for the project.
The main difference is, npm install does install the packages taking packge.json as a reference. Where in the case of npm ci, it does install the packages taking package-lock.json as a reference, making sure every time the exact package is installed.
You probably have something like:
"typescript":"~2.1.6"
in your package.json which npm updates to the latest minor version, in your case being 2.4.1
Edit: Question from OP
But that doesn't explain why "npm install" would change the lock file. Isn't the lock file meant to create a reproducible build? If so,
regardless of the semver value, it should still use the same 2.1.6
version.
Answer:
This is intended to lock down your full dependency tree. Let's say typescript v2.4.1 requires widget ~v1.0.0. When you npm install it
grabs widget v1.0.0. Later on your fellow developer (or CI build)
does an npm install and gets typescript v2.4.1 but widget has been
updated to widget v1.0.1. Now your node module are out of sync. This
is what package-lock.json prevents.
Or more generally:
As an example, consider
package A:
{ "name": "A", "version": "0.1.0", "dependencies": {
"B": "<0.1.0" } }
package B:
{ "name": "B", "version": "0.0.1", "dependencies": {
"C": "<0.1.0" } }
and package C:
{ "name": "C", "version": "0.0.1" }
If these are the only versions
of A, B, and C available in the registry, then a normal npm install A
will install:
A#0.1.0 -- B#0.0.1
-- C#0.0.1
However, if B#0.0.2 is published, then a fresh npm install A will install:
A#0.1.0 -- B#0.0.2
-- C#0.0.1 assuming the new version did not modify B's dependencies. Of course, the new version of B could include a new
version of C and any number of new dependencies. If such changes are
undesirable, the author of A could specify a dependency on B#0.0.1.
However, if A's author and B's author are not the same person, there's
no way for A's author to say that he or she does not want to pull in
newly published versions of C when B hasn't changed at all.
OP Question 2: So let me see if I understand correctly. What you're
saying is that the lock file specifies the versions of the secondary
dependencies, but still relies on the fuzzy matching of package.json
to determine the top-level dependencies. Is that accurate?
Answer: No. package-lock locks the entire package tree, including the
root packages described in package.json. If typescript is locked
at 2.4.1 in your package-lock.json, it should remain that way until it is
changed. And lets say tomorrow typescript releases version 2.4.2.
If I checkout your branch and run npm install, npm will respect the
lockfile and install 2.4.1.
More on package-lock.json:
package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json. It describes the exact tree that was generated, such that subsequent installs are able to generate identical trees, regardless of intermediate dependency updates.
This file is intended to be committed into source repositories, and serves various purposes:
Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.
Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.
To facilitate greater visibility of tree changes through readable source control diffs.
And optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.
https://docs.npmjs.com/files/package-lock.json
EDIT: the name "lock" is a tricky one, its NPM trying to catch up with Yarn. It isn't a locked file whatsoever. package.json is a user-fixed file, that once "installed" will generate node_modules folder tree and that tree will then be written in package-lock.json. So you see, its the other way around - dependency versions will be pulled from package.json as always, and package-lock.json should be called package-tree.json
(hope this made my answer clearer, after so many downvotes)
A simplistic answer: package.json have your dependencies as usual, while package-lock.json is "an exact, and more importantly reproducible node_modules tree" (taken from npm docs itself).
As for the tricky name, its NPM trying to catch up with Yarn.
There is an open issue for this on their github page: https://github.com/npm/npm/issues/18712
This issue is most severe when developers are using different operating systems.
Npm install detects any changes made to package.json file to reflect the dependency list accordingly.
Ex. If user added or removed a new dependency, the build will download or remove the dependencies in the local computer. We can compare this to .m2 repository in java where maven keeps track of pom.xml file constantly to update the dependencies.
package-lock.json is a replica of package.json used at run-time by internal processes, only difference is package-lock.json is read-only to user.
Imagine the situation:
We have a project with node_modules gitignored. While fixing a task developer1 updated package.json to newer version of some module, e.g.
"dependencies": {
"async": "^1.5.2", // was 1.5.1
...
Then he runned npm install locally to get updated module, performed tests, finished task and pushed changes on the server.
Developer2 pulled changes from server and get app broken because still having previous version of async locally (1.5.1). And developer2 can waste a huge amount of time finding what's exactly goes wrong. Until they do npm i.
Can you suggest any npm package or ready-to-use hook that can check versions in package.json against actual versions of node_modules ?
It will be really helpful!
PS: I'm aware of https://www.npmjs.com/package/npm-check but it does not do what I need.
The package check-dependencies might do what you want.
$ check-dependencies
url-loader: installed: 0.5.8, expected: 0.4.0
Invoke npm install to install missing packages
At my current day job we had exactly this problem.
We fixed it by creating an easy script that pulled the new source and after that directly executes npm update.