I am building an app with Angular 4 cli, using the Dragula drag-and-drop stystem ng2-dragula. I want to update to this particular forked repo which provides some specific augmentation.
But I am struggling to install this with node. I can't simply run
npm install git://github.com/nguyenj/dragula.git
I think because in the ng2-dragula package.json, the dependency is specified as
"dependencies": {
"dragula": "^3.7.2"
},
I have tried to address this using npm-shrinkwrap, using the solution posted here (See section The Real Solution)
But it isn't working; Even if I manually change the dependencies section of the ng2-dragula package.json, running npm install just installs the original dragula, NOT the version I want from https://github.com/nguyenj/dragula
Why? How can I solve this?
Actually, this was something quite simple- no need to be overwriting anything- I just needed to specify the branch of the forked repo:
npm install git://github.com/nguyenj/dragula.git#feature/axis
I just recently upgraded to npm#5. I now have a package-lock.json file with everything from package.json. I would expect that, when I run npm install that the dependency versions would be pulled from the lock file to determine what should be installed in my node_modules directory. What's strange is that it actually ends up modifying and rewriting my package-lock.json file.
For example, the lock file had typescript specified to be at version 2.1.6. Then, after the npm install command, the version was changed to 2.4.1. That seems to defeat the whole purpose of a lock file.
What am I missing? How do I get npm to actually respect my lock file?
Update 3: As other answers point out as well, the npm ci command got introduced in npm 5.7.0 as additional way to achieve fast and reproducible builds in the CI context. See the documentation and npm blog for further information.
Update 2: The issue to update and clarify the documentation is GitHub issue #18103.
Update 1: The behaviour that was described below got fixed in npm 5.4.2: the currently intended behaviour is outlined in GitHub issue #17979.
Original answer (pre-5.4.2): The behaviour of package-lock.json was changed in npm 5.1.0 as discussed in issue #16866. The behaviour that you observe is apparently intended by npm as of version 5.1.0.
That means that package.json can override package-lock.json whenever a newer version is found for a dependency in package.json. If you want to pin your dependencies effectively, you now must specify the versions without a prefix, e.g., you need to write them as 1.2.0 instead of ~1.2.0 or ^1.2.0. Then the combination of package.json and package-lock.json will yield reproducible builds. To be clear: package-lock.json alone no longer locks the root level dependencies!
Whether this design decision was good or not is arguable, there is an ongoing discussion resulting from this confusion on GitHub in issue #17979. (In my eyes it is a questionable decision; at least the name lock doesn't hold true any longer.)
One more side note: there is also a restriction for registries that don’t support immutable packages, such as when you pull packages directly from GitHub instead of npmjs.org. See this documentation of package locks for further explanation.
I've found that there will be a new version of npm 5.7.1 with the new command npm ci, that will install from package-lock.json only
The new npm ci command installs from your lock-file ONLY. If your package.json and your lock-file are out of sync then it will report an error.
It works by throwing away your node_modules and recreating it from scratch.
Beyond guaranteeing you that you'll only get what is in your lock-file it's also much faster (2x-10x!) than npm install when you don't start with a node_modules.
As you may take from the name, we expect it to be a big boon to continuous integration environments. We also expect that folks who do production deploys from git tags will see major gains.
Short Answer:
npm install honors package-lock.json only if it satisfies the requirements of package.json.
If it doesn't satisfy those requirements, packages are updated & package-lock is overwritten.
If you want the install to fail instead of overwriting package-lock when this happens, use npm ci.
Here is a scenario that might explain things (Verified with NPM 6.3.0)
You declare a dependency in package.json like:
"depA": "^1.0.0"
Then you do, npm install which will generate a package-lock.json with:
"depA": "1.0.0"
Few days later, a newer minor version of "depA" is released, say "1.1.0", then the following holds true:
npm ci # respects only package-lock.json and installs 1.0.0
npm install # also, respects the package-lock version and keeps 1.0.0 installed
# (i.e. when package-lock.json exists, it overrules package.json)
Next, you manually update your package.json to:
"depA": "^1.1.0"
Then rerun:
npm ci # will try to honor package-lock which says 1.0.0
# but that does not satisfy package.json requirement of "^1.1.0"
# so it would throw an error
npm install # installs "1.1.0" (as required by the updated package.json)
# also rewrites package-lock.json version to "1.1.0"
# (i.e. when package.json is modified, it overrules the package-lock.json)
Use the newly introduced
npm ci
npm ci promises the most benefit to large teams. Giving developers the ability to “sign off” on a package lock promotes more efficient collaboration across large teams, and the ability to install exactly what is in a lockfile has the potential to save tens if not hundreds of developer hours a month, freeing teams up to spend more time building and shipping amazing things.
Introducing npm ci for faster, more reliable builds
Use the npm ci command instead of npm install.
"ci" stands for "clean install".
It will install the project dependencies based on the package-lock.json file instead of the lenient package.json file dependencies.
It will produce identical builds to your team mates and it is also much faster.
You can read more about it in this blog post:
https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable
It appears this issue is fixed in npm v5.4.2
https://github.com/npm/npm/issues/17979
(Scroll down to the last comment in the thread)
Update
Actually fixed in 5.6.0. There was a cross platform bug in 5.4.2 that was causing the issue to still occur.
https://github.com/npm/npm/issues/18712
Update 2
See my answer here:
https://stackoverflow.com/a/53680257/1611058
npm ci is the command you should be using when installing existing projects now.
In the future, you will be able to use a --from-lock-file (or similar) flag to install only from the package-lock.json without modifying it.
This will be useful for CI, etc. environments where reproducible builds are important.
See https://github.com/npm/npm/issues/18286 for tracking of the feature.
Probably you should use something like this
npm ci
Instead of using npm install
if you don't want to change the version of your package.
According to the official documentation, both npm install and npm ci install the dependencies which are needed for the project.
The main difference is, npm install does install the packages taking packge.json as a reference. Where in the case of npm ci, it does install the packages taking package-lock.json as a reference, making sure every time the exact package is installed.
You probably have something like:
"typescript":"~2.1.6"
in your package.json which npm updates to the latest minor version, in your case being 2.4.1
Edit: Question from OP
But that doesn't explain why "npm install" would change the lock file. Isn't the lock file meant to create a reproducible build? If so,
regardless of the semver value, it should still use the same 2.1.6
version.
Answer:
This is intended to lock down your full dependency tree. Let's say typescript v2.4.1 requires widget ~v1.0.0. When you npm install it
grabs widget v1.0.0. Later on your fellow developer (or CI build)
does an npm install and gets typescript v2.4.1 but widget has been
updated to widget v1.0.1. Now your node module are out of sync. This
is what package-lock.json prevents.
Or more generally:
As an example, consider
package A:
{ "name": "A", "version": "0.1.0", "dependencies": {
"B": "<0.1.0" } }
package B:
{ "name": "B", "version": "0.0.1", "dependencies": {
"C": "<0.1.0" } }
and package C:
{ "name": "C", "version": "0.0.1" }
If these are the only versions
of A, B, and C available in the registry, then a normal npm install A
will install:
A#0.1.0 -- B#0.0.1
-- C#0.0.1
However, if B#0.0.2 is published, then a fresh npm install A will install:
A#0.1.0 -- B#0.0.2
-- C#0.0.1 assuming the new version did not modify B's dependencies. Of course, the new version of B could include a new
version of C and any number of new dependencies. If such changes are
undesirable, the author of A could specify a dependency on B#0.0.1.
However, if A's author and B's author are not the same person, there's
no way for A's author to say that he or she does not want to pull in
newly published versions of C when B hasn't changed at all.
OP Question 2: So let me see if I understand correctly. What you're
saying is that the lock file specifies the versions of the secondary
dependencies, but still relies on the fuzzy matching of package.json
to determine the top-level dependencies. Is that accurate?
Answer: No. package-lock locks the entire package tree, including the
root packages described in package.json. If typescript is locked
at 2.4.1 in your package-lock.json, it should remain that way until it is
changed. And lets say tomorrow typescript releases version 2.4.2.
If I checkout your branch and run npm install, npm will respect the
lockfile and install 2.4.1.
More on package-lock.json:
package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json. It describes the exact tree that was generated, such that subsequent installs are able to generate identical trees, regardless of intermediate dependency updates.
This file is intended to be committed into source repositories, and serves various purposes:
Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.
Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.
To facilitate greater visibility of tree changes through readable source control diffs.
And optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.
https://docs.npmjs.com/files/package-lock.json
EDIT: the name "lock" is a tricky one, its NPM trying to catch up with Yarn. It isn't a locked file whatsoever. package.json is a user-fixed file, that once "installed" will generate node_modules folder tree and that tree will then be written in package-lock.json. So you see, its the other way around - dependency versions will be pulled from package.json as always, and package-lock.json should be called package-tree.json
(hope this made my answer clearer, after so many downvotes)
A simplistic answer: package.json have your dependencies as usual, while package-lock.json is "an exact, and more importantly reproducible node_modules tree" (taken from npm docs itself).
As for the tricky name, its NPM trying to catch up with Yarn.
There is an open issue for this on their github page: https://github.com/npm/npm/issues/18712
This issue is most severe when developers are using different operating systems.
Npm install detects any changes made to package.json file to reflect the dependency list accordingly.
Ex. If user added or removed a new dependency, the build will download or remove the dependencies in the local computer. We can compare this to .m2 repository in java where maven keeps track of pom.xml file constantly to update the dependencies.
package-lock.json is a replica of package.json used at run-time by internal processes, only difference is package-lock.json is read-only to user.
I have already seen this question npm-install-vs-update-whats-the-difference
My question is around using install vs update for private github repo using git+ urls.
I have a private github repo which is used in the grunt. This repo receives frequent updates. I installed this repo using git+ssh url as mentioned here npm install git remote url
Everything works fine when installing. Problem comes when updating the package. As per my understanding and question mentioned above, npm updates the package to latest version when doing npm install but this doesnt seem to be case with package installed from github. I had to use npm update to get the latest version. I dont mind using npm update but I have observed that its slow compared to npm install. Can anyone put their thoughts why this might be happening.
My package.json looks like following
{
"name": "My Project",
"version": "1.0.0",
"dependencies": {
"grunt": "^0.4.5",
//Relevent package
"my-tasks": "git+ssh://git#github.com:Flutterbee/my-tasks.git"
}
}
PS : Using npm 3.3.3 (if that makes difference)
Can you try with specifying the branch name with in your package.json like this
"my-tasks": "git+ssh://git#github.com:Flutterbee/my-tasks.git#master"
I seem to understand how to clone a repository, but I'm not sure I'm fully grasping how to keep that repository current with what's on Github. I'm using Basscss for example, and I believe my local project is slightly out of date with what's on Github.
I'm trying to update Basscss to the latest version using npm updatebut even after running that command my package.json file remains the same with the version stuck at "4.2.1".
"name": "basscss",
"version": "4.2.1",
...
Am I doing something wrong here? Any help with this is appreciated. Thanks in advance!
npm update will update the module's files listed in your project. if you want to update your package.json file with the newer version. use the save option.
npm update --save
edit
npm install basscss#latest --save
I would try to re-install latest and save to package file. BTW, your package.json should reference the dep's like this:
"name" : "YOUR_PROJECT",
"version" : "0.0.1",
"dependencies": {
"basscss": "^4.2.4",
}
The problem drove me crazy, there is a package in npm database, but it has some bugs, which are already fixed in github, how could I make use of the fixed version(github version)?
Edit:
You can install directly from the GitHub repository, even just using the GitHub username and the repository name:
npm install LearnBoost/socket.io
You can also add a <commit-ish>, specifying e.g. a commit hash or a version tag, like so:
npm install LearnBoost/socket.io#1.7.x
Without a protocol, this will be interpreted as git://github.com/LearnBoost/socket.io. You can also prefix the repo with gitlab:, gist: or bitbucket:, respectively. For more information, see Using git URLs as dependencies.
You can install directly from a URL, example:
npm install https://github.com/LearnBoost/socket.io/tarball/master
You can find the URL on Github under "Downloads" on any project page. Select the "Download as tar.gz" link.
Or you can install a tarball:
npm install foo.tar.gz
See npm install(1).
Edit:
I should mention that this works equally well in package.json files. Specify the URL instead of the version in your dependencies, like so:
...
"dependencies": {
"foo": "http://example.com/foo.tar.gz",
"bar": "1.2.x",
...
}
Other temporary solution, get the github project and use npm link (http://npmjs.org/doc/link.html) to link the local folder obtained through git to your node_modules folder in your own project. Anyway in the end, you'll have to wait for the project maintainer to do a npm publish.
Either add the module as a git sub-module (using git submodule) to your project or tell the module maintainer to update the version and trigger a npm publish to update the npm repository.
When using the sub-module way, be aware that you cannot update the reference using npm-commands.