Why does npm install with git+ssh install differently than https? - node.js

What is the difference between installing a npm package via https and ssh? My expectation is that the downloaded package would be the same but this is not the case. For example:
// package.json
"dependencies": {
"lodash": "^4.17.19"
// vs
"lodash": "git#github.com:lodash/lodash.git#semver:^4.17.19"
}
When I use the first option, the actual npm package gets installed.
When I install via the second option, I get only the files that are whitelisted from the repo but not the actual package itself.
I don't see a good explanation in the npm documentation. Why aren't these installing the same thing? Is there a way to install the actual package via ssh and not the commit itself?

Two ways of installing dependencies.
From NPM repository itself (specify the version)
From github (specify a branch OR commit and tag)
It is advisable to publish to the registry the minified/compiled version of the library than the source unless it is necessary. So, it is possible that what you get from the NPM is different than the source repository itself.
It is really question of the "place" (npm or github) than the method (http or ssh)

Related

Overwriting node dependencies with github forked repo: npm shrinkwrap...?

I am building an app with Angular 4 cli, using the Dragula drag-and-drop stystem ng2-dragula. I want to update to this particular forked repo which provides some specific augmentation.
But I am struggling to install this with node. I can't simply run
npm install git://github.com/nguyenj/dragula.git
I think because in the ng2-dragula package.json, the dependency is specified as
"dependencies": {
"dragula": "^3.7.2"
},
I have tried to address this using npm-shrinkwrap, using the solution posted here (See section The Real Solution)
But it isn't working; Even if I manually change the dependencies section of the ng2-dragula package.json, running npm install just installs the original dragula, NOT the version I want from https://github.com/nguyenj/dragula
Why? How can I solve this?
Actually, this was something quite simple- no need to be overwriting anything- I just needed to specify the branch of the forked repo:
npm install git://github.com/nguyenj/dragula.git#feature/axis

npm package from private github repo, install vs update (package.json)

I have already seen this question npm-install-vs-update-whats-the-difference
My question is around using install vs update for private github repo using git+ urls.
I have a private github repo which is used in the grunt. This repo receives frequent updates. I installed this repo using git+ssh url as mentioned here npm install git remote url
Everything works fine when installing. Problem comes when updating the package. As per my understanding and question mentioned above, npm updates the package to latest version when doing npm install but this doesnt seem to be case with package installed from github. I had to use npm update to get the latest version. I dont mind using npm update but I have observed that its slow compared to npm install. Can anyone put their thoughts why this might be happening.
My package.json looks like following
{
"name": "My Project",
"version": "1.0.0",
"dependencies": {
"grunt": "^0.4.5",
//Relevent package
"my-tasks": "git+ssh://git#github.com:Flutterbee/my-tasks.git"
}
}
PS : Using npm 3.3.3 (if that makes difference)
Can you try with specifying the branch name with in your package.json like this
"my-tasks": "git+ssh://git#github.com:Flutterbee/my-tasks.git#master"

How can I automatically link local npm package?

I'm buidling two private npm packages that depends on each other.
Say that I have :
project
/my-commons
package.json :
{
name : "my-commons",
version : "0.0.1"
...
}
/my-server
package.json :
{
dependencies : {
"my-commons" : "0.0.1"
}
}
I can use 'npm link' to install the 'commons' package. So anyone willing to
start working on server has to do :
checkout project
cd my-server
npm link ../my-commons
npm install
And a symlink to ../my-commons is added in /my-server/node_modules, and everything's fine.
Is there however a way to tell npm that 'my-commons' package will always be in that folder out there, so that you could just do :
checkout project
cd my-server
npm install
Or am I missing something obvious here ?
Thanks
Maybe.
But first: If my-commons is needed by my-server then it is most likely a good idea to keep it in my-server/node_modules even if that is redundant. In most cases it's best keep a module's dependencies isolated from the rest of your application.
In that scenario, npm link can be used during development, when you're working on my-commons and want to use the changes in my-server without having to npm publish my-commons.
In production you will not want to use npm link, because dependent modules will lose control over which version of the linked module they end up with. If my-server depends on my-commons 0.1.0, but you npm linked your 1.0.1-pre-release version of the my-commons module all hell might break loose.
However, since version 1.2.10 (shipping with node 0.8.19) NPM supports peer dependencies.
Peer dependencies allow you to specify that my-server requires that my-commons be installed "besides" my-server. This does not enable you to require("my-commons") inside my-server but could be useful if my-server is a plugin for my-commons.
For more information: http://blog.nodejs.org/2013/02/07/peer-dependencies/
And finally, since you said that you are developing private packages: If installing the packages is what is causing trouble for you, because you can't publish your packages to the public NPM registry, have a look at alternative ways to specify dependencies (git-, http-URLs, TGZ files): https://docs.npmjs.com/files/package.json#dependencies

How to install a private NPM module without my own registry?

I've taken some shared code and put it in an NPM module, one I don't want to upload to the central registry. The question is, how do I install it from other projects?
The obvious way is probably to set up my own NPM registry, but according to the documentation, that involves a lot of hassle.
Can I just install an NPM module that sits on the local filesystem, or perhaps even from git?
npm install --from-git git#server:project
In your private npm modules add
"private": true
to your package.json
Then to reference the private module in another module, use this in your package.json
{
"name": "myapp",
"dependencies": {
"private-repo": "git+ssh://git#github.com:myaccount/myprivate.git#v1.0.0",
}
}
cd somedir
npm install .
or
npm install path/to/somedir
somedir must contain the package.json inside it.
It knows about git too:
npm install git://github.com/visionmedia/express.git
Can I just install an NPM package that sits on the local filesystem, or perhaps even from git?
Yes you can! From the docs https://docs.npmjs.com/cli/install
A package is:
a) a folder containing a program described by a package.json file
b) a gzipped tarball containing (a)
c) a url that resolves to (b)
d) a <name>#<version> that is published on the registry with (c)
e) a <name>#<tag> that points to (d)
f) a <name> that has a "latest" tag satisfying (e)
g) a <git remote url> that resolves to (b)
Isn't npm brilliant?
Update January 2016
In addition to other answers, there is sometimes the scenario where you wish to have private modules available in a team context.
Both Github and Bitbucket support the concept of generating a team API Key. This API key can be used as the password to perform API requests as this team.
In your private npm modules add
"private": true
to your package.json
Then to reference the private module in another module, use this in your package.json
{
"name": "myapp",
"dependencies": {
"private-repo":
"git+https://myteamname:aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4#bitbucket.org/myprivate.git",
}
}
where team name = myteamname, and API Key = aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4
Here I reference a bitbucket repo, but it is almost identical using github too.
Finally, as an alternative, if you really don't mind paying $7 per month (as of writing) then you can now have private NPM modules out of the box.
FWIW: I had problems with all of these answers when dealing with a private organization repository.
The following worked for me:
npm install -S "git+https://username#github.com/orgname/repositoryname.git"
For example:
npm install -S "git+https://blesh#github.com/netflix/private-repository.git"
I'm not entirely sure why the other answers didn't work for me in this one case, because they're what I tried first before I hit Google and found this answer. And the other answers are what I've done in the past.
Hopefully this helps someone else.
Structure your code in an accessible fashion like below. If this is possible for you.
NodeProjs\Apps\MainApp\package.json
NodeProjs\Modules\DataModule\package.json
Within MainApp # NodProjs\Apps\MainApp\
npm install --S ../../Modules/DataModule
You may need to update package.json as:
"dependencies": {
"datamodule": "../../Modules/DataModule"
}
This worked for my situation.
I had this same problem, and after some searching around, I found Reggie (https://github.com/mbrevoort/node-reggie). It looks pretty solid. It allows for lightweight publishing of NPM modules to private servers. Not perfect (no authentication upon installation), and it's still really young, but I tested it locally, and it seems to do what it says it should do.
That is... (and this just from their docs)
npm install -g reggie
reggie-server -d ~/.reggie
then cd into your module directory and...
reggie -u http://<host:port> publish
reggie -u http://127.0.0.1:8080 publish
finally, you can install packages from reggie just by using that url either in a direct npm install command, or from within a package.json... like so
npm install http://<host:port>/package/<name>/<version>
npm install http://<host:port>/package/foo/1.0.0
or..
dependencies: {
"foo": "http://<host:port>/package/foo/1.0.0"
}
Npm now provides unlimited private hosted modules for $7/user/month used like so
cd private-project
npm login
in your package json set "name": " #username/private-project"
npm publish
then to require your project:
cd ../new-project
npm install --save #username/private-project
This was what I was looking for - get the latest from "private repo" :
GitHub :
$ npm install git+https://token:x-oauth-basic#github.com/username/my-new-project.git
$ npm install git+ssh://git#github.com/username/my-new-project.git
Bitbucket :
$ npm install git+https://username:password#bitbucket.org/username/my-new-project.git
$ npm install git+ssh://git#bitbucket.org/username/my-new-project.git
Starting with arcseldon's answer, I found that the team name was needed in the URL like so:
npm install --save "git+https://myteamname#aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4#bitbucket.org/myteamname/myprivate.git"
And note that the API key is only available for the team, not individual users.
I use the following with a private github repository:
npm install github:mygithubuser/myproject
Very simple -
npm config set registry https://path-to-your-registry/
It actually sets registry = "https://path-to-your-registry" this line to /Users/<ur-machine-user-name>/.npmrc
All the value you have set explicitly or have been set by default can be seen by - npm config list
You can use Verdaccio for this purpose which is a lightweight private npm proxy registry built in Node.js. Also it is free and open-source. By using Verdaccio it does not involve that much hassle as a plain private npm registry would.
You can find detailed information about how to install and run it on their website but here are the steps:
It requires node >=8.x.
// Install it from npm globally
npm install -g verdaccio
// Simply run with the default configuration that will host the registry which you can reach at http://localhost:4873/
verdaccio
// Set the registry for your project and every package will be downloaded from your private registry
npm set registry http://localhost:4873/
// OR use the registry upon individual package install
npm install --registry http://localhost:4873
It also has a docker so you can easily publish it to your publicly available docker and voila you have a private npm repository that can be distributed to others in a way as you configure it!
Config to install from public Github repository, even if machine is under firewall:
dependencies: {
"foo": "https://github.com/package/foo/tarball/master"
}
Obviously, setting up the private npm registry is the most scalable and long-term solution, although it's a bit of hassle in the beginning.
Also, you can install using the git+https/ssh as mentioned in the other answers. But if you have the private repo, and you're building the image in the cloud, let's say using google cloud build, you have to set up the GitHub ssh connection.
The simplest solution for one-off case like this can be solved using the following approach.
Clone and modify or create your own library from scratch.
Generate the archive file(package code along with its dependencies), using
yarn install && yarn pack
this will produce file like
rich-markdown-editor-v11.13.117.tgz
move this file to libs folder and add this entry in dependencies object of package.json.
"rich-markdown-editor": "file:libs/rich-markdown-editor-v11.13.117.tgz",
Now, install the package.
yarn install
Make sure to add that file in your vcs and the installation process in docker image creation should work in cloud as well.
Note: if you frequently update the package and commit in your vcs, it will increase your repo size(while cloning with full history).
Publish your module under an organization name using the standard "#my-org/my-module" (by default all organization modules are private).
From your npm profile create a read-only access token under "Access Tokens"
Next in your project directory root create a .npmrc file, and inside the file write the following:
//registry.npmjs.org/:_authToken=${Your_Access_Token}
Note: this also should work for others packaging services that follow the same standard.

npm to install packages from local position rather than from web?

The problem drove me crazy, there is a package in npm database, but it has some bugs, which are already fixed in github, how could I make use of the fixed version(github version)?
Edit:
You can install directly from the GitHub repository, even just using the GitHub username and the repository name:
npm install LearnBoost/socket.io
You can also add a <commit-ish>, specifying e.g. a commit hash or a version tag, like so:
npm install LearnBoost/socket.io#1.7.x
Without a protocol, this will be interpreted as git://github.com/LearnBoost/socket.io. You can also prefix the repo with gitlab:, gist: or bitbucket:, respectively. For more information, see Using git URLs as dependencies.
You can install directly from a URL, example:
npm install https://github.com/LearnBoost/socket.io/tarball/master
You can find the URL on Github under "Downloads" on any project page. Select the "Download as tar.gz" link.
Or you can install a tarball:
npm install foo.tar.gz
See npm install(1).
Edit:
I should mention that this works equally well in package.json files. Specify the URL instead of the version in your dependencies, like so:
...
"dependencies": {
"foo": "http://example.com/foo.tar.gz",
"bar": "1.2.x",
...
}
Other temporary solution, get the github project and use npm link (http://npmjs.org/doc/link.html) to link the local folder obtained through git to your node_modules folder in your own project. Anyway in the end, you'll have to wait for the project maintainer to do a npm publish.
Either add the module as a git sub-module (using git submodule) to your project or tell the module maintainer to update the version and trigger a npm publish to update the npm repository.
When using the sub-module way, be aware that you cannot update the reference using npm-commands.

Resources