I believe the answer is no, but hoping for something different. =)
I'm experimenting with Gemfury as a provider, and as such moving some private packages into it, but not ready to move all of them. Some of them need to reside in Github Packages for a while longer.
In the mean time, consumers need to be able to install from the regular npm registry, Github Packages and now Gemfury.
public npm + Github Packages -> works
public npm + Gemfury -> works
public npm + Github Packages + Gemfury -> haven't found a way to get this to work
Is this possible? Are there docs or guides on how to accomplish this ultra-blended-index? From https://gemfury.com/help/npm-registry/ I can't derive how to set up the version that combines all 3.
The closest I got to this to working was with the following .npmrc
#ourscope:registry=<<github_packages_url>>
registry=<<gemfury_proxy_url>>
//n<<gemfury_proxy_url>>/:_authToken=${NPM_GEMFURY_AUTH_TOKEN}
engine-strict=true
The above works locally, but doesn't seem to work on CI though
Since I couldn't find any documentation to support it, I'm not even sure if it should be working locally or if it's just a happy accident of sorts.
Any insights on whether this is possible and how to adopt it would be appreciated!
Related
Okay, so I get it that I might be looking for a totally unexpected behavior.
My current company has nothing for me to do for a few weeks, so I would like to start a side project. The problem is, the firewall is really strong here, so I cannot download anything with Git or Npm. I also am not allowed to do any request to the IT support, since I am not in my company's office but some offices owned by a client (that applies said strict policy). In short, I am stuck with firewall and proxy policies that I cannot modify.
I may download a module's zipped archive through the browser, and install it from there. However, it has multiple dependencies, that themselves have dependencies, and so forth. And since I cannot run npm install to retrieve the dependencies, I'm stuck.
I saw multiple possible solutions in order to solve those dependencies issues:
make NPM run all requests and downloads through the browser, since browsers are allowed to access to the network. I don't see any options for that so far.
Download all required dependencies as tarball and step-by-step install each of them. Because of the potential number of dependencies being huge, I am looking for a huge bulk of modules commonly used to download once.
Most solutions I find make the assumption that I may use npm install properly, while my proxy doesn't allow it.
I wouldn't like to spend days on Chrome's built in game. Any idea?
maybe you can create the project somewhere else, and then "import" it on your office local machine:
npm init
edit package.json with your dependecies
npm install
put everything on usb stick and put it on your local computer at work.
or
send an archive of the code via email and download the tarball from email at office.
I found only this thread when an user ask similar question but without answer. ( https://github.com/npm/npm-registry-couchapp/issues/17 )
I know two sites that are repositories of NPM packages:
https://www.npmjs.com
http://libraries.io
And I can't find a way to obtain an rss for single package in a way like it is done for github or sourceforge:
https://github.com/<author_name>/<package_name>/releases.atom
http://sourceforge.net/projects/<project_name>/rss?limit=20
And yes, i know that I might check what is the github repository for given NPM and do it that way, but I specifically ask for a way to reach this info from some kind of NMP repository that tracks if a given package changed its git address (not necessarily these two I added here as example.)
It is hard to find these tricks by google because it thinks I'm searching for some kind of NPM package that deals with RSS/ATOM.
I've recently added a release atom feed to all packages on libraries.io, simply add /versions.atom to the end of any project page url, for example:
https://libraries.io/npm/node-sass/versions.atom
With npm modules it should never be more than 10 minutes delayed in showing the newest version.
You need to hit your requests against an NPM registry, i.e. https://registry.npmjs.org/-/rss.
For single packages, the URL should be https://registry.npmjs.org/-/rss/browserfiy, but the responses aren't being constructed correctly at the time of writing.
Alternatively, you could go for a JSON response, i.e. https://registry.npmjs.org/browserify. For a more programmatic way of accessing package details, you can use npm-registry-client.
How should we deal with local packages that are a dependency in other local packages?
For simplicities sake, say we have the follow packages
api - express application
people - a package to deal with people
data-access - a package that deals with data access
And then the dependencies are
api depends on people
people depends on data-access
Currently we have these dependencies setup as file dependencies.
I.e. api package.json would have
"dependencies": {
"people": "file:../people"
}
Trouble with this is that we're finding it a PITA when we make updates to one package and want those changes in the other packages that depend on it.
The options we have thought of are:
npm install - but this won't overwrite previously installed packages if changes are made, so we have to delete the old one from the node_modules directory and re-run npm install... which can be niggly if the package dependency is deep.
npm link - we're not sold on the idea because it doesn't survive version control... Just thinking about it now, maybe we have some kind of local build script that would run the npm link commands for us... this way it could survive version control. Would that be a grunt job?
grunt - we haven't dived too deep into this one yet, but it feels like a good direction. A little bit of googling we came accross this: https://github.com/ahutchings/grunt-install-dependencies
So, what option would work best for our situation?
Are there other options that we haven't thought of yet?
Ps. we're a .NET shop doing a PoC in node, so assume we know nothing!
Pps. if you strongly believe we're setting up our project incorrectly and we shouldn't have smaller individual packages, let me know in the comments with a link to some reading on the subject.
So, I agree that going with 'many small packages' is usually a good idea. Check out 12factor.net if you haven't already.
That said, in specific answer to your question I'd say your best bet is to consider mainly how you want to maintain them.
If the 'subcomponents' are all just parts of your app (as, for example, data-access implies), then I'd keep them in the same folder structure, not map them in package.json at all, and just require them where you need them. In this case, everything versions together and is part of the same git repository.
If you really want to or need to keep them all in separate git repositories, then you can do npm link, but to be honest I've found it more useful to just use the URL syntax in package.json:
dependencies: {
"people" : "git://path.to.git:repo#version.number"
}
Then, when you want to explicitly update one of your dependencies, you just have to bump the version number in your package.json and run npm install again.
I'm using nTwitter to access twitter api using node.
However, there is a bug in the search utility, and someone already forked and fixed it, but it wasn't pulled yet.
How can I use the fixed version s.t all my team will have the fixed version (meaning, just fixing it locally won't do the trick), but still use it as an npm module? can it be done at all?
Install that commit like this:
npm install git://github.com/AvianFlu/ntwitter#e496bc07b9d0138f65902a43bc267796ab1a74d1
Or install using package.json:
{
....
"dependencies": {
....
"ntwitter" : "git://github.com/AvianFlu/ntwitter#e496bc07b9d0138f65902a43bc267796ab1a74d1"
}
}
You can set up a private npm repository and upload your package there under a unique version, something like 1.2.5-yourcompanyname, so it'll be installed instead for everyone who is using this registry.
Pros: it works just like npm registry for everyone who is using it
Cons: it is usable only in closed groups, i.e. within a team or a company
You can set up a git dependency as damphat pointed out.
Pros: it will work for most of the people out of the box
Cons: it will require git to be installed on every user's machine
You can check it to your git repository either as a submodule or just like regular files. Use bundleDependencies when you're publishing it.
Pros: faster to install, and usually work for everyone
Cons: takes up space in git repository, longer checkouts, etc.
We went for the first solution, and usually are trying to avoid second one because we don't have git on production. Third one is quite popular as well.
I'm writing a number of pieces of code (for internal use) using node.js and want to store the modules (packaged up for npm) in a package repository for each distribution to the various machines they will be installed on.
Ideally, I'd like a solution similar to Debian's apt repositories in which I can run a private repository server and configure npm to use a list of repositories to install from (When installing "foo", if "foo" is known by my private server install it from there, otherwise install it from the public server).
However, it looks like the npm registry configuration key only accepts a single URL.
Is there a way to achieve what I want?
The closest I've been able to find have been:
Mirroring the public repository locally and adding my packages on top of it… but I don't want to keep that amount of data (2.5G and still downloading) replicated on AWS.
Hosting all my packages in git repositories and installing from there (which is more of a hassle).
Hosting static packages on HTTP (as far as I can tell, this would prevent me from automatically getting "the latest version". I suppose I could do something with symlinks, but that is still less flexible than git, requires full URLs (which need to be kept up to date), and doesn't give a searchable repository.
I just set this up for my work. Here's what I did:
Setup empty NPM registry: I followed the instructions from this fork of npmjs.org, which adds much improved documentation.
Setup Kappa: I used Kappa, a great npm proxy from Paypal. (I'm guessing they have a very similar use case to most people who want a private repository; this was exactly what I wanted).
Setup npm_lazy (optional): I wanted a nice cache of frequently used packages in case npmjs.org went down, so I added npm_lazy in front of the whole thing, as a caching layer.
Whole thing took two days(ish) to get up and running. As a side note, if you're worried about people pushing to the public registry by accident, I recommend adding this to your package.json:
"publishConfig": { "registry": "http://my-registry.example.com" },
This really is just a bit of paranoia; once you setup your npm to point to your Kappa/npm_lazy instance, Kappa handles publishing to your private repository for you.
Note: Kappa will only every publish to the first repository in it's config. If you need to publish to both your private registry, and the public, you will need to work out your own solution.
In your package.json, you can use any url that points to a valid npm packed module. I use an s3 with a bucket name that is hard to guess.
npm pack
s3cmd put *.tgz s3://path-to-your bucket
S3 is just an example, you could use any mean that can place a file on a web server, it can even be protected via basic auth.
I believe Paypal's Kappa project would suit your need.
Here is an article describing Paypal's Kraken project and how Kappa fits in.
I understand it wasn't available at the time of Quentin's question, but perhaps this will be useful for others that come along here.
npm-registry-client GitHub issue #42 lists several ways how to create your own repository mirror. namely:
https://github.com/cnpm/cnpmjs.org
https://github.com/rlidwka/sinopia
https://github.com/rlidwka/sinopia#similar-existing-things
https://github.com/davglass/registry-static
Overall it seems to me that you can get best answers by searching through issues in repositories owned by https://github.com/npm or by asking your question there
Based only on listening to a recent episode of NodeUp (#37?), I think you may want to have a look at irisnpm. From what I remember it's a service which gives you a merged set of the public modules and your own private modules.
As Dominic Barnes suggested, we can replicate only _design documents (CouchDB table schemes)
How to replicate design documents only?
However, it needs to check if some data needed.
You could replicate the modules you need and then write a proxy server which looks for a module in your replication. If a module doesn't exist, it could pipe the request to NPM and return the result from there.