Given Alex Birsan's revelation that he was able to run code internally within major private companies via dependency confusion (uploading packages to a public repo with the same name as a company's private/internal package, but with a higher version number), does Gemfury have protections against this?
I know the blended index goes a long way because all dependencies will be pulled from the Gemfury repo first if applicable, but does this include version checks? For example: if my Gemfury repo contains package A with version 1.1 and the public repo contains package A with version 1.2, and my package.json is set to pull the package with the highest minor number, is Gemfury smart enough to pull version 1.1 from my local repo instead of 1.2 from the public repo?
The answer is dependent on the package manager. For package managers that do multi-repo resolution on the client side, this will be dependent on the implementation of the package manager. For some, you can explicitly specify the source for each dependency, which avoids repo confusion. Given Alex Birsan's revelation, various package managers are addressing this individually.
As far as Gemfury's blending indexes for npm and Go Modules, the blending is done at the package level, so if a named package exists in your private account, only private versions are surfaced to the client for that package. This avoids versions in the public index interfering with a private package.
Related
Recently I started to use a private npm packages for backend.
They stored on our private packages server (an npm repository I deployed among application components.
And while I mostly like how it all works and happens, but there is a gotcha.
Imagine a packages like:
#example/logging, #example/utilities, #example/validation, #example/database, #example/security - low-level packages. And many of these low-level packages depends on #selfcad/logging package.
#example/updates-queue, #example/subscriptions, #example/auth-tools, #example/cloud-functions and #example/backend-core packages are representing high-level abstractions and components. Each of high-level packages depending on few or sometimes all of low-level packages.
Some of these packages are referenced in cloud functions via #example/cloud-functions package, especially low-level packages. Other used directly in a cloud functions.
Plus there is a backend component where most of packages are used directly.
So, true headache starts, when a change must be made e. g. in the low-level package like #example/logging or #example/validation.
And what is the most easier way to spread the update or fix to widely referenced lower package? With nested dependencies.
Like so: a package, e. g. #example/validation referenced directly and indirectly, like #example/configuration uses #example/validation, #example/database uses #example/validation and I updated #example/validation internals. How to spread such update across packages?
Maybe you should have a look to Yarn Workspaces. This is a useful feature of yarn to link dependent packages each others using a mono repo. This feature also includes auto-discovery/auto-linking.
I alter some code in a package at
C:\Users\usr\AppData\Local\.meteor\packages\accounts-ui-unstyled\1.3.0\web.browser\login_buttons.js
The thing is , after I alter the code and run “meteor” in the command line the changes are not implemented, I even deleted the whole package mentioned before and run the app and it was like … nothing happened, it’s like the application have some sort of a cache of the packages that he doesn’t have to go to that path to get them , instead it uses what it had from it before.
Can anyone please explain this to me ? What’s happening here ?
The correct way of "changing" a package is to git clone the package from git (or otherwise retrieve it's source) into either a project internal /packages folder or a project external folder (requires environment variable METOER_PACKAGE_DIRS).
If the package is, as in your case, a Meteor internal package, you can also copy only the package into your project and even add it to your versioning.
In this package you then apply your changes. It will be used in favor of the atmosphere package.
A good practice is to also increment the package version, so it is known for everyone that a custom version is in use.
Why you should not change packages inside the users \Users\...\.meteor installation packages folder?
This is the path to packages, that will be used as defaults for every new meteor project you create. Deep changes can create deep damage to your projects since changing a package will apply to all dependent projects.
Think also about project specific customization. The above described method will allow this, too.
When browsing the packagist.org repositories you see packages with these version numbers e.g. If you look at the Phpunit repo
There are a few instances
4.5.x-dev
4.3.x-dev
4.2.x-dev
Do these packages contain the current work the developers are performing towards basic updates, security and bugfixes etc on an otherwise basically stable package?
These are the dev branches. These are unstable and contain bug fixes, etc. This will eventually be released as 4.5.8 for instance (if the library is still supported).
You can get it by using 4.5.x-dev or 4.5.*#dev as version constraint.
I am in the process of introducing NuGet into our software dev process, both for external binaries (eg Moq, NUnit) and for internal library projects containing shared functionality.
TeamCity is producing NuGet packages from our internal library projects, and publishing them to a local repository. My modified solution files use the local repository for accessing the NuGet packages.
Consider the following source code solutions:
Company.Interfaces.sln builds Company.Interfaces.1.2.3.7654.nupkg.
Company.Common.sln contains a reference to Company.Interfaces via its NuGet package, and builds Company.Common.1.1.1.7655.nupkg, with Company.Interfaces.1.2.3.7654 included as a dependency.
The Company.DataAccess.sln uses the Company.Common nupkg to add
Company.Interfaces and Company.Common as references. It builds
Company.DataAccess.1.0.8.7660.nupkg, including Company.Common.1.1.1.7655 as a dependent component.
Company.Product.A is a website solution that contains references to all three library projects (added by selecting the
Company.DataAccess NuGet package).
Questions:
If there is a source code change to Company.Interfaces, do I always need to renumber and rebuild the intermediate packages (Company.Common and Company.DataAccess) and update the packages in Company.Product.A?
Or does that depend on whether the source code change was
a bug fix, or
a new feature, or
a breaking change?
In reality, I have 8 levels of dependent library packages. Is there tooling support for updating an entire tree of packages, should that be necessary?
I know about Semantic Versioning.
We are using VS2012, C#4.0, TeamCity 7.1.5.
It is a good idea to update everything on each check-in, in order to test it early.
What you're describing can be easily managed using artifact dependencies (http://confluence.jetbrains.com/display/TCD7/Artifact+Dependencies) and "Finish Build" build triggers (or even solely "Nuget Dependency Trigger").
We wrote our own build configuration on the base project (would be Company.Interfaces.sln in this case) which builds and updates the whole tree in one go. It checks in updated packages.config files and .nuspec files along the way. I can't say how much of a time-saver this ended up being for us, even if it might sound like overkill at the beginning.
One thing to watch out for: the script we wrote checks in the files even if the chain fails somewhere in between, to give us the chance of fixing it on our local machine, check in the fix and restart the publishing.
I'm writing a number of pieces of code (for internal use) using node.js and want to store the modules (packaged up for npm) in a package repository for each distribution to the various machines they will be installed on.
Ideally, I'd like a solution similar to Debian's apt repositories in which I can run a private repository server and configure npm to use a list of repositories to install from (When installing "foo", if "foo" is known by my private server install it from there, otherwise install it from the public server).
However, it looks like the npm registry configuration key only accepts a single URL.
Is there a way to achieve what I want?
The closest I've been able to find have been:
Mirroring the public repository locally and adding my packages on top of it… but I don't want to keep that amount of data (2.5G and still downloading) replicated on AWS.
Hosting all my packages in git repositories and installing from there (which is more of a hassle).
Hosting static packages on HTTP (as far as I can tell, this would prevent me from automatically getting "the latest version". I suppose I could do something with symlinks, but that is still less flexible than git, requires full URLs (which need to be kept up to date), and doesn't give a searchable repository.
I just set this up for my work. Here's what I did:
Setup empty NPM registry: I followed the instructions from this fork of npmjs.org, which adds much improved documentation.
Setup Kappa: I used Kappa, a great npm proxy from Paypal. (I'm guessing they have a very similar use case to most people who want a private repository; this was exactly what I wanted).
Setup npm_lazy (optional): I wanted a nice cache of frequently used packages in case npmjs.org went down, so I added npm_lazy in front of the whole thing, as a caching layer.
Whole thing took two days(ish) to get up and running. As a side note, if you're worried about people pushing to the public registry by accident, I recommend adding this to your package.json:
"publishConfig": { "registry": "http://my-registry.example.com" },
This really is just a bit of paranoia; once you setup your npm to point to your Kappa/npm_lazy instance, Kappa handles publishing to your private repository for you.
Note: Kappa will only every publish to the first repository in it's config. If you need to publish to both your private registry, and the public, you will need to work out your own solution.
In your package.json, you can use any url that points to a valid npm packed module. I use an s3 with a bucket name that is hard to guess.
npm pack
s3cmd put *.tgz s3://path-to-your bucket
S3 is just an example, you could use any mean that can place a file on a web server, it can even be protected via basic auth.
I believe Paypal's Kappa project would suit your need.
Here is an article describing Paypal's Kraken project and how Kappa fits in.
I understand it wasn't available at the time of Quentin's question, but perhaps this will be useful for others that come along here.
npm-registry-client GitHub issue #42 lists several ways how to create your own repository mirror. namely:
https://github.com/cnpm/cnpmjs.org
https://github.com/rlidwka/sinopia
https://github.com/rlidwka/sinopia#similar-existing-things
https://github.com/davglass/registry-static
Overall it seems to me that you can get best answers by searching through issues in repositories owned by https://github.com/npm or by asking your question there
Based only on listening to a recent episode of NodeUp (#37?), I think you may want to have a look at irisnpm. From what I remember it's a service which gives you a merged set of the public modules and your own private modules.
As Dominic Barnes suggested, we can replicate only _design documents (CouchDB table schemes)
How to replicate design documents only?
However, it needs to check if some data needed.
You could replicate the modules you need and then write a proxy server which looks for a module in your replication. If a module doesn't exist, it could pipe the request to NPM and return the result from there.