In another project I managed to setup Artifacts for NuGet and it worked better than I expected. I was able to publish my private NuGet package and use the artifacts to get the private package along with the nuget.org too:
Now I want to do the same for npm.
So I set up my artifact and then went to my pipeline and updated the task:
Notice how there is no "Use packages from npm" flag.
Anyway, I thought it would work without that flag and I ran my pipeline and I got this error:
10691 error 404 '#babel/helper-validator-identifier#7.9.5' is not in the npm registry.
Which is really annoying. It's erroring out at the first package it searches for.
I tried to be smart and had two tasks for installing packages. One for my artifact private package and the other using npm defaults but if it errors, it doesn't try to get the next package :(
Does anyone know how I can get around this.
PS: I should stipulate that I don't want to add a .npmrc file to my project. There are many many many projects that use this private package and it would be a massive ballache to have to add that file to them all.
Related
We are in the process of migrating from npmjs to GitHub Packages for our private npm packages. To try and smooth this process, I am trying to have our CI process publish packages to both registries while projects make the switch. We've been using pretty vanilla semantic-release to do our versioning/publishing and I'm trying to preserve that but keep getting stuck trying to publish to more than one place.
My first attempt had a workflow that went:
build/test/etc.
npx semantic-release with NPM_CONFIG_REGISTRY={github} and appropriate creds
npx semantic-release with NPM_CONFIG_REGISTRY={npmjs} and appropriate creds
This results in an error in step 3 because semantic-release tries to re-tag/release to github and fails because it already did that in step 2.
My second attempt was:
build/tests/etc.
npx semantic-release (same github settings as previously)
npm publish with npmjs settings
This works if step two did a release - step 3 pushes the just-created release to npmjs and all is well. However, if step 2 did not do a release (typically because there were no commits that should cause one based on semantic-releases rules), step 3 ends up trying to re-publish whatever the previously released version was, resulting in an error.
Googling for topics related to "publishing to multiple registries with semantic-release" doesn't seem to yield much, so I'm also wondering if I'm just going about this migration the wrong way. Alternative suggestions welcome.
Trying to set up a proof-of-concept for the place I work using a private npm registry to limit the packages developers can download. I set up a feed on Azure Artifacts, and set the official npm registry (https://registry.npmjs.org) as the only upstream source. This feed was set as the registry in the npmrc file, and the project is correctly identifying that as the registry source. (per npm config get registry).
When a user (with permissions to install from upstream) tries to install a package from the empty feed, it installs the package (from the upstream) correctly along with all of its dependencies. It also saves the package to the Artifacts feed, but only some of its dependencies are saved to the feed. There seems to be no rhyme or reason as to which dependencies it saves, as it changes almost every time I install the same exact package.
When a user that does not have permission to install from an upstream source tries to install that same package, it fails on one of the dependencies that wasn't saved, giving a 404 error for the artifacts feed, saying that the package was not found in the registry.
I've set up quite a few different feeds, both project-scoped and organization-scoped to see if I perhaps fiddled with the wrong settings/set something up wrong, but I get the same behavior with every feed I set up.
Are there certain criteria that determine whether or not a dependency is downloaded, and is there a way that I can make it so all dependencies are saved to the feed when a package is installed from the upstream?
Are there certain criteria that determine whether or not a dependency is downloaded
npm has a local cache. You'll want to run npm cache clean before testing. Otherwise, there's no guarantee that the package will be downloaded. It may be installed from the cache instead.
and is there a way that I can make it so all dependencies are saved to the feed when a package is installed from the upstream?
I suppose you can try disabling the cache, but that will likely greatly inflate installation times for your users. You may only want to do that while testing. That said, there are various somewhat-hacky ways to do it more permanently-ish. You can use the force config option but that has other side effects. I imagine you can set the cache to be /dev/null or something like that, although I've never tried that. There are other ideas in the answers provided to the "Disable npm cache" Stackoverflow question.
I have set up an Azure DevOps Artifacts Feed for NPM.
I followed the Instructions on https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=windows
In the next Step, I wanted to publish packages from a “node_modules” Directory within a Visual Studio Project that got its packages from the Public source.
I thought if I run “npm publish” next to my custom “.npmrc” and “package.json” File it would publish all my libraries from the “node_modules” directory. Instead, it published my VisualStudio Project, which uses these libraries. It even followed the Git Ignore rules to not include the “node_modules” folder …
What would be the default way to publish the packages I depend on?
Do I have to write a script to do it for every single Package manually?
What do I with prebuild requiring packages?
After I ran a simple script a couple packages failed.
script:
for /d %i in (C:\Path\node_modules\*) do ( cd "%i" & npm publish )
Error:
…
6 warn prepublish-on-install As of npm#5, `prepublish` scripts are deprecated.
7 warn prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
8 warn prepublish-on-install See the deprecation note in `npm help scripts` for more information.
…
23 error code ELIFECYCLE
24 error errno 1
25 error xml-name-validator#3.0.0 prepublish: `node scripts/generate-grammar.js < lib/grammar.pegjs > lib/generated-parser.js`
25 error Exit status 1
26 error Failed at the xml-name-validator#3.0.0 prepublish script.
…
I saw these packages have their own sub packages : /
PS: My DevOps server and workstation do not have direct access to public Networks!
Thanks for any Help!
What would be the default way to publish the packages I depend on?
You would not re-publish all your project's dependencies (= modules already published by other people) but let users of your module load them automatically by installing your project as a dependency.
Do I have to write a script to do it for every single Package manually?
Again, you would not publish other people's packages.
You might use a bundler like Parcel, Rollup or Webpack to include all your dependencies' build code within your own build artefact, so it will not have external dependencies anymore. To tell your module's users about that fact, you would also have to tweak the package.json of your project (i.e. dependencies become devDependencies) and you also should take care of licenses (some of them require you to include legal headers inside your artefact, you publish other people's work under your own name).
Furthermore you break with the modularity of the ecosystem, so don't expect overall efficiency.
PS: My DevOps server and workstation do not have direct access to public Networks!
I don't know if this was already possible at the time of your post:
Within an Azure Feed you can define Upstream sources.
So your Feed will provide a proxy to ( and cache for ...) npmjs.org, where the dependencies of your project are published / hosted (no need for direct access to public Networks because you will download from npmjs.org through your Feed).
I'm not able to install packages from npm, since I have placed a .npmrc file for install a private library.
This library is hosted by azure work space.
So i just placed the new config file to install. And it works fine in localhost.
How can i keep 2 registry in nprmc file, one for private and another one of npm registry?
error 404 Not Found - GET https://registry.npmjs.org/mm-core - Not found
mm-core is my private library hosted in azure, without my .npmrc file i receive this error.
With .npmrc file npm packages can't be installed.
any help?
It actually should work if you follow the official documents correctly. And we don't need to keep 2 registry in nprmc file, one for private and another one of npm registry.
Solution:
Keep the .npmrc file which presents the azure devops artifacts feed. And sign-in the azure devops web portal to configure the feed settings:
In feed settings, go Upstream sources and make sure you have npmjs as Upstream source. If it not exists, click the Add upstream source to add npmjs.
Then you only need to hold one registry for private library. If the package is not found in your private feed, since we've configured npmjs as upstream source, it will fetch the missing package there automatically!
More details about magic upstream source please refer to this document.
I'm trying to implement a developer workflow with docker, with the ability to develop offline (as in, not having to run npm install when you switch between branches that have differing dependencies)
The most intuitive way to do that is to store dependencies in source control. This has its own issues especially when using modules that compile dependencies. I have tried nearly everything I could think of and find:
npm packing my projects dependencies, storing in source but this doesn't store my dependencies' dependencies
storing node_modules in source, copying this to the container and running npm rebuild but it doesn't actually trigger a rebuild
running npm install --no-registry so t triggers a rebuild but doesn't try to call out, but it actually calls out to the public registry anyway
other solutions I've seen like Node-PAC seem abandoned
npmbox looks the most promising but it requires that it's installed on the target globally, which would work in a container I can build but not production, unless we start deploying containers in production.
Is this a fruitless effort? Lack of network access is rare and would only really be needed when installing a new module or moving between revisions that have differing dependencies
Another option is to setup a private npm repository and to configure it to cache public repository. There are several options to implement this, I would recommend to try Nexus: https://www.sonatype.com/nexus-repository-oss