I'm used to deploy code depending on Composer (PHP's NPM cousing), that sports .json and .lock files. The first one describes the package and your version constraints, and the second one lists exactly what was installed. Always there's a lock file and you run composer install you're sure to receive the same set of packages; running composer update will re-read the json file, install new versions, and update the lock file.
That's awesome for production deployment, since you don't need to checkout your dependencies to your versioning system and you're sure to have the exact same set of dependencies in production as you have in development.
My question is: how to best automate deployment of NPM-dependent code? Is it possible to achieve a method similar to Composer? I've noticed that npm install only installs what's first available in the package.json file. After the first run, i.e. if you change a version constraint you must manually npm update that package - and that would render automate deployment useless, as there's no way to check in to versioning "update this package here to a new version"...
npm shrinkwrap is a analog of composer.lock file. It will generate a npm-shrinkwrap.json, that have all deps with version in it, so you can use it to deploy to production env. Also you can try a various libs from npm to lock versions or search for updates of it without changing packages.json.
Related
Trying to set up a proof-of-concept for the place I work using a private npm registry to limit the packages developers can download. I set up a feed on Azure Artifacts, and set the official npm registry (https://registry.npmjs.org) as the only upstream source. This feed was set as the registry in the npmrc file, and the project is correctly identifying that as the registry source. (per npm config get registry).
When a user (with permissions to install from upstream) tries to install a package from the empty feed, it installs the package (from the upstream) correctly along with all of its dependencies. It also saves the package to the Artifacts feed, but only some of its dependencies are saved to the feed. There seems to be no rhyme or reason as to which dependencies it saves, as it changes almost every time I install the same exact package.
When a user that does not have permission to install from an upstream source tries to install that same package, it fails on one of the dependencies that wasn't saved, giving a 404 error for the artifacts feed, saying that the package was not found in the registry.
I've set up quite a few different feeds, both project-scoped and organization-scoped to see if I perhaps fiddled with the wrong settings/set something up wrong, but I get the same behavior with every feed I set up.
Are there certain criteria that determine whether or not a dependency is downloaded, and is there a way that I can make it so all dependencies are saved to the feed when a package is installed from the upstream?
Are there certain criteria that determine whether or not a dependency is downloaded
npm has a local cache. You'll want to run npm cache clean before testing. Otherwise, there's no guarantee that the package will be downloaded. It may be installed from the cache instead.
and is there a way that I can make it so all dependencies are saved to the feed when a package is installed from the upstream?
I suppose you can try disabling the cache, but that will likely greatly inflate installation times for your users. You may only want to do that while testing. That said, there are various somewhat-hacky ways to do it more permanently-ish. You can use the force config option but that has other side effects. I imagine you can set the cache to be /dev/null or something like that, although I've never tried that. There are other ideas in the answers provided to the "Disable npm cache" Stackoverflow question.
I want to try and make some changes to a package published in npm? (I've suggest some changes as an issue but I think they are simple enough for me to attempt them).
https://www.npmjs.com/package/bt-presence#contributing--modifying
The author supplies some information on how to modify the package, but not really enough for someone doing it for the first time.
Where should I clone the GitHub repo to? The folder where the package is installed? I tried it in my home folder and that would not build (unmodified).
The command npm run build - where is this run from? The root folder of the package where the package.json is?
Will I need to modify the package.json?
In general what is the best way to develop something like this for npm? I've worked on packages before but they were simply Javascript.
If you want to work on the bt-presence package in isolation, you can put the cloned repository anywhere. If you want to use your modified version of bt-presence in combination with an application, my recommended approach is to register bt-presence as a dependency in the application's package.json file with the version set to a relative path to your bt-presence repository; then running npm install in the application will make a symlink from node_modules/bt-presence in the application to your bt-presence repository.
npm run build should indeed be run from the root folder that contains the package.json of bt-presence.
If you just want to change the code of bt-presence, you won't need to modify its package.json. You would only modify the package.json if you need to change any of the settings in there, e.g, if you need to add additional dependencies to your version of bt-presence.
None of the above is really specific to TypeScript. (Some JavaScript packages have build processes too if they need to transform or package the JavaScript files in some way.)
I have a query on how others accomplish this and what, if any, the best practices are. I have an application that comprises three packages and they are setup in Bamboo for CI/CD.
The issue I am experiencing is that of how to automate the update of the package version on each build, e.g. npm version patch to bump the package version.
What I would like to know is how other accomplish these, I have thought about manually running npm version patch but that is not ideal and would be prone to errors (eg. forgetting to run this). I had also thought of adding a step in Bamboo to run npm version patch during the build but Bamboo clones the repository with out remotes and there unable to commit the change, otherwise every build would be the same version and still no better off.
An example is that we have a package which is an Electron application and when the build runs generates an MSI and .yml file for the auto updater, without incrementing the version the MSI will always be the same version and therefore the auto update would not work.
So what are the strategies for automating the version of an npm package for each build? What do others do in similar situations?
I have a custom npm module that I am working on, and it has a GitHub repo. I'm also working on a project that uses the custom module. When working on the larger project, it is nice to use npm link so I can make changes to the module and see them right away in the main project.
To deploy to staging or production, I use shrinkwrap and shrinkpack so I can do an npm install after every deploy (some of the dependencies need binaries, and dev systems aren't the same as production systems, so they do need to be installed and not just kept in source control). Edit: I'm crossing this out as the answer below technically solves my issue, even though it doesn't solve for this particular point, but that wasn't as important as the rest of it.
Of course, since the module is linked to my main project and not listed in package.json, a deploy and install misses it entirely. I can go ahead and list it in package.json and have it point to the appropriate GitHub repo, but then every time I need to test a change in the main project I would have to commit and push those changes, then update the main project, kill and restart the app...that would get tiresome pretty quickly.
I guess I need something like the opposite of "devDependencies"; something where I can have it not install the module on dev, but do install it from GitHub when doing npm install on staging or production. Other than remembering to manually change package.json every time I need to go back and forth, is there a better way to do this?
you can specify a github repository as your package to install, in your package.json file:
{
dependencies: {
"my-library": "githubusername/my-library"
}
}
this will work in your production environment.
in your development environment, use "npm link".
from within the "my-library" folder, run npm link directly. that will tell npm on your local box that "my-library" is avaialable as a link.
now, in your project that uses "my-library", run npm link my-library. this will create a symlink to your local development version of "my-library", allowing you to change code in that repository and have it work in your other project that needs it.
once you are ready to push to production, push "my-library" to your github repository, and then you can npm install on your servers, like normal.
It happens occasionally that the development version of a module works in my development workspace and passes on Travis-CI but after publishing to npm it turns-out the end-user package is broken.
For example if you use a sub module that should be in dependencies but had it in devDependencies then CI will pass (but there are plenty other possible breakages).
How do you automate testing this? Do you use external rigging? Is there a secret module? Do you have a user acceptance test suite?
I use Github with Travis-CI but the standard setup uses the development install.
Once upon a time I discovered that npm would let me publish packages that are uninstallable. So I've added a target to my Gruntfile that does this:
Issue npm pack to create a package from my source.
Into a directory created (automatically by my Gruntfile) just for testing install the new package using npm install <path to the package created in the previous step>.
I have a target for publishing a new version that will publish only if the steps above are successful.
The steps above would not catch the dependency problem you mentioned in the question but they could easily be extended to catch it. To do this, I'd add one or more tests that cause the package installed in step 2 above to call require with all that it depends on.
I would suggest to set up your own CI server that does essentially one thing, npm install package ; cd node_modules/package ; npm test. This would ensure that your package is installable at least on your server.
I heard that Jenkins is good for this (at least, that's what node.js core team seems to be using), but don't have any first hand experience yet. We're just planning to set in up in a couple of weeks.
Also, having some external module that depends on you and testing it helps a bit. :)