I have a query on how others accomplish this and what, if any, the best practices are. I have an application that comprises three packages and they are setup in Bamboo for CI/CD.
The issue I am experiencing is that of how to automate the update of the package version on each build, e.g. npm version patch to bump the package version.
What I would like to know is how other accomplish these, I have thought about manually running npm version patch but that is not ideal and would be prone to errors (eg. forgetting to run this). I had also thought of adding a step in Bamboo to run npm version patch during the build but Bamboo clones the repository with out remotes and there unable to commit the change, otherwise every build would be the same version and still no better off.
An example is that we have a package which is an Electron application and when the build runs generates an MSI and .yml file for the auto updater, without incrementing the version the MSI will always be the same version and therefore the auto update would not work.
So what are the strategies for automating the version of an npm package for each build? What do others do in similar situations?
Related
I am interested in understanding how to install node-sqlite3, but provide by own precompiled package of sqlite3: I just want to install the Node client and skip the build phase entirely during install.
How can I do this?
(Reasoning: I am going to test the module in multiple environments and have already read countless posts of people having build issues in various environments, so I'd rather manually compile myself.)
It turns out that I was looking for a package like this one:
dblite on npm, GitHub
We build a web application and our project uses various npm packages for development, testing and run-time.
The project is built as part of a large project in TFS. TFS runs ant to build the project. Our build.xml first runs npm install, then transpiles and minifies the TypeScript and Sass files (using Grunt tasks) and then builds the final war fie.
This all works OK, but our TFS is not allowed to access the Internet during the build, only our local network. Therefore, we have all the npm libraries we use copied to a file server in our network, and our package.json dependencies point to paths on that file server.
Does this seems like a reasonable solution?
The problem we have is that the npm install takes about 10 minutes to get all the >50 packages we use (which includes karma, grunt, sass, tslint, etc. – total is 170MB).
We are now looking for way to reduce the TFS build time. One option is to but the node_modules in our source control and skip the npm install step, but is seems wrong to put third-party code in our source control.
I’d love to hear other ideas to handle this and have shorter build time.
Note that on developers machine the project builds in no time, as all packages are already installed, but TFS builds start by getting a clean environment from source control, so nothing is installed.
Tough problem. You could have TFS check if your package.json checksum has changed in order to determine if a "clean" is necessary. You'd still have a 10 minute build whenever package.json is updated, but package.json changes are usually infrequent.
The lines become blurred when you host your own npm libraries since this is essentially taking a snapshot of only the dependencies you need. Therefore, if you added a dependency, colors, you'd have to update your npm repo. That could be viewed as updating the node_modules folder on your npm repo. It's a static list of available dependencies which essentially defeats the purpose of a package.json (unless of course other internal apps use the internal npm repo).
BUT, I digress, I'd argue that the best option is to have a package.json checksum for TFS to know if it should bother rebuilding node_modules.
I'm used to deploy code depending on Composer (PHP's NPM cousing), that sports .json and .lock files. The first one describes the package and your version constraints, and the second one lists exactly what was installed. Always there's a lock file and you run composer install you're sure to receive the same set of packages; running composer update will re-read the json file, install new versions, and update the lock file.
That's awesome for production deployment, since you don't need to checkout your dependencies to your versioning system and you're sure to have the exact same set of dependencies in production as you have in development.
My question is: how to best automate deployment of NPM-dependent code? Is it possible to achieve a method similar to Composer? I've noticed that npm install only installs what's first available in the package.json file. After the first run, i.e. if you change a version constraint you must manually npm update that package - and that would render automate deployment useless, as there's no way to check in to versioning "update this package here to a new version"...
npm shrinkwrap is a analog of composer.lock file. It will generate a npm-shrinkwrap.json, that have all deps with version in it, so you can use it to deploy to production env. Also you can try a various libs from npm to lock versions or search for updates of it without changing packages.json.
It happens occasionally that the development version of a module works in my development workspace and passes on Travis-CI but after publishing to npm it turns-out the end-user package is broken.
For example if you use a sub module that should be in dependencies but had it in devDependencies then CI will pass (but there are plenty other possible breakages).
How do you automate testing this? Do you use external rigging? Is there a secret module? Do you have a user acceptance test suite?
I use Github with Travis-CI but the standard setup uses the development install.
Once upon a time I discovered that npm would let me publish packages that are uninstallable. So I've added a target to my Gruntfile that does this:
Issue npm pack to create a package from my source.
Into a directory created (automatically by my Gruntfile) just for testing install the new package using npm install <path to the package created in the previous step>.
I have a target for publishing a new version that will publish only if the steps above are successful.
The steps above would not catch the dependency problem you mentioned in the question but they could easily be extended to catch it. To do this, I'd add one or more tests that cause the package installed in step 2 above to call require with all that it depends on.
I would suggest to set up your own CI server that does essentially one thing, npm install package ; cd node_modules/package ; npm test. This would ensure that your package is installable at least on your server.
I heard that Jenkins is good for this (at least, that's what node.js core team seems to be using), but don't have any first hand experience yet. We're just planning to set in up in a couple of weeks.
Also, having some external module that depends on you and testing it helps a bit. :)
There are solutions to increment the version number in different application. Cocoa apps have agvtool and maven has the maven-release-plugin which increments the version number on releases. Are there similar tools for nodejs?
I think such a tool seems excessively heavy-handed when the only thing you need to do to increment the version number in Node.js is something as simple as:
sed -i 's/0.1.2/0.2.4/' package.json
Will change the version for your Node.js package?
If you're talking about something that will deploy your code when you explicitly mark a new version, I'd be more inclined to use git hooks and write a script to detect when a git tag is created and then start the deployment process.
You could use the pre-applypatch hook to detect when a tag is being created to run the sed script listed above and run npm publish for you automatically, but again, I don't see the point of having a heavyweight tool handle all of that when it's a simple script away (and said script could be written in Node.js, too!)
I think the most correct answer is to use npm version.
For example, to increment the major version of your package:
npm version major
This will increment the major version of your package and if it is a git repository it will also commit that change and create a tag of the same name of the version.
Follow up with:
git push
git push --tags
grunt-bump provides version bumping, as well as git tagging/commit/push. There are similar packages for other build tools such as gulp-bump.
Similar functionality is also now available with npm version, but with fewer configuration options.