At our company we have at the moment 5 web applications that are built using Gulp. For Gulp, we have a common buildfile that all applications use (and override certain parts of it if needed).
This makes it very easy to add features or fix bugs in all projects at the same time. However, I still need to edit the package.json file in each project separatly if I want to add a new npm dependency or bump a version for an existing one.
What I would like to accomplish is to a "base file" where all the common dependencies are configured, and the I would like to import that into the "local" package.json in each project. It would also be nice if each project could add more dependencies than the ones registered as common.
Is it possible to do this?
No, and it's a good thing that it isn't. You need to declare your dependencies explicitly on each project.
What you can do, if your build process is a shared API, is to extract your build script into an npm package of its own, and include that in the package.json of all other projects, and use it in them (coding it in a way that allows for overrides)
Then when you need a new dependency for your common build, you only need to change it once. (Note that with this, you'd still need to make sure your build package version is up to date in all other applications)
Related
I have a large project that contains multiple sub-modules, many of which are optional. And I have each sub-module sitting in its own folder with local package.json.
I'm trying to figure out the best way to create an integral package.json that would be able to run npm install on a specific subset of those modules.
Question
Is there a more civilized approach these days than just running npm install from every sub-module folder? I want to be able to halt the integral installation when it fails for one module, and hopefully do it on a declarative level within package.json, rather than implementing such logic myself. Plus, I want postinstall-s run on each package (if present), and then one of the integration package.
Other than that, maybe there is a better way today to manage such projects? I'm open to any suggestions!
We know how the dependencies work in NPM. My question is, how can I create a common-library that provides dev dependencies to multiple other components?
Let's say I have a microservice architecture where all the components use mocha and nyc for running the tests.
I don't want to go into each separate component to add them, but to manage this "testing framework" from one central library in devDependencies - however since they are not transitive, they are not installed. I cannot also put them in dependencies because they are not relevant in production mode...
Is it possible in NPM?
I've Searched for the same but sadly, the response is :
no, this isn't possible in NPM.
Workaround
Adding them directly inside the peerDependencies list and counting on the three shaking to make it work.
Feature-request
https://github.com/npm/rfcs/issues/553
I've built a custom style "skin" on top of bootstrap that I intend to use in multiple UI projects.
Rather than copying/pasting the UI styles/components (built using sass and typescript in my current Angular 5 project), I want to build an NPM package that I can install these styles and components I've built in new projects, thus allowing updates to be done to the NPM package (maybe extending the controls within for example) without breaking the UI's using them or needing to update files within.
I've never setup an NPM project before. I've found a number of examples of how to build NPM packages, for example https://codeforgeek.com/2014/08/how-to-create-nodejs-npm-package/ but it seems to be for vanilla JS in this example. I need an example which:
Builds on a dependency, in this case bootstrap
Is to be used in Angular (I'm using version 5)
Is installable and updatable via NPM or maybe Yarn
Has anyone any top tips on achieving the above? Or any really clear guides where someone has done this before?
I realise this question is relatively broad but really I just need some pointers to get started and I will document the process further when I have a better understanding.
Thanks in advance!
So you should move your theme into a separate project. Then all you have to do is run npm init in that directory and you have a npm.
As for sharing it between projects, I would create a repo on Github for this theme npm. Push all of your changes there. Then you can reference it in the package.json of all your projects by linking to the Github repo. Here is a good Stack question about how to do that.
Finally, if you want to develop your theme locally inside one of your projects, you can use npm link. Here are he docs on that.
How should we deal with local packages that are a dependency in other local packages?
For simplicities sake, say we have the follow packages
api - express application
people - a package to deal with people
data-access - a package that deals with data access
And then the dependencies are
api depends on people
people depends on data-access
Currently we have these dependencies setup as file dependencies.
I.e. api package.json would have
"dependencies": {
"people": "file:../people"
}
Trouble with this is that we're finding it a PITA when we make updates to one package and want those changes in the other packages that depend on it.
The options we have thought of are:
npm install - but this won't overwrite previously installed packages if changes are made, so we have to delete the old one from the node_modules directory and re-run npm install... which can be niggly if the package dependency is deep.
npm link - we're not sold on the idea because it doesn't survive version control... Just thinking about it now, maybe we have some kind of local build script that would run the npm link commands for us... this way it could survive version control. Would that be a grunt job?
grunt - we haven't dived too deep into this one yet, but it feels like a good direction. A little bit of googling we came accross this: https://github.com/ahutchings/grunt-install-dependencies
So, what option would work best for our situation?
Are there other options that we haven't thought of yet?
Ps. we're a .NET shop doing a PoC in node, so assume we know nothing!
Pps. if you strongly believe we're setting up our project incorrectly and we shouldn't have smaller individual packages, let me know in the comments with a link to some reading on the subject.
So, I agree that going with 'many small packages' is usually a good idea. Check out 12factor.net if you haven't already.
That said, in specific answer to your question I'd say your best bet is to consider mainly how you want to maintain them.
If the 'subcomponents' are all just parts of your app (as, for example, data-access implies), then I'd keep them in the same folder structure, not map them in package.json at all, and just require them where you need them. In this case, everything versions together and is part of the same git repository.
If you really want to or need to keep them all in separate git repositories, then you can do npm link, but to be honest I've found it more useful to just use the URL syntax in package.json:
dependencies: {
"people" : "git://path.to.git:repo#version.number"
}
Then, when you want to explicitly update one of your dependencies, you just have to bump the version number in your package.json and run npm install again.
What exactly should I put in .npmignore?
Tests? Stuff like .travis.yml, .jshintrc? Anything that isn't needed when running the module (except the readme)?
I can't find any guidance on this.
As you probably found, NPM doesn't really state specifically what should go in there, rather they have a list of ignored-by-default files. Many people don't even use it as everything in your .gitignore is ignored in npm by default if .npmignore doesn't exist. Additionally, many files are already ignored by default regardless of settings and some files are always excluded from being ignored, as outlined in the link above.
There is not much official on what always should be there because it is basically a subset of .gitignore, but from what I gather from using node for 5-ish years, here's what I've come up with.
Note: By production I mean any time where your module is used by someone and not to develop on the module itself.
Pre-release cross-compiled sources
Pros: If you are using a language that cross-compiles into JavaScript, you can precompile before release and not include .coffee files in your package but keep tracking them in your git repository.
Build file leftovers
Pros: People using things like node-gyp might have object files that get generated during a build that never should go into the package.
Cons: This should always go into the .gitignore anyway. You must place these things inside here if you are using a .npmignore file already as it overrides .gitignore from npm's point of view.
Tests
Pros: Less baggage in your production code.
Cons: You cannot run tests on live environments in the slim chance there is a system-specific failure, such as an out of date version of node running that causes a test to fail.
Continuous integration settings/Meta files
Pros: Again, less baggage. Things such as .travis.yml are not required for using, testing, or viewing the code.
Non-readme docs and code examples
Pros: Less baggage. Some people exist in the school-of-thought where if you cannot express at least minimum viable functionality in your Readme, your module is too big.
Cons: People cannot see exhaustive documentation and code examples on their own file system. They would have to visit the repository (which also requires an internet connection).
Github-pages objects
Pros: You certainly don't need to litter your releases with CNAME files or placeholder index.htmls if you use your module serves double-duty as a gh-pages repository as well.
bower.json and friends
Pros: If you decide to build in your dependencies prior to release, you don't need the end-user to install bower then install more things with that. I would, personally, keep that stuff in the package. When I do an npm install, I should only be relying on npm and no other external sources.
Basically, you should ever use it if there is something you wish to keep out of your npm package but checked-in to your module's repo. It's not a long list of items, but npm would rather build in the functionality than having people stuck with irrelevant objects in their package.
I agree with lante's short and syntetic answer and SamT's big answer:
You should not include your tests in your package.
Your package should only contains production runtime files.
That will make your package more straightforward and faster to be dowloaded.
My contribution to those answers:
.npmignore is the blacklist way to achieve package file selection. But in a more practical way, you can whitelist files you need to include in your package using the files field in your package.json:
{
"files": [
"lib/",
"index.js"
]
}
I think that's simpler, future proof and have better semantics ;)
Just to clarify, anytime someone do npm install your-library, npm will download all source files that the package includes. Those files that were included in the .npmignore file in the source code of the package your-library will be excluded when publishing the lib, so users of your-library won't download them.
Know that people installing your library will need just your library running, anything else will be not necessary.
For example, when someone installs a library, its probably that he/she doesn't care about your .travis.yml or your .jshintrc files, or even some images, Grunt files, documentation, etc.
.npmignore could let your npm package to have less files, and faster to be downloaded
Don't include your tests. Oftentimes tests are like 5x the size of the actual codebase. As long as your tests are on Github, etc, that's good enough.
But what you absolutely should do is test your NPM package in its published format. Create some smoke tests that reside in the actual codebase, but are not part of the test suite.
You can read about testing your package after tarballing it, here:
https://github.com/ORESoftware/r2g
How to test an `npm publish` result, without actually publishing to NPM?