As a bit of background for my reasoning around this request:
I have created a set of Gulp based tasks and have packaged them in a NPM styled package. Having this package allows me to import my tasks into my future projects so that I don't need to be copying and pasting my Gulp tasks around between them. I have this working great.
The issue that I currently have is that I am creating a fairly wide and generic set of tasks that can service a array of web project configurations. I don't use all of these tasks in every project, but when installing my Gulptools package I end up getting all of it's dependencies installed. Currently at 250mb worth, but I expect this to increase with age,
It would be really great if I could do some sort of lazy dependency installation, that would pull and install my package's dependencies when they are first required during runtime. Subsequent requests would be much faster then as the dependency would be installed.
Does anyone know how to do this? If so, you are awesome! :)
Update 1
I have found the following package!
https://www.npmjs.com/package/lazy-install
I will investigate this and report. :)
Related
Here are some of my current thoughts, but would also like to understand how the community is doing it now
Use caching to reduce build time
Motivation
Currently, every time you change something in the libs, someone else needs to initialize it again when pulling it through git. If you know which package it's in, you can just run the initialize command for the specified package. If you don't know, you need to run the root initialize command, which is actually very slow because it re-runs all npm packages that contain the intialize command, regardless of whether they have changed or not.
reduce initialize time and improve collaborative development experience
support for ci/cd caching of built libs to speed up build time
Current state
lerna run
First initialization of new project 198.54s
Re-initialize for non-new projects 90.17s
Requirements
Execute commands as concurrently as possible according to the dependency graph, and implement caching based on git changes
Execute commands in modules that specify module dependencies
Execute commands in all submodules
Implementation ideas
lerna api is currently severely undocumented: https://github.com/lerna/lerna/issues/2013
scan lerna.json for all modules
run the specified commands in parallel as much as possible according to the dependencies
check git when running the command to see if it needs to be skipped
Schematic: https://photos.app.goo.gl/aMyaewgJxkQMhe1q6
The ultra-runner already exists, so maybe it's better to support this feature based on it
There seems to be no support for running commands other than build at the moment, see: https://github.com/folke/ultra-runner/issues/224
Known solutions
Trying to use Microsoft rush -- hoping for a better solution without replacing the main framework lerna+yarn
I've built a custom style "skin" on top of bootstrap that I intend to use in multiple UI projects.
Rather than copying/pasting the UI styles/components (built using sass and typescript in my current Angular 5 project), I want to build an NPM package that I can install these styles and components I've built in new projects, thus allowing updates to be done to the NPM package (maybe extending the controls within for example) without breaking the UI's using them or needing to update files within.
I've never setup an NPM project before. I've found a number of examples of how to build NPM packages, for example https://codeforgeek.com/2014/08/how-to-create-nodejs-npm-package/ but it seems to be for vanilla JS in this example. I need an example which:
Builds on a dependency, in this case bootstrap
Is to be used in Angular (I'm using version 5)
Is installable and updatable via NPM or maybe Yarn
Has anyone any top tips on achieving the above? Or any really clear guides where someone has done this before?
I realise this question is relatively broad but really I just need some pointers to get started and I will document the process further when I have a better understanding.
Thanks in advance!
So you should move your theme into a separate project. Then all you have to do is run npm init in that directory and you have a npm.
As for sharing it between projects, I would create a repo on Github for this theme npm. Push all of your changes there. Then you can reference it in the package.json of all your projects by linking to the Github repo. Here is a good Stack question about how to do that.
Finally, if you want to develop your theme locally inside one of your projects, you can use npm link. Here are he docs on that.
I lately helped out on a project, where I added a really small dependency - in fact, it only contained a regular expression (https://www.npmjs.com/package/is-unc-path).
The feedback I got from the developer of the project was that he tries to minimize third-party dependencies if they can be implemented easily - whereby he - if I understand it correctly - asks me to just copy the code instead of adding another dependency.
To me, adding a new dependency looks just like putting some lines of code into an extra file in the repo. In addition, the developers will get informed by an update if the code needs a change.
Is it just a religious thought that drives a developer to do this? Are there maybe any costs (performance- or space-wise, etc) when adding a dependency?
I also had some disputes with my managers once concerning the third party libraries, the problem was even greater he got into believing that you should version the node_modules folder.
The source of any conflict usually is the ignorance.
His arguments were:
you should deliver to the client a working product not needing for him to do any other jobs like npm install
if github, npm is down in the moment when you run npm install on the server what you will do ?
if the library that you install has a bug who will be responsible
My arguments were:
versioning node_modules is not going to work due to how package dependencies work, each library will download his own node_modules dependencies and then your git repository will grow rapidly to hundreds of mb. Deploy will become more and more slow, downloading each time half a gb of code take time. NPM does use a module caching mechanism if there are no changes it will not download code uselessly.
the problem with left-pad was painfull but after that npm implemented a locking system and now for each package you just lock to a specific commit hash.
And Github, and npm does not have just a single instance service, they run in cloud.
When installing a dependency you always have some ideas behind and there are community best practices, usually they resume to: 1. Does the repo has unit tests. 2. The download number 3. When was the latest update.
Node.js ecosystem is built on modularity, it is not that node is so popular cause of some luck, but cause of how it was designed to create modules and reuse them. Sometimes working in node.js environment feels like putting lego pieces together and building your toy. This is the main cause of super fast development in node.js. People just reuse stuff.
Finally he stayed on his own ideas, and I left the project :D.
I am new to angular2 and I have never used angular1.
I have been trying this tutorial https://angular.io/guide/quickstart and
i wonder if all the recommended dependencies are really necessary because my project size gets big after 'npm install' (more than 100Mb).
You won't necessarily use all of the #angular dependencies during the Quickstart, but if you continue on with their tutorial and as your app gets more complex, you will definitely utilize them.
The other packages are necessary. They consist of typescript compilers, libraries for Observables, and more Angular 2 features. Here is a quick read for info on some of these.
If you're using angular-cli, calling ng serve will also run a series of automated tests that depend on packages.
How should we deal with local packages that are a dependency in other local packages?
For simplicities sake, say we have the follow packages
api - express application
people - a package to deal with people
data-access - a package that deals with data access
And then the dependencies are
api depends on people
people depends on data-access
Currently we have these dependencies setup as file dependencies.
I.e. api package.json would have
"dependencies": {
"people": "file:../people"
}
Trouble with this is that we're finding it a PITA when we make updates to one package and want those changes in the other packages that depend on it.
The options we have thought of are:
npm install - but this won't overwrite previously installed packages if changes are made, so we have to delete the old one from the node_modules directory and re-run npm install... which can be niggly if the package dependency is deep.
npm link - we're not sold on the idea because it doesn't survive version control... Just thinking about it now, maybe we have some kind of local build script that would run the npm link commands for us... this way it could survive version control. Would that be a grunt job?
grunt - we haven't dived too deep into this one yet, but it feels like a good direction. A little bit of googling we came accross this: https://github.com/ahutchings/grunt-install-dependencies
So, what option would work best for our situation?
Are there other options that we haven't thought of yet?
Ps. we're a .NET shop doing a PoC in node, so assume we know nothing!
Pps. if you strongly believe we're setting up our project incorrectly and we shouldn't have smaller individual packages, let me know in the comments with a link to some reading on the subject.
So, I agree that going with 'many small packages' is usually a good idea. Check out 12factor.net if you haven't already.
That said, in specific answer to your question I'd say your best bet is to consider mainly how you want to maintain them.
If the 'subcomponents' are all just parts of your app (as, for example, data-access implies), then I'd keep them in the same folder structure, not map them in package.json at all, and just require them where you need them. In this case, everything versions together and is part of the same git repository.
If you really want to or need to keep them all in separate git repositories, then you can do npm link, but to be honest I've found it more useful to just use the URL syntax in package.json:
dependencies: {
"people" : "git://path.to.git:repo#version.number"
}
Then, when you want to explicitly update one of your dependencies, you just have to bump the version number in your package.json and run npm install again.