I'm trying to figure out where dependency injection has it's place in Node. I can't seem to get my head around it even though I know how it works in Java and I've been reading countless blogs.
The examples on the net imo are to trivial. they don't really show Why DI is needed. I'd prefer a complicated example.
I've looked at the following frameworks:
https://github.com/young-steveo/bottlejs
http://inversify.io/
Now, Node uses the module pattern. When I do an import it receives a singleton since that's what node does, it caches modules, unless the factory pattern is used to return a new instance (return new MyThing()).
Now dependency injections primary function is to decouple everything.
When people say that, I get the notion that the goal is... To remove all the imports from the top of a module.
How I write today:
'use strict';
// node modules
import os from 'os';
...8 more modules here
import fs from 'fs';
// npm modules
import express from 'express';
...8 more modules here
import _ from 'lodash';
// local modules
import moduleOne from './moduleOne';
...8 more modules here
import moduleTen from './moduleTen';
//...rest of my code
Having 30 imports is a pain to change. Having the same 30 in multiple files is an even bigger pain.
I was reading https://blog.risingstack.com/fundamental-node-js-design-patterns/ and I looked at the dependency injection area. In the example 1 dependency is passed, fine. What about 30? I don't think that would be good practice?
How would one structure such an application with so many dependencies? And make it friendly for unit testing and mocking?
Implements an Ioc pattern as the dependency injection is in your projects always is a very good choice, this allows you to decouple and granulate your software making it more flexible and less rigid. with the node js module patter is very hard to implements abstraction in your code, that always is required in good architectures, also, doing it, make your code meets with the D [Dependency Inversion] from SOLID and make more easy implements the S-O-I.
If you wanna see an use case for DI, see the readme for this repository also a DI library for node Jems DI, it void the long importing list in modules, and do not make you depend 100% on it with things like putting metadata or forcing you to write extra code in your modules that depend on the DI libraries or sometimes are not needed for your business logic, always you put some abstraction between that DI library and your instances activation.
Related
Different people have told me that in order to improve my Python programming skills, it helps to go and look how existing projects are implemented. But I am struggeling a bit to navigate through the projects and find the parts of the code I'm interested in.
Let's say I'm using butter of the scipy.signal package, and I want to know how it is implemented, so I'm going to scipy's github repo and move to the signal folder. Now, where is the first place I should start looking for the implementation of butter?
I am also a bit confused about what a module/package/class/function is. Is scipy a module? Or a package? And then what is signal? Is there some kind of pattern like module.class.function? (Or another example: matplotlib.pyplot...)
It sounds like you have two questions here. First, how do you find where scipy.signal.butter is implemented? Second, what are the different hierarchical units of Python code (and how do they relate to that butter thing)?
The first one actually has an easy solution. If you follow the link you gave for the butter function, you will see a [source] link just to the right of the function signature. Clicking on that will take you directly to the source of the function in the github repository (pinned to the commit that matches the version of the docs you were reading, which is probably what you want). Not all API documentation will have that kind of link, but when it does it makes things really easy!
As for the second question, I'm not going to fully explain each level, but here are some broad strokes, starting with the most narrow way of organizing code and moving to the more broad ways.
Functions are reusable chunks of code that you can call from other code. Functions have a local namespace when they are running.
Classes are ways of organizing data together with one or more functions. Functions defined in classes are called methods (but not all functions need to be in a class). Classes have a class namespace, and each instance of a class also has its own instance namespace.
Modules are groups of code, often functions or methods (but sometimes other stuff like data too). Each module has a global namespace. Generally speaking, each .py file will create a module when it is loaded. One module can access another module by using an import statement.
Packages are a special kind of module that's defined by a folder foo/, rather than a foo.py file. This lets you organize whole groups of modules, rather than everything being at the same level. Packages can have further sub-packages (represented with nested folders like foo/bar/). In addition to the modules and subpackages that can be imported, a package will also have its own regular module namespace, which will be populated by running the foo/__init__.py file.
To bring this back around to your specific question, in your case, scipy is a top-level package, and scipy.signal is a sub-package within it. The name butter is a function, but it's actually defined in the scipy/signal/_filter_design.py file. You can access it directly from scipy.signal because scipy/signal/__init__.py imports it (and all the other names defined in its module) with from ._filter_design import * (see here).
The design of implementing something in an inner module and then importing it for use in the package's __init__.py file is a pretty common one. It helps modules that would be excessively large to be subdivided, for ease of their developers, while still having a single place to access a big chuck of the API. It is, however, very confusing to work out for yourself, so don't feel bad if you couldn't figure it out yourself. Sometimes you may need to search the repository to find the definition of something, even if you know where you're importing it from.
I have app.js here:
let x = 5;
const print = require('./print')
print()
I have print.js here
module.exports = function(){
console.log(x)
}
Is there a good way I can use the variables in app.js in the print.js function? I'm working with much more variables in my actual application so I would like to avoid using parameters.
You can't or shouldn't. That's not how the modular architecture in node.js works. Unlikely the messy world of the browser where many bodies of code just declare a zillion things in the global scope that everything can then use, node.js is modular. Each file is typically its own module. It must export things that it wishes to share and other modules must import it's exports in order to use them.
If you want to share data with another module, you call some function and pass it that data.
While there are globals in node.js (you can assign things like global.x = 5 and then reference those anywhere), using globals are strongly, strongly discouraged for anything except built-in node.js functionality defined by the platform (not defined by an app).
There are dozens of reasons why globals are problematic and it gets worse the larger a project is, the more people there are working on it or the more 3rd party code libraries you use. If you really want to know more about that, you can just search "why are globals bad in Javascript" and find all sorts of discussion on the topic.
The modular architecture of node.js is one of the reasons that we have NPM and tens of thousands of easily reusable modules of code that work in node.js. These pieces of code all use the same modular architecture making them able to uniformly be used in any app with far less risk of symbol conflicts or modules conflicting with existing code. And, the modular architecture clearly defines dependencies on other pieces of code so a module can be self-contained and load any of the modules it depends on, even different versions of a module than other code in the project is using. None of that works when sharing code via globals.
I will repeat. In node.js, if you want to share a function or data between modules, you export that in one module and import it in another module.
Assign those variables to the global object.
global.x = 5;
const print = require('./print')
print()
module.exports = function(){
console.log(global.x)
}
Right now I'm going through my application, changing instances of this pattern:
import {Grid, Row, Col} from 'react-bootstrap'
into:
import {Grid, Row, Col} from '../react-bootstrap'
where react-bootstrap.js is a simple file at the root of my project, with a selective import of the ES6 modules I need from that NPM package:
import Grid from 'react-bootstrap/es/Grid'
import Col from 'react-bootstrap/es/Col'
import Row from 'react-bootstrap/es/Row'
export {Grid, Col, Row}
Doing this for a number of packages, I was able to reduce my bundle file size by more than 50%.
Is there a WebPack module or plugin that can do it automatically for any package?
If this transformation (that is, only including in the bundle what is explicitly imported, instead of the entire library) was applied recursively to the entire package tree, I bet we would see a dramatic size difference.
Edit: as Swivel points out, this is known as Tree Shaking and is supposed to be performed automatically by Webpack 3+ with UglifyJSPlugin, which is included in the production configuration from react-scripts that I'm using.
I'm not sure if this is a bug in either of those projects, but I'm seeing big size gains by doing selective imports manually, which shouldn't be the case if Tree Shaking was being performed.
I used to search for this and found this article. It is very helpful. I hope it works for you.
Whatever tool that would be would be equivalent to implementing tree shaking, and it would need to be integrated into your bundler. So, no.
For the record, dead code elimination is not the same thing as tree shaking. Tree shaking is breaking unused dependencies between modules. Dead code elimination is within a single module. Uglify.js only knows about one module at a time, so it cannot do tree shaking: it just does dead code elimination. So the fact that you are using the UglifyJSPlugin is irrelevant to whether or not your build environment has tree shaking.
I really like the way NodeJS (and it's browser-side counterparts) handle modules:
var $ = require('jquery');
var config = require('./config.json');
module.exports = function(){};
module.exports = {...}
I am actually rather disappointed by the ES2015 'import' spec which is very similar to the majority of languages.
Out of curiosity, I decided to look for other languages which implement or even support a similar export/import style, but to no avail.
Perhaps I'm missing something, or more likely, my Google Foo isn't up to scratch, but it would be really interesting to see which other languages work in a similar way.
Has anyone come across similar systems?
Or maybe someone can even provide reasons that it isn't used all that often.
It is nearly impossible to properly compare these features. One can only compare their implementation in specific languages. I collected my experience mostly with the language Java and nodejs.
I observed these differences:
You can use require for more than just making other modules available to your module. For example, you can use it to parse a JSON file.
You can use require everywhere in your code, while import is only available at the top of a file.
require actually executes the required module (if it was not yet executed), while import has a more declarative nature. This might not be true for all languages, but it is a tendency.
require can load private dependencies from sub directories, while import often uses one global namespace for all the code. Again, this is also not true in general, but merely a tendency.
Responsibilities
As you can see, the require method has multiple responsibilities: declaring module dependencies and reading data. This is better separated with the import approach, since import is supposed to only handle module dependencies. I guess, what you like about being able to use the require method for reading JSON is, that it provides a really easy interface to the programmer. I agree that it is nice to have this kind of easy JSON reading interface, however there is no need to mix it with the module dependency mechanism. There can just be another method, for example readJson(). This would separate the concerns, so the require method would only be needed for declaring module dependencies.
Location in the Code
Now, that we only use require for module dependencies, it is a bad practice to use it anywhere else than at the top of your module. It just makes it hard to see the module dependencies when you use it everywhere in your code. This is why you can use the import statement only on top of your code.
I don't see the point where import creates a global variable. It merely creates a consistent identifier for each dependency, which is limited to the current file. As I said above, I recommend doing the same with the require method by using it only at the top of the file. It really helps to increase the readability of the code.
How it works
Executing code when loading a module can also be a problem, especially in big programs. You might run into a loop where one module transitively requires itself. This can be really hard to resolve. To my knowledge, nodejs handles this situation like so: When A requires B and B requires A and you start by requiring A, then:
the module system remembers that it currently loads A
it executes the code in A
it remembers that is currently loads B
it executes the code in B
it tries to load A, but A is already loading
A is not yet finished loading
it returns the half loaded A to B
B does not expect A to be half loaded
This might be a problem. Now, one can argue that cyclic dependencies should really be avoided and I agree with this. However, cyclic dependencies should only be avoided between separate components of a program. Classes in a component often have cyclic dependencies. Now, the module system can be used for both abstraction layers: Classes and Components. This might be an issue.
Next, the require approach often leads to singleton modules, which cannot be used multiple times in the same program, because they store global state. However, this is not really the fault of the system but the programmers fault how uses the system in the wrong way. Still, my observation is that the require approach misleads especially new programmers to do this.
Dependency Management
The dependency management that underlays the different approaches is indeed an interesting point. For example Java still misses a proper module system in the current version. Again, it is announced for the next version, but who knows whether this will ever become true. Currently, you can only get modules using OSGi, which is far from easy to use.
The dependency management underlaying nodejs is very powerful. However, it is also not perfect. For example non-private dependencies, which are dependencies that are exposed via the modules API, are always a problem. However, this is a common problem for dependency management so it is not limited to nodejs.
Conclusion
I guess both are not that bad, since each is used successfully. However, in my opinion, import has some objective advantages over require, like the separation of responsibilities. It follows that import can be restricted to the top of the code, which means there is only one place to search for module dependencies. Also, import might be a better fit for compiled languages, since these do not need to execute code to load code.
I'm new to Node.js, but quite like the module system and require().
That being said, coming from a C background, it makes me uneasy seeing the same module being require()'d everywhere. All in all, it leads me to some design choices that deviate from how things are done in C. For example:
Should I require() mongoose in every file that defines a mongoose model? Or inject a mongoose instance into each file that defines a model.
Should I require() my mongoose models in every module that needs them? Or have a model provider that is passed around and used to provide these models.
Ect. For someone who uses dependency injection a lot - my gut C feeling is telling me to require() a module only once, and pass it around as needed. However, after looking at some open-source stuff, this doesn't seem to be Node way of things. require() does make things super easy..
Does it hurt to overuse this mechanism?
require() caches modules when you use it. When you see the same file or module required everywhere it's only being loaded once, and the stored module.exports is being passed around instead. This means that you can use require everywhere and not worry about performance and memory issues.
As cptroot states requiring a module everywhere you need it instead of passing it around as an argument is safe to do and is also much easier. However, you should view any require call as a hardcoded dependency which you can't change easily. E.g. if you want to mock a module for testing these hardcoded dependencies will hurt.
So passing a module instance around as an argument instead of just requiring it again and again reduces the amount of hardcoded dependencies because you inject this dependency now. E.g. in your tests you will benefit from easily injecting a mock instead.
If you go down this road you will want to use a dependency injection container that helps you injecting all your dependencies and get rid of all hardcoded require calls. To choose a dependency injection container appropriate for your project you should read this excellent article. Also check out Fire Up! which I implemented.