When mocking Node dependencies, I've happened upon the following libraries:
Proxyquire
Rewire
SandboxedModule
Sinon
They all seem to do more-or-less the same thing: allow you to mock require() calls (with the exception of Sinon which mocks pretty much everything). They all seem to require some pretty elaborate setup, noting the exact syntax of the string passed to require -- not great during refactoring.
What are the pros and cons of each library? When would I choose one over the other? What are example use-cases where each library excels? What are other products in this space that are better?
It totally feels like cheating, but since no one else is answering the question, here goes:
Proxyquire takes over require and lets you inject fakes anywhere in the dependency chain. For requires you don't take over and for methods you don't define for requires you do take over, it'll fall back to the original. This can be disabled with noCallThru. Thus it still loads the original, just replaces things with the things you define. Unlike Rewire and SandboxedModule, you can't define global variables for your require overloads.
Rewire takes over require and injects __get__ and __set__ properties into each module. If you know the private variable name, you can replace it. Think dependency injection.
SandboxedModule is nearly identical to Proxyquire except it runs the entire process in a new V8 vm. (There is a per-test performance cost to this approach.) It also has a nasty bug in v 1.0 that makes it fail when something you didn't replace references a native module it doesn't support. See https://github.com/robrich/sandboxed-module-graceful-fs.
Sinon doesn't take over require like the other 3. Rather, it's a more traditional mocking framework. Replace specified methods with fakes, or create a mock that tracks when it was called.
Related
How do I set up shared assets across test files?
For example, I have several variables, defined objects, that several tests should use. I also have many jest.mock() to also share across the bunch.
I tried an import of a class or something but nothing happens. To be fair, not sure what type of object I should create: class, function, unicorn, etc.?
I looked into creating my own TestEnvironment but can't see how this gets referenced by my tests. Everything I've seen says it just works. LIES!
This is a React app, latest versions of React and Jest, using create-react-app.
For example, I have several variables, defined objects, that several tests should use.
We are using normal ES6 imports for that. Also it makes sense to try to avoid tight coulping for test suites.
I also have many jest.mock() to also share across the bunch.
You can use manual mock feature for that. Manual mocks are defined by writing a module in a mocks/ subdirectory immediately adjacent to the module.
To be fair, not sure what type of object I should create: class, function, unicorn, etc.?
Mock should be the same type and have the same interface as mocked entity.
I really like the way NodeJS (and it's browser-side counterparts) handle modules:
var $ = require('jquery');
var config = require('./config.json');
module.exports = function(){};
module.exports = {...}
I am actually rather disappointed by the ES2015 'import' spec which is very similar to the majority of languages.
Out of curiosity, I decided to look for other languages which implement or even support a similar export/import style, but to no avail.
Perhaps I'm missing something, or more likely, my Google Foo isn't up to scratch, but it would be really interesting to see which other languages work in a similar way.
Has anyone come across similar systems?
Or maybe someone can even provide reasons that it isn't used all that often.
It is nearly impossible to properly compare these features. One can only compare their implementation in specific languages. I collected my experience mostly with the language Java and nodejs.
I observed these differences:
You can use require for more than just making other modules available to your module. For example, you can use it to parse a JSON file.
You can use require everywhere in your code, while import is only available at the top of a file.
require actually executes the required module (if it was not yet executed), while import has a more declarative nature. This might not be true for all languages, but it is a tendency.
require can load private dependencies from sub directories, while import often uses one global namespace for all the code. Again, this is also not true in general, but merely a tendency.
Responsibilities
As you can see, the require method has multiple responsibilities: declaring module dependencies and reading data. This is better separated with the import approach, since import is supposed to only handle module dependencies. I guess, what you like about being able to use the require method for reading JSON is, that it provides a really easy interface to the programmer. I agree that it is nice to have this kind of easy JSON reading interface, however there is no need to mix it with the module dependency mechanism. There can just be another method, for example readJson(). This would separate the concerns, so the require method would only be needed for declaring module dependencies.
Location in the Code
Now, that we only use require for module dependencies, it is a bad practice to use it anywhere else than at the top of your module. It just makes it hard to see the module dependencies when you use it everywhere in your code. This is why you can use the import statement only on top of your code.
I don't see the point where import creates a global variable. It merely creates a consistent identifier for each dependency, which is limited to the current file. As I said above, I recommend doing the same with the require method by using it only at the top of the file. It really helps to increase the readability of the code.
How it works
Executing code when loading a module can also be a problem, especially in big programs. You might run into a loop where one module transitively requires itself. This can be really hard to resolve. To my knowledge, nodejs handles this situation like so: When A requires B and B requires A and you start by requiring A, then:
the module system remembers that it currently loads A
it executes the code in A
it remembers that is currently loads B
it executes the code in B
it tries to load A, but A is already loading
A is not yet finished loading
it returns the half loaded A to B
B does not expect A to be half loaded
This might be a problem. Now, one can argue that cyclic dependencies should really be avoided and I agree with this. However, cyclic dependencies should only be avoided between separate components of a program. Classes in a component often have cyclic dependencies. Now, the module system can be used for both abstraction layers: Classes and Components. This might be an issue.
Next, the require approach often leads to singleton modules, which cannot be used multiple times in the same program, because they store global state. However, this is not really the fault of the system but the programmers fault how uses the system in the wrong way. Still, my observation is that the require approach misleads especially new programmers to do this.
Dependency Management
The dependency management that underlays the different approaches is indeed an interesting point. For example Java still misses a proper module system in the current version. Again, it is announced for the next version, but who knows whether this will ever become true. Currently, you can only get modules using OSGi, which is far from easy to use.
The dependency management underlaying nodejs is very powerful. However, it is also not perfect. For example non-private dependencies, which are dependencies that are exposed via the modules API, are always a problem. However, this is a common problem for dependency management so it is not limited to nodejs.
Conclusion
I guess both are not that bad, since each is used successfully. However, in my opinion, import has some objective advantages over require, like the separation of responsibilities. It follows that import can be restricted to the top of the code, which means there is only one place to search for module dependencies. Also, import might be a better fit for compiled languages, since these do not need to execute code to load code.
Seems like Bluebird overlaps Co in generator/coroutine related functionality. Bluebird is espoused to have exceptional speed-performance, so for discussion sake, (assuming the aforementioned overlap premise is true) if one wanted to substitute Bluebird for Co in Koa (Node.js context), could it be easily be done without diminishing Koa's functionality, and if so how?
(My guess is it can't practically be done since it seems Koa is built over Co and doesn't explicitly expose it, but facades it. Such a substitution it seems would be tantamount to replacing jQuery with something else in Bootstrap)
First of all, bluebird and co are not comparable like that. You mean Bluebird.coroutine vs co (short for coroutine).
Now, the difference between Bluebird.coroutine and co is that co only allows you to yield a certain set of hard-coded types. While Bluebird.coroutine can be configured to support yielding arbitrary types, the documentation for example contains examples how you can add support for yielding thunks and callbacks.
Async generators are so trivial that the only differences there can be between implementations is what types you can yield and how it performs. Not much room to be better or worse.
However bluebird.coroutine is only a fraction of bluebird features.
Generators only solve the problem of making a sequence of actions less verbose. There is a lot of useful functionality for more advanced needs like resource management, concurrency coordination, error handling, cancellation+timeouts and long stack traces which are impossible or extremely painful if you only have async generators powered by thunks/callbacks/minimal promises.
You can make a drop-in replacement for co by configuring all the yield types that co supports and then just using bluebird.coroutine:
var co = require("bluebird").coroutine;
// Configure all yield types you need using co.addYieldHandler
// See documentation for examples
module.exports = co;
However this doesn't really make any sense since very little code actually should run directly in your request handler - the functions that the request handler calls however do. And those functions are not helped by koa (hmm so what is the point of koa again? :D), so they can be bluebird coroutines directly.
esailija said this about Bluebird,
a feature is being added that allows not only yielding callbacks, thunks etc but any arbitrary thing that comes to your mind. Bluebird is also the fastest. So after this version koa should be just using bluebird indeed. See https://github.com/petkaantonov/bluebird/issues/131#issuecomment-36975495
That said, I don't believe him. And, I don't believe a bluebird wrapper would be faster than Co -- if such a thing were possible. Co.js works, and there is no way possible to get Bluebird.js to pass the tests currently. If you're using ES6, ignore Bluebird entirely and use Co.
I'm new to Node.js, but quite like the module system and require().
That being said, coming from a C background, it makes me uneasy seeing the same module being require()'d everywhere. All in all, it leads me to some design choices that deviate from how things are done in C. For example:
Should I require() mongoose in every file that defines a mongoose model? Or inject a mongoose instance into each file that defines a model.
Should I require() my mongoose models in every module that needs them? Or have a model provider that is passed around and used to provide these models.
Ect. For someone who uses dependency injection a lot - my gut C feeling is telling me to require() a module only once, and pass it around as needed. However, after looking at some open-source stuff, this doesn't seem to be Node way of things. require() does make things super easy..
Does it hurt to overuse this mechanism?
require() caches modules when you use it. When you see the same file or module required everywhere it's only being loaded once, and the stored module.exports is being passed around instead. This means that you can use require everywhere and not worry about performance and memory issues.
As cptroot states requiring a module everywhere you need it instead of passing it around as an argument is safe to do and is also much easier. However, you should view any require call as a hardcoded dependency which you can't change easily. E.g. if you want to mock a module for testing these hardcoded dependencies will hurt.
So passing a module instance around as an argument instead of just requiring it again and again reduces the amount of hardcoded dependencies because you inject this dependency now. E.g. in your tests you will benefit from easily injecting a mock instead.
If you go down this road you will want to use a dependency injection container that helps you injecting all your dependencies and get rid of all hardcoded require calls. To choose a dependency injection container appropriate for your project you should read this excellent article. Also check out Fire Up! which I implemented.
I'm working on my 1st Node.js module, and having to do common utility stuff like check types, looping etc.
The native JS for some of this stuff is pretty ugly. Underscore.js makes it more readable and adds a lot of new features too. But if I don't need the new stuff, should I use Underscore or just do it the hard way?
Thanks!
In node.is you can rely on having some ES5 stuff, array iteration functions and utility functions like isArray. In my node modules I never used underscore and had, due to array iteration functions like map, forEach never the need to use underscore or lodash.
I would not avoid a underscore dependency in case I'd really need it. The node.js platform relies on small modules depending on a couple of small modules itself. So why not depend on underscore.
I see no reason to avoid using a module that makes your life easier. And, it just so happens, that underscore.js is the most depended upon package in the npm registry (as of the time of this answer, according to https://npmjs.org/). So yea, no reason to avoid it.
I've never used underscore nor async on real projects. Once you know how to code good javascript it's not necessary to use any helper library. For example, functions that should execute in serie and are asynchronous it's pretty easy to do with a simple "recursive while loop", you don't need to load any library.
But at the end this is a personal preference. Use external libraries if you feel comfortable with them.
Advice: Don't look at the github starts or npm installations to decide which module to use. Being popular doesn't mean being good. I've tried a lot of popular modules and about a 40% of them are just bad/bugged/not really useful. There are a lot of modules that are not popular that are really good. Being popular helps to take a decision but you should not install and use a module just because it's popular.
Underscore does the right thing, which is check for all the native es5 methods first, meaning you won't have much in the way of performance loss on native methods getting replaced with slower non-native versions that basically do the same thing (code here):
var
nativeForEach = ArrayProto.forEach,
nativeMap = ArrayProto.map,
nativeReduce = ArrayProto.reduce,
nativeReduceRight = ArrayProto.reduceRight,
nativeFilter = ArrayProto.filter,
nativeEvery = ArrayProto.every,
nativeSome = ArrayProto.some,
nativeIndexOf = ArrayProto.indexOf,
nativeLastIndexOf = ArrayProto.lastIndexOf,
nativeIsArray = Array.isArray,
nativeKeys = Object.keys,
nativeBind = FuncProto.bind;
Note: prototypes assigned to "Proto" vars earlier.
That said, I'm pretty sure V8 has most if not all of these. Being of client-side dev origins I'd be delighted simply to use the raw naked thing without having to think about how or what library is best for dragging IE kicking and screaming out of the stone age this time, providing the built-in methods aren't as ugly as the DOM API and I would say these aren't.
If underscore does more for you than the above then by all means use it. If it doesn't, I'd consider it a waste of space. All it really does on the browser is give you fallback methods for the older browsers which aren't a going concern in Node. It's light though. I wouldn't object either way if you were on my team and didn't want to write your own versions of something uniquely handled by underscore but would prefer the direct native method names/args, etc. in my own code on the principle of disliking dependencies anywhere I don't need them.
I use underscore in modules that are shared with the browser, not to depend on ES5. Also Underscore has quite a few very useful methods that are not available in ES5, so it would make sense to read their manual page.