Is vm.runInNewContext considered black magic like eval?
Is there a
significant performance difference between require and reading a
file and using vm to run it or is the the same under the hood (if
you implemented caching etc and just wanted to add some variables to
the context)
If you check out the code that implements loading Modules in node.js, you'll see that require uses vm.runInNewContext or vm.runInThisContext under the hood. The require however, does some other extra things, like caching the module.
The node documentation shows how the behavior is similar and different between the vm commands and eval.
So, require, eval and vm are all a little bit different, but all can be used to load code. They all have similar security issues if you are loading arbitrary code that comes from the client.
runInNewContext is not meant to be used as a replacement of require or eval, but instead as a way to create a sandbox environment where you can safely run other scripts.
Disadvantages are that it's slow (creation takes ~10 ms.) and takes up a couple megabytes. So no, don't use it as a require replacement.
Related
To simplify, let's have an entry.js, which requires common.js (with very universal functions), and routines.js (which contains project-specific functions). routines.js too requires common.js.
I don't expect the above scenario to be a problem, but in general, are the multiple requires, by modules which require one another a bad practice and is there a better approach? I'm thinking requiring within the entry.js and passing over to functions. Or is this microoptimizing and multiple requires are no big deal?
I don't imagine this being a big deal as Node caches each "require" for you as long as the resolved filename is exactly the same (in most instances it will be, however the node docs list some cases where it may not be)
I found this post to have a nice simple explanation of how the module/require system in node works.
In React, some packages allow you to import Components using either individual assignment: import Card from "#material-ui/core/Card", or via object destructuring: import { Card } from "#material-ui/core".
I read in a Blog that using the object destructuring syntax can have performance ramifications if your environment doesn't have proper tree-shaking functionality. The result being that every component of #material-ui/core is imported, not just the one you wanted.
In what situations could using object destructuring imports cause a decline in application performance and how serious would the impact be? Also, in an environment that does have all the bells and whistles, like the default create-react-app configuration, will using one over the other make any difference at all?
Relying on package internal structure is often discouraged but it's officially valid in Material UI:
import Card from '#material-ui/core/Card';
In order to not depend on this and keep imports shorter, top-level exports can be used
import { Card } from "#material-ui/core"
Both are interchangeable, as long as the setup supports tree-shaking. In case unused top-level exports can be tree-shaken, the second option is preferable. Otherwise the the first option is preferable, it guarantees unused package imports to not be included into the bundle.
create-react-app uses Webpack configuration that supports tree-shaking and can benefit from the second option.
Loading in extra code, such as numerous components from material-ui you may not need, has two primary performance impacts: Download time, and execution time.
Download time is simple: Your JS file(s) are larger, therefore take longer to download, especially over slower connections such as mobile. Properly slimming down your JS using mechanisms like tree shaking is always a good idea.
Execution time is a little less apparent, but also has a similar effect, this time to browsers with less computing power available - again, primarily mobile. Even if the components are never used, the browser must still parse and execute the source and pull it into memory. On your desktop with a powerful processor and plenty of memory you'll probably never notice the difference, but on a slower/older computer or mobile device you may notice a small lag even after the file(s) finish downloading as they are processed.
Assuming your build tooling has properly working tree shaking, my opinion is generally they are roughly equivalent. The build tool will not include the unused components into the compiled JS, so it shouldn't impact either download or execution time.
I'm developing a 'widget', for lack of a better word, that will be loaded in many different sites that I don't control.
We're using RequireJS to keep things easy, but this has the side effect of breaking A LOT of sites that don't already use/support it.
The be clear - we don't control the sites, and the cause is that many of the sites existing libraries are loading into RequireJS instead of globally, and the code on these sites expects them to be loaded globally.
The only practical solution I can think of so far is to rename RequireJS' require() and define() (and perhaps others), then edit every library we rely on (using sed, of course) to load using the 'new' functions.
Has anyone else dealt with this? Is there a better method I'm missing?
Michael
For anyone who stumbles upon this, here's what I ended up doing...
There isn't a good solution for this at the moment as:
1) All libraries that load into RequireJS need define() to exist in their scope at execution time
and
2) There isn't any mechanism for asynchronously loading scripts that would allow define to be defined (pun not intended) and undefined before/after execution, aside from eval(), and that's just not a good option.
This means that, it's not really possible to have some type of scoped RequireJS without it possibly interfering with other scripts on-page that CAN use RequireJS, but are intended to load globally on that particular site.
So... here's the hacky solution I did...
Instead of loading the JS libraries myself, I bundled them on the fly, along with RequireJS, and wrapped in an immediately executing function.
The reason for doing this on-the-fly, is that some site specific data is necessary for the program to function, and it saves an HTTP request to obtain it (at the expense of a larger file download).
This allowed me to:
1) Use libraries that need to run under RequireJS (or similar) to work property
2) Avoid cluttering up the global namespace for stuff like jQuery
3) Avoid editing library source (eg. changing define() to my_special_define() or similar)
I hope this helps someone if they're trying to do the same thing as me :)
I really like the way NodeJS (and it's browser-side counterparts) handle modules:
var $ = require('jquery');
var config = require('./config.json');
module.exports = function(){};
module.exports = {...}
I am actually rather disappointed by the ES2015 'import' spec which is very similar to the majority of languages.
Out of curiosity, I decided to look for other languages which implement or even support a similar export/import style, but to no avail.
Perhaps I'm missing something, or more likely, my Google Foo isn't up to scratch, but it would be really interesting to see which other languages work in a similar way.
Has anyone come across similar systems?
Or maybe someone can even provide reasons that it isn't used all that often.
It is nearly impossible to properly compare these features. One can only compare their implementation in specific languages. I collected my experience mostly with the language Java and nodejs.
I observed these differences:
You can use require for more than just making other modules available to your module. For example, you can use it to parse a JSON file.
You can use require everywhere in your code, while import is only available at the top of a file.
require actually executes the required module (if it was not yet executed), while import has a more declarative nature. This might not be true for all languages, but it is a tendency.
require can load private dependencies from sub directories, while import often uses one global namespace for all the code. Again, this is also not true in general, but merely a tendency.
Responsibilities
As you can see, the require method has multiple responsibilities: declaring module dependencies and reading data. This is better separated with the import approach, since import is supposed to only handle module dependencies. I guess, what you like about being able to use the require method for reading JSON is, that it provides a really easy interface to the programmer. I agree that it is nice to have this kind of easy JSON reading interface, however there is no need to mix it with the module dependency mechanism. There can just be another method, for example readJson(). This would separate the concerns, so the require method would only be needed for declaring module dependencies.
Location in the Code
Now, that we only use require for module dependencies, it is a bad practice to use it anywhere else than at the top of your module. It just makes it hard to see the module dependencies when you use it everywhere in your code. This is why you can use the import statement only on top of your code.
I don't see the point where import creates a global variable. It merely creates a consistent identifier for each dependency, which is limited to the current file. As I said above, I recommend doing the same with the require method by using it only at the top of the file. It really helps to increase the readability of the code.
How it works
Executing code when loading a module can also be a problem, especially in big programs. You might run into a loop where one module transitively requires itself. This can be really hard to resolve. To my knowledge, nodejs handles this situation like so: When A requires B and B requires A and you start by requiring A, then:
the module system remembers that it currently loads A
it executes the code in A
it remembers that is currently loads B
it executes the code in B
it tries to load A, but A is already loading
A is not yet finished loading
it returns the half loaded A to B
B does not expect A to be half loaded
This might be a problem. Now, one can argue that cyclic dependencies should really be avoided and I agree with this. However, cyclic dependencies should only be avoided between separate components of a program. Classes in a component often have cyclic dependencies. Now, the module system can be used for both abstraction layers: Classes and Components. This might be an issue.
Next, the require approach often leads to singleton modules, which cannot be used multiple times in the same program, because they store global state. However, this is not really the fault of the system but the programmers fault how uses the system in the wrong way. Still, my observation is that the require approach misleads especially new programmers to do this.
Dependency Management
The dependency management that underlays the different approaches is indeed an interesting point. For example Java still misses a proper module system in the current version. Again, it is announced for the next version, but who knows whether this will ever become true. Currently, you can only get modules using OSGi, which is far from easy to use.
The dependency management underlaying nodejs is very powerful. However, it is also not perfect. For example non-private dependencies, which are dependencies that are exposed via the modules API, are always a problem. However, this is a common problem for dependency management so it is not limited to nodejs.
Conclusion
I guess both are not that bad, since each is used successfully. However, in my opinion, import has some objective advantages over require, like the separation of responsibilities. It follows that import can be restricted to the top of the code, which means there is only one place to search for module dependencies. Also, import might be a better fit for compiled languages, since these do not need to execute code to load code.
I'm new to Node.js, but quite like the module system and require().
That being said, coming from a C background, it makes me uneasy seeing the same module being require()'d everywhere. All in all, it leads me to some design choices that deviate from how things are done in C. For example:
Should I require() mongoose in every file that defines a mongoose model? Or inject a mongoose instance into each file that defines a model.
Should I require() my mongoose models in every module that needs them? Or have a model provider that is passed around and used to provide these models.
Ect. For someone who uses dependency injection a lot - my gut C feeling is telling me to require() a module only once, and pass it around as needed. However, after looking at some open-source stuff, this doesn't seem to be Node way of things. require() does make things super easy..
Does it hurt to overuse this mechanism?
require() caches modules when you use it. When you see the same file or module required everywhere it's only being loaded once, and the stored module.exports is being passed around instead. This means that you can use require everywhere and not worry about performance and memory issues.
As cptroot states requiring a module everywhere you need it instead of passing it around as an argument is safe to do and is also much easier. However, you should view any require call as a hardcoded dependency which you can't change easily. E.g. if you want to mock a module for testing these hardcoded dependencies will hurt.
So passing a module instance around as an argument instead of just requiring it again and again reduces the amount of hardcoded dependencies because you inject this dependency now. E.g. in your tests you will benefit from easily injecting a mock instead.
If you go down this road you will want to use a dependency injection container that helps you injecting all your dependencies and get rid of all hardcoded require calls. To choose a dependency injection container appropriate for your project you should read this excellent article. Also check out Fire Up! which I implemented.