I'm trying to cleanly write some universal javascript code, for node and browser.
Most of the code is env-agnostic, however, some implementation parts detect the environment (node or browser) and conditionally execute different code.
I would like to activate node typings ONLY for those specific files. However, I couldn't find a way to do so. Either:
node typings, when referenced in even one file only, are made effective for all files (bad, since I could inadvertently rely on node specificities)
if not referencing node typings at all, typescript obviously complains about a lot of unknown definitions, which would be painful to patch by hand
Do anyone has a clean way of activating some type definitions for a selected set of files ?
It's not possible at this time.
A solution: building node-dependant and node-independant files separately. This could be done automatically with a script.
Related
By default, when editing a JavaScript file in VSCode, it will assume that the fetch function and related types exist in the current context. This makes sense for JavaScript designed to run in the browser, but when running on node.js the fetch function does not exist unless it is installed via node-fetch. I find that in this context, VSCode is misleading, as it will not highlight an error when you trying calling the fetch function, or access other types such as Request and Response, even though they do not exist unless you have node-fetch installed.
How can I configure vscode to assume a node.js context, and therefore assume that fetch does not exist unless I explicitly import it from node-fetch?
Why web types are there by default
From the docs for tsconfig.json compilerOptions.lib:
TypeScript includes a default set of type definitions for built-in JS APIs (like Math), as well as type definitions for things found in browser environments (like document).
How to change the defaults
Create a tsconfig.json or jsconfig.json, and set the compilerOptions.lib array to not contain "DOM", which means that lib.dom.d.ts (the "DOM standard library" type definitions file that comes with TypeScript) will not be assumed. You should specify which ECMA Script standard you want to write your source code in.
The config file also has fields to control what files it takes effect on: files, include, and exclude. If you specify none of them, include will default to **, which matches everything beside and recursively under subdirectories beside the config file.
Having to create this file could be seen as annoying if you just want to write a single JS file (ie. now you have a whole config file just for one source file!). I don't know if there are alternatives that are more convenient for such a use case. If anyone knows, please edit this answer.
I looked briefly into TypeScript triple-slash directives, which allow specifying things on a per-file basis, but I think you can only add things (ie. I don't think you can use them to remove a lib).
At the time of this writing, there are VS Code settings that can be applied at the user-settings scope that affect settings for implicit projects (JS/TS files which don't have a project config file) (js/ts.implicitProjectConfig.*), but none of them are for setting the compileOptions.lib field, and my gut says it's probably not going to happen (but don't quote me on that).
You probably also want types for the Node API
Use npm to install a version of #types/node. Make sure the major version number of the version you install matches the major version number of the version of Node JS you want to script to be runnable on.
Fun irrelevant facts to this question
Continuing on the point about VS Code's user-settings for implicit projects, VS Code puts some defaults in effect (on top of those that TypeScript itself does) if no project is detected. You can poke through the code at github.dev/microsoft/vscode by doing "Find in Files", using extensions/typescript-language-features/**/* as the "files to include" field, and compilerOptions as the find query. compilerOptions.lib seems to not be something that VS Code touches in such a scenario.
If you use something like $FlowIssue it's not guaranteed to be in everyone's .flowconfig file. If you declare a library interface, that seems to only work for the given project, not in other projects that import your package (even if you provide the .flowconfig and interface files in your NPM package).
Here's the code I'm trying to suppress errors for in apps that use my package:
// $FlowIssue
const isSSRTest = process.env.NODE_ENV === 'test' // $FlowIssue
&& typeof CONFIG !== 'undefined' && CONFIG.isSSR
CONFIG is a global that exists when tests are run by Jest.
I previously had an interface declaration for CONFIG, but that wasn't honored in user applications--perhaps I'm missing a mechanism to make that work?? With this solution, at least there is a good chance that user's have the $FlowIssue suppression comment. It's still not good enough though.
What's the idiomatic solution here for packages built with Flow?
Declaring a global variable
This is the way to declare a global variable:
declare var CONFIG: any;. Instead of any you could/should use the actual type.
Error Suppression
With flow v0.33 they introduced this change:
suppress_comment now defaults to matching // $FlowFixMe if there are
no suppress_comments listed in a .flowconfig
This means that there is a greater chance of your error being suppressed if you use $FlowFixMe.
Differences in .flowconfig between your library and your consumers' code are a real issue, and there is no way to make it so that your code can be dropped into any other project and be sure it will typecheck. On top of that, even if you have identical .flowconfigs, you may be running different versions of Flow. Your code may typecheck in one version, but not in another, so it may be the case that consumers of your library will be pinned to a specific version of Flow if they want to avoid getting errors reported from your library.
Worse, if one library type checks only in one version of Flow, and another type checks only in another version, there may be no version of Flow that a consumer can choose in order to avoid errors.
The only way to solve this generally is to write library definition files and publish them to flow-typed. Unfortunately, this is currently a manual process because there is not yet any tooling that can take a project and generate library definitions for it. In the mean time, simply copying your source files to have the .js.flow extension before publishing will work in some cases, but it is not a general solution.
See also https://stackoverflow.com/a/43852211/901387
I am building a custom binary of NodeJS from the latest code base for an embedded system. I have a couple modules that I would like to ship as standard with the binary - or even run a custom script the is compiled into the binary and can be invoked through a command line option.
So two questions:
1) I vaguely remember that node allowed to include custom modules during build time but I went through the latest 5.9.0 configure script and I can't see anything related - or maybe I am missing it.
2) Did someone already do something similar? If yes, what were the best practices you came up with?
I am not looking for something like Electron or other binary bundlers but actually building into the node binary.
Thanks,
Andy
So I guess I figure it out much faster that I thought.
For anyone else, you can add any NPM module to it and just add the actual source files to the node.gyp configuration file.
Compile it and run the custom binary. It's all in there now.
> var cmu = require("cmu");
undefined
> cmu
{ version: [Function] }
> cmu.version()
'It worked!'
> `
After studying this for quite a while, I have to say that the flyandi's answer is not quite true. You cannot add any NPM module just by adding it to the node.gyp.
You can only add pure JavaScript modules this way. To be able to embed a C++ module (I deliberately don't use the word "native", because that one is quite ambiguous in nodeJS terminology - just look at the sources).
To summarize this:
To embed a JS module to your custom nodejs, just add it in the library_files section of the node.gyp file. Also note that it should be placed within the lib folder, otherwise you'll have troubles requiring the module. That's because the name/path listed in node.gyp / library_files is used to encode the id of the module in the node_javascript.cc intermediate file which is then used when searching for the built-in modules.
To embed a native module is much more difficult. The best way I have found so far is to build the module as a static library instead of dynamic, which for cmake(-js) based module you can achieve by changing the SHARED parameter to STATIC like this:
add_library(${PROJECT_NAME} STATIC ${SRC})
instead of:
add_library(${PROJECT_NAME} SHARED ${SRC})
And also changing the suffix:
set_target_properties(
${PROJECT_NAME}
PROPERTIES
PREFIX ""
SUFFIX ".lib") /* instead of .node */
Then you can link it from node.gyp by adding this section:
'link_settings': {
'libraries' : [
"path/to/my/library.lib",
#...add other static dependencies
],
},
(how to do this with node-gyp based project should be quite ease to google)
This allows you to build the module, but you won't be able to require it, because require() function in node can only be used to load built-in JS modules, external JS modules or external dynamic node modules. But now we have a built-in C++ module. Well, lot of node integrated modules are C++, but they always have a JS wrapper in /lib, and those wrappers they use process.binding() to load the C++ module. That is, process.binding() is sort of a require() function for integrated C++ modules.
That said, we also need to call require.binding() instead of require to load our integrated module. To be able to do that, we have to make our module "built-in" first.
We can do that by replacing
NODE_MODULE(mymodule, InitAll)
int the module definition with
NODE_BUILTIN_MODULE_CONTEXT_AWARE(mymodule, InitAll)
which will register it as internal module and from now on we can process.binding() it.
Note that NODE_BUILTIN_MODULE_CONTEXT_AWARE is not defined in node.h as NODE_MODULE but in node_internals.h so you either have to include that one, or copy the macro definition over to your cpp file (the first one is of course better because the nodejs API tends to change quite often...).
The last thing we need to do is to list our newly integrated module among the others so that the node knows to initialize them (that is include them within the list of modules used when searching for the modules loaded with process.binding()). In node_internals.h there is this macro:
#define NODE_BUILTIN_STANDARD_MODULES(V) \
V(async_wrap) \
V(buffer) \
V(cares_wrap) \
...
So just add the your module to the list the same way as the others V(mymodule).
I might have forgotten some step, so ask in the comments if you think I have missed something.
If you wonder why would anyone even want to do this... You can come up with several reasons, but here's one most important to me: Those package managers used to pack your project within one executable (like pkg or nexe) work only with node-gyp based modules. If you, like me, need to use cmake based module, the final executable won't work...
I'm trying to avoid relative require() calls in my express setup. I'd also like to avoid placing my code in the node_modules folder. In short, I'm trying to implement any of the methods described in this gist.
Any of those solutions will work fine for executing code with node or npm. However, I'm trying to find a solution that will also be supported by Intellij IDEA's code resolver, i.e. trying to make sure "go to declaration" and autocomplete hinting works.
I've tried the following
Setting NODE_PATH in the run configuration.
Using a global prefix, i.e. require( global.__base + "mylib").
Adding a symlinked folder to node_modules/.
Adding a symlink from a lib/ folder to node_modules/lib/ does work, but comes with two caveats:
Changes to the source files aren't picked up automatically, so I have to manually "synchronize" node_modules/lib, and
When "going to declaration", IntelliJ (of course) opens node_modules/lib/mylib instead of lib/mylib. This can lead to confusion as the actual file and the symlinked file can be open in separate windows.
Instead of a different way to require local paths (all these methods do work with node after all), I'd be happy with a way to hint to IDEA that it should search the lib/ folder for sources.
So, I realised that if you add a library through Project Structure > Libraries, it won't actually be enabled.
Instead, go to Preferences > Languages & Frameworks > Javascript > Libraries and add a new library. Set the framework type to node_modules, Visibility to Project and add your lib folder.
After adding it, make sure the Enabled checkbox is checked.
That's it, Intellij can now resolve your require('mylib') paths.
Use whatever method from the gist mentioned in the question to actually get node to resolve the paths.
I was wondering whether Node.js/npm include any kind of exension mechanism comparable to Python setuptools' "entry points".
So, in short:
is there any way I can do dynamic discovery of services provided by other packages using npm?
if not, what would be the best way to implement something similar? Specifying the extension name in the main module's configuration file seems to be the logical solution, but I wonder whether something "automatic" can be done.
I'm not aware of any builtin mechanism to do this.
One viable way of doing it yourself:
I made a small tool (Jumpstart) to quickly create project scaffolding from templates with placeholders, and I used a kind of plugin mechanism for that. It basically comes down to that the Jumpstart script searches for modules named jumpstart-* "adjacent" to where the module itself is installed. So it would work for both local and global installations. If installed locally, it would search the other local modules (on the same level) and if global, it searches the other global modules.
Note that here, "search" comes down to a simple fs.exists check to see if there's a Jumpstart template module with a particular name installed. However, nothing would stand in the way to actually get a full list of all installed packages matching the jumpstart-* pattern, and loading all at once. I could also search up the entire directory tree for node_modules directories and do the same. There's no point in doing this for this particular program, however.
See https://npmjs.org/package/jumpstart for docs.
The only limitation to this technique is that all modules must be named in a consistent fashion. Start with some string, end with some string, something like that. Any rogue packages polluting the namespace could be detected by doing further checks on a package contents: What files does it contain? What kind of object does its main module export? etc.
Brunch also uses a plugin mechanism. This one actually deals with file extensions, so is more relevant: https://github.com/brunch/brunch/wiki/Plugins . See for example source of the CoffeeScript plugin https://github.com/brunch/coffee-script-brunch/blob/master/src/index.coffee .