I have been working with TypeScript off and on for two years now. I am not an expert by any means, but I've spent some time in the eco-system: with VS2015 and node tools, with VSCode, and at the command line with tsc and typings (also used by VSCode).
And I have struggled with the correct way to get static typing and auto-complete etc in the code I've been writing.
I have a repo I maintain (EasyNodeQ) which was the start of my TypeScript experience and I sort of got that to a manageable place with ///reference and DefinitelyTyped. But any time I tried to use that within another project I had issues.
Things got a little better when I started using typings (rather than downloading the *.d.ts files myself) and especially with the ambient flag.
But I still have lots of issues trying to use that package within other projects. Depending on the approach I take, I get lots of Duplicate Identifier's, or module not found's or...
And this can range from Node definitions, to packages I use in both places (like node-uuid).
All I want to understand is this: how to I structure EasyNodeQ so that as I'm working on it, I get the static typings benefits of TypeScript but also have it be seamlessly included in other projects which can then also get these benefits?
Does that make sense?
The basic use cases are: npm install a package and get its typings, use my EasyNodeQ package and get its typings, work in a new project that uses EasyNodeQ and other packages and easily manage those typings.
Preferably in a VSCode or command line way...
This is with ES6 and the latest version of TypeScript (though an answer that works with ES5 would be nice - just not required).
I hope this makes sense. I've looked all over and I can't cobble together an answer that works.
UPDATE
I'm not convinced I've done this the "right" way, but it's working now so I thought I'd post the various things I've done (generally in the order I think they mattered and not inclusive because I may have forgotten some).
I hadn't npm'd the dependency package (easynodeq) and was just using a git url in the package.json - so I created a proper npm package and now install that package from npm
Instead of trying to use Bus.ts as both code and definition, I made Bus.js the "meat" of the npm, and built a Bus.d.ts file (also in the npm)
Embraced typings, using non-ambient definitions where possible and a mix of ambient definitions downloaded (via git) from DefinitelyTyped and "--ambient" definitions for the rest (because I'm still confused about the difference). The ambient definitions ended-up being the majority: node, express, serve-static, express-serve-static-core, mime, amqplib, when vs just bluebird and node-uuid, even though most of them were found by "typings search ..." Am I doing something wrong?
Modified package.json to also do "typings install"
Cleaned-up the git repo
There are several ways to make this work today. As you mentioned, using Typings works for definitions that aren't natively included in their NPM packages. For ones that do, you can using the typings field in package.json and it'll work with node module resolution. When you combine this together, you can publish packages that use both typings.json and relies on packaged typings - though this now forces your consumers to be using Typings to install the definitions. None of this, however, works with "ambient" definitions as they can not be namespaced properly.
This may be useful: https://github.com/typings/typings/blob/master/docs/faq.md#should-i-use-the-typings-field-in-packagejson. There's also dozens of examples I have using both workflows: https://github.com/blakeembrey/change-case/blob/master/package.json#L6 which uses node module resolution all the way down and https://github.com/blakeembrey/popsicle/blob/master/typings.json which uses Typings instead. Using Typings is only possible because it works to create namespaced ambient modules for you, but they won't conflict.
Related
I am working on a project under NodeJS that uses ES6 imports/exports (hence having set type: module in its package.json) and want to automate some tasks using Grunt. Now, I absolutely love grunt, but it appears to me that even in 2022 it is still not able to work nicely with ES6 modules? I always get an error saying "require() of ES Module /vagrant/Gruntfile.js from /vagrant/node_modules/grunt/lib/grunt/task.js not supported."
I understand where this is coming from, and I do understand there are workaround options - in particular, renaming Gruntfile.js to Gruntfile.cjs and passing it to grunt with the --gruntfile command line option. But that is incredibly annoying - it makes the command six times as long as it would be if I could just run grunt and be done with it. Pretty much the same goes for transpiling with something like Babel: That is exactly the kind of thing grunt is intended to handle in the first place, so it feels a bit like the horse riding the jockey. I feel like this should "just work".
Am I missing something here, or is grunt really unable to handle ES6 imports out of the box?
Actually, looking at the grunt github page it appears that a recent commit has addressed it.
I guess this issue will therefore be resolved in their next update.
I have a library coming up deep in the node module hierarchy which is causing security issues. I am not directly referencing that module in my package.json. One of the module which I reference is loading up another module and that module is loading this module. So it's the third layer in the dependency tree. I can find out the library dependence tree using npm ls.
I tried updating package.json, but that's not correct I think.
How can I update the version of this particular module without touching the top modules? Should I have to use shrinkwrap?
One (horrible way) (to answer your question directly) you could carefully manage all of those dependencies on your own and build that structure outside of NPM. I hate it. There is a ton of dependency management overhead and no guarantee any of the hand assembled components would work together - so testing overhead too. but in "theory" it could work. FWIW I don't think shrinkwrap helps with sub dependencies at all.
I recommend this course (I understand this isn't what you asked for - but it is the best approach IMO):
Fork/Branch the library and make the change there.
Then issue a pull request (Back to the main branch)
Until it is is merged back in, you cab reference it via the GIT url in your package.json
from: https://docs.npmjs.com/files/package.json
git+ssh://git#github.com:npm/npm.git#v1.0.27
git+ssh://git#github.com:npm/npm#semver:^5.0
git+https://isaacs#github.com/npm/npm.git
git://github.com/npm/npm.git#v1.0.27
Has anybody got a setup working in which module 1 using typescript is referencing another module 2 also with typescript and both use tsd types such as node.d.ts ?
I have no problem compiling both and using them, as long as i simply require module 2. But as soon as i use import instead, i get into duplicate identifier hell, due to the fact that source files in both modules import e.g. node.d.ts typings from obviously different paths. Classes in both projects use e.g 'streams' or 'lodash' thus both use them in typings and thus use the /// reference syntax. The tsconfig in both projects excludes typings.
Typescript has come a long way since this was asked, and it's now much easier. You can link the proper files in package.json:
{
"main": "library-commonjs.js",
"module": "library-es6-module.js",
"typings": "library-commonjs.d.ts"
}
main is used by packages using CommonJS and Node.js module resolution. module is used by packages supporting ES6 imports. And typings is followed by Typescript (currently Typescript 2.2) to resolve type definitions for the package.
After struggling with this, I spent some time creating a typescript boilerplate/starter project that demonstrates how to do it properly:
https://github.com/bitjson/typescript-starter
That project includes a lot more documentation, and several examples of how to import code from typescript projects.
EDIT: With TS 2.2 this has gotten quite a lot better. See the accepted answer.
It seems this is not really possible yet in typescript 1.8.x. But they seem to work on it via https://github.com/Microsoft/TypeScript/issues/7156.
Also the problem is supposedly mitigated by using a jsconfig.json which should be used by VScode (see https://blogs.msdn.microsoft.com/vscode/2015/07/06/visual-studio-code-es6/). Sadly i didnt get it working yet.
I think it would be great to directly import modules from node_modules directory without need to manually provide a declaration file for it (let us assume that declaration is provided with module itself). I guess that problem are dependencies that come with declarations (file paths could be resolved relative to the module, but that would cause duplicates and compiler can't handle that).
Currently working with node modules is very inconvenient because simple install from npm repository just isn't enough and we have to manually search for declarations and provide them in our project. Let's say that our project is dependent on 10 node modules (all of them have declarations) and after a year we would like to update them. We would have to manually search for new declarations and let's say that we have around 20 projects like this (it would become a nightmare). Maybe there should be an option to directly import .ts file from node module?
Do you have any suggestions?
This is a re-post from CodePlex to hear your opinions ...
If you use grunt-typescript then I've got a pull request which solves this at least for me. See https://github.com/k-maru/grunt-typescript/pull/36
From the pull request README
Working with modules in node_modules (i.e. npm)
The standard way to use npm packages is to provide a definition file
that specifies the package to the typescript and import the module from there.
///<reference path="path/to/mod.d.ts" />
import mod = module('mod')
The typescript compiler will convert the import to a nodejs require.
var mod = require('mod')
This is pretty unwieldy as you need to know the precise path to the
npm installed package and npm can put the package at pretty much any
level when you are working with multiple levels of dependencies.
With the node_modules option in the grunt config you can just
import a npm package without need to know the exact level where the
package has been installed by npm as long as it is installed locally
and not globally.
To import an npm module in your typescript source do
import npmModule = module('node_modules/npmModule/foo')
Mostly due to a lucky chance this works. Typescript compiler will read
the typescript definition file node_modules/npmModule/foo.d.ts if it
is present at some point on the way towards the root and the resulting
javascript file will contain a require for npmModule/foo if needed.
I don't think that node modules will ever contain built-in typescript support. The language still is a 0.x release and officially described as an alpha version.
Nevertheless there are means to ease the configuration process for typescript. Github already contains huge collections of .d.ts files such as:
https://github.com/borisyankov/DefinitelyTyped
or
https://github.com/soywiz/typescript-node-definitions
You might want to take a look at this tool: https://github.com/Diullei/tsd .
I've never used it but it seems like it's almost what you're looking for.
Moreover I've heard that an official database of .d.ts files is planned. Unfortunately I couldn't find the link but it will probably be some time before this is implemented anyways.
Is there a way to specify OS specific dependencies in a npm package.json file?
For example, I would only want to install 'dbus' (https://npmjs.org/package/dbus) as a dependency for my module if the user is running Linux. I would have a different dependency for Mac and Windows.
There's a possible good way of doing this, depending on your setup.
npm package.json supports an os key,
and also optionalDependencies
os can be used to specify which OS a module can be installed on.
optionalDependencies are module dependencies that if they cannot be installed, npm skips them and continues installing.
In this way you can have your module have an optional dependency for each OS, and only the one which works will be loaded/installed ^.^
EDIT: As #Sebastien mentions below, this approach is dangerous.
For any given OS, at least one of your dependencies is "required" and the rest "optional". Making all versions of the dependency optional means that if your installation fails for a legitimate reason, it will silently skip installation and you will be missing a dependency you really need.
I think the short answer is no. I can think of a couple of workarounds though - the simplest is to just add everything to package.json regardless of OS, and then require() the correct one at runtime.
If that doesn't work for you, you might be able to use an install script to get the result you're going for - https://docs.npmjs.com/misc/scripts
I haven't tested this but I think it would work:
Add something like this to your package.json:
,"scripts": {
"install": "node install_dependencies.js"
}
And then add a install_dependencies.js file that checks the OS and runs the appropriate npm install ... commands.
There's also the bindings-shyp module:
https://www.npmjs.com/package/bindings-shyp
Helper module for loading your native module's .node file
This is a helper module for authors of Node.js native addon modules. It is basically the "swiss army knife" of require()ing your native module's .node file.
Throughout the course of Node's native addon history, addons have ended up being compiled in a variety of different places, depending on which build tool and which version of node was used. To make matters worse, now the gyp build tool can produce either a Release or Debug build, each being built into different locations.
This module checks all the possible locations that a native addon would be built at, and returns the first one that loads successfully.
Quoting #npm_support at:
https://twitter.com/npm_support/status/968195526989512705
2/2 If you'd like to avoid installation problems related to dependencies, one route is for you to write a wrapper that's required as a regular dependency, and to make sure that it has optionalDeps (and also ensure that the wrapper verifies you have everything needed to work).
But IMHO it looks more like a workaround than solving the problem for real.
I can understand that npm wants to preserve portability and avoid to deal with platform specifics, but it has to be done anyway and IMHO doing this at runtime is not optimal (specialty if one wants do optimize code size).
So today I have no optimal solution to share but an open discussion for proposal.
Can't "conditional dependencies" be supported in npm ?
The 1st thing that came to my mind was to to add a "override" section that will change (+add, -remove, =replace) current parsed sections.
For example:
dependencies: { "common-stuff": "*" }
overrides: {
"os: { linux: { dependencies: { "+best-linux-module" } } }
}
And other option suggested by a developer I know, would be to introduce a provides keyword, then several modules could provide a same semantic than would be satisfied by resolver (a la debian), but it's generating similar overhead.
I am looking for a generic approach not only focused on OS support but also on other flavors of package (depending on engines for instance).
Do you know any related issue in NPM tracker ? if not I am considering to file a bug to be tracked at:
https://github.com/npm/npm/issues?q=dependencies+conditional
Feedback welcome on this idea.