I'm trying to fully understand how Typescript works in a Node.js project. To accomplish this I have created my own custom_modules folder with a separate #types folder underneath for my declarations while the actual implementation is under the custom_modules attempting to mimic the structure of node_modules. My goal is to make this module usable in the project non-relatively with separate declaration and implementation. I have been able to setup a project that compiles with this setup, but running it errors with:
Cannot find module 'foo'
The source is available here:
https://github.com/anorborg/learn-ts
The node_modules folder is a somewhat special case in how typings get handeled. This is a result of how nodejs works. Take a look at the [module-resolution] doucmentation (https://www.typescriptlang.org/docs/handbook/module-resolution.html), it describes more in depth how module resolution work in typescript.
But in short to answer your question: you can not use non-relative module paths in this way. Node will look for the file in node_modules at runtime, and will not find it there. The paths property in the tsconfig.json is there to solve problems that can occur in other cases, as when targeting RequireJS or SystemJS for example, but not when targeting node.
Related
I have an NPM module that uses another local NPM module containing shared code. Neither of them are public, this is all local.
I import the shared module in my package.json like so:
"my-shared": "file:../my-shared-code"
When I npm install, my-shared-code gets imported correctly, and I can import code like:
import Blah from 'my-shared/src/sharedTypes';
Problem
I have to use the word "src" in the import. However, when I build I create a build directory, which breaks all these imports!
I was wondering if I could use NPM to map the imports somehow?
Can I make it so I don't have to use the word "src" at all?
Could I just do:
import Blah from 'my-shared/sharedTypes';
and then it magically figures out whether to use the "src" or "build" dirs?
What I tried
I looked into the options for package.json, and there is a "files" property that I thought might help. However I think that just whitelists files.
I also looked into the "main" property, however I'm not "exporting a module". I just have a load of utility files that I wanna be able to import into multiple other projects.
What I'm actually trying to achieve
I'm using typescript, and I've got a front-end and a backend that both share types for certain models, but also share some utility functions.
I want my typescript/react front-end and my typescript backend to be able to import typescript files from another node_package, however it needs to know to use "src" for development and "build" when built for production.
While it's not exactly what you're asking for, you might be looking for npm link. It's not intuitive, but it's what I use for your above situation.
npm-link is a way of registering local projects so that others can reference them. Just beware that reading the docs is important -- using it may impact your local environment in non-obvious ways (especially if you build and publish directly from your machine--versus CI/CD).
In the consumer project you can use path mapping --> https://www.typescriptlang.org/docs/handbook/module-resolution.html#path-mapping
As for the package, you can re-export using export * from './my-file' syntax, in a index.ts file (you may need one at each directory). But this is tedious if you have a lot of files. (imperfect example)
Another solution seems to make a script to copy the package.json into the dist folder and make the build there.
There is a similar discussion there: How to npm publish specific folder but as package root
I'm just starting to learn Angular, I installed in my Ubuntu Linux:
Angular CLI: 7.1.4. and
Node: 10.14.2.
My question is, why one project is too large? I mean a simple "helloworld" is 334MB, I don't know why, Is it possible resize it? or delete any folder that is unnecessary? Or Is there something wrong with my installation? (how can I detect that?)
The bigger folder is node_modules, and when I create my projects, generates a lot of folders.
I was reading about "angular lazy loading", but I'm new.. It is not totally clear for me.
this is the folder spaces:
Please... I hope somebody can help me...
best regards
You might be using big bundles which are not needed, so you can break them up:
https://medium.com/#julienetienne/javascript-bundles-are-too-big-break-up-the-libraries-3be87d5ef487
In modern JavaScript, projects require modules that themselves require modules that require modules... resulting in node_modules directory with hundreds of dependencies. A 100Kb library might be included because someone needed one function from it. JavaScript is not compiled, so all that source tends to be rather big. This is an unfortunate, but unavoidable truth; your Angular project directories will be big, and there's nothing you can do about it. It is perfectly normal.
The good part: modern JavaScript deployment typically includes packing up those libraries with Webpack or Parcel or similar code bundlers. Most of them implement "tree shaking", which analyses the code to find only the functions that are potentially utilised starting from the entry point, and only bundle those, ignoring the rest. This means that 100Kb library whose one function is used is transformed into just that one function in the final distribution bundle.
Not all of the bundlers are equally good at it at this point. For example, Webpack cannot tree-shake the CommonJS modules, only ES6 ones.
You can remove node_modules folder when you are not going to use the app.
And, when you need work on the application, you can re-generate node_modules using the command: npm install
Those are just node-modules, they are needed for building the project, but not necessarily everything inside of them will be deployed to production. As Amadan said, there is a process of tree-shaking (filtering only used modules) and also in production you use the minified version of the same JS code (where for example whitespace is missing and variable-names are shortened), among other optimizations. A production-optimized Angular project should not be more than a 100KB for a hello-world application.
In the provided image I see packages like
selenium-webdriver
protractor
Those belong to dev-dependencies (see your package.json file) because they are used for testing. When building for production, no code from dev-dependencies should be included. The typescript package (which is nr.2 in size in your screenshot) will also not be present in production because types like string are only used for writing Typescript code, but the browser receives Javascript, which it is compiled to.
so far i've learned a bit about NodeJS. But now i want to write a huge enterprise app with it and i'm wondering how to setup the structure correctly? Coming from other languages like PHP and Java, i imagine, i would split my project in different NPM modules. For example #mybigproject/customer, #mybigproject/cart and #mybigproject/checkout and so on.
But those submodules would be installed in the node_modules folder of the application skeleton. How would i tell for example Express, that the template files are in the different module directories? Or for example i use TypeORM for data access. So each module would have it's own set of models. How do those models know the database configuration data, as it's only in the main application skeleton, or the other way around, how does the application skeleton should know where to find the models?
Don't use npm modules for different parts of your project.
This components is integral part of your project and usually depend on your global config / schema / routing / etc
Just put it in different files and require it where you need it.
You can get an idea for folders structure from projects like Sail.JS
Use npm modules if you writing some utility that going to serve you for different apps and you want an easy way to upgrade the utility code once for all your apps (or in case you want to share that utility as open source for all of us)
NPM can install your local folder as a dependency. (ref)
npm install <folder>:
Install the package in the directory as a symlink in the current
project. Its dependencies will be installed before it's linked. If
sits inside the root of your project, its dependencies may be
hoisted to the toplevel node_modules as they would for other types of
dependencies.
Your module keeps its original location after installed and a symlink is created as the same name of your module folder in the top level node_modules folder.
In these custom sub-modules, you can use __dirname and relative paths to locate you configuration files to feed to database or other data consumers.
But remember that, sub-modules often serve as utility functions for the main module. They should be independent from the project context.
I am building a RESTful API with the serverless framework to run on AWS (API Gateway + Lambda + DynamoDB). It's my first Node project and my first serverless production project and I have no idea how to structure it. So far I have the following
|--Functions
|-----Function1
|--------InternalModule
|-----Function2
|-----Function3
|--------InternalModule
|-----Function4
|--Shared
|-----Module1
|-----Module2
|-----Module3
|--Tests
|-----Functions
|--------Function1
|-----------InternalModule
|--------Function2
|-----------InternalModule
|--------Function3
|-----------InternalModule
|--------Function4
|-----------InternalModule
|-----Modules
|--------Module1
|-----------InternalModule
|--------Module2
|-----------InternalModule
|--------Module3
|-----------InternalModule
I keep my API endpoints (Lambda handlers) in Functions. Some of them have internal modules which they only use and there are some who use modules from Shared. I want to have unit tests for all my modules - inner and shared as well as API testing on the lambda functions. I am using mocha and chai and want to integrate everything in a pipeline which on a git push runs the linters and tests and if they are successful deploy the API to the appropriate stage. The problem is that in order to test each module I have to have chai as a local node module in every folder where I have a test file and reference the modules to be tested by a relative path. In most cases it looks really ugly because of the nesting. If I want to test an internal module from
Tests/Functions/Function1/InternalModule
and I require it on top of the test like so
require('../../../../Tests/Functions/Function1/InternalModule')
+ I have to install chai in every folder so it's reachable. The same goes for mocha and all the dependencies needed for the test and I haven't even mentioned configuration. The main idea I am now considering is weather or not I should bring all modules to a folder called modules and require them when needed - worst case
from Functions/Function1
require('../../Modules/Module1')
Also keep the test files in the module folder and run them inside, but that would require the assertion library installed in every folder. I've read about npm link and symlinks but I want to keep track of what dependencies each folder has so I can install them on the CI environment after the clean project is downloaded from GitHub where I can't do links (or I've got the whole concept wrong?)
If anyone can suggest a better solution I would highly appreciate it!
the way Node uses require is so much more than I thought!
First, Node.js looks to see if the given module is a core module - Node.js comes with many modules compiled directly into the executable binary (ex. http, fs, sys, events, path, etc.). These core modules will always take precedence in the loading algorithm.
If the given module is not a core module, Node.js will then begin to search for a directory named, "node_modules". It will start in the current directory (relative to the currently-executing Javascript file in Node) and then work its way up the folder hierarchy, checking each level for a node_modules folder.
read more here
I will try out Putting all modules in a separate folder each with it's own Folder prefixed with FunctionName_ so I know where each module is used, and the test file + package.json file. Then if I need a module I can require it from the functions with shallow nesting which would look not so bad:
from Functions/Function1
require('module-1');
with package.json
"dependencies":{
"module-1":"file:../../Modules/Function1_Module1"
}
and have a separate folder Shared where I keep the shared Modules.
I am still open for better ideas!
I am a relative newbie to node.js and typescript and am looking for tutorials and examples for building custom libraries for typescript. I am currently working on a project(switched to WebStorm GUI) that requires I build several microservices and several of the microservices will need to share certain code, i.e., base repository functionality, etc. My though would be to move the code they have in common to a series of separate libraries(projects) and make them into typings and have the projects that need them to install them. Following the example I found here, and using grunt: What is the story for creating and consuming TypeScript libraries?, I was able to generate the .js file and the .d.ts files in a dist folder on the project. There are a few areas that I am confused about:
1) The example makes reference to a "main": "./dist/my.service.js" section in the jsconfig file, how necessary is this file and what should go into it?
2) If all of the files are transpiled and added to the dist folder, will the dependent code still be able to access it or do I have to transpile all of the files to root?
3) After I have have all of the file successfully transpiled and moved to the proper location, how do I install them to the dependent project if they are local and not up on the npm or Definitely Typed, etc.?
Well if you would like to see an example project that I am currently working on that at the moment can be installed with npm is binary-type-tree. There are some things I need to fix but overall the project is working great. You can see my setup for Jest in my package.json. Along with how to set up the types and the main.
Depending on what your library will be focused on you will want to choose the appropriate module system. Right now commonJS is the most common for npm packages. Although you can only compile to one file with system or amd.
As for compiling you do not need gulp/grunt you can use Webstorms IDE to compile your files. Simply open up the Webstorm settings "Languages and Frameworks" and select "Typescript" there you will see enable Typescript compiler.
The example given is very old. Typescript had a major update to how typings work back in October of 2016, they moved to Typescript 2 I think, or maybe it was November. Anyway when you google search Typescript I would set the filter to not before that time.
1.
The main in the package.json of the Typescript project should point to the entry point of your library, or your executable. Since mine is a library it points to the compiled folder's index file. The file does not exist in my repo but it is there upon compilation.
2.
This I believe is answered depending on your build. In my build as an example I use module system es6 as you can see in my tsconfig.json file. This uses the ES6 module system.
if you would like to use commonjs the structure of my project will still work properly except you will need something like Systemjs or Babel on the front end.
3.
This one is a bit unknown to me but what I have done for only testing is copy my directory into the node modules of another project I am working on. Now to require the project you do not need a #types since I have "declaration": true in my tsconfig.json file. The package.json of my project has a "types": section which points to the compiled .d.ts file. The project that is requiring this new TS package, if also written in TS, will have to have the typeRoots and types section in the tsconfig.json file. Once this is all set up you should be able to require it just fine.
Make sure that the src of all your TS files is declared in the includes section of the tsconfig if your files are not compiled to root. Otherwise your project will have to require in files in an odd format like import * as BTT from "lib/basic-node/avl-tree";. Which is not what I wanted, once I added this it became import * as BTT from 'binary-type-tree';.
The "main" field in your package.json tells Node's module system what file to require when calling require(), so you will definitely need it. You point that field to a transpiled file, which will also contained transpiled references to your other transpiled TypeScript files, so you shouldn't have to add anything to root.
If you're trying to use the package from another project, you simply reference it as a dependency using NPM the same way you would with any other JS project.
{
"name": "dependent-project",
"dependencies": {
"bar": "file:../typescript-project"
}
}
Again, Node will know how to load the TypeScript project because you've specified the transpiled entry point in the main field.
There are several things you must consider:
In package.json, you must set some things up (example):
main property to point to your UMD compatible bundle
module property for ES5 module. Then modern workflows can benefit from it, for example to apply Tree Shaking
typings pointing to your .d.ts file (which should be generated)
That counts on a build process, which can be made with a module bundler, such as RollupJS or Webpack. They can generate source maps and so on.
As per question 3: you can install packages from local or even from Git repos. In your package.json, for example:
"your-library": "git+https://github.com/alexjoverm/typescript-library-starter.git"
I'd suggest you to take a look at TypeScript Library Starter. You can find there everything you need. It has configured out of the box:
Automatic releases
Package.json configuration
Universal module bundles
Source Maps
Typings (.d.ts) auto generated
Docs using TypeDoc
Tests and coverage